YOLO-based GNN for multi-person pose estimation

Ming Gong; Ruixu Liu; Vijayan K. Asari

doi:10.1117/12.3016568

7 June 2024 YOLO-based GNN for multi-person pose estimation

Ming Gong, Ruixu Liu, Vijayan K. Asari

Proceedings Volume 13040, Pattern Recognition and Prediction XXXV; 130400F (2024) https://doi.org/10.1117/12.3016568
Event: SPIE Defense + Commercial Sensing, 2024, National Harbor, Maryland, United States

Abstract

2D multi-person pose estimation is a well-studied problem for understanding humans in an image. This involves keypoint detection, which requires to detect and localize the points of interest (human joints). Multi-person pose estimation remains challenging because of occlusion of body parts, non-rigidity of human body, variable number of persons in an image and various scales. The most common existing method for keypoint detection is heatmap-based regression. However, there are several drawbacks. The precision relies on the resolution of the output heatmap; the computation is costly for post-process or pre-process for high resolution heatmap; the overlapping heatmap signals of spatial closely keypoints could not be distinguished. Therefore, heatmap-free pose estimation was emerged to tackle these problems. KAPAO and YOLO-Pose are the representations. They both utilized YOLO for keypoint detection since YOLO is an extremely fast object detection method with high accuracy. A graph consists of a collection of nodes and a collection of edges that connect the nodes. A human pose could be referred to a graph, where human joints are nodes and corresponding connection will draw the pose. Graph neural network (GNN) is designed for data with graph structure. Inspired by these, we introduce a YOLO-based GNN, a heatmap-free approach for 2D multi-person pose estimation. YOLO-based network is leveraged for keypoint detection. The detected keypoints and connections will be then re-arranged and refined by GNN. We tested our framework on COCO-2017 dataset and preliminary results show superior performance in accuracy and efficiency.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Ming Gong, Ruixu Liu, and Vijayan K. Asari "YOLO-based GNN for multi-person pose estimation", Proc. SPIE 13040, Pattern Recognition and Prediction XXXV, 130400F (7 June 2024); https://doi.org/10.1117/12.3016568

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available