|
We propose a real-time novel framework for tracking specific joints in the human body on low resolution imagery using optical flow based Kalman tracker without the need of a depth sensor. Body joint tracking is necessary for a variety of surveillance based applications such as recognizing gait signatures of individuals, identifying the motion patterns associated with a particular action and the corresponding interactions with objects in the scene to classify a certain activity. The proposed framework consists of two stages; the initialization stage and the tracking stage. In the initialization stage, the joints to be tracked are either manually marked or automatically obtained from other joint detection algorithms in the first few frames within a window of interest and appropriate image descriptions of each joint are computed. We employ the use of a well-known image coding scheme known as the Local Binary Patterns (LBP) to represent the joint local region where this image coding removes the variance to non-uniform lighting conditions as well as enhances the underlying edges and corner. The image descriptions of the joint region would then include a histogram computed from the LBP-coded ROI and a HOG (Histogram of Oriented Gradients) descriptor to represent the edge information. Next the tracking stage can be divided into two phases: Optical flow based detection of joints in corresponding frames of the sequence and prediction /correction phases of Kalman tracker with respect to the joint coordinates. Lucas Kanade optical flow is used to locate the individual joints in consecutive frames of the video based on their location in the previous frame. But more often, mismatches can occur due to the rotation of the joint region and the rotation variance of the optical flow matching technique. The mismatch is then determined by comparing the joint region descriptors using Chi-squared metric between a pair of frames and depending on this statistic, either the prediction phase or the correction phase of the corresponding Kalman filter is called. The Kalman filter for each joint is modeled and designed based on a linear approximation of the joint trajectory where its true form is mostly sinusoidal in fashion. The framework is tested on a private dataset provided by Air Force Institute of Technology. This dataset consists of a total of 21 video sequences, with each sequence containing an individual walking across the face of the building and climbing up / down a flight of stairs. The challenges associated in this dataset are the very low-resolution imagery along with some interlacing effects. The algorithm has being successfully tested on some sequences of this dataset and three joints mainly, the shoulder, the hip and the elbow are tracked successfully within a window of interest. Future work will involve using these three perfectly trackable joints to estimate positions of other joints which are difficult to track due to their small size and occlusions.
|
