To guarantee the safety and efficiency of industrial production and prevent accidents or losses caused by personnel negligence or negligence, this work proposes a personnel on-duty status recognition method. The method combines a human pose estimation algorithm and a target detection algorithm, which can automatically discriminate six states of personnel on duty. First, the original image is processed using a high-resolution network (HRNet) to generate human pose keypoint maps. Then SE-VGG16 is constructed by combining the squeeze-excitation network and VGG16 for feature extraction of human pose keypoint maps. Finally, the design of the lightweight convolutional neural network for primary classification and you only look once version 5 is used for reclassification for behaviors with similar action features. The experimental results show that the method has an average recognition accuracy of 98.27% with good robustness and generalization ability for six kinds of personnel on-duty status in multiple environments.