基于CNN的手势姿态估计在手势识别中的应用

发布时间：2018-10-29 09:05

【摘要】：手势识别是人机交互领域一个重要的研究方向。使用手势作为人机交互接口,自然、直观,又贴近人类交流习惯,其应用前景广泛。针对手势识别算法应用于人机交互时,往往要求使用者手势所在平面平行于摄像头成像平面即垂直于水平面才能进行识别这个问题,本文提出一种应用手势姿态估计方法于手势识别之中的算法。通过应用卷积神经网络对深度图中人手进行姿态估计,得到关键关节点的空间坐标再使用它们进行手势识别,使得非典型手势可以同典型手势一样被识别。本论文主要做了如下三部分工作:1.基于Kinect获取深度信息,跟踪并分割出复杂场景中的手势部分。对分割出来的手势深度图进行形态学处理及数据归一化得到能够输入卷积神经网络的手势深度图。2.对于用于姿态估计的卷积网络模型,通过加入非线性手势模型中间层以及使用多分辨率的手势深度图作为网络输入提高准确率;通过减少需要估计的手势关节点数只保留有关键作用的关节点以提高检测速度。实验证明,本文提出的网络模型使手势姿态估计平均误差降低了2.21mm。3.基于指尖点到指根距离与手指中间关节点到指根距离之间的比例可以表征手指的弯曲程度,而经过手势姿态估计步骤能够得到手势各关节点空间坐标,进一步求出关节点之间距离,所以本文以手指弯曲程度比例为特征应用于猜拳手势识别。本文手势识别算法平均识别率为95.8%,非典型性手势识别率94.6%。
[Abstract]:Gesture recognition is an important research direction in the field of human-computer interaction. Gesture is used as human-computer interface. It is natural, intuitive and close to human communication habits, so it has a wide application prospect. When the gesture recognition algorithm is applied to human-computer interaction, it is often required that the user's gesture plane be parallel to the camera imaging plane, that is, perpendicular to the horizontal plane. This paper presents an algorithm for gesture recognition using gesture attitude estimation method. By using convolutional neural network to estimate the human hand attitude in the depth map, the spatial coordinates of the key points are obtained and then used for gesture recognition, so that the atypical gestures can be recognized as typical gestures. The main work of this paper is as follows: 1. Get depth information based on Kinect, track and segment hand gesture in complex scene. The gesture depth map which can input convolutional neural network can be obtained by morphological processing and data normalization. 2. 2. For the convolution network model for attitude estimation, the accuracy is improved by adding the middle layer of the nonlinear gesture model and using the multi-resolution gesture depth map as the network input. The detection speed is improved by reducing the number of gesture nodes that need to be estimated. Experimental results show that the proposed network model can reduce the average error of gesture attitude estimation by 2.21mm. 3. Based on the ratio between finger tip distance and finger root distance, the curvature of finger can be represented by the ratio of finger tip to finger root distance, and the spatial coordinates of each node can be obtained by gesture attitude estimation. The distance between the knots is further calculated, so this paper applies the ratio of finger bending to the recognition of guessing hand gesture. The average recognition rate of gesture recognition algorithm is 95.8 and the recognition rate of atypical gesture is 94.6.
【学位授予单位】：南昌大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.41

【参考文献】