基于多模态输入的手势识别算法研究

发布时间：2018-08-06 19:33

【摘要】：作为新一波科技浪潮的排头兵,人工智能正以前所未有的速度渗透到人类生活的方方面面。其中,人机交互技术作为人工智能领域的重要组成部分,受到广泛的关注。在众多的人机交互手段中,手势交互是最接近人类交流习惯也是最自然的一种交互方式,相关手势识别技术可以被用于聋哑人教学、智能家居和虚拟现实等应用场合,具有广泛的应用前景。在上述背景下,本文对基于视觉的静态及动态手势识别问题进行了重点研究,取得了一些富有实际意义的研究成果。本文的主要工作与创新点如下:1.深入研究了静态手势识别问题。针对传统的手势检测方法不能对前臂、手掌和手指区域进行很好的区分,导致手势识别效果低下的问题,提出了一种有效的、基于直线检测的冗余手臂去除方法。实验结果验证了方法的有效性。2.现有的静态手势识别算法大都首先利用形状分解方法提取手指特征,然后利用模板匹配技术实现对手势的分类。因此,手指检测算法性能的好坏会对整个系统的识别性能产生直接影响。为此,本文从以下三个方面对手指检测与识别算法进行了改进:(1)提出了一种新的融合形态学处理和曲率信息的手指区域分割算法:(2)提出了一种基于多参数的改进相似性度量方法;(3)提出了一种基于分层模板匹配的手势识别方法。实验结果表明,本文所提出的手势检测与识别方法能有效克服杂乱背景、类肤色区域等不利因素的影响,取得较为理想的检测与识别效果。3.提出了一种基于多卷积神经网络融合的动态手势识别方法。该方法从给定的深度图像序列出发,首先提取运动信息,然后将其送入到不同结构的卷积神经网络以预测相关的三维时序信息,据此可以从空间和时间的维度去捕捉连续运动特征,进而实现对动态手势的分类。定性和定量的实验结果验证了本文所提出的动态手势识别算法的性能。
[Abstract]:As the vanguard of a new wave of science and technology, artificial intelligence is permeating every aspect of human life at an unprecedented speed. Among them, as an important part of artificial intelligence field, human-computer interaction technology has received extensive attention. Among the many human-computer interaction methods, gesture interaction is the most close to human communication habits and the most natural way of interaction. Related gesture recognition technology can be used in deaf and mute people teaching, smart home and virtual reality and other applications. It has wide application prospect. Under the above background, this paper focuses on static and dynamic gesture recognition based on vision, and obtains some meaningful research results. The main work and innovation of this paper are as follows: 1. The problem of static gesture recognition is studied in depth. Aiming at the problem that the traditional hand gesture detection method can not distinguish the forearm, palm and finger regions well, which leads to the low performance of gesture recognition, this paper proposes an effective method for removing redundant arms based on line detection. The experimental results show that the method is effective. Most of the existing static gesture recognition algorithms first use shape decomposition method to extract finger features and then use template matching technology to achieve gesture classification. Therefore, the performance of finger detection algorithm will have a direct impact on the recognition performance of the whole system. To that end, This paper improves the algorithm of finger detection and recognition in the following three aspects: (1) A new algorithm of finger region segmentation based on morphological processing and curvature information is proposed; (2) an improved similarity based on multiple parameters is proposed. (3) A method of gesture recognition based on hierarchical template matching is proposed. The experimental results show that the proposed gesture detection and recognition method can effectively overcome the influence of clutter background, skin color region and other adverse factors, and achieve a more ideal detection and recognition effect. 3. A dynamic gesture recognition method based on multi-convolution neural network fusion is proposed. In this method, the motion information is extracted from the given depth image sequence, and then sent to the convolutional neural network with different structures to predict the related 3D temporal information. The continuous motion features can be captured from the dimension of space and time, and the classification of dynamic gestures can be realized. The qualitative and quantitative experimental results verify the performance of the proposed dynamic gesture recognition algorithm.
【学位授予单位】：中国科学技术大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.41

【参考文献】