基于Kinect手势识别的虚拟环境体感交互技术研究

发布时间：2018-05-06 12:21

本文选题：虚拟环境 + 体感交互　；参考：《重庆邮电大学》2016年硕士论文

【摘要】：目前大多数虚拟环境中的人机交互应用仍然是基于鼠标、键盘、头盔显示器、数据手套、操纵杆等交互设备,难以让用户获得真实、自然的感觉,严重地制约了虚拟环境的实用价值。体感技术与虚拟环境技术的有机结合,可以生成一个完全逼真的,集视、听、触、情等多种感知的智能虚拟环境,给予人们自然、真实感受的同时,更能从数字化虚拟空间自由获得丰富、准确的信息与知识。本文面向虚拟环境自然人机交互应用需求,结合体感技术与虚拟环境技术,设计了一种基于手势识别的虚拟环境体感交互方法。基于Kinect体感器,获取手势数据,并采用不同的算法分别对静态及动态手势进行识别。研究了符合人体行为习惯的手势设计方法,设计了基于手势识别的虚拟矿井自然交互系统,实现了与虚拟矿井更加自然的人机交互,增加了用户的自然感及真实感。主要包括以下内容:首先,研究了基于手势识别的虚拟环境自然交互设计方法。提出一种基于手势识别的虚拟环境体感交互框架。在此基础上,研究了虚拟环境自然交互手势设计方法,以及基于Kinect的深度信息获取、人体骨骼追踪等关键技术。其次,研究了基于Kinect静态手势识别方法。基于深度图-灰度直方图分割出手势,根据手腕像素特征分割出手掌,并采用中值滤波进行平滑处理,并提出一种融合K曲率与凸缺陷的指尖检测方法。在此基础上,研究、实现了基于Kinect的静态手势识别算法,通过在不同环境下的测试,验证了算法的准确性和鲁棒性。然后,研究了基于Kinect动态手势识别方法。根据关节点信息实现对动态手势轨迹的追踪,在获取的关节点信息基础上,研究了动态手势特征提取及动态手势表示方法。提出了一种改进的动态时间规整(IDTW)动态手势识别算法,对传统的动态时间规整算法(DTW)的搜索路径进行约束与特征加权,并在公开的手势数据集上进行测试,结果表明,所提出的算法减少运算量的同时也提高了整体识别率。最后,面向虚拟矿井自然人机交互应用需求,设计了基于Kinect手势识别的虚拟矿井体感交互系统,定义虚拟矿井交互手势。通过典型交互场景,测试、验证了所提基于手势识别的虚拟环境体感交互方法的准确性和有效性。
[Abstract]:At present, most human-computer interaction applications in virtual environments are still based on interactive devices such as mouse, keyboard, helmet display, data gloves, joystick, etc. It is difficult for users to get real and natural feelings. It restricts the practical value of virtual environment seriously. The organic combination of body feeling technology and virtual environment technology can create a completely realistic, intelligent virtual environment with visual, listening, touching, feeling and so on, giving people natural and real feelings at the same time. More from the digital virtual space free access to rich, accurate information and knowledge. In this paper, a virtual environment interaction method based on gesture recognition is designed to meet the needs of the application of natural human-computer interaction in virtual environment, combined with the technology of body sense and virtual environment. Based on Kinect somatosensors, gesture data are obtained, and different algorithms are used to recognize static and dynamic gestures. This paper studies the gesture design method in accordance with human behavior habits, designs a virtual mine natural interaction system based on gesture recognition, realizes a more natural human-computer interaction with virtual mine, and increases the user's sense of nature and reality. The main contents are as follows: firstly, the design method of natural interaction in virtual environment based on gesture recognition is studied. This paper presents a framework of virtual environment somatosensory interaction based on gesture recognition. On this basis, the design method of natural interactive gesture in virtual environment, the key technologies of depth information acquisition based on Kinect, human skeleton tracking and so on are studied. Secondly, the static gesture recognition method based on Kinect is studied. Hand gesture is segmented based on depth map and gray histogram, palm is segmented according to the pixel feature of wrist, and the median filter is used to smooth the hand. A method of finger tip detection based on K-curvature and convex defect is proposed. On this basis, a static gesture recognition algorithm based on Kinect is implemented. The accuracy and robustness of the algorithm are verified by testing in different environments. Then, the dynamic gesture recognition method based on Kinect is studied. According to the information of the gate node, the tracking of the dynamic gesture track is realized. Based on the obtained information of the node, the feature extraction and the representation method of the dynamic gesture are studied. An improved dynamic gesture recognition algorithm based on dynamic time warping (IDTW) is proposed. The search path of the traditional dynamic time warping algorithm (DTW) is weighted by constraints and features, and tested on the open gesture data set. The results show that, The proposed algorithm reduces the computational complexity and improves the overall recognition rate. Finally, to meet the requirement of virtual mine natural human-computer interaction, a virtual mine body sensation interaction system based on Kinect gesture recognition is designed, and the virtual mine interaction gesture is defined. Through the typical interactive scenes and tests, the accuracy and effectiveness of the proposed method based on gesture recognition for virtual environment somatosensory interaction are verified.
【学位授予单位】：重庆邮电大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP391.41

【参考文献】