基于Kinect行为识别技术的体感辅助形体交互模式研究与实现

发布时间：2019-01-04 07:31

【摘要】：随着科技的迅猛发展,在产品设计领域,以满足基本功能需求为目标的设计逐渐转变为以关注使用者感性需求为导向。近年来,虚拟现实和体感交互技术的巨大发展也为计算机辅助产品设计创造了更加丰富多样的方式,极大地提高了产品设计的效率和人机交互友好性。本研究以人机交互、计算机图形学等理论为基础,基于Kinect平台开发一种基于行为识别和虚拟现实技术的体感辅助形体交互系统(Somatosensory Aided Shape Interaction,简称SASI),将体感技术引入到产品设计系统中,从而增加产品设计系统的可操作性以及可用性,对于提升产品设计系统的交互体验具有一定意义。对SASI系统的框架进行搭建,通过分析用户对产品设计和人机交互中的功能需求和非功能需求的分析,得到SASI系统的基本要素;对系统场景进行设计,建立场景间的层级式逻辑关系,对各个分场景及相互的接口进行分析和设计,研究各个场景的效果设置和实现的功能;基于图形学相关原理,得到光线追踪和碰撞检测算法综合计算得到的交点处的空间信息;通过对相关数据进行坐标变换,获得它们在相应坐标系下的坐标,从而完成与SASI系统UI控件进行交互的功能。针对现有动作识别算法识别高相似度动作能力的不足,提出一种稳定高效的高相似度动作搜索与识别算法。利用Kinect的深度数据流和骨骼数据流获得相关关节的信息,利用该信息定义人体结构特征向量模之比和结构特征角,从而获得相关数据的稳定的特征量来描述目标动作。分析变量之间的关联性,将相关向量数据输入改进后的BP神经网络进行学习,进而建立行为识别算法的完整模型,并验证所提取特征向量的有效性。其中,特征向量的提取方法与神经网络学习是该算法的关键。对SASI系统的人机交互模块的各项功能进行研究和实现,分析Kinect硬件与SASI系统的交互原理和相关接口功能的实现方法,研究Kinect API提供的包括彩色数据流和深度数据流在内的重要数据流的工作原理和使用方法,并通过Kinect相关数据流获得人体关节位置信息和旋转信息,从而完成对人体目标行为的识别,完成对SASI系统的手势交互。最后基于上述研究实现SASI系统的各个模块的人机交互功能并对其进行实例验证。
[Abstract]:With the rapid development of science and technology, in the field of product design, the design which aims to meet the basic functional needs is gradually changed to focus on the emotional needs of users. In recent years, the great development of virtual reality and body sense interaction technology has also created more rich and diverse ways for computer-aided product design, greatly improving the efficiency of product design and human-computer interaction friendliness. Based on the theories of human-computer interaction and computer graphics, and based on Kinect platform, this paper develops a kind of body interaction system (Somatosensory Aided Shape Interaction, (SASI),) based on behavior recognition and virtual reality technology. In order to increase the operability and usability of the product design system, it is significant to introduce the technology of body feeling into the product design system, which is helpful to enhance the interactive experience of the product design system. The framework of the SASI system is built, and the basic elements of the SASI system are obtained by analyzing the functional and non-functional requirements in product design and human-computer interaction. The design of the system scene, the establishment of the hierarchical logical relationship between the scenes, the analysis and design of each sub-scene and their interfaces, the study of the effect of each scene setting and implementation of the function; Based on the principle of graphic correlation, the spatial information of intersection is obtained by the integrated calculation of ray tracing and collision detection algorithm. Through the coordinate transformation of the related data, the coordinate of them in the corresponding coordinate system is obtained, and the function of interacting with the UI control of the SASI system is completed. A stable and efficient motion search and recognition algorithm with high similarity is proposed to overcome the shortcomings of the existing motion recognition algorithms. The depth data stream and bone data stream of Kinect are used to obtain the information of related joints, and the ratio of structural feature vector modules and the structural feature angle are defined by this information, and the stable feature quantity of the relevant data is obtained to describe the target action. The correlation between variables is analyzed and the correlation vector data is input to the improved BP neural network for learning. Then the complete model of the behavior recognition algorithm is established and the validity of the extracted feature vectors is verified. The extraction method of feature vector and the learning of neural network are the key of the algorithm. The functions of man-machine interaction module of SASI system are studied and realized. The interaction principle of Kinect hardware and SASI system and the realization method of related interface function are analyzed. This paper studies the working principle and using method of important data flow, including color data stream and deep data stream provided by Kinect API, and obtains joint position information and rotation information through Kinect data stream. Thus the recognition of human target behavior and gesture interaction to SASI system are completed. Finally, the man-machine interaction function of each module of SASI system is realized based on the above research and verified by an example.
【学位授予单位】：哈尔滨工业大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.52

【相似文献】