视频中人体行为识别若干问题研究

发布时间：2018-10-30 18:38

【摘要】：行为识别是计算机视觉、机器学习、人工智能等领域的热点与重点研究方向。该方向对图像、视频数据中的人体行为进行分析识别,其研究成果在安全监控、病残监护、多媒体内容理解、人机交互、虚拟现实等方面得到了切实应用。然而,现有的行为识别技术在实际应用中存在很多局限性。为满足实际需求,本文针对如下四个关于视频中人体行为识别问题展开研究。1)在特定场景下,某些行为的样本极难收集,如何利用极少的样本快速的对特定行为进行识别;2)在行人可检测的较复杂场景中,如何有效的对特定行为进行识别;3)在行人可检测的较复杂场景中,如何快速有效的对多类行为进行识别;4)在行人不可有效检测的复杂场景中,如何有效的对多类行为进行识别。本文从实际应用问题出发,以模式识别、机器学习等理论为基础,开展了一系列创新性的研究,并提出了如上四个问题的解决方法。本文主要的研究工作和贡献如下:1)提出了一种基于霍夫投票的全局行为表征方法,即位移直方图序列表示法。该方法首先对行为视频中的运动区域进行粗略估计;然后根据运动区域中连续多帧图像中的兴趣点的匹配情况,使用二维的位移直方图表征这些连续图像中人体的运动信息;最后根据位移直方图序列,采用矩阵余弦相似度的度量方式对行为进行识别;对于识别的行为,匹配的兴趣点精确地定位了行为发生的时空位置。实验结果表明,在静态或背景较均匀场景下,该方法能够有效的对特定行为进行检测识别。此外,该方法采用从粗到细的行为定位方式,有效的提高了行为的表征速度。该方法解决了在样本极少情况下,特定行为的识别与检测问题。2)提出了一种在新视角下对人体行为进行时空特征学习的方法。该方法首先对行为人体进行检测与跟踪,并使用多限制玻尔兹曼机(RBM)对人体各部位的时序形状特征进行时空特征编码;然后将人体各部位的时空特征编码通过RBM神经网络整合为行为视频的全局时空特征表征;最后通过训练的支持向量机分类器对行为进行识别。大量实验验证了该方法的有效性。这种从人体各部位的形状特征序列中提取时空特征的方法,开辟了行为特征提取的新视角。该方法解决了较复杂场景下,特定行为的识别问题。3)提出了一种基于倒排索引的快速的多类行为识别算法。该方法首先对检测与跟踪到的行为人体的兴趣区域,提取形状运动特征,并通过层级聚类的方法利用这些特征构建行为状态二叉树;基于状态二叉树,快速的将行为表征为行为状态序列;然后,通过构建的行为状态倒排索引表与行为状态转换倒排索引表,计算行为状态序列对应于各行为类别的两个分值向量;最后根据加权的分值向量来识别行为。实验表明,该方法能够快速的对多类行为进行识别。行为状态二叉树的应用,加快了对行为视频的行为状态序列表征;倒排索引表的使用,明显提高了多类行为的识别速度。该方法解决了较复杂场景下,多类行为的快速识别问题。4)提出了一种基于独立子空间分析网络,利用从视频中学习的空间特征对视频行为进行时空特征编码的方法。首先,该方法利用引入规则化约束的独立子空间分析网络,学习了一组时间缓慢不变的空间特征;对从采样的视频块中提取的此类特征在时间域与空间域上进行池化处理,得到了能够有效的识别行为的局部时空特征。然后,基于特征袋(BOF)模型使用提取的局部时空特征对行为进行表征。最后采用非线性的支持向量机分类器识别多类行为。实验结果表明,时间缓慢不变规则化约束与去噪准则的引入,使学习的空间特征及提取的局部时空特征对混乱背景,遮挡等因素具有较强的鲁棒性。该方法解决了复杂场景下,多类行为的识别问题。
[Abstract]:Behavior recognition is the focus and focus of computer vision, machine learning, artificial intelligence and so on. The analysis and recognition of human behavior in image and video data in this direction have been applied in safety monitoring, disability monitoring, multimedia content understanding, human-computer interaction and virtual reality. However, the existing behavior recognition technology has many limitations in practical application. In order to meet the practical needs, this paper studies the problem of human behavior recognition in video. 1) In a certain scenario, the samples of some behaviors are extremely difficult to collect and how to use very few samples to quickly identify specific behaviors. 2) how to effectively identify specific behaviors in a more complex scene detectable by a pedestrian; 3) how to quickly and effectively identify a multi-class behavior in a more complex scene detectable by a pedestrian; and 4) in a complex scene where the pedestrian is not effectively detected, How to identify the multi-class behavior effectively. Based on the theory of pattern recognition, machine learning and so on, this paper develops a series of innovative research on the basis of the theory of pattern recognition, machine learning and so on. The main research work and contribution of this paper are as follows: 1) A global behavior representation method based on Hov voting is proposed, i.e., the representation of the displacement histogram sequence. The method comprises the following steps of: roughly estimating the motion area in the behavior video; then, using a two-dimensional displacement histogram to characterize the motion information of the human body in the continuous images according to the matching condition of the points of interest in the continuous multi-frame image in the moving area; and finally, according to the displacement histogram sequence, The behavior is identified by a measure of the similarity of the matrix cosine; for the identified behavior, the matching interest points accurately locate the spatiotemporal positions of the behavior. The experimental results show that the method can detect the specific behavior effectively under the static or background more uniform scene. In addition, the method adopts a coarse-to-fine behavior positioning mode, and effectively improves the characterization speed of the behavior. the method solves the problem of identification and detection of specific behaviors in rare cases of samples. The method comprises the following steps of: firstly detecting and tracking a human body, and performing space-time feature coding on the sequence shape characteristics of each part of the human body by using a multi-limiting Boltzmann machine (RBM); then the space-time feature codes of each part of the human body are coded by the RBM neural network as the global space-time feature representation of the behavioral video; and finally the behavior is identified by the trained support vector machine classifier. A large number of experiments verify the effectiveness of the method. The method for extracting the time-space features from the shape characteristic sequence of each part of the human body opens up a new perspective of behavioral feature extraction. A fast multi-class behavior recognition algorithm based on inverted index is proposed in this paper. The method comprises the following steps of: firstly, detecting and tracking an area of interest of a human body to be tracked, extracting shape motion characteristics, and constructing a behavior state binary tree by utilizing the characteristics through a hierarchical clustering method; based on the state binary tree, the behavior is characterized as a behavior state sequence rapidly; then, calculating the behavior state sequence corresponding to the two score vectors of each behavior category by constructing the behavior state inverted index table and the behavior state transition inverted index table; and finally, identifying the behavior according to the weighted score vector. Experiments show that the method can quickly identify the multi-class behavior. the application of the behavior state binary tree accelerates the characterization of the behavior state sequence of the behavior video, and the use of the inverted index table obviously improves the recognition speed of the multi-class behavior. The method solves the problem of fast recognition of multi-class behavior in complex scenes. 4) A method based on independent subspace analysis network is proposed for space-time feature coding of video behavior using spatial features learned from video. firstly, the method utilizes an independent subspace analysis network introduced with regularization constraint to study a set of spatial features which are slowly invariant in a set of time; and performing pooled processing on the features extracted from the sampled video blocks in a temporal domain and a spatial domain, and the local time-space characteristics of the identification behavior can be effectively identified. Then, the behavior is characterized using the extracted local space-time feature based on the feature bag (BOF) model. Finally, the nonlinear support vector machine classifier is adopted to identify the multi-class behavior. The experimental results show that the time-invariant regularization constraints and the introduction of de-noising criteria make the spatial features of learning and the extracted local time-space features have strong robustness to the clutter background, occlusion and other factors. The method solves the problem of multi-class behavior recognition in complex scenes.
【学位授予单位】：电子科技大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP391.41

【相似文献】