基于结构化认知计算的群体行为分析

发布时间：2018-05-07 18:20

  本文选题：计算机视觉 + 群体行为分析　； 参考：《哈尔滨工业大学》2017年博士论文

【摘要】：随着人口的快速增长、人群活动更加多样以及社会化进程的迅速发展,群体场景变得更加普遍,于是建模、分析和理解视频中群体行为数据的需求日益增强。相比于以往的视频内容分析的工作,群体视频中人群数量增大,场景更为复杂等因素使得对视频中的群体行为的分析问题面临着巨大的挑战。与此同时,群体行为中蕴含着很多跨学科领域问题的重要线索,理解群体行为的形成机理早已成为社会学和自然科学重要的研究课题之一。群体行为分析的研究可以为很多关键工程应用提供支持和相应的解决方案,如智能视频监控,人群异常监测,公共设施规划等。这使得对群体行为的高层语义理解和分析变得越来越迫切。对于视频中的群体场景人群行为的分析,简称群体行为分析,主要目的是以普通的监控视频为基础进行群体场景语义内容的理解和分析。在对研究现状分析的基础上发现现有方法的发展主要受到两方面挑战的制约,主要是群体认知机理匮乏和结构化语义缺失。本文以结构化认知信息在群体行为分析中的作用为出发点,以结构化认知信息在表达、协同、挖掘的三个阶段的表现形式为主线,对高效的群体行为计算框架和算法模型展开研究:基于结构化交互属性,期望获得描述群体行为交互作用的表达模型;利用结构化语义信息进行群组建模,探索群组的共生结构和形成结构的一致性以及多属性融合的群组协同模型;面向群体行为高层语义知识,挖掘群体情感和建模注意选择机制。研究群体行为在视频内容智能分析领域的具体应用,试图挖掘真实场景视频数据中出现的动态群体模式和行为。具体来说,本文的研究内容和主要贡献可以概括如下:首先,针对群体行为认知机理的缺乏,即“所提取的底层运动特征与高层群体语义之间需要认知机理来填补语义鸿沟”的问题,本文提出了一种基于结构化交互属性的群体认知表达模型,以刻画群体行为的交互作用来增强现有的群体表达的判别力和丰富性。现有的群体行为表达模型缺乏对社会性交互作用的深层建模,需诉诸于属性或概念特征,构建从底层运动描述到中层对象交互的特定语义表示。通过借鉴社会化群体行为认知机理,本文系统地提出了结构化交互属性的组织和表示方法,将量化后的属性作为群体表达,并从结构化属性自身特点出发提出了在线融合策略。在UMN、UCSD、UCF-Web多个数据集上进行了群体行为异常检测任务的比较实验。结果证明了基于结构化交互属性的群体表达模型的有效性。其次,本文就结构化语义信息在群组表达中的缺失,即“如何利用群组的结构及关联特性和多属性信息”的问题进行了拓展研究,通过协同建模来提高结构化的语义表示。本文提出了基于结构一致性图挖掘的群组检测方法,其中包含基于共生结构一致性的轨迹图词包模型来进行群体事件的刻画,以及基于形成结构一致性的密集子图模型进行群组结构的描述。在UMN、PETS等数据集上的实验结果表明,所提方法可以在群体事件识别和群组检测中有效地提高性能。更进一步,可以通过多种属性来全面地描述群组轮廓,包括同质、异质、拓扑属性等。本文探讨如何将群组的多属性信息进行融合,进而提出基于深度属性嵌入图学习的群体描述方法,来进行群体视频检索。所提方法整合多种属性到图排序的框架中,同时进行排序分数、属性权重和深度转换矩阵的优化。在CUHK-Crowd数据集上进行了群体视频检索实验,实验结果表明了所提方法的优异性能。最后,本文以结构化认知表示为基础,提出了面向群体行为高层语义知识挖掘的包括群体情感和注意选择机制的建模方法。针对群体情感,本文探讨结构化轨迹特征和情感空间的映射关系,进而提出基于结构化轨迹学习的群体情感建模方法。通过结构化的轨迹学习提取连贯的轨迹特征,进一步加权回归学习将特征映射到情感空间来构建群体情感曲线表示。实验结果表明,所提方法可以有效地进行群体情感的分类匹配等任务。另外,从群体场景的显著度建模出发,本文探讨群体场景中的注意选择机理,并提出了基于级联深度网络的群体显著度预测方法。实验结果表明,所提方法同时考虑到群体和显著度的感知特性,同主流方法相比更为有效。通过以上研究,本文对面向视频内容分析的群体行为表达和计算模型进行了深入的探索,为群体行为分析研究中所面临的关键问题提供了切实的解决方案。结果表明:结构化的认知因素在群体行为表达和应用中起到重要作用。通过引入结构化交互属性,可以提取出更丰富和易于理解的特征增强对象级的描述,从而提升异常检测任务的准确率;特定的群组结构化语义中具有共生和形成结构一致性,综合考虑利用一致性以及协同多属性优化可以显著提升群组模式分析的性能;结合群体行为认知机理,可以进一步对群体行为的情感和注意机制等高层语义进行合理解释和建模,同时能够有效地解决群体事件识别、情感分类、显著度预测等实际的应用问题。
[Abstract]:With the rapid growth of the population, the more diverse activities of the population and the rapid development of the process of socialization, the group scene has become more common, so the demand for modeling, analysis and understanding of the group behavior data in video is increasing. Compared with the previous video content analysis, the number of people in group video is increasing and the scene is more complex. Factors make a great challenge to the analysis of group behavior in video. At the same time, group behavior contains many important interdisciplinary clues. Understanding the formation mechanism of group behavior has already become one of the most important research topics in sociology and natural science. The key engineering applications provide support and corresponding solutions, such as intelligent video surveillance, crowd anomaly monitoring, public facility planning, etc.. This makes the high-level semantic understanding and analysis of group behavior becoming more and more urgent. Understanding and analyzing the semantic content of group scene based on monitoring video. Based on the analysis of the present situation, it is found that the development of the existing methods is mainly restricted by two challenges, mainly the lack of group cognitive mechanism and the lack of structured semantics. This paper is based on the role of structured cognitive information in group behavior analysis. On the basis of structured cognitive information in the three stages of expression, collaboration and mining, the main line is to study the efficient computing framework and algorithm model of group behavior: Based on structured interaction properties, we expect to obtain an expression model describing the interaction of group behavior, and use structured semantic information to model groups and explore groups. The conformance of the symbiotic structure and the formation structure of the group and the group cooperative model of multi attribute fusion, the high level semantic knowledge of group behavior, the mining of group emotion and the attention selection mechanism of modeling, and the application of group behavior in the field of video content intelligence analysis, trying to dig out the dynamic groups in the video data of real scene. In particular, the research content and main contributions of this paper can be summarized as follows: firstly, in view of the lack of cognitive mechanism of group behavior, that is, "the underlying movement features and the high-level group semantics need cognitive mechanism to fill the gap of complement meaning", this paper proposes a structured interactive attribute. The model of group cognitive expression to characterize the interaction of group behavior to enhance the discriminatory power and richness of the existing group expression. The existing model of group behavior expression lacks the deep modeling of social intercourse interaction. It needs to resort to attribute or conceptual features, and constructs a specific semantic representation from the bottom transport description to the middle object interaction. By drawing on the cognitive mechanism of social group behavior, this paper systematically proposes the organization and representation of structured interactive attributes, expresses the quantized attributes as groups, and proposes an online fusion strategy from the characteristics of structured attributes itself. The task of group behavior anomaly detection is carried out on multiple data sets of UMN, UCSD and UCF-Web. The results demonstrate the effectiveness of the group expression model based on structured interaction properties. Secondly, this paper extends the research on the lack of structured semantic information in the group expression, that is, "how to use the structure of groups and association characteristics and multi attribute information", and through collaborative modeling to improve the structure of the language. In this paper, a group detection method based on the structure consistency graph mining is proposed, which includes the trajectory graph packet model based on the conformance of symbiotic structure to depict the group events, and the description of the group structure based on the dense subgraph model based on the conformance of the structure. The experimental result table on the data set of UMN, PETS and so on The proposed method can effectively improve performance in group event recognition and group detection. Further, the group profiles can be described comprehensively through a variety of attributes, including homogeneity, heterogeneity and topological properties. This paper discusses how to integrate the multi attribute information of groups, and then proposes a group based on the depth attribute embedding graph to learn the group. Description method to carry out group video retrieval. The proposed method integrates multiple attributes to graph sorting framework, and performs sorting score, attribute weight and depth conversion matrix optimization. A group video retrieval experiment on CUHK-Crowd data sets is carried out. The experimental results show the excellent performance of the proposed method. Finally, this paper is structured. On the basis of cognitive representation, a modeling method including group emotion and attention selection mechanism for group behavior high-level semantic knowledge mining is proposed. Based on group emotion, this paper discusses the mapping relationship between structural trajectory characteristics and emotional space, and then proposes a group affective modeling method based on structured trajectory learning. Trajectory learning extracts coherent trajectory features, and further weighted regression learning maps features to emotional space to construct a group emotional curve. The experimental results show that the proposed method can effectively carry out the task of classification and matching of group emotions. In addition, from the modeling of the group scene, this paper discusses the group scene. The method of group saliency prediction based on cascade depth network is proposed. The experimental results show that the proposed method takes into account the perception characteristics of group and saliency, which is more effective than the mainstream method. Deep exploration provides a practical solution for the key problems in group behavior analysis. The results show that structured cognitive factors play an important role in the expression and application of group behavior. By introducing structured interaction properties, a more rich and understandable feature enhancement object level can be extracted. In order to improve the accuracy of the exception detection task, the specific group structure semantics have symbiotic and structural consistency. Considering the use of consistency and cooperative multi attribute optimization, the performance of the group pattern analysis can be improved significantly, and the cognitive mechanism of group behavior can be used to further the emotion and attention mechanism of group behavior. Such high-level semantics can be reasonably interpreted and modeled, and at the same time, it can effectively solve the practical application problems such as group event recognition, sentiment classification, saliency prediction and so on.

【学位授予单位】：哈尔滨工业大学
【学位级别】：博士
【学位授予年份】：2017
【分类号】：TP391.41
，

本文编号：1857972