基于图神经网络的姿态时空特征提取与匹配

发布时间：2024-12-19 04:28

　　随着技术的发展,视频和照片数据快速增加。对这些图片和视频中的特征进行分析有助于我们了解人类的行为,具有重要的理论和实践意义,因此我们需要分析姿势和姿势序列的特征。在这篇论文中,我们研究了如何使用图神经网络从2D人体姿态中提取特征,匹配相应的目标。先前的方法已经探索了如何利用图卷积来从2D姿势回归相应的3D姿态以及识别相应的动作等,但是它们假定了邻接矩阵中人类骨骼的自然拓扑结构,这使得图卷积的接收域受限.另外,这些方法仅利用2D姿势的位置信息,无法克服信息不足导致的深度歧义问题。同时,室外3D姿态标注是极其困难的,这大大限制了现有姿态提取与匹配模型的在室外的泛化性。为了解决这些问题,我们主要提出了两点改进,一是提出了自适应语义图卷积算子,在学习人体骨架自然连接的强度的同时学习不直接相连关节点之间的联系。二是我们提出利用序数深度信息,也就是关节点相对于其父节点是否离摄像头更近的信息,来构建骨架图。一方面,这有助于减少2D姿态固有的深度歧义,另一方面,这有助于克服野外环境难以获得3D关节点标注的难题。我们在三个不同的姿态时空特征提取与匹配任务上进行了实验:从单张2D骨架回归相应的3D姿态、从2...

【文章页数】：92 页

【学位级别】：硕士

【文章目录】：
摘要
Abstract
Chapter 1 Introduction
    1.1 Pose Feature Extraction and Matching
    1.2 Graph Neural Network
    1.3 Challenges
    1.4 Contribution
    1.5 Thesis Outline
Chapter 2 Literature Review
    2.1 Three Dimensional Human Pose Estimation from a Single Monocular Image
    2.2 Three Dimensional Human Pose Estimation from a Monocular Video
    2.3 Skeleton-based Action Recognition
Chapter 3 Three Dimensional Human Pose Estimation from a Monocular Image
    3.1 Semantic graph convolution networks
        3.1.1 Graph Convolution
        3.1.2 Semantic Graph Convolution
        3.1.3 Non-Local Layer
        3.1.4 Semantic Graph Convolutional Networks for 3D Human Pose Regression
    3.2 Adaptive Semantic Graph Convolution
    3.3 Utilizing Ordinal Depth Information
    3.4 Adaptive Semantic Graph Convolution Networks for 3D Human Pose Estimation
    3.5 Experiments
        3.5.1 Datasets and Protocols
        3.5.2 Implementation Details
        3.5.3 Experimental Results
        3.5.4 Ablation Experiments
        3.5.5 Visualization of the Adaptive Adjacent Matrix
        3.5.6 Qualitative Results
    3.6 Summary
Chapter 4 Three Dimensional Human Pose Estimation from a Monocular Video
    4.1 Introduction
    4.2 Problem Definition
    4.3 Spatial Temporal Graph Convolution
    4.4 Adaptive Semantic Graph Convolution and Using Ordinal Depth Information
        4.4.1 Utilizing Ordinal Depth Information
        4.4.2 Adaptive Semantic Graph Convolution for Spatial-Temporal Pose Graph
    4.5 Experiments
        4.5.1 Implementation details
        4.5.2 Experimental Results
        4.5.3 Ablation Study
    4.6 Summary
Chapter 5 Action Recognition Based on 2D Skeleton with Ordinal Depth
    5.1 Introduction
    5.2 Method
        5.2.1 3D Human Pose Estimation
        5.2.2 Spatial-Temporal Graph Convolutional Network for Action Recognition
    5.3 Experiments
        5.3.1 Dataset and Evaluation Protocol
        5.3.2 Implementation Details
        5.3.3 Experimental Results
    5.4 Summary
Conclusions
结论
References
致谢



本文编号：4017657