用于视频动作检测的时空多任务神经网络

发布时间：2018-09-18 12:08

【摘要】：近些年来,通过使用深度学习技术,视频中的动作检测任务已经取得了十分显著的进步。在实际的应用中,更多的需求是在未裁剪的长视频中进行动作检测任务,然而由于在时间维度上定位一个动作的难度较大,目前已有方法的准确率都并不能令人满意。为了解决这一挑战,本文提出了一个基于时空特征的、多任务的、三维卷积神经网络,用于在未剪裁长视频中进行动作检测(包括时序定位)。首先,本文提出了一个融合网络结构,用于在训练阶段提取视频级时空特征。通过在动作识别任务数据集上评价这一融合网络结构,实验结果说明了视频级时空特征的有效性。第二,基于这一融合结构,本文提出了一个时空的多任务神经网络结构,这一网络结构中包含两个分支输出层分别用处动作分类和动作的时序定位。为了达到较高的时序定位的准确率,本文展现了一个新的时序回归方法,用于校正包含一个动作的时序候选框。与此同时,为了更好的利用视频中丰富的运动信息,本文引入了一个新的视频表示方法,交错图像,作为神经网络的一个另外的输入。综合以上本文提出的方法,最终本文的模型在标准数据集上的动作识别任务和动作检测任务上都超过了目前最好的方法。
[Abstract]:In recent years, the motion detection task in video has made remarkable progress through the use of deep learning technology. In practical applications, more demands are made on the task of motion detection in long uncut video. However, due to the difficulty of locating an action in time dimension, the accuracy of existing methods is not satisfactory. To solve this challenge, this paper proposes a spatio-temporal, multi-task, three-dimensional convolution neural network for motion detection (including timing localization) in unclipped video. Firstly, a fusion network structure is proposed to extract video-level temporal and spatial features during the training phase. By evaluating the fusion network structure on the action recognition task data set, the experimental results show the effectiveness of the space-time features at the video level. Secondly, based on this fusion structure, a spatio-temporal multitasking neural network structure is proposed, which consists of two branch output layers, which are used for action classification and timing localization respectively. In order to achieve high accuracy of timing location, a new time series regression method is presented to correct the timing candidates with one action. At the same time, in order to make better use of the abundant motion information in the video, this paper introduces a new video representation method, interlaced image, as another input of the neural network. Finally, the model of this paper outperforms the best methods in the task of motion recognition and action detection on the standard data set.
【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.41;TP183

【相似文献】