基于深度相机人脸与行人感知系统的设计与实现
发布时间：2018-10-11 10:18
【摘要】：计算机视觉是人工智能的重要研究领域,目标检测作为计算机视觉的基础任务,是学术界和工业界的研究热点。其中,关于人的感知更是具有广泛的应用意义,尤其是在智能安防、无人驾驶和移动机器人等行业。在这些行业的解决方案中很多使用深度相机进行人的感知,以达到快速准确以及三维定位的目的。在一些商业化及开源代码中,一般只针对单一设备或特定场景,并且难以根据用户需求进行二次开发或功能扩展与删减。基于以上原因,本文提出基于深度相机,易于扩展且方便开发的人脸与行人感知系统。本系统主要分为四个模块:硬件层、驱动层、应用层、可视化层。层间,层内的功能单元相互独立,接口格式统一,易于调用,方便功能单元以插件形式扩展与删减;硬件层兼容异构的相机设备,包括多种深度相机,彩色相机;驱动层统一相机的接口;应用层内的某个单元可以方便被其他单元调用,例如检测单元可为跟踪单元服务;可视化层使用机器人操作系统的3D可视化工具,能够以多种显示方式查看结果。本系统针对相机个数可分为单深度相机系统与多深度相机系统,前者的感知应用包括人脸检测与识别,行人检测与跟踪;后者克服单深度相机系统覆盖面小的缺点,组成相机网络,实现对行人的跨区域长时间的跟踪。本系统针对人脸感知集成了快速人脸检测与识别算法,方便部署于低功耗设备。对于RGB相机,集成Dlib人脸检测器。对于深度相机,本文提出了基于Dlib训练器并联合RGB-D信息进行人脸(头)检测的方法,可使用深度相机准确检测人脸(头)。人脸识别模块使用特征脸与费希尔脸的方法。对于行人感知,本系统使用了传统的基于RGB-D的算法和基于多模态深度学习模型的方法,前者使用强大的三维图像处理库PCL进行开发,后者基于当前快速而高效的faster R-CNN框架。本系统的跟踪模块利用Tracking-by-Detection的思想,并使用扩展卡尔曼滤波的方法以达到抗遮挡的效果。最后,本系统使用多个深度相机组成网络,使用相机标定的方法使得每个相机知道其它相机以及地面的位置,从而重构相机所覆盖到的三维世界,实现跨区域长时间的行人跟踪。
[Abstract]:Computer vision is an important research field of artificial intelligence. As a basic task of computer vision, target detection is a hot research topic in academia and industry. Among them, human perception has a wide range of applications, especially in intelligent security, unmanned and mobile robots and other industries. In many solutions in these industries, depth cameras are used for human perception to achieve rapid, accurate and three-dimensional positioning. In some commercial and open source code, it is generally only for a single device or a specific scenario, and it is difficult to carry out secondary development or functional expansion and deletion according to user requirements. For the above reasons, this paper proposes a face and pedestrian perception system based on depth camera, which is easy to expand and develop. The system is mainly divided into four modules: hardware layer, driver layer, application layer, visual layer. Among the layers, the function units in the layers are independent of each other, the interface format is uniform, easy to call, the function units are easily extended and deleted in the form of plug-in, the hardware layer is compatible with heterogeneous camera equipment, including various depth cameras and color cameras; The driver layer unifies the camera interface; one unit in the application layer can be easily invoked by other units, such as the detection unit, which serves the tracking unit; the visualization layer uses the 3D visualization tool of the robot operating system. Ability to view results in multiple displays. According to the number of cameras, the system can be divided into single depth camera system and multi-depth camera system. The former includes face detection and recognition, pedestrian detection and tracking, and the latter overcomes the shortcomings of small coverage of single depth camera system. A camera network is formed to track pedestrians across areas for a long time. This system integrates fast face detection and recognition algorithms for face perception, which is convenient to deploy in low power equipment. For RGB camera, Dlib face detector is integrated. For the depth camera, this paper presents a method of face (head) detection based on Dlib trainer and RGB-D information, which can accurately detect the face (head) by using the depth camera. The method of feature face and Fisher face is used in face recognition module. For pedestrian perception, the system uses the traditional algorithm based on RGB-D and the method based on multi-modal depth learning model. The former is developed using a powerful 3D image processing library PCL, and the latter is based on the current fast and efficient faster R-CNN framework. The tracking module of this system uses the idea of Tracking-by-Detection and the method of extended Kalman filter to achieve the effect of anti-occlusion. Finally, the system uses multiple depth cameras to form a network, using camera calibration method to make each camera know the location of the other cameras and the ground, so as to reconstruct the 3D world covered by the camera. Long time pedestrian tracking across areas is achieved.
【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.41
[Abstract]:Computer vision is an important research field of artificial intelligence. As a basic task of computer vision, target detection is a hot research topic in academia and industry. Among them, human perception has a wide range of applications, especially in intelligent security, unmanned and mobile robots and other industries. In many solutions in these industries, depth cameras are used for human perception to achieve rapid, accurate and three-dimensional positioning. In some commercial and open source code, it is generally only for a single device or a specific scenario, and it is difficult to carry out secondary development or functional expansion and deletion according to user requirements. For the above reasons, this paper proposes a face and pedestrian perception system based on depth camera, which is easy to expand and develop. The system is mainly divided into four modules: hardware layer, driver layer, application layer, visual layer. Among the layers, the function units in the layers are independent of each other, the interface format is uniform, easy to call, the function units are easily extended and deleted in the form of plug-in, the hardware layer is compatible with heterogeneous camera equipment, including various depth cameras and color cameras; The driver layer unifies the camera interface; one unit in the application layer can be easily invoked by other units, such as the detection unit, which serves the tracking unit; the visualization layer uses the 3D visualization tool of the robot operating system. Ability to view results in multiple displays. According to the number of cameras, the system can be divided into single depth camera system and multi-depth camera system. The former includes face detection and recognition, pedestrian detection and tracking, and the latter overcomes the shortcomings of small coverage of single depth camera system. A camera network is formed to track pedestrians across areas for a long time. This system integrates fast face detection and recognition algorithms for face perception, which is convenient to deploy in low power equipment. For RGB camera, Dlib face detector is integrated. For the depth camera, this paper presents a method of face (head) detection based on Dlib trainer and RGB-D information, which can accurately detect the face (head) by using the depth camera. The method of feature face and Fisher face is used in face recognition module. For pedestrian perception, the system uses the traditional algorithm based on RGB-D and the method based on multi-modal depth learning model. The former is developed using a powerful 3D image processing library PCL, and the latter is based on the current fast and efficient faster R-CNN framework. The tracking module of this system uses the idea of Tracking-by-Detection and the method of extended Kalman filter to achieve the effect of anti-occlusion. Finally, the system uses multiple depth cameras to form a network, using camera calibration method to make each camera know the location of the other cameras and the ground, so as to reconstruct the 3D world covered by the camera. Long time pedestrian tracking across areas is achieved.
【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.41
【相似文献】
相关期刊论文 前10条
1 张R，
本文编号：2263826
本文链接：https://www.wllwen.com/kejilunwen/ruanjiangongchenglunwen/2263826.html
