基于Hadoop的视频云转码系统的研究与设计

发布时间：2018-05-30 08:48

本文选题：hadoop + 云计算　；参考：《南京邮电大学》2017年硕士论文

【摘要】：当前市场上由于终端设备类型的多样化、各种视频媒体格式以及网络环境的差异性,需要一种能够解决海量视频转码、实时快速处理视频的技术。而且随着数据量的快速增加,传统单机视频转码系统的难以扩展性不足以达到处理能力随着数据量线性增加的要求。为了解决这个问题,本文采用了云计算技术中的Hadoop云计算平台,它是一个开源框架,同时支持分布式计算、海量数据处理与存储。Hadoop开源框架的出现使得解决面向多种终端设备、多种视频格式、多种码流要求的海量视频转码问题成为可能。本论文的主要工作是研究和设计基于Hadoop的视频云转码系统,将云计算技术和视频转码技术进行融合。本系统由两部分组成:Web服务器子系统和Hadoop云平台子系统,采用了Hadoop中的HDFS分布式文件系统进行视频文件存储,Hadoop中的MapReduce并行计算框架与Avidemux开源视频切片工具、FFMPEG视频转码工具、视频合并工具Mencoder共同完成视频转码。为了提高系统的转码性能,本文提出了改进后的用于对视频数据进行序列化的方法video Seralizable()和Hadoop云平台容错处理机制,目的是提高系统的转码性能,增强用户体验。通过测试,验证了本系统基本能够充分利用现有计算机的资源,实现大规模视频数据在Hadoop云计算平台上的分布式转码。与单机转码系统相比,达到提高转码效率,降低转码时间的目的。针对本文的研究现状提出了优化方案,在未来的工作中将具体实施。
[Abstract]:Due to the diversity of terminal device types, the differences of various video media formats and network environment, a technology is needed to solve the problem of mass video transcoding and real-time and fast processing of video. With the rapid increase of data volume, the scalability of traditional single-machine video transcoding system is not enough to meet the requirements of processing capacity linearly increasing with the amount of data. In order to solve this problem, this paper adopts Hadoop cloud computing platform in cloud computing technology, which is an open source framework, and supports distributed computing. The emergence of mass data processing and storage. Hadoop open source framework makes the solution for a variety of terminal devices. It is possible to solve the problem of mass video transcoding in various video formats and bitstream requirements. The main work of this thesis is to study and design the video cloud transcoding system based on Hadoop, and combine cloud computing technology with video transcoding technology. This system consists of two parts: Web server subsystem and Hadoop cloud platform subsystem. The HDFS distributed file system in Hadoop is used to store video files and the MapReduce parallel computing framework in Hadoop and the Avidemux open source video slicing tool FFMPEG video transcoding tool. Video merge tool Mencoder to complete the video transcoding. In order to improve the transcoding performance of the system, this paper proposes an improved method for serializing video data, video Seralizable, and the fault-tolerant processing mechanism of the Hadoop cloud platform. The purpose is to improve the transcoding performance of the system and enhance the user experience. Through the test, it is verified that the system can make full use of the resources of the existing computer and realize the distributed transcoding of large-scale video data on the Hadoop cloud computing platform. Compared with single-machine transcoding system, the efficiency of transcoding is improved and the time of transcoding is reduced. According to the present situation of this paper, the optimization scheme is put forward, which will be carried out in the future.
【学位授予单位】：南京邮电大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TN919.8

【参考文献】