NGB智能终端数据访问关键技术研究

发布时间：2018-03-14 09:13

本文选题：分布式数据去重复　切入点：差分压缩　出处：《中国科学技术大学》2013年博士论文　论文类型：学位论文

【摘要】：随着下一代广播电视网(the Next Generation Broadcasting network, NGB)建设进程的推进和三网融合业务的开展,开放业务平台上多样化的双向交互应用将广播电视网的终端用户从单一的数据被动接收者转变为数据生产者和消费者的双重身份,这就向NGB提出了数据存储需求。基于用户对数据方便、快捷、可靠共享方式的强烈需求,以及大部分广播电视网智能终端自身存储、计算资源有限的状况,采用高可扩展、按需分配的集中式存储资源池乃是符合信息产业技术发展趋势的低成本、高效率解决方案。在这一需求背景下,在NGB中建设适合自身网络特性的存储服务具有重要意义。另一方面,广播电视网数字化整体转换过程中形成的有服务器质量保证的高清视频类业务是广播电视网区别于互联网和宽带电信网的特色业务。与其他数据的共享需求不同,高清视频主要通过直播、点播和录播的方式欣赏,录制的大容量视频文件保存在高清媒体终端本地的存储设备中。由于存储设备普遍使用价格可接受、但访问性能提升逐渐趋于平缓的传统硬盘,不足以应付视频分辨率的几何式增长速度；加快高清视频录制播放时硬盘文件系统的并行读写速度,降低文件读写时处理器的使用率,保证用户欣赏时的流畅体验就显得尤为重要。针对这两方面的问题,本文的主要工作和创新点如下： 1.利用广播电视网络下行带宽大、且可管理分配的优势,提出一种参考差分压缩的去重复传输优化策略。在该传输策略中,存储服务器根据客户端发送的分块摘要值在数据库中查找相似文件和相似分块,并将相似分块回复给客户端,客户端将相似分块与本地数据进行差分压缩编码后上传更少量的数据。通过将一部分数据量从低带宽的一端移到高带宽的一端,而又不显著增加总的传输量,参考差分的去重传输策略减少了数据传输的总时间,加快了数据同步的速度。 2.提出一种Hash表与B+树相结合的目录索引结构HB-Index。使用Hash表索引的方式,可以在O(1)时间内找到文件名对应的索引节点；另外仅在目录文件中保存子文件的索引节点号,并由B+树组织这些子索引节点号,支持O(logdN)时间内文件的创建和删除。实测结果表明,HB-Index不仅能够快速地处理包含大量文件的单个文件夹,在分区内文件总数很大时,也有优越的全局性能。 3.提出一种磁盘文件系统-Boaz,用于提升高清视频的快速连续读写速度。它将元数据与数据分区域存储,设置合适的最小分配单元,在保证大文件最低连续读写带宽和最大磁盘空间利用率的同时,降低了写文件时的元数据更新频率和CPU资源消耗。另外,基于预期文件大小的数据块预分配策略增加了文件元数据的连续性,降低磁盘的老化速度。在高清视频播放器上的测试结果表明,Boaz在顺序读写和并行访问大文件时,能够获得比现有磁盘文件系统更快的访问速度和更低的CPU利用率。本文针对几个关键问题提出的优化解决方法,或许可以为NGB中终端用户数据的存储研究起到砖引玉的作用,在进一步完善之后能够为NGB的示范建设进程提供一些实际可应用的技术。
[Abstract]:With the next generation of broadcast television networks (the Next Generation Broadcasting network, NGB) and the advance of the construction of triple play business, open service platform interactive application diversification will end users of radio and television networks data from passive receivers into a single double identity data producers and consumers, it is proposed to NGB the data storage needs.
Based on data for user convenience, fast, strong demand for reliable sharing, as well as most radio and television network intelligent terminal storage, limited computing resources, a highly scalable, centralized storage resource pool according to needs is the low cost development trend of information technology industry, high efficiency in this demand solutions. Under the background, the construction of storage services for their own network characteristics has important significance in NGB.
On the other hand, the formation of the overall conversion of digital broadcast television network in the process of HD video service server to ensure the quality of radio and television network from the Internet and broadband telecommunication network business characteristics. Different from other data sharing requirements, mainly through direct broadcast high-definition video on demand, appreciation and taped the way large video files the record stored in the HD media storage device terminal in the local storage device. Due to the widespread use of the price is acceptable, but the access performance gradually flatten the traditional hard disk, is not enough to cope with geometric video resolution growth rate; speed up the parallel read and write speed HD video recording playback when the hard disk file system, reduce the use of documents to read write processor rate, to ensure that users enjoy the smooth experience is particularly important.
In view of these two aspects, the main work and innovation of this paper are as follows:
1. use of radio and television network downlink bandwidth, and management advantages, the paper proposes a reference differential compression to repeat transmission optimization strategy. In this scheme, the storage server according to block the client value in the database to find similar documents and similar blocks, and similar block reply the client, the client will be similar to the block and the local data are differential data upload less compression encoding. Through a portion of the amount of data from one end of the move to the end of the low bandwidth high bandwidth, but not significantly increase the total amount of transmissions, reference differential transmission strategy to reduce the total time of data transmission and the speed of data synchronization.
HB-Index. directory structure 2. presents a Hash table and B+ tree combined with Hash index method, O (1) found in the index node file name corresponding to the time; save the inode number sub file only in the directory file, and the B+ tree to organize these sub index nodes. Support O (logdN) to create and delete files in time. The measured results show that the single folder HB-Index can not only quickly deal with large numbers of files, the total partition file is very large, there are overall superior performance.
3. presents a disk file system -Boaz, to enhance the high-definition video rapid sequential read and write speed. It will metadata and data storage area, set the minimum allocation unit is appropriate, in which the minimum file sequential read and write bandwidth and maximum disk space utilization and reduce the metadata update frequency and write files CPU resource consumption. In addition, the expected file size of the data block allocation strategy based on increasing the continuity of file metadata, reduce the aging speed of the disk. The test results in the HD video player on the show, Boaz read and write access to large files in parallel order, can be obtained than the existing disk file system faster access and the lower CPU utilization rate.
The optimization solution proposed for several key problems in this paper may play a role of brick and jade in the research of end user data storage in NGB. After further improvement, it can provide some practical and applied technologies for NGB's demonstration and construction process.

【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2013
【分类号】：TN946.3;TP333.35

【相似文献】