面向图像特征标识的存储系统查询机制研究

发布时间：2018-02-04 11:40

本文关键词： 查询服务局部灵敏哈希 MapReduce编程模型存储系统　出处：《华中科技大学》2012年硕士论文　论文类型：学位论文

【摘要】：随着数据规模不断增加和处理难度增大，大数据（Big Data）变得非常重要。存储系统对图像文件等非结构化的大数据查询服务缺乏有力支持，传统的文件元数据信息没有表达图像文件内容，区分度低。图像文件作为存储系统中一类增长迅速的文件数据，面临数据规模增加和查询性能优化的双重挑战。从这一点出发，提出一种面向图像文件的分布式存储系统中的索引策略，能够有效地提高图像文件查询效率，并能保证查询准确率。全文设计并实现面向图像文件的分布式查询系统，对用户的图像文件查询请求进行响应。采用执行速度快、具有良好稳定性的加速稳健特征算法提取图像的特征标识，对文件元数据进行辅助，增强对图像的表达力度。采用局部灵敏哈希方法（Locality Sensitive Hashing，LSH）解决高维的图像特征标识数据的最近邻居查询问题。研究的主要内容是结合局部灵敏哈希计算与MapReduce编程模型，提出一种分布式的索引策略，，在分布式环境的多节点上并行采集图像特征标识，并使用多节点并发的哈希计算生成索引记录，解决局部灵敏哈希方法在时间和空间上开销大的问题，提高存储系统中图像文件的查询效率。最后，为了将索引记录的生成和用户数据处理分离，采用关系型数据库存储结构化的图像文件索引信息，进一步缩短用户查询响应时间。实验结果表明，分布式的索引构建和图像特征标识采集可以显著提高系统时间性能，对用户查询响应快，能够保证查询准确率。
[Abstract]:Big data Big data becomes very important as the size of the data grows and processing becomes more difficult. The storage system lacks support for unstructured big data query services such as image files. The traditional file metadata information does not express the image file content, and the distinction is low. The image file is a kind of fast growing file data in the storage system. Facing the dual challenges of increasing data scale and optimizing query performance, an index strategy for image file oriented distributed storage system is proposed, which can effectively improve the efficiency of image file query. And can ensure the query accuracy. A distributed query system for image files is designed and implemented in this paper. An accelerated robust feature algorithm with good stability extracts the feature identification of the image and assists the file metadata. The local sensitive hashing method was used to enhance the expression of the image. The localization Sensitive Hashing was used. LSH) is used to solve the problem of nearest neighbor query for high dimensional image feature identification data. The main content of this study is to combine local sensitive hash computation with MapReduce programming model. A distributed indexing strategy is proposed, in which image feature identification is collected in parallel on multi-nodes in distributed environment, and index records are generated by multi-node concurrent hash computation. To solve the problem that the local sensitive hashing method is expensive in time and space, and improve the query efficiency of image files in storage system. Finally, in order to separate the generation of index records from user data processing. The relational database is used to store the structured image file index information, which further shortens the response time of user query. The experimental results show that the distributed index construction and image feature identification collection can significantly improve the time performance of the system, respond quickly to the user query, and ensure the accuracy of the query.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333;TP391.41

【共引文献】