基于搜索引擎网络信息的压缩系统的研究和实现

发布时间：2018-06-17 21:14

本文选题：搜索引擎 + 压缩　；参考：《华中科技大学》2012年硕士论文

【摘要】：随着互联网的飞速发展，，网络信息呈现大爆炸的趋势，获取这些海量信息的搜索引擎应运而生。面对搜索引擎获取的海量信息，研究如何利用有效的技术来存储和检索这些信息具有重要的意义。目前搜索引擎中常见的方法是利用数据库或索引来存储这些信息并提供接口来检索。本文实现了一种压缩技术和索引相结合的技术方法来解决搜索引擎的网络信息存储和检索问题。本文首先介绍了搜索引擎相关技术，并详细介绍了本文系统的方案所运用的Zip和GZip压缩算法原理，为最终系统的实现奠定了理论基础。然后再针对搜索引擎的网络信息特点进行重点分析。这种信息具有信息量大，时效性强，多媒体化信息，超链接方式等鲜明特点。设计了针对文本信息和非文本信息分别采用Zip算法和GZip算法的方案，进而设计了基于搜索引擎网络信息的压缩系统的方案。接着实现了基于搜索引擎网络信息的压缩系统。系统分为三部分，首先采集网络信息，然后通过压缩算法处理并存储这些数据，最后检索时将数据解压还原为原始信息。最后通过实验测试，从准确性、压缩率、压缩速度三个指标对本文设计的基于搜索引擎信息压缩方案的效能进行评估。实验数据证明了该压缩系统方案的压缩算法具有很高的压缩率和压缩速度且具有实用性，从而有效地解决了搜索引擎的数据存储和检索问题。
[Abstract]:With the rapid development of the Internet, the network information presents the trend of big explosion, the search engine that obtains these massive information emerges as the times require. In the face of the massive information obtained by search engines, it is of great significance to study how to use effective technology to store and retrieve these information. At present, the common method in search engine is to use database or index to store this information and provide interface to retrieve it. In this paper, a compression technique combined with index is implemented to solve the problem of web information storage and retrieval in search engines. This paper first introduces the search engine technology, and introduces in detail the Zip and GZip compression algorithms used in the scheme of the system, which lays a theoretical foundation for the final implementation of the system. Then focus on the characteristics of the search engine network information. This information has a large amount of information, strong timeliness, multimedia information, hyperlinks and other distinct characteristics. The scheme of Zip algorithm and GZip algorithm for text information and non-text information are designed, and then the scheme of compression system based on search engine network information is designed. Then the compression system based on search engine network information is implemented. The system is divided into three parts. Firstly, the network information is collected, then the data is processed and stored by the compression algorithm, and the data is decompressed and restored to the original information when the data is retrieved. Finally, the efficiency of the information compression scheme based on search engine is evaluated from three indexes: accuracy, compression ratio and compression speed. The experimental data show that the compression algorithm of this scheme has high compression rate, compression speed and practicability, thus effectively solving the problem of data storage and retrieval in search engines.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP391.3

【参考文献】