社区网络实时搜索引擎的研究

发布时间：2018-02-21 04:43

本文关键词： 搜索引擎社区网络网络爬虫全文搜索　出处：《哈尔滨工业大学》2012年硕士论文　论文类型：学位论文

【摘要】：随着互联网技术的不断发展，出现了各式各样具有很多丰富功能的网站，人们对网络的需求也不只满足与以往的看新闻，查资料，越来越多的人喜欢在网络中记录自己日常的生活，用简短的状态来表达自己的心情，或者对某种事情的看法。网络不仅是一个展现数据的平台，而且变成了展现用户的一个窗口。这部分由用户所创造的数据与之前的经过专业编辑创建的数据不同，其具有数据更自由，方式更灵活，内容更丰富，角度更全面，响应更迅速的特点，因此对这类数据的研究有着很大的意义。然而，当前的搜索引擎因为一些技术上的一些限制很难有效地获取这类数据。文章将搜索引擎划分为数据抓取，索引建立，查询处理，数据展示四个模块，分析了每个模块在抓取这类数据时遇到的难题，并针对这些困难，提出了新的理论和解决方案。在数据抓取部分，以往学术界认为网页的变化遵循泊松过程，而本论文分析了不同时间断对网页变化规律的影响，并利用用户之间的相互亲密度修正该变化规律，提出了新的网页变化模型。在索引建立方面，，提出了使用多种索引的方式，不但提高了结果的时效性，并且可以支持时间段内的统计数据查询。在数据排序中，改进了原有的以网页为基础的PageRank，考虑到了社区数据的新的属性，评论和回复，并且加入了用户的重要程度作为排序的指标。在数据的展示方面，提出了利用情绪将数据结果分类，以便于展示给用户更直观的数据。其次本论文以这些解决方案为基础，设计并实现了一个新型的面向社区网络的搜索引擎。文章的最后给出了实验结果，验证了系统具有很好的性能。
[Abstract]:With the continuous development of Internet technology, a variety of websites with a lot of rich functions have emerged. People's demand for the network is not only to meet the needs of the past, but also to read the news and check the materials. More and more people like to record their daily life on the Internet, to express their feelings in a brief state, or to view something. The Internet is not only a platform for displaying data. And become a window to show the user. This part of the data created by the user is different from the previous data created by professional editors. It has the characteristics of freer data, more flexible way, richer content, more comprehensive angle, and faster response. Therefore, the research on this kind of data has great significance. However, the current search engine is very difficult to obtain this kind of data effectively because of some technical limitations. In this paper, the search engine is divided into four modules: data capture, index building, query processing and data display. The difficulties encountered by each module in capturing such data are analyzed, and a new theory and solution are put forward in view of these difficulties. In the part of data capture, the academic circles used to think that the changes of web pages follow the Poisson process. However, this paper analyzes the influence of different time breaks on the changing rules of web pages, and uses the mutual affinity between users to correct the rule of change. A new web page change model is put forward. In the aspect of index building, the method of using multiple indexes is put forward, which not only improves the timeliness of the results, but also supports the query of statistical data in the time period. Improved the existing Page Rank-based page, taking into account the new attributes, comments and responses of community data, and added the importance of the user as a ranking indicator. In order to display more intuitionistic data to the user, the data result is classified by emotion. Secondly, based on these solutions, a new type of search engine for community network is designed and implemented in this paper. Finally, the experimental results are given to verify the good performance of the system.
【学位授予单位】：哈尔滨工业大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP391.3

【参考文献】