基于用户行为分析的核能领域垂直检索系统研究

发布时间：2018-07-02 14:40

本文选题：用户行为分析 + 相似度评分算法　；参考：《合肥工业大学》2015年硕士论文

【摘要】：中国核网具有资讯及时、信息全面的特点。如何从杂乱无章的庞大数据和强干扰的信息中搜索出有价值的核能信息,这对人类的智能信息处理能力提出了挑战。由于网络资源的复杂性,通用搜索引擎已经不能满足用户信息检索的准确性需要。为了更及时、更精准地获取核能信息与核能数据,为中国核网高效及时提供精准的数据素材,保证这一核能领域门户网站的数据及时性、精确性、全面性。本文针对核能领域海量数据的高效获取、深度分类和信息检索开展研究。本文首先研究了针对核能领域专用数据的主题爬虫,实现了对核能主流数据的抓取、去噪。其次,基于支持向量机(Support Vector Machine)技术和IKAnalyzer分词技术对已爬取的海量数据按照中国核网的频道进行分类。最后,针对中国核网用户来源、搜索行为具有较为明显的聚类特性,对Lucene框架的检索排序算法进行改进,提出一种基于用户行为分析的检索排序算法,该算法很大程度上提高了搜索引擎的查询效率,为中国核网提供更有效、更精确的核能信息。相似度评分算法是全文信息检索的核心步骤,能够高效的对返回结果进行展示。本文对Lucene内部相似度评分算法进行研究之后,针对Lucene内部相似度排序算法仅考虑查询词出现频率而未考虑潜在用户心理这一问题,本文实现了面向核能领域的垂直检索系统,该系统基于改进的相似度评分算法,并且能够通过对用户最近一段时间内的检索与点击行为进行分析,获取一组用户偏好关键词,对查询词与偏好关键词进行组合评分。论文实验表明,该系统能够高效获取核能领域信息,并在保证正确率的情况下对信息进行分类,有效节约人力资源。通过对爬取的数据索引并验证其检索性能,可知改进相似度评分算法对检索时间延长30-40ms,却在在大众用户接受时间内,但该算法对搜索结果质量有一定提高。
[Abstract]:China's nuclear network has the characteristics of timely information and comprehensive information. How to search the valuable nuclear energy information from the huge data and the strong interference information is a challenge to the intelligent information processing ability of human beings. Because of the complexity of network resources, general search engine can not meet the accuracy of user information retrieval. In order to obtain nuclear energy information and nuclear energy data more timely and accurately, provide accurate data material for China nuclear network efficiently and timely, ensure the data timeliness, accuracy and comprehensiveness of this nuclear energy field portal. This paper focuses on the efficient acquisition, depth classification and information retrieval of massive data in nuclear power field. In this paper, firstly, we study the topic crawler of special data in nuclear energy field, and realize the acquisition and denoising of the mainstream data of nuclear energy. Secondly, based on support Vector Machine (SVM) and IKAnalyzer participle technology, the crawling mass data is classified according to the channel of China Nuclear Network. Finally, in view of the Chinese nuclear network user source, the search behavior has the obvious clustering characteristic, carries on the improvement to the Lucene frame retrieval sorting algorithm, proposes a retrieval sorting algorithm based on the user behavior analysis. The algorithm greatly improves the search efficiency of the search engine and provides more effective and accurate nuclear energy information for the Chinese nuclear network. Similarity scoring algorithm is the core step of full text information retrieval, which can efficiently display the returned results. After the study of Lucene internal similarity scoring algorithm, aiming at the problem that Lucene internal similarity ranking algorithm only considers the frequency of query words but not the psychology of potential users, this paper implements a vertical retrieval system for nuclear energy domain. Based on the improved similarity scoring algorithm, the system can obtain a group of user preference keywords and score the combination of query words and preference keywords by analyzing the retrieval and click behavior of users in the most recent period of time. The experimental results show that the system can obtain nuclear energy information efficiently and classify the information under the condition of ensuring correct rate, which can save human resource effectively. By indexing the crawled data and validating its retrieval performance, we can see that the improved similarity scoring algorithm can prolong the retrieval time by 30-40ms, but it is in the time of acceptance by the public users, but the quality of the search results is improved to some extent by the improved similarity scoring algorithm.
【学位授予单位】：合肥工业大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP391.3

【相似文献】