面向开源软件的聚类搜索系统设计与实现

发布时间：2018-05-20 21:00

本文选题：开源软件 + 聚类搜索　；参考：《国防科学技术大学》2012年硕士论文

【摘要】：利用开源软件来提高软件的开发效率和质量,已成为在软件工程领域的重要发展趋势。随着开源软件的快速发展和广泛应用，互联网上出现了大量面向开源软件开发和共享的开源社区。目前，，种类繁多、数量巨大的开源软件广泛分布于互联网的众多开源社区，这对开源软件的搜索和选择带来严峻挑战。如何自动收集和检索互联网开源社区中的海量开源数据，并对检索到的数据结果进行聚类分析，为用户提供一种面向开源软件的跨社区聚类搜索服务，是具有重要研究和实践价值的课题。本文深入分析了搜索引擎和聚类搜索相关技术，针对开源软件数据在互联网上的分布规律和数据特点，设计了面向开源社区数据爬取、属性抽取与索引、搜索结果聚类分析的开源软件搜索系统Influx，能够有效支持开源软件的跨社区聚类搜索。本文的工作主要包括：首先，本文对搜索引擎和聚类搜索相关技术进行了比较分析，针对开源社区搜索系统的特殊需求，提出一种面向开源软件的聚类搜索系统体系结构Influx，将此类聚类搜索系统结构划分为数据存储、数据检索、数据分析和数据访问四个层次，具有良好可扩展性。其次，设计了开源软件聚类搜索系统的信息检索机制和聚类分析机制。其中，基于Heritrix和Lucene平台设计了高效的开源软件信息爬取、信息抽取和属性索引机制，基于K-means算法设计一种改良的搜索结果聚类机制，以供用户选择性的浏览搜索结果。最后，实现了面向开源软件的搜索系统Influx并进行了实验，对系统功能和性能进行了验证。实验结果表明，Influx搜索系统能够有效支持在互联网范围进行跨社区开源软件搜索和搜索结果的聚类分析。
[Abstract]:The use of open source software to improve the efficiency and quality of software development has become an important development trend in the field of software engineering. With the rapid development and wide application of open source software, there are a large number of open source communities for open source software development and sharing on the Internet. At present, a wide variety of open source software is widely distributed in many open source communities on the Internet, which brings serious challenges to the search and selection of open source software. How to automatically collect and retrieve the massive open source data in the open source community of the Internet, and analyze the result of the data retrieval, so as to provide users with a cross-community clustering search service oriented to open source software. Is an important research and practical value of the subject. In this paper, the related technologies of search engine and clustering search are deeply analyzed. According to the distribution rule and data characteristics of open source software data on the Internet, this paper designs a method for data crawling, attribute extraction and indexing in open source community. Influx, an open source software search system for clustering analysis of search results, can effectively support cross-community clustering search of open source software. The work of this paper mainly includes: First of all, this paper makes a comparative analysis of search engine and cluster search technology, aiming at the special needs of open source community search system. A cluster search system architecture named Influx for open source software is proposed. The cluster search system is divided into four levels: data storage, data retrieval, data analysis and data access. Secondly, the information retrieval mechanism and clustering analysis mechanism of open source clustering search system are designed. Among them, based on Heritrix and Lucene platform, an efficient open source software information crawling, information extraction and attribute indexing mechanism is designed. Based on K-means algorithm, an improved search result clustering mechanism is designed for users to browse search results selectively. Finally, the open source software oriented search system Influx is implemented and tested, and the function and performance of the system are verified. The experimental results show that the Influx search system can effectively support cross-community open source software search and clustering analysis of search results.
【学位授予单位】：国防科学技术大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP311.52

【参考文献】