面向领域的Deep Web查询接口发现研究

发布时间：2018-05-16 18:44

本文选题：Deep + Web　；参考：《暨南大学》2014年硕士论文

【摘要】：深层网络指的是位于表层网络之下所隐藏的数据，需要用户填写表单发送查询请求才能获取，其数据量远远超过表层网络且信息价值巨大。由此原因，如何挖掘出位于深层网络中的海量数据成为了研究热点，特别是Deep Web的信息集成研究尤为重要。Deep Web数据集成中的第一步是Web数据库的发现，也就是查询接口的发现。但由于深层网络的数据位于众多的web数据库中，并且处于不断的变化中，相应的接口也可能随之改变，增大了获取的难度。其中最为突出的技术难点是：一，Web数据库分布广泛且数量巨大，获取包含查询接口的网页信息的效率问题有待提高；二，查询接口都是以表单的形式存在，但并非所有的表单都是查询接口，如何从中正确地筛选出Deep Web查询接口、提高分类正确性也是亟待解决的问题。围绕着Deep Web查询接口发现中的两个难题，本文主要做了以下工作：首先，对Deep Web进行研究，其中包括Deep Web的概念、规模、存在方式、获得方法以及Deep Web查询接口发现中的一些关键问题，提出本文研究的研究方向和内容。其次，对查询接口发现中用到的相关技术进行分析，，包括通常用的DOM解析和启发式规则研究，然后分析了查询接口发现的主要算法，并进行比较。再次，针对面向领域的Deep Web查询接口获取的效率问题，本文提出了一种查询接口发现算法，包括基于单线程和多线程算法，并进行试验对比，结果显示基于多线程的算法效率提升显著。最后，为了从获取的网页表单中正确地筛选出Deep Web查询接口，本文在前人研究的基础上，提出了基于启发式规则的K最近邻算法，用于从表单中正确识别出Deep Web查询接口，为了进行实验验证，本文从多种途径多个领域取得查询接口和非查询接口，并分别进行实验，实验结果表明，该算法能明显提高对Deep Web查询接口的辨别能力，特别市在面向图书领域的实例中，在查重率和查全率方面都有明显提升。
[Abstract]:The deep network refers to the data hidden under the surface network, which needs the user to fill out the form to send a query request to obtain, and the amount of data is far more than the surface network and the value of information is huge. For this reason, how to mine the massive data located in the deep network has become a research hotspot, especially the information integration research of Deep Web is especially important. The first step in the data integration of Deep Web is the discovery of Web database, that is, the discovery of query interface. However, because the data of the deep network is located in many web databases, and is in constant change, the corresponding interface may also change, which increases the difficulty of obtaining. Among them, the most prominent technical difficulties are: first, the Web database is widely distributed and the number is huge, the efficiency of obtaining the web page information containing the query interface needs to be improved; second, the query interface exists in the form of form. However, not all forms are query interfaces, so how to select the Deep Web query interface correctly and improve the classification accuracy is an urgent problem to be solved. Around the two difficult problems in Deep Web query interface discovery, this paper mainly does the following work: Firstly, this paper studies Deep Web, including the concept, scale, existing mode, acquisition method and some key problems of Deep Web query interface discovery, and puts forward the research direction and content of this paper. Secondly, the related techniques used in query interface discovery are analyzed, including DOM parsing and heuristic rules, and then the main algorithms of query interface discovery are analyzed and compared. Thirdly, aiming at the efficiency of domain oriented Deep Web query interface acquisition, a query interface discovery algorithm is proposed in this paper, including single thread and multithread algorithms, and compared with each other. The results show that the efficiency of multithreading algorithm is improved significantly. Finally, in order to select the Deep Web query interface correctly from the obtained web page form, this paper proposes a K-nearest neighbor algorithm based on heuristic rules, which is used to correctly identify the Deep Web query interface from the form. In order to validate the experiment, the query interface and the non-query interface are obtained from many fields. The experimental results show that the algorithm can obviously improve the ability of discriminating the Deep Web query interface. Special city in the book-oriented field of examples, in the search rate and recall rate has improved significantly.
【学位授予单位】：暨南大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.09

【相似文献】