面向特定领域的Deep Web数据获取技术研究
发布时间：2018-07-25 06:03
【摘要】：随着互联网技术的飞速发展,隐藏在Web数据库中高质量的信息资源因为结构完整、数据量巨大而受到广泛的关注。然而这类信息资源只有用户向Web查询接口提交查询后才会以HTML页面的方式展现出来,使得传统的搜索引擎无法获取,因而被称为Deep Web。为了提高Deep Web资源的利用程度,需要将隐藏在查询接口之后的数据展现到查询结果页面中,并将其抽取成为结构化的数据。 本文对特定领域的Deep Web数据获取关键技术进行了研究。研究主要分为两部分：数据表面化以及数据记录抽取。主要研究内容如下： 1)针对Deep Web查询接口中的范围型属性,提出了一种基于采样的值域划分方法。该方法有效地提高了Top-k查询接口中数据表面化的效率。 2)针对查询接口中的分类型属性,改进一种基于层次树模型的数据表面化方法。该方法通过调整分类型属性的提交顺序,有效的减少了查询提交的次数。 3)针对查询接口中的文本型属性,本文采用了一种候选值筛选的方法。该方法利用候选属性值在样本库中的分布,对候选值进行筛选,增加了平均查询收益。 4)根据查询结果页面中特征节点的分布信息,提出了一种数据区域定位算法。该算法将页面的结构信息和数据记录的属性特征结合起来,弱化了网页结构变更对抽取效果的影响。 5)在数据记录抽取阶段,本文讨论了一种特征序列划分和树相似度相结合的数据记录抽取方法。该方法不但可以提高数据记录抽取的准确率,而且能够对齐数据记录的属性。 本文通过实验验证了上述算法的有效性,并设计了面向电子商务领域的Deep Web信息集成原型系统。
[Abstract]:With the rapid development of Internet technology, the high quality information resources hidden in Web database have received extensive attention because of its complete structure and huge amount of data. However, this kind of information resource is only displayed in the way of HTML page after the user submits the query to the Web query interface, which makes the traditional search engine unable to obtain, so it is called Deep Web. In order to improve the utilization of Deep Web resources, the data hidden behind the query interface should be displayed in the query result page and extracted into structured data. In this paper, the key technologies of Deep Web data acquisition in specific fields are studied. The research is mainly divided into two parts: data surface and data record extraction. The main research contents are as follows: 1) aiming at the range attributes in Deep Web query interface, a range partition method based on sampling is proposed. This method effectively improves the efficiency of data surfacing in the Top-k query interface. 2) aiming at the classification attributes in the query interface, a hierarchical tree model based data surfacing method is improved. This method reduces the number of query submissions effectively by adjusting the submission order of the type attributes. 3) aiming at the text type attributes in the query interface, this paper adopts a candidate value filtering method. The method uses the distribution of candidate attribute values in the sample database to filter the candidate values and increase the average query income. 4) based on the distribution information of feature nodes in the query result page, a data region location algorithm is proposed. The algorithm combines the structure information of the page with the attribute feature of the data record, which weakens the influence of the page structure change on the extraction effect. 5) in the data record extraction stage, In this paper, a method of data record extraction based on feature sequence partitioning and tree similarity is discussed. This method can not only improve the accuracy of data record extraction, but also align the attributes of data record. In this paper, the effectiveness of the above algorithm is verified by experiments, and a prototype system of Deep Web information integration oriented to the field of electronic commerce is designed.
【学位授予单位】：苏州大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP393.09
本文编号：2142827
[Abstract]:With the rapid development of Internet technology, the high quality information resources hidden in Web database have received extensive attention because of its complete structure and huge amount of data. However, this kind of information resource is only displayed in the way of HTML page after the user submits the query to the Web query interface, which makes the traditional search engine unable to obtain, so it is called Deep Web. In order to improve the utilization of Deep Web resources, the data hidden behind the query interface should be displayed in the query result page and extracted into structured data. In this paper, the key technologies of Deep Web data acquisition in specific fields are studied. The research is mainly divided into two parts: data surface and data record extraction. The main research contents are as follows: 1) aiming at the range attributes in Deep Web query interface, a range partition method based on sampling is proposed. This method effectively improves the efficiency of data surfacing in the Top-k query interface. 2) aiming at the classification attributes in the query interface, a hierarchical tree model based data surfacing method is improved. This method reduces the number of query submissions effectively by adjusting the submission order of the type attributes. 3) aiming at the text type attributes in the query interface, this paper adopts a candidate value filtering method. The method uses the distribution of candidate attribute values in the sample database to filter the candidate values and increase the average query income. 4) based on the distribution information of feature nodes in the query result page, a data region location algorithm is proposed. The algorithm combines the structure information of the page with the attribute feature of the data record, which weakens the influence of the page structure change on the extraction effect. 5) in the data record extraction stage, In this paper, a method of data record extraction based on feature sequence partitioning and tree similarity is discussed. This method can not only improve the accuracy of data record extraction, but also align the attributes of data record. In this paper, the effectiveness of the above algorithm is verified by experiments, and a prototype system of Deep Web information integration oriented to the field of electronic commerce is designed.
【学位授予单位】：苏州大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP393.09
【参考文献】
相关期刊论文 前8条
1 胡东东,孟小峰;一种基于树结构的Web数据自动抽取方法[J];计算机研究与发展;2004年10期
2 马安香;张斌;高克宁;齐鹏;张引;;基于结果模式的Deep Web数据抽取[J];计算机研究与发展;2009年02期
3 申德荣;马也;聂铁铮;寇月;于戈;;一种应用于Deep Web数据集成系统中的查询松弛策略[J];计算机研究与发展;2010年01期
4 田建伟;李石君;;基于层次树模型的Deep Web数据提取方法[J];计算机研究与发展;2011年01期
5 李效东,顾毓清;基于DOM的Web信息提取[J];计算机学报;2002年05期
6 刘玉奎;周立柱;范举;;中文深度万维网数据库的现状研究[J];计算机学报;2011年02期
7 刘伟;孟小峰;凌妍妍;;一种基于图模型的Web数据库采样方法[J];软件学报;2008年02期
8 凌妍妍;孟小峰;刘伟;;基于属性相关度的Web数据库大小估算方法[J];软件学报;2008年02期
相关硕士学位论文 前2条
1 杨舟;特定领域的Deep Web数据抽取与语义标注研究[D];苏州大学;2011年
2 陈洪平;面向Deep Web的数据抽取与语义标注技术研究[D];苏州大学;2010年
，本文编号：2142827
本文链接：https://www.wllwen.com/kejilunwen/sousuoyinqinglunwen/2142827.html
