基于表单特性的深层网络数据源分类方法研究

发布时间：2018-03-19 04:00

本文选题：深层网络　切入点：数据源自动分类　出处：《哈尔滨工程大学》2012年硕士论文　论文类型：学位论文

【摘要】：当前，Deep Web中蕴含着高质量的海量信息并且其数量还在不断地增长，由于DeepWeb具有分布、异构、自治等特点，用户高效、快捷地获取自己感兴趣的信息面临巨大挑战。然而，将Deep Web数据源按领域分类是解决这一挑战的基础。因此，研究DeepWeb数据源的组织问题具有重要意义。本文通过Web字典、课题组开发的深层网络数据源自动抽取工具以及搜索引擎收集了大量深层网络数据源，它们分别来自于航空订票、图书销售、汽车和房地产等四个领域。针对其中的200多个数据源进行统计和分析发现：第一，“主题词”能够较好地区分深层网络数据源。具体来说，查询接口源代码中，绝大多数title标记含有内容，，而且这部分内容中的有些词往往只出现在某个领域并且在一定程度上反映了该查询接口的主题，即所属的相关领域；第二，同一领域查询接口间相似属性的个数往往较多，不同领域接口间相似属性的个数则较少，或者几乎没有；第三，对于每个领域，随着深层网络数据源的增长，查询接口中属性出现的总的词汇量往往趋向于一个较小水平，平均大概在60左右；第四，深层网络中大部分为结构化的数据源。受此启发，基于表单特性——主题和表单属性信息，本文提出了一种新的深层网络数据源分类方法以及改进的查询接口相似性度量方法，实现了按照现实领域自动组织大规模深层网络数据源的目的。该方法主要由四大模块组成：预处理模块、标记策略模块、半监督K-Means聚类模块和后分类模块。本文还提出了一种查询接口标记策略，以降低随机选择初始中心点所产生的影响。实验结果表明：该方法能够有效、通用地解决深层网络数据源的分类问题并且具有较高的准确率和召回率。
[Abstract]:At present, deep Web contains mass information of high quality and its quantity is still growing. Because of the characteristics of DeepWeb, such as distribution, heterogeneity, autonomy, etc., it is a great challenge for users to obtain information of their own interest efficiently and quickly. However, Classification of DeepWeb data sources by domain is the basis to solve this challenge. Therefore, it is of great significance to study the organization of DeepWeb data sources. This paper collects a lot of deep network data sources through Web dictionary, automatic extraction tool of deep network data source developed by our research group and search engine, which come from airline ticket booking, book sales, etc. According to the statistics and analysis of more than 200 data sources, first, the "subject words" can better distinguish the deep network data sources. Specifically, in the source code of the query interface, The vast majority of title tags contain content, and some of the words in this content tend to appear only in one domain and to some extent reflect the subject of the query interface, that is, the related domain to which it belongs; second, The number of similar attributes among interfaces in the same domain is often more than that among interfaces in different domains. Third, for each domain, as the number of deep network data sources increases, the number of similar attributes among interfaces in different domains is less or less. The total vocabulary of attributes in the query interface tends to be smaller, with an average of about 60; 4th, most of the deep network is a structured data source. Inspired by this, this paper proposes a new classification method for deep network data sources and an improved method for measuring similarity of query interfaces based on form feature-topic and form attribute information. This method is mainly composed of four modules: preprocessing module, marking strategy module, and so on. Semi-supervised K-Means clustering module and post-classification module. This paper also proposes a query interface marking strategy to reduce the impact of random selection of initial center points. Experimental results show that the proposed method is effective. It solves the classification problem of deep network data sources and has high accuracy and recall rate.
【学位授予单位】：哈尔滨工程大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP393.09

【参考文献】