移动数据同步机制与真值发现算法研究

发布时间：2018-07-09 22:15

本文选题：移动计算 + 数据同步　；参考：《南京邮电大学》2016年硕士论文

【摘要】：目前数据同步业务已经成为信息化时代最重要的业务之一,移动设备的多样化和性能的提高都促进了移动数据同步业务的发展。然而移动计算环境的复杂性、动态性、弱连接性以及通信延迟与带宽相对有限等特征使得移动节点间的数据同步效率降低,基于传统集中式架构的移动数据同步方案难以高效地实现数据的一致性。为了解决上述问题,本文总结了主流的数据同步技术以及同步架构,针对移动计算环境提出一种混合式数据同步机制(Hybrid Data Synchronization Mechanism,HDSM),将集中式架构和Ad hoc架构有机融合,设置自组织域(Self-Organization Domain,SOD)为同步进程管理单位,通过数据通信局域化,减少了同步数据通信量,同时减轻了数据同步服务器负载。为了缩短同步响应时间,加快SOD中同步数据的分发速度,本文还提出了一种基于节点能力值的数据分发策略(Capacity-value-based Data Distribution Strategy,CDDS),根据移动节点综合处理能力值来建立SOD树分发路径,实现了SOD中各移动节点对同步数据的高效分发。针对增量数据捕获问题,本文调研了常用的增量同步方法例如时间戳法、触发器法、日志法、快照法、API法等并提出了一种基于轨迹变更的增量捕获策略(Increment Capture Strategy based on Track Changes,ICSTC),采用触发器捕获操作日志,并用净化方法合并操作日志得到净增量数据,减少了同步数据量,降低了数据的传输成本以及缩短了同步响应时间。如今互联网已然成为了一个巨型数据集,它的发展给人们生活和工作带来便利的同时,其提供的数据质量问题也日益突出。为了确保用户搜集数据的质量,提供给用户或决策者最为准确的数据,真值发现问题受到了广泛的关注。针对现有真值发现算法复杂难懂、准确性不高的问题,本文提出了一种基于实体属性的真值发现算法(Truth Finder Algorithm based on Entity Attribute,TFAEA),在数据源可靠性和事实准确性迭代计算机制的基础上,考虑事实相互支持度和数据源之间的依赖关系两个因素,简化了真值发现算法。为了提高真值发现算法的准确性,本文采用单向文本相似度与事实冲突程度相结合的方法来计算事实相互支持度,并提出了数据源对称饱和度方法来计算数据源之间的依赖程度。实验结果表明,TFAEA算法不仅稳定,而且有很高的准确性。最后,在基于混合式数据同步机制和基于实体属性的真值发现算法基础上,设计并实现了商品比购系统,详细阐述了系统的功能架构,分模块介绍了系统的具体实现。
[Abstract]:At present, data synchronization service has become one of the most important services in the information age. The diversification of mobile devices and the improvement of performance have promoted the development of mobile data synchronization services. However, the complexity, dynamic, weak connectivity and limited communication delay and bandwidth of mobile computing environment make the efficiency of data synchronization between mobile nodes reduced. Mobile data synchronization scheme based on traditional centralized architecture is difficult to achieve data consistency efficiently. In order to solve the above problems, this paper summarizes the mainstream data synchronization technology and synchronization architecture, and proposes a hybrid data synchronization mechanism (HDSM) for mobile computing environment, which combines the centralized architecture and Ad hoc architecture. Self-Organization Domain SOD (SOD) is set as the management unit of synchronous process. Through the localization of data communication, the amount of synchronous data communication is reduced, and the load of data synchronization server is reduced at the same time. In order to shorten the synchronous response time and speed up the distribution of synchronous data in SOD, this paper also proposes a new data distribution strategy based on node capability value (Capacity-value-based data Distribution Strategy CDDS), which establishes the SOD tree distribution path according to the comprehensive processing ability of mobile nodes. The efficient distribution of synchronous data between mobile nodes in SOD is realized. Aiming at the problem of incremental data capture, this paper investigates the commonly used incremental synchronization methods such as timestamp method, trigger method, log method, etc. In this paper, an incremental capture strategy based on trajectory change (Increment capture Strategy based on track change) is proposed, which uses trigger to capture operation log, and combines operation log with purification method to obtain net increasing data, which reduces the amount of synchronous data. The data transmission cost is reduced and the synchronous response time is shortened. Nowadays the Internet has become a huge data set, its development brings convenience to people's life and work, at the same time, the problem of data quality is becoming more and more prominent. In order to ensure the quality of data collected by users and provide the most accurate data to users or decision-makers, the problem of truth-finding has been paid more and more attention. Aiming at the problem that the existing truth discovery algorithms are difficult to understand and the accuracy is not high, a truth finding Algorithm based on entity attribute algorithm (TFAEA) is proposed in this paper, which is based on the iterative computing mechanism of data source reliability and factual accuracy. Considering the fact mutual support degree and the dependence between data sources, the truth discovery algorithm is simplified. In order to improve the accuracy of the truth discovery algorithm, this paper uses the unidirectional text similarity and the fact conflict degree to calculate the fact mutual support degree. A method of data source symmetry saturation is proposed to calculate the degree of dependence between data sources. The experimental results show that the TFAEA algorithm is not only stable but also accurate. Finally, based on the mixed data synchronization mechanism and the truth-value discovery algorithm based on entity attributes, the commodity purchase system is designed and implemented. The function structure of the system is described in detail, and the implementation of the system is introduced in detail.
【学位授予单位】：南京邮电大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TN929.5

【相似文献】