关系数据中函数依赖检测方法

发布时间：2018-06-07 15:22

本文选题：数据质量 + 函数依赖　；参考：《计算机学报》2017年01期

【摘要】：在数据质量研究中函数依赖被广泛用于关系数据不一致性的修复.然而,不一致修复问题面临的一个主要挑战是如何从包含有错误的关系数据中自动发现有效的函数依赖(Functional Dependence,FD).目前基于统计度量置信度的FD自动发现方法经常找出大量近似成立但无效的FD.如果直接利用这些FD修复数据,会产生更多错误.针对该问题,文中提出了一种基于数据语义分析的函数依赖检测方法.该方法通过条件概率来分析属性值和元组的数据置信度,进而计算函数依赖成立的置信度.文中同时提出了利用关系数据构建马尔科夫毯贝叶斯网络用以计算数据置信度的方法.最后文中通过实验在模拟数据和真实数据上验证了基于数据语义的置信度计算方法在自动检测中的精确度优于基于统计的计算方法,并且在交互式检测应用场景中数据语义的置信度所需用户工作量少于基于统计的方法.
[Abstract]:In data quality research, functional dependencies are widely used to repair the inconsistency of relational data. However, one of the major challenges to the problem of fixing inconsistencies is how to automatically find effective functional dependency from relational data containing errors. Current FD automatic discovery methods based on statistical metric confidence often find a large number of approximate but invalid FDs. If these FD fixes the data directly, it will produce more errors. To solve this problem, a method of functional dependency detection based on data semantic analysis is proposed. The confidence degree of attribute value and tuple is analyzed by conditional probability, and the confidence degree of functional dependency is calculated. At the same time, a method of using relational data to construct Markov blanket Bayesian network to calculate the confidence of data is proposed. Finally, the accuracy of the confidence degree calculation method based on data semantics is proved to be superior to that of statistical calculation method in automatic detection through experiments on simulated data and real data. Moreover, in the interactive detection application scenario, the user workload is less than the statistical method.
【作者单位】：西北工业大学计算机学院;
【基金】：国家“九七三”重点基础研究发展规划项目基金(2012CB316203) 国家自然科学基金(61332006,61472321) 西北工业大学基础研究基金(3102014JSJ0013,3102014JSJ0005)资助~~
【分类号】：TP311.13

【相似文献】