面向概念漂移数据流的分类算法研究

发布时间：2018-12-11 17:00

【摘要】：在信息通信技术如此发达的今天,气象监控、网络安全、电子商务等众多应用领域都产生了庞大的数据流,这些数据流中蕴含着大量有价值的信息,而传统的静态数据挖掘技术已无法适应高速、连续、无限、多变的数据流,研究适合、高效的数据流挖掘技术已成为数据挖掘领域的热点。分类是数据挖掘的一个重要分支,与之对应的数据流分类挖掘,更是亟待深入分析并具有现实意义的研究方向。数据流动态多变的特性使得其中隐含的目标或规律可能会随着时间或环境的改变而变化,即发生概念漂移。另一方面,在无线传感器网络、信用卡欺诈检测、网络监控等大量现实应用数据流中的数据项,由于设备精度、重复抽样、数据过时、隐私等原因,通常具有不同程度的不确定性,并不是已知并精确的。本文主要针对数据流分类中的概念漂移和数据不确定性这两个问题展开研究。针对数据流中概念会重复出现、历史概念和当前概念存在着一定的联系以及概念间的相互转换关系等特点,提出了ECA-RC算法。该算法运用集成分类的思想处理数据流中的概念漂移,但在学习过程中不会将暂时失效的概念及对应基分类器删除,而是把它们的基本信息存储起来,方便以后调用。考虑到永久存储全部历史信息占用的内存量较大,采用周期性删除出现频率较低的历史分类信息。另外,可根据概念间的转换关系预测即将到来的概念,在提高分类精度的同时又提高了时间效率。使用传统数据流分类算法处理数据流中存在的不确定数据项时,得到的结果往往不尽如人意。为有效利用数据流中的不确定信息,提出了一种针对不确定数据流的集成分类算法ECA-UCD,该算法将不确定数据用区间及其概率分布函数表示,仍用集成分类思想来解决分类问题,在合理处理数据流中不确定性的同时,还能有效解决数据流中隐含的概念漂移问题。最后通过实验结果验证所提算法具有较好的鲁棒性和较高的分类准确率。
[Abstract]:With the development of information and communication technology, meteorological monitoring, network security, electronic commerce and many other application fields have produced huge data flow, which contains a lot of valuable information. However, the traditional static data mining technology can not adapt to the high-speed, continuous, infinite and changeable data flow. The research on the suitable and efficient data stream mining technology has become a hot spot in the field of data mining. Classification is an important branch of data mining, and the corresponding data stream classification mining is an urgent need for in-depth analysis and practical significance of the research direction. Because of the dynamic characteristics of data flow, the implicit object or law may change with the change of time or environment, that is, the concept drift occurs. On the other hand, in wireless sensor networks, credit card fraud detection, network monitoring, and other practical applications, data items in the data flow, due to equipment accuracy, repeated sampling, outdated data, privacy, and so on, Often with varying degrees of uncertainty, it is not known and accurate. This paper focuses on the conceptual drift and data uncertainty in data stream classification. In view of the reappearance of concepts in data flow, the relationship between historical concepts and current concepts, and the relationship between concepts, a ECA-RC algorithm is proposed. The algorithm uses the idea of integrated classification to deal with the concept drift in the data stream, but does not delete the temporarily invalid concept and the corresponding base classifier in the learning process, but stores their basic information for later call. In view of the large amount of memory consumed by the permanent storage of all historical information, the history classification information with low frequency of periodic deletion is adopted. In addition, the coming concepts can be predicted according to the transformation relationship between concepts, which not only improves the classification accuracy but also improves the time efficiency. When the traditional data stream classification algorithm is used to deal with the uncertain data items in the data flow, the results are often unsatisfactory. In order to effectively utilize the uncertain information in the data stream, an ensemble classification algorithm (ECA-UCD,) for uncertain data flow is proposed. The algorithm uses the interval and its probability distribution function to represent the uncertain data. The idea of integrated classification is still used to solve the classification problem, which can deal with the uncertainty in the data flow reasonably and effectively solve the problem of the concept drift in the data stream at the same time. Finally, the experimental results show that the proposed algorithm has better robustness and higher classification accuracy.
【学位授予单位】：天津工业大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP311.13

【相似文献】