复杂数据流分析关键技术研究

发布时间：2018-10-11 15:00

【摘要】：近些年，在传感器网络、互联网监控、金融交易等领域，出现了数据流分析的应用需求。然而，传统的数据处理方式不能很好地解决这种数据流分析应用，针对这些应用的数据流分析技术在学术上和工业上都受到了高度关注。随着复杂数据流分析的需求增加，研究复杂数据流分析的关键技术也越来越重要。在进行复杂数据流分析时，现有的解决方法通常是设计特定的复杂概要数据分析算法来解决。但对于越来越多的各种复杂数据流分析应用，每次都开发相应的分析算法来解决的方式显然不能很好地满足需求，一种更好的方式是重复利用一些比较通用的分析算法，通过与其他算法的组合来完成复杂数据流分析。然而，在组合分析时一个分析算子的输出数据如何很好地传给后续分析算子的问题需要得到解决，也就是两个算子之间的数据交换问题。结合数据流分析要求的高效的时间空间性能，，设计了较为高效和通用的数据交换方式。数据交换采用了基于模式匹配的星状交换方式，该方式中需要解决中间数据模型，数据模式的描述，模式匹配过程和模式转换描述几个关键问题。结合数据流分析中关系数据和复杂数据的不同特点，设计了中间数据的元数据模型；同时设计了基于XML标签对该数据模型中的数据模式进行描述；给出了模式匹配的两个要点：结构匹配与类型匹配，同时为了最佳匹配考虑了名称匹配的问题；最后利用可根据数据模式预分配实际数据的特点，设计了简单高效的模式转换描述。介绍了研究中实现的数据流实时分析平台系统，同时介绍了其中数据交换的测试分析。对于复杂数据流分析中数据交换的设计能够比较高效和通用地解决组合数据流分析中的数据交换问题。
[Abstract]:In recent years, applications of data flow analysis have emerged in sensor networks, Internet monitoring, financial transactions and other fields. However, the traditional data processing method can not solve this kind of data flow analysis application well, and the data flow analysis technology for these applications has received great attention both academically and industrially. With the increasing demand of complex data flow analysis, the key technology of complex data flow analysis is becoming more and more important. In complex data flow analysis, the existing solution is usually to design a specific complex summary data analysis algorithm to solve the problem. However, for more and more complex data flow analysis applications, it is obvious that the corresponding analysis algorithms are developed to solve the problem. A better way is to reuse some common analysis algorithms. The complex data flow analysis is accomplished by combining with other algorithms. However, the problem of how to transmit the output data of an analysis operator to the subsequent analysis operator in combinatorial analysis needs to be solved, that is, the problem of data exchange between the two operators. Combining the efficient time and space performance required by data flow analysis, a more efficient and universal data exchange method is designed. Data exchange is based on pattern matching, which needs to solve several key problems, such as intermediate data model, data pattern description, pattern matching process and pattern transformation description. According to the different characteristics of relational data and complex data in data flow analysis, the metadata model of intermediate data is designed, and the data pattern in the data model is described based on XML tag. Two key points of pattern matching are given: structure matching and type matching, and the name matching is considered for the best matching. Finally, the characteristics of the actual data can be pre-assigned according to the data pattern. A simple and efficient description of pattern transformation is designed. This paper introduces the real-time analysis platform system of data flow, and introduces the test and analysis of data exchange. The design of data exchange in complex data flow analysis can solve the problem of data exchange efficiently and generically.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.06

【相似文献】