数据流与磁盘存贮表的连接计算算法

发布时间：2018-01-16 13:29

本文关键词：数据流与磁盘存贮表的连接计算算法　出处：《控制工程》2014年06期 　论文类型：期刊论文

【摘要】：数据流与存贮表的连接查询经常出现在主动式数据仓库的维护中,与传统的关系数据库的连接计算不同,数据流快速处理的要求不允许将数据流先存入磁盘再计算,而计算机内存无法存储无界增长的数据流,因此数据流查询采用先处理再存储结果的计算方式。数据流与存贮表的连接计算算法重点要解决内存开销和处理速率二个问题。MESHJOIN算法最早提出将存贮表划分为若干个数据块,将数据块交替放入内存与数据流窗口完成连接计算。在MESHJOIN算法思想的基础上将存贮表的内存数据块也划分为若干逻辑分区,每次连接计算仅替换其中的一个逻辑分区,有效地降低了数据流滑动窗口所需的I/O代价,从而提高滑动窗口的计算速率。最后通过实验对二种算法在内存开销和计算速率进行了比较。
[Abstract]:The connection query between data stream and storage table often appears in the maintenance of active data warehouse, which is different from the traditional connection calculation of relational database. The requirement for fast data flow processing does not allow data streams to be stored on disk before computing, while computer memory cannot store unbounded growing data streams. Therefore, the data stream query is calculated by processing the results before storing the results. The algorithm for computing the connection between data stream and storage table focuses on solving the two problems of memory overhead and processing rate. The MeshJOIN algorithm was first proposed. The storage table is divided into several data blocks. The data blocks are placed alternately into memory and data stream windows to complete the connection calculation. The memory data blocks of the storage table are divided into several logical partitions on the basis of the MESHJOIN algorithm. Each connection computation replaces only one of these logical partitions, which effectively reduces the I / O cost required for the sliding window of the data flow. In order to improve the calculation rate of sliding window, the memory overhead and computing rate of the two algorithms are compared by experiments.
【作者单位】：广东警官学院计算机系;广州市公共安全重点研究基地;武汉大学软件工程国家重点实验室;暨南大学计算机系;
【基金】：公安部技术研究计划项目(2014JSYJB048) 广东省自然科学基金(S2012010008831) 武汉大学软件工程国家重点实验室开放基金(SKLSE2012-09-37)
【分类号】：TP311.13;TP333
【正文快照】： 1引言源源不断产生的数据流与磁盘存贮表的连接查询经常出现在主动式数据仓库的实时维护中,如数据仓库集成端的代理主码分配、重复元组检测等,该类增量维护可表示为S嗷喙cR,S是数据源产生的增量流,R是磁盘存贮表,且R往往不能一次全部放入主存,R的更新相对S的流速可以忽略。传

【参考文献】