MapReduce并行加速数据流多模式相似性搜索

发布时间：2018-04-27 18:00

本文选题：时间序列 + 数据流　；参考：《计算机应用》2017年01期

【摘要】：设计时间序列数据在Hadoop分布式文件系统(HDFS)中的有效存储方式,利用分布式缓存工具Distributed Cache将各子序列分发到Hadoop集群的计算节点上,将动态时间弯曲距离矩阵划分成多个子矩阵,采取并行迭代计算每条反对角线上子矩阵的方法,基于MapReduce编程模型,实现高效并行计算时间序列动态弯曲距离,通过改进剪裁冗余计算方法,设计实现一种数据流多模式相似性搜索并行算法。中国雪深长时间序列数据集的实验结果表明,当每条时间序列的长度达到5 000以上时,并行计算动态弯曲距离所需时间少于串行计算所需时间,当每条时间序列的长度达到9 000以上时,参与计算的集群节点越多,并行计算所需时间越少;当模式长度达到4 000、参与计算的集群节点数达5个以上时,从数据流中并行搜索出与模式匹配的相似子序列所需时间约为串行搜索所需时间的20%。
[Abstract]:The effective storage method of time series data in Hadoop distributed file system (HDFS) is designed. Each sub-sequence is distributed to the computing node of Hadoop cluster by using distributed cache tool Distributed Cache, and the dynamic time bending distance matrix is divided into several sub-matrices. Based on the MapReduce programming model, the dynamic bending distance of time series can be calculated efficiently by using the method of parallel iteration to calculate the submatrix on every anti-angle line. The method of cutting redundancy is improved. A parallel algorithm for data stream multi-pattern similarity search is designed and implemented. The experimental results of the Chinese snow depth time series data set show that when the length of each time series is more than 5,000, the time required for parallel calculation of dynamic bending distance is less than the time required for serial computation. When the length of each time series is more than 9,000, the more cluster nodes participate in the computation, the less time is required for parallel computing; when the length of the mode reaches 4,000, the number of cluster nodes participating in the calculation is more than five. The time required for parallel searching of similar sub-sequences matching patterns from the data stream is about 20 times of the time required for serial search.
【作者单位】：广西大学计算机与电子信息学院;广西科技信息网络中心;
【基金】：广西自然科学基金资助项目(2014GXNSFAA118396)~~
【分类号】：TP311.13;O211.61

【相似文献】