基于压缩网络编码的WSN流式计算技术研究

发布时间：2018-07-25 08:10

【摘要】：在大范围部署的无线传感器网络(WSN)中,节点产生大量实时的环境特征数据流,而如何从这些大规模实时数据流中高效地发现异常数据正是流式计算的重要研究内容。对此,本文提出了在Spark Stream上构建WSN流式计算系统以检测异常环境特征数据;为优化数据流的传输和计算效率,本文引入了压缩网络编码技术,旨在改进大数据环境下的系统数据处理性能。本文主要研究内容如下:首先,设计实现了 WSN的末梢数据终端以感知测量环境特征数据,包括各类节点的物理结构和网络拓扑;还分析了节点间的基础数据传输模型和协议。WSN末梢数据终端网络规划为环带结构,各环带内按簇汇聚数据并向上传输,形成广义蝶形网络以获得压缩网络编码增益。其次,构建了流式计算平台来快速发现异常数据,该平台通过数据云网关接收WSN末梢数据终端汇聚的原始环境特征数据流,将同步后的数据记录推送到Spark Stream上的流式k-means程序中实时聚类,以快速发现大批数据流中的异常类簇。然后,改进了 k-means算法的微批内聚类更新方式。提出在Spark上实现k-means安全区间更新优化算法,削减单个微批数据流中的流式计算时间,使系统及时响应数据流累积后的聚类模型更新。最后,在系统的传输和处理阶段分别引入压缩网络编码和译码重构技术。传输阶段中,压缩链路传输流量,以提高系统传输效率;处理阶段中,利用Spark框架实现解码重构计算,充分利用大数据框架的计算性能,减少WSN节点性能消耗。本文形成了一个在大数据环境下进行WSN流式计算的异常数据实时发现系统,在系统的高效传输,快速处理,可靠保证等三方面进行了系统优化。
[Abstract]:In the large-scale deployed wireless sensor network (WSN), nodes generate a large number of real-time environmental feature data streams, and how to efficiently find abnormal data from these large-scale real-time data streams is an important research content of flow computing. In this paper, a WSN streaming computing system based on Spark Stream is proposed to detect the abnormal environment characteristic data, in order to optimize the transmission and computational efficiency of the data flow, the compressed network coding technology is introduced in this paper. The purpose of this paper is to improve the performance of system data processing in big data environment. The main contents of this paper are as follows: firstly, the end data terminal of WSN is designed and implemented to perceive and measure the environmental characteristic data, including the physical structure and network topology of all kinds of nodes; The basic data transmission model between nodes and the protocol. WSN terminal network is designed as a ring band structure. The data in each ring band is aggregated by clusters and transmitted upward to form a generalized butterfly network to obtain the coding gain of the compressed network. Secondly, a flow computing platform is constructed to quickly discover the abnormal data. The platform receives the raw data stream of the WSN terminal through the data cloud gateway. The synchronous data record is pushed to the streaming k-means program on Spark Stream to cluster in real time, so as to quickly find a large number of abnormal clusters in the data stream. Then, the update method of k-means algorithm is improved. An optimization algorithm for k-means security interval updating based on Spark is proposed to reduce the flow computing time in a single microbatch data stream and to enable the system to respond to the cluster model update after data flow accumulation in a timely manner. Finally, compression network coding and decoding reconstruction techniques are introduced in the transmission and processing stages of the system. In the transmission phase, the transmission flow of the link is compressed to improve the transmission efficiency of the system. In the processing stage, the decoding and reconfiguration computation is realized by using the Spark framework to make full use of the computational performance of the big data framework and the performance consumption of the WSN nodes is reduced. In this paper, a real-time discovery system of abnormal data based on WSN flow calculation in big data environment is developed. The system is optimized in three aspects: efficient transmission, fast processing and reliability assurance.
【学位授予单位】：南京理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP212.9;TN929.5

【参考文献】