基于Hadoop的电梯安全大数据挖掘研究

发布时间：2018-03-18 17:16

本文选题：电梯安全　切入点：数据挖掘　出处：《西安科技大学》2017年硕士论文　论文类型：学位论文

【摘要】：近年来,随着电梯的使用频率越来越高,电梯安全事故频发,由此引发的电梯安全问题引起了广泛的关注。电梯远程监管系统是一个综合性电梯管理平台,所采集到的电梯信息会通过GPRS无线网络传输到服务器。随着4G时代的到来,数据传输速率增加,数据库中积累的数据越来越多,电梯也进入了大数据时代。本课题的主要内容是基于Hadoop对海量的电梯数据进行挖掘分析,设计并实现了一个基于Hadoop的数据挖掘系统,主要包括四个模块:数据导入导出模块,数据预处理模块,数据挖掘模块和调度模块。其中,数据导入导出模块的作用是实现关系型数据库SQL Server与HDFS的数据传输;数据预处理模块的主要目的是对电梯数据进行清洗,其原因是原始的数据中包含有大量的“脏数据”,若不进行预处理会对挖掘结果造成一定的影响;数据挖掘模块是本课题的核心模块,其作用是实现对电梯数据的挖掘工作,该模块实现了两种改进的算法,一种是聚类分析算法一 K-Means,另一种是关联规则算法—Apriori。最后通过调度模块将所有的模块整合起来,使其成为一个完整的系统。本课题以电梯远程监管系统数据库中的数据作为数据源,对改进前后的两种算法分别在Hadoop平台上进行实验,并从加速比和可扩展性两方面进行了比较分析,结果表明两种改进后的算法的性能均有很大的提升。利用改进后的两种算法分别对电梯数据进行聚类分析与关联规则挖掘,不但提高了算法的效率,而且保证了数据挖掘的充分性,为电梯安全可靠性分析提供了支持。
[Abstract]:In recent years, with the increasing frequency of elevator use and frequent accidents of elevator safety, the elevator safety problems caused by it have aroused widespread concern. The elevator remote supervision system is a comprehensive elevator management platform. The elevator information collected will be transmitted to the server via the GPRS wireless network. With the arrival of the 4G era, the data transmission rate will increase, and more and more data will accumulate in the database. Elevator has also entered big data era. The main content of this subject is to mine and analyze mass elevator data based on Hadoop, and design and implement a data mining system based on Hadoop, which includes four modules: data import and export module. Data preprocessing module, data mining module and scheduling module. Among them, the function of data import and export module is to realize the data transmission between relational database SQL Server and HDFS, the main purpose of data preprocessing module is to clean elevator data. The reason is that the raw data contains a large number of "dirty data", if not pre-processing will have a certain impact on the mining results; data mining module is the core module of this topic, its role is to achieve the elevator data mining work, This module implements two improved algorithms, one is clustering analysis algorithm K-Means, the other is association rule algorithm -Apriori.Finally, all the modules are integrated by scheduling module. In this paper, the data in the database of elevator remote supervision system is used as the data source, and the two algorithms before and after the improvement are experimented on the Hadoop platform. The comparison and analysis of speedup and extensibility show that the performance of the two improved algorithms is greatly improved. The two improved algorithms are used for clustering analysis and association rule mining of elevator data respectively. It not only improves the efficiency of the algorithm, but also ensures the adequacy of data mining, and provides support for elevator safety and reliability analysis.
【学位授予单位】：西安科技大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.13;TU857

【参考文献】