基于大数据的城市排污监管系统的研究与实现

发布时间：2018-04-24 02:13

本文选题：大数据 + 排污监管　；参考：《江苏大学》2017年硕士论文

【摘要】：随着我国经济的快速发展,人民生活及工业用水量大幅提升,产生的污水排放也随之增多,尤其是工业废水的排放问题十分严重。对此,建立城市排污监管系统能够有效地提前发现问题并从源头解决问题,是未来此类问题解决的必然趋势。目前,污水产生的源头分布十分广泛、排放时间不固定,而且污水的监测判定指标很多,传统解决方案存在采集周期长、评判指标数量大且维度高、系统处理效率低等问题。针对上述问题,本课题在深入分析污水数据特性的基础上,结合大数据处理技术,研究并实现了一种基于大数据分析的城市排污监管系统。本文的主要工作包括以下内容:1)提出一种污水大数据的实时监测技术。首先,基于集成学习Adaboost算法的思想,采用FNN_Adaboost模型来预测未知的复杂污水指标,提高对未知指标值的预测精度。然后,在降低污水数据维度基础上,通过改进k-means聚类算法增强污水大数据的聚类效果。最后,基于Spark Streaming流式数据处理框架实现对工厂排污超标实时监测。2)设计一种污水大数据的存储体系架构。针对污水数据的数量大、多样性等特征,采用主从式架构搭建Hadoop集群,并在HBase数据库中设计了面向列的污水数据存储模型,不仅能存储海量的污水数据信息,而且有效提高了数据处理效率。3)提出一种污水大数据的预测分析技术。针对工厂排污情况的差异性及污水数据的动态性,在进行序列化处理与分析的基础上,对比采用线性和非线性时间序列预测模型进行数据预测的效果,并选取最优模型来实现污水大数据的预测分析。最后通过实际应用分析验证了该技术的可行性和有效性。4)在对上述技术进行研究的基础上,本文设计并实现了一种基于大数据分析的城市排污监管系统。该系统借鉴MVC的设计思想,实现了数据解析,数据分析处理,数据存储,数据查询,预测报警等功能模块,不仅能够对城市工厂排污情况的全面智能管理,而且可以进行大数据关联分析,从而积极推进城市环保大数据的建设和发展。
[Abstract]:With the rapid development of our country's economy, the amount of water used in people's life and industry has been greatly increased, and the sewage discharge has increased, especially the problem of industrial wastewater discharge is very serious. Therefore, it is an inevitable trend to set up urban sewage supervision system to find out the problem and solve it from the source effectively in the future. At present, the source of sewage is widely distributed, the discharge time is not fixed, and there are many monitoring and judging indexes of sewage. The traditional solution has many problems, such as long collection period, large number of evaluation indexes and high dimensionality, low efficiency of system treatment and so on. In view of the above problems, based on the in-depth analysis of the characteristics of sewage data, combined with big data treatment technology, this paper studies and implements a city sewage supervision system based on big data analysis. The main work of this paper includes the following contents: 1) A real-time monitoring technology for wastewater big data is proposed. Firstly, based on the idea of integrated learning Adaboost algorithm, FNN_Adaboost model is used to predict the unknown complex wastewater index, and the prediction accuracy of unknown index value is improved. Then, on the basis of reducing the dimension of sewage data, the improved k-means clustering algorithm is used to enhance the clustering effect of sewage big data. Finally, a storage system of wastewater big data is designed based on Spark Streaming streaming data processing framework. Aiming at the characteristics of large quantity and diversity of sewage data, Hadoop cluster is constructed by master-slave architecture, and a column-oriented sewage data storage model is designed in HBase database, which can not only store massive sewage data information. Moreover, the efficiency of data processing is improved. 3) A kind of forecast and analysis technology of sewage big data is put forward. In view of the difference of sewage discharge in factories and the dynamic nature of sewage data, on the basis of serialization and analysis, the effects of linear and nonlinear time series prediction models for data prediction are compared. And select the best model to realize the forecast and analysis of the sewage big data. Finally, the feasibility and effectiveness of the technology are verified by practical application analysis. On the basis of the research on the above technology, this paper designs and implements a kind of urban sewage supervision system based on big data analysis. Using the design idea of MVC for reference, the system realizes the function modules of data analysis, data analysis and processing, data storage, data query, prediction and alarm, etc. And can carry on big data relevance analysis, thus actively promote the construction and development of urban environmental protection big data.
【学位授予单位】：江苏大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.13;TP311.52

【参考文献】