基于紫外光谱的管网水水质异常检测若干关键问题研究

发布时间：2018-10-09 10:22

【摘要】：水质安全问题关系国计民生,水质污染事件不仅破坏了当地的水体环境,也严重影响到居民的管网水饮用安全。传统的化学法等水质异常检测手段往往费时且可能会造成二次污染。利用紫外吸收光谱法具有可现场原位检测、耗时较少、无二次污染等特性和优点,本文开展了基于紫外吸收光谱的在线水质异常检测方法研究,着重研究讨论了应用紫外吸收全光谱分析方法如何克服噪声干扰和基线漂移、水质背景波动、工况突变等因素的影响,从而提升对水质污染异常的检出和判别能力。论文的主要工作和创新点如下:(1)开展了面向噪声干扰和基线漂移的水质光谱异常检测方法研究,提出了分析紫外吸收全光谱数据的水质异常检测算法。该方法首先采用Savitzky-Golay(S-G)卷积平滑法进行管网水质紫外吸收光谱平滑滤波,并利用光谱数据均值中心化去除野值点和散射,应用非对称最小二乘法进行了光谱基线校正;进而采用主成分分析法(PCA)对光谱数据进行降维和特征提取,对正常水质构成的训练矩阵通过自适应优化后建立PCA模型,然后结合残差空间的Q统计量检出水质离群点(序列),最后通过分析序列中离群点的密度等时序特征来确定水质异常概率。通过管网水污染物注入实验对不同预处理方法的异常检测效果进行了对比分析,验证了所研究方法对于消除不同类型噪声和基线漂移状况是有效的。(2)针对长期水质背景波动的影响,对水质紫外光谱异常检测方法进行了优化。首先分析了长期水质背景波动的原因,讨论了原水水质变化和供水管网用水量变化等因素对紫外光谱检测数据所产生的缓慢时变波动特性;然后针对水质背景波动特点提出了滑动窗PCA法,通过注入污染物事件进行了分析和验证,同常规PCA法相比较,滑动窗PCA法更能适应水质背景长期趋势变化的影响,并根据不同窗口长度下的检出效果结合目标函数不断对"最佳"窗口长度进行优化。(3)针对工况突变导致的水质异常误报问题,进行了水质光谱异常分类方法的研究。分析了工况突变导致的紫外光谱数据波动特点,研究了水质异常分类的技术方案,提出建立不同工况突变导致的水质异常模式知识库,并对检出后的异常进行在线匹配。考虑到紫外光谱的维数大、波长之间相关度高等特点,论文采用主成分分析和Fisher线性判别分析法来进行特征提取和异常模式分类,将污染物异常和工况突变异常加以区分。全文围绕基于紫外吸收全光谱进行水质异常检测过程中的各类干扰问题,对异常检测方法进行了研究和改进完善。基于论文研究成果,开发了基于紫外光谱水质预警系统软件并得到了应用。
[Abstract]:Water quality safety is related to the national economy and people's livelihood. Water pollution not only destroys the local water environment, but also seriously affects the safety of drinking water in the pipe network of residents. Traditional methods such as chemical method are time-consuming and may cause secondary pollution. Ultraviolet absorption spectrometry has the advantages of in situ detection, less time consuming and no secondary pollution. In this paper, the on-line detection method of water quality anomaly based on UV absorption spectrum is studied. How to overcome the influence of noise interference and baseline drift, water quality background fluctuation, working condition mutation and so on is discussed in this paper, so as to improve the ability of detecting and discriminating water pollution anomalies. The main work and innovations of this paper are as follows: (1) the detection method of water quality spectral anomaly based on noise interference and baseline drift is studied, and an algorithm for detecting water quality anomaly based on ultraviolet absorption data is proposed. In this method, Savitzky-Golay (S-G) convolution smoothing method is used to filter water quality UV absorption spectrum smoothing, and the outliers and scattering are removed by means of the mean value of spectral data, and the spectral baseline is corrected by asymmetric least square method. Then the principal component analysis (PCA) is used to extract the dimensionality of spectral data, and the PCA model is established by adaptive optimization of the training matrix of normal water quality. Then the outliers of water quality are detected by the Q statistics in the residual space and the probability of water quality anomalies is determined by analyzing the temporal characteristics of the outliers such as the density of the outliers in the series. The results of abnormal detection of different pretreatment methods were compared and analyzed by water pollutant injection experiment in pipe network. The results show that the proposed method is effective to eliminate different noise and baseline drift. (2) aiming at the influence of long-term water quality background fluctuation, the method of UV spectrum anomaly detection is optimized. Firstly, the reason of long-term water quality background fluctuation is analyzed, and the slow time-varying characteristics of raw water quality change and water consumption change of water supply network on UV spectrum detection data are discussed. According to the characteristics of water quality background fluctuation, a sliding window PCA method is put forward, which is analyzed and verified by injecting pollutant events. Compared with the conventional PCA method, the sliding window PCA method is more suitable for the influence of the long-term trend change of water quality background. According to the detection effect of different window length and the objective function, the optimal window length is optimized. (3) aiming at the problem of water quality anomaly misreporting caused by the sudden change of working condition, the classification method of water quality spectral anomaly is studied. In this paper, the characteristics of ultraviolet spectrum data fluctuation caused by operating condition mutation are analyzed, the technical scheme of water quality anomaly classification is studied, and the knowledge base of water quality anomaly pattern caused by water quality anomaly under different working conditions is proposed, and the anomaly after detection is matched online. Considering the characteristics of ultraviolet spectrum such as large dimension and high correlation between wavelengths, principal component analysis (PCA) and Fisher linear discriminant analysis (Fisher) are used to extract features and classify abnormal patterns. In this paper, the method of anomaly detection is studied and improved based on all kinds of interference in the process of water quality anomaly detection based on UV absorption spectrum. Based on the research results, the software of water quality early warning system based on UV spectrum is developed and applied.
【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：X832

【相似文献】