EM算法在不完全监测数据处理中的应用研究

发布时间：2018-11-01 18:14

【摘要】：众所周知,在开展测量工作时会受到地形条件、天气、环境、人为等因素的影响,这些因素常常会导致观测数据缺失或者含有粗差,使得观测数据变得不完全。现在,对于变形监测的数据处理方法大多数都是基于完全数据的,若不对缺失数据处理,往往会影响到结果的准确性。在出现数据缺失这种情况时,常常采用删除法、普通填补法、拟合法或预测法对缺失数据进行处理,然后再利用常规方法对数据进行建模分析。但是这些方法都有一定的局限性,删除法实施起来简单、快捷,但是它导致了资源的浪费,当缺失数据较多或处于较重要的位置时,该方法可能导致结果的错误。普通填补法、拟合法和预测法虽然能够一定程度上提高变形监测数据处理质量,但是得到的并一定是最优的结果,因为这些方法都是先对缺失的数据进行填补,然后再进行建模分析预测,由于这些方法自身的局限性,其填补的数据也存在一定的不确定性,再利用它来进行建模分析可能就会导致结果产生偏差。基于以上情况,本文在分析了缺失数据的机制以及缺失数据处理方法之后,根据统计领域中的经典算法EM算法(Expectation-maximization Algorithm,又译作最大期望化算法)原理,探讨了当变形监测数据出现缺失时利用EM算法对其进行处理的方法。本文针对EM算法在不完全监测数据处理中的研究主要做了以下的工作:(1)论述了变形监测中常用的数据处理方法,并对这些方法进行了总结比较,分析各种方法在测绘数据处理中的适用情形和优缺点。(2)根据数据缺失机制介绍了测量中常用的不完全测量数据的处理方法,通过对比各种缺失数据处理方法,分析了各种方法的适用性。(3)对EM算法的原理和性质进行了介绍,详细的介绍了EM算法同常用的预报模型AR(p)模型结合处理不完全监测数据的步骤。通过对比删除法和回归填补法在单一缺失和多重缺失情况下的预报效果,证实了EM算法用于变形监测不完全数据处理中的可靠性。(4)在完全数据情况下分别采用GM(1,1)灰色模型和BP神经网络模型对沉降数据进行预测,对比在单一缺失数据与多重缺失情况下采用EM算法估计的AR(p)模型的预测结果。通过对比分析发现,4种方法的预测效果相差不大,综合比较EM算法估计的AR(p)模型的预测精度较高。
[Abstract]:As we all know, the survey work will be affected by the terrain conditions, weather, environment, human factors and other factors, these factors will often lead to the lack of observation data or contain gross error, make the observation data become incomplete. Nowadays, most of the data processing methods for deformation monitoring are based on complete data. If the missing data is not processed, the accuracy of the results will be affected. In the case of missing data, deletion method, general filling method, fitting method or prediction method are often used to process the missing data, and then the data are modeled and analyzed by the conventional method. However, these methods have some limitations. Deletion method is simple and fast to implement, but it leads to the waste of resources. When there are more missing data or in a more important position, the method may lead to the error of the result. The common filling method, the fitting method and the prediction method can improve the processing quality of deformation monitoring data to some extent, but the result must be the best, because these methods are to fill the missing data first. Then the modeling analysis and prediction are carried out. Due to the limitations of these methods there are some uncertainties in the data filled by these methods. Using them for modeling and analysis may lead to the deviation of the results. Based on the above situation, this paper analyzes the mechanism of missing data and the methods of processing missing data, and according to the principle of EM algorithm, a classical algorithm in the field of statistics, which is translated as maximum expectation algorithm, The method of using EM algorithm to deal with deformation monitoring data when it is missing is discussed. In this paper, the research of EM algorithm in incomplete monitoring data processing has been done as follows: (1) the data processing methods commonly used in deformation monitoring are discussed, and these methods are summarized and compared. The application of various methods in surveying and mapping data processing and their advantages and disadvantages are analyzed. (2) according to the mechanism of missing data, the common methods of incomplete measurement data processing are introduced, and the methods of missing data processing are compared. The applicability of various methods is analyzed. (3) the principle and properties of the EM algorithm are introduced, and the steps of processing incomplete monitoring data by combining the EM algorithm with the commonly used prediction model AR (p) model are introduced in detail. By comparing the prediction results of deletion method and regression filling method in the case of single deletion and multiple deletions, The reliability of EM algorithm used in incomplete data processing of deformation monitoring is confirmed. (4) in the case of complete data, GM (1k-1) grey model and BP neural network model are used to predict the settlement data, respectively. The prediction results of AR (p) model estimated by EM algorithm under the condition of single missing data and multiple deletions are compared. By comparison and analysis, it is found that the prediction effect of the four methods is not different, and the prediction accuracy of the AR (p) model estimated by comprehensive comparison with EM algorithm is higher.
【学位授予单位】：成都理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：P207

【参考文献】