面向生产制造的大数据分析平台技术研究

发布时间：2018-03-01 10:46

本文关键词： 大数据平台分布式并行化 STKmeans 不均衡数据　出处：《山东大学》2017年硕士论文　论文类型：学位论文

【摘要】：随着大数据技术逐渐成为工业4.0的标配技术之一,生产制造企业迈入了智能制造时代,基于数据的生产和管理需求越来越高。由于基于物联网技术的数据采集和控制模块的广泛应用,形成了极为庞大的数据源,传统数据库技术对这种大规模的数据的存储、管理以及分析能力显得捉襟见肘。而利用大数据技术,可以给企业提供高效的数据分析能力,提高质量管控、工艺提升、服务升级等方面的管理和决策能力。为解决生产制造企业无法通过传统数据分析方法从企业大数据中获取有价值信息的痛点,本文实现了一套大数据分析解决方案并进行了实例分析。首先介绍了平台的基础架构,然后针对大数据平台核心功能模块如数据获取、大数据分析模块、平台可视化等模块进行了详细设计,尤其针对大数据平台的运行特点以KNN为例对分析模块给出了分布式并行化的实现思路。针对某生产制造企业具体的业务需求,分别给出了大数据平台支撑下的数据预处理、特征提取、聚类模型及算法设计和实现,结合企业拧紧工艺的数据分析案例进行了详细阐述。在此过程中,针对该企业数据特征,提出了一种改进的针对大规模不均衡数据的聚类分析算法—STKmeans(Second Time Kmeans),企业数据的分析结果证明了其聚类效果远远高于传统聚类方法,同时验证了平台的实用性。
[Abstract]:With big data technology becoming one of the standard technologies in industry 4.0, manufacturing enterprises have entered the era of intelligent manufacturing. Because of the wide application of data acquisition and control module based on the Internet of things, a huge data source has been formed, and the traditional database technology stores this kind of large-scale data. The ability of management and analysis is overstretched. By using big data technology, we can provide efficient data analysis ability, improve quality control, and improve technology. In order to solve the problem that manufacturing enterprises can not obtain valuable information from big data through traditional data analysis, This paper has realized a set of big data analysis solution and carried on the example analysis. First, introduced the platform infrastructure, then aimed at the big data platform core function module such as the data acquisition, the big data analysis module, The platform visualization module is designed in detail, especially according to the running characteristics of big data platform, taking KNN as an example, the realization of distributed parallelization is given. The data preprocessing, feature extraction, clustering model and algorithm design and implementation based on big data platform are presented, and the data analysis cases of enterprise tightening process are described in detail. This paper presents an improved clustering algorithm for large scale unbalanced data-STKmeans second Time Kmeansan. The results of enterprise data analysis show that the clustering effect of this algorithm is much higher than that of the traditional clustering method, and the practicability of the platform is verified at the same time.
【学位授予单位】：山东大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.13

【相似文献】