农业大数据平台的实现与数据分析算法

发布时间：2018-06-17 00:07

本文选题：农业大数据平台 + 数据可视化　；参考：《东北农业大学》2017年硕士论文

【摘要】：随着农业现代信息化进程的不断发展以及农业种植、畜牧业、渔业、农产品加工、气象等数据的不断积累,农业数据正以前所未有的速度不断增长并形成了海量数据。我国农业领域数据具有数据实时性强、维度高、数据存储分散、难于综合分析等特性,一方面是因为我国农业结构复杂,农业数据涉及多个领域,另一方面农业数据又容易受到地理环境、土壤、天气、作物、病虫害等的影响。这些数据的有效利用需要相应的大数据平台作为支撑,大数据平台可以整合农业领域的数据,提供查询、下载、上传、可视化等功能;平台的数据挖掘方法可以挖掘隐藏在农业数据中的知识,发现规律;大数据平台还可以为农业工作者提出决策意见和指导建议。所以开发具有以上功能的农业大数据平台具有重要现实意义。本文根据农业大数据的性质,分析了农业大数据平台的主要技术,对农业数据分析和可视化工具进行了较深入的探讨,提出了基于改进的谱聚类算法,并搭建了具有挖掘功能的农业大数据平台。论文主要工作如下:在农业数据收集方面我们通过中国统计年鉴、高校的实验数据和相关农业网站下载了一定量的农业数据,同时还使用网络爬虫技术抓取了一些农业相关网站数据,并利用这些数据建立了农业数据库。在农业数据平台数据挖掘和可视化两项关键技术研究方面,我们提出了针对农业大数据的基于闵可夫斯基测量相似程度的改进谱聚类算法,在UCI数据集中的seeds和soybean数据集的仿真实验结果表明论文提出的算法在聚类精度和运算速度上都有一定的提高。我们还使用多维数据的数据可视化技术,实现平台的数据可视化功能。在平台开发方面我们根据平台需求给出了系统设计方案并使用J2EE相关技术实现了平台的全部功能,在系统测试方面我们利用农业机械、畜牧业数据进行了实验仿真。本文开发的农业大数据平台,用户界面友好使用简单,在数据收集方面我们通过网络爬虫模块,实现了数据自动获取和存储。除可以实现目前农业数据平台查询、上传、下载的功能外,还具有性能优越的数据挖掘模块和简单易懂的数据可视化模块。平台设计合理且实用。农业大数据关键技术的研究和平台的开发对农业信息化和智慧农业的发展具有重要的参考价值和推动作用。
[Abstract]:With the continuous development of modern agricultural information process and the continuous accumulation of agricultural planting, animal husbandry, fishery, processing of agricultural products and meteorological data, agricultural data is growing at an unprecedented rate and forming massive data. The data of agricultural field in our country have high real-time data, high dimension, scattered data storage and difficult to integrate. On the one hand, it is because of the complex agricultural structure in China, the agricultural data are involved in many fields, and on the other hand, the agricultural data are easily affected by the geographical environment, soil, weather, crops, diseases and pests. The effective utilization of these data needs the corresponding large data platform as support, and the large data platform can integrate the agricultural field. Data, providing the functions of query, downloading, uploading, visualization and so on. The data mining method of the platform can discover the knowledge hidden in the agricultural data and discover the rules. The large data platform can also provide advice and advice for the agricultural workers. Therefore, it is of great practical significance to develop a large agricultural data platform with the above functions. According to the nature of large agricultural data, the main technology of agricultural data platform is analyzed, and the agricultural data analysis and visualization tools are discussed. The improved spectral clustering algorithm is proposed and a large agricultural data platform with mining function is set up. The main work of this paper is as follows: in the field of agricultural data collection, I We have downloaded a certain amount of agricultural data through the Chinese Statistical Yearbook, the experimental data of the University and the related agricultural websites. At the same time, we also use the web crawler technology to capture some agricultural related website data, and use these data to establish the agricultural database. In the field of data mining and visualization of the agricultural data platform, two key technologies are studied. We propose an improved spectral clustering algorithm based on the similarity degree of Minkowski measurement for large agricultural data. The simulation results of seeds and soybean data sets in the UCI data set show that the proposed algorithm has a certain improvement in clustering accuracy and computing speed. We also use multidimensional data visualization techniques. In the aspect of platform development, we give the system design scheme according to the platform requirements and use J2EE related technology to realize all the functions of the platform. In the system testing, we use agricultural machinery and animal husbandry data to imitate true. The agricultural big data platform and the user community developed in this paper. The face is friendly and simple. In data collection, we have realized data acquisition and storage through the network crawler module. In addition to the functions of agricultural data platform query, uploading and downloading, it also has excellent data mining module and simple and easy to understand data visualization module. The platform is designed to be reasonable and practical. The key technology research and platform development of big data has important reference value and impetus to the development of agricultural informatization and intelligent agriculture.
【学位授予单位】：东北农业大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：F323.3

【参考文献】