基于Hodoop的农业博客研究与实现

发布时间：2018-04-14 02:24

本文选题：农业博客 + 分布式存储　；参考：《西北农林科技大学》2014年硕士论文

【摘要】：在互联网技术飞速发展的时代，人们获取信息主要来源于网络资源，农业领域也是如此，但农业用户与其他用户不同，，计算机使用知识的匮乏使农业用户难以从网络上获得所需信息，尤其是在当今网络信息数据量不断增加的情况下，如何从海量的复杂数据中得到所需的相关农业信息更是具有一定的难度。因此，为广大农业用户研发一个具有实际意义的互动交流平台是十分有必要的。博客是一种热门的用于网络信息交流的互联网应用平台，然而现有的博客网站多为综合性交流平台且实时性不高，并不适合农业用户使用。针对这些问题，本研究以分布式存储和并行计算技术为支撑，研究与实现了一个具有分布式、高实时和可扩展性的农业博客。主要研究内容及结论如下：（1）农业博客基础模型构建。首先根据广大农业用户对农业博客的应用需求，设计了农业博客系统的整体系统架构。然后针对其整体架构改进了现有的博客开发模式，构建了农业博客的分布式存储模型及数据并行处理模型，并对传统的推-拉模式进行了改进。（2）农业博客系统设计与实现。首先根据系统的功能需求、服务体系以及基本的模型规划了农业博客系统的总体功能架构，设计了总体数据流，并对系统的主要功能模块进行详细的设计，同时对主要的数据处理过程（存储与访问）进行了设计说明。然后对数据库模式进行了详细的设计与说明。最后利用MapReduce实现了数据的并行处理，同时对Coprocessor数据处理与数据访问服务的实现进行了具体的说明。（3）原型测试与分析。首先根据Hadoop的框架部署了分布式测试集群并整合了MyEclipse开发环境，然后对农业博客系统的用户界面进行了系统功能的测试，最后在模拟大数据集的条件下完成了对系统的性能测试分析。原型测试结果表明系统具有可行性和高效性。
[Abstract]:In the era of rapid development of Internet technology, people obtain information mainly from the network resources, and so is the agricultural field, but agricultural users are different from other users.The lack of knowledge about the use of computers makes it difficult for agricultural users to obtain the required information from the network, especially when the amount of data on the network is increasing.It is more difficult to get the relevant agricultural information from the massive complex data.Therefore, it is necessary to develop a practical interactive communication platform for agricultural users.Blog is a popular Internet application platform for the exchange of information on the Internet. However, most of the existing blog sites are comprehensive communication platforms and the real-time performance is not high, so it is not suitable for agricultural users.To solve these problems, a distributed, real-time and scalable agricultural blog is studied and implemented based on distributed storage and parallel computing technology.The main contents and conclusions are as follows:1) constructing the basic model of agriculture blog.Firstly, according to the application demand of agricultural blog, the whole system structure of agricultural blog system is designed.Then, the existing blog development model is improved according to its overall architecture, the distributed storage model and data parallel processing model of agricultural blog are constructed, and the traditional push-pull model is improved.Agricultural blog system design and implementation.Firstly, according to the functional requirements of the system, the service system and the basic model, the overall functional framework of the agricultural blog system is planned, the overall data flow is designed, and the main functional modules of the system are designed in detail.At the same time, the main data processing procedures (storage and access) are described.Then the database schema is designed and explained in detail.Finally, the parallel processing of data is realized by using MapReduce, and the realization of Coprocessor data processing and data access service is explained in detail.Prototype testing and analysis.First, the distributed test cluster is deployed according to the framework of Hadoop and the MyEclipse development environment is integrated, then the user interface of the agricultural blog system is tested.Finally, the performance of the system is tested and analyzed under the condition of simulating big data set.The prototype test results show that the system is feasible and efficient.
【学位授予单位】：西北农林科技大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.092

【参考文献】