基于分布式文件系统的农业数据云存储研究
发布时间：2018-11-22 09:43
【摘要】：目前，我国传统农业正在向现代化农业、数字化农业和信息化农业转变。《2006—2020国家信息化发展战略》指出：“重点建设农村重要信息系统，增强信息资源共享服务能力。”随着我国农业信息化建设的不断深入，全国各个地区纷纷建立起自己网站和农业信息服务站。各类农业数据资源呈现爆炸式增长，并且具有多元化、异构化的特点。目前，我们可以通过购置高端的昂贵的存储器和服务器来管理这些数据；然而，将来会出现没有一台单独的设备能够存储这么多数据的局面。另外，由于农业数据涉及领域较多，区域性强、季节性强，我国农业信息资源分布在不同区域的PC机、服务器和数据存储设备上，各个地区之间缺少联系，使得各种农业信息资源独立存在，形成一个个的“信息孤岛”，共享性较差。如何解决农业信息化过程中农业数据存储的瓶颈问题，是当前农业信息化过程中要研究的主要内容。 论文阐述了研究农业数据云存储的目的和意义，，并对国内外云存储在农业方面的应用做了总结。然后比较了几种文件系统的特点，最终确定借鉴HDFS文件系统设计了分布式文件系统。分布式文件系统主要采用了文件备份、负载均衡和并发控制这三种技术。通过研究如何对文件复制形成多个副本，并存储在不同的设备上，实现了数据的冗余，使得系统更加安全可靠，避免了因为某台机器的故障造成文件无法访问的现象，并为负载均衡提供实现基础；通过研究对用户上传下载文件有效均衡的管理，使文件在各个节点上的存储较为均衡，实现用户对文件的访问有效地分流到不同的节点上，从而提高了系统性能；通过研究文件传输的并发性和远程调用的并发性，实现了云存储系统的并发控制。在此基础上，基于分布式文件系统，设计并研发了农业数据的云存储系统。系统用Java来开发，用Oracle数据库来存储数据，通过调用分布式文件系统提供的接口，实现了农业文件数据操作的各个功能和对系统的管理功能，实现了农业文件数据资源按照统一标准的存储和检索，解决了大量农业文件数据资源分散和共享性差的问题。在技术上，本文将云存储技术用到农业数据管理上，是较为新颖的。
[Abstract]:At present, the traditional agriculture of our country is changing to modern agriculture, digital agriculture and information agriculture. The National Informatization Development Strategy 2006-2020 points out: "it is important to construct the important information system in rural areas and to enhance the ability of information resources sharing and service." With the development of agricultural information construction in China, all regions have established their own websites and agricultural information service stations. All kinds of agricultural data resources show explosive growth, and have the characteristics of diversification and isomerization. At present, we can manage this data by purchasing high-end and expensive storage and servers; however, no single device will be able to store so much data in the future. In addition, agricultural information resources are distributed on PC machines, servers and data storage devices in different regions because of the large number of agricultural data, strong regional and seasonal, and lack of connection between different regions. All kinds of agricultural information resources exist independently, forming one by one "information isolated islands", the sharing is poor. How to solve the bottleneck problem of agricultural data storage in the process of agricultural informatization is the main content to be studied in the process of agricultural informatization. The purpose and significance of research on cloud storage of agricultural data are expounded, and the application of cloud storage in agriculture at home and abroad is summarized. Then the characteristics of several file systems are compared and the distributed file system is designed based on the HDFS file system. The distributed file system mainly adopts three technologies: file backup, load balance and concurrency control. By studying how to make multiple copies of files and store them on different devices, the redundancy of data is realized, which makes the system more secure and reliable, and avoids the phenomenon that the files cannot be accessed because of the failure of a machine. And provides the realization foundation for the load balancing; By studying the effective and balanced management of users uploading and downloading files, the storage of files on each node is more balanced, and users' access to files is effectively diverted to different nodes, thus improving the performance of the system. By studying the concurrency of file transfer and remote call, the concurrency control of cloud storage system is realized. Based on the distributed file system, a cloud storage system for agricultural data is designed and developed. The system is developed by Java and stored by Oracle database. By calling the interface provided by the distributed file system, the functions of agricultural file data operation and the management function of the system are realized. The storage and retrieval of agricultural file data resources according to unified standards are realized, and the problems of dispersion and poor sharing of a large number of agricultural file data resources are solved. Technically, this paper applies cloud storage technology to agricultural data management, which is relatively novel.
【学位授予单位】：山东农业大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333
本文编号：2348909
[Abstract]:At present, the traditional agriculture of our country is changing to modern agriculture, digital agriculture and information agriculture. The National Informatization Development Strategy 2006-2020 points out: "it is important to construct the important information system in rural areas and to enhance the ability of information resources sharing and service." With the development of agricultural information construction in China, all regions have established their own websites and agricultural information service stations. All kinds of agricultural data resources show explosive growth, and have the characteristics of diversification and isomerization. At present, we can manage this data by purchasing high-end and expensive storage and servers; however, no single device will be able to store so much data in the future. In addition, agricultural information resources are distributed on PC machines, servers and data storage devices in different regions because of the large number of agricultural data, strong regional and seasonal, and lack of connection between different regions. All kinds of agricultural information resources exist independently, forming one by one "information isolated islands", the sharing is poor. How to solve the bottleneck problem of agricultural data storage in the process of agricultural informatization is the main content to be studied in the process of agricultural informatization. The purpose and significance of research on cloud storage of agricultural data are expounded, and the application of cloud storage in agriculture at home and abroad is summarized. Then the characteristics of several file systems are compared and the distributed file system is designed based on the HDFS file system. The distributed file system mainly adopts three technologies: file backup, load balance and concurrency control. By studying how to make multiple copies of files and store them on different devices, the redundancy of data is realized, which makes the system more secure and reliable, and avoids the phenomenon that the files cannot be accessed because of the failure of a machine. And provides the realization foundation for the load balancing; By studying the effective and balanced management of users uploading and downloading files, the storage of files on each node is more balanced, and users' access to files is effectively diverted to different nodes, thus improving the performance of the system. By studying the concurrency of file transfer and remote call, the concurrency control of cloud storage system is realized. Based on the distributed file system, a cloud storage system for agricultural data is designed and developed. The system is developed by Java and stored by Oracle database. By calling the interface provided by the distributed file system, the functions of agricultural file data operation and the management function of the system are realized. The storage and retrieval of agricultural file data resources according to unified standards are realized, and the problems of dispersion and poor sharing of a large number of agricultural file data resources are solved. Technically, this paper applies cloud storage technology to agricultural data management, which is relatively novel.
【学位授予单位】：山东农业大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333
【参考文献】
相关期刊论文 前10条
1 李光达;郑怀国;谭翠萍;邱琳;赵静娟;;基于云计算的农业信息服务研究[J];安徽农业科学;2011年27期
2 拓守恒;;云计算与云数据存储技术研究[J];电脑开发与应用;2010年09期
3 蔡键;王树梅;;基于Google的云计算实例分析[J];电脑知识与技术;2009年25期
4 周轶男;王宇;;Hadoop文件系统性能分析[J];电子技术;2011年05期
5 曹丽英;张晓贤;赵月玲;陈桂芬;;云计算在农业信息资源整合模式中的应用[J];中国农机化;2012年03期
6 史强;;GFS云存储技术可靠性简介[J];福建电脑;2012年01期
7 王鑫;王占江;;浅谈分布式数据库架构[J];计算机光盘软件与应用;2012年08期
8 杨德志,黄华,张建刚,许鲁;大容量、高性能、高扩展能力的蓝鲸分布式文件系统[J];计算机研究与发展;2005年06期
9 黄华,张建刚,许鲁;蓝鲸分布式文件系统的分布式分层资源管理模型[J];计算机研究与发展;2005年06期
10 郭东;杜勇;胡亮;;基于HDFS的云数据备份系统[J];吉林大学学报(理学版);2012年01期
本文编号：2348909
本文链接：https://www.wllwen.com/kejilunwen/jisuanjikexuelunwen/2348909.html
