基于Hadoop交通信息并行处理云平台的设计与实现
发布时间：2018-08-19 06:33
【摘要】：随着社会的发展和城市化的加快,机动车的数量呈现爆炸式的增长。但有限的地理空间和紧缺的土地资源,使得人、车、路之间的交通矛盾日益突出。并且随着计算机网络和信息采集技术的发展,日常需要处理的交通数据也逐步增多。而传统平台在面对大数据问题时显得捉襟见肘,这使得所面临的交通问题越来越严重。可喜的是,目前出现了许多成熟的大数据处理技术,为智能交通系统的发展带来了新的思路。本文设计的主要出发点是通过将Hadoop、HBase与现有交通云平台进行整合,实现新一代智慧交通云平台,从而增加平台大数据处理的能力,改善现有的交通系统。本文研究的主要内容有:1.Hadoop、HBase集群多节点部署:本文在基于v Sphere所构建的多虚拟节点上搭建Hadoop与HBase集群,作为新一代交通云平台的底层基础。2.交通基础服务的设计与实现:本文给出了两种交通服务的设计与实现方案,一种是针对原始交通数据所开发的数据中心统一化服务,从而解决数据输入的问题。另一种是为使已有云平台获得大数据处理能力而设计的并行计算和大数据存储服务。3.交通应用架构设计与案例实现:本文设计的交通应用架构体系体包括交通核心应用和交通智慧应用,其作用是验证新一代交通云平台功能。其中交通核心应用主要功能是基于Map Reduce框架对交通数据做计算与分析,而交通智慧应用是将计算结果以可视化的角度展示给用户。通过应用的成功部署、运行从而可以验证平台整合方案的可行性、正确性。4.Hadoop、HBase集群与原有交通云平台整合:本文在基于Socket通信方式的基础上,实现了Hadoop、HBase集群与原有云平台功能的整合,使得新一代智慧交通云平台拥有大数据处理的能力。用户通过整合后的平台入口上传并运行应用,从而获取大数据计算和存储能力。本文提出了通过融合Hadoop、HBase集群构建交通信息并行处理云平台的设计方案,从而解决交通领域大数据的难题。同时在此基础上给出了交通应用架构和服务设计方案,为智能交通系统的发展做出了贡献。
[Abstract]:With the development of society and the acceleration of urbanization, the number of motor vehicles is increasing explosively. However, limited geographical space and scarce land resources make traffic conflicts between people, cars and roads increasingly prominent. With the development of computer network and information collection technology, traffic data need to be processed gradually. The traditional platform is overstretched in the face of big data problem, which makes the traffic problem more and more serious. Fortunately, there are many mature big data processing technologies, which bring new ideas for the development of its. The main starting point of this design is to realize a new generation of intelligent transportation cloud platform by integrating Hadoop HBase with the existing transportation cloud platform, so as to increase the big data processing capacity of the platform and improve the existing transportation system. The main contents of this paper are as follows: 1. The multi-node deployment of Hadoop Sphere cluster: this paper builds Hadoop and HBase cluster on the multi-virtual nodes based on v Sphere, which is the base of the next generation traffic cloud platform. Design and implementation of basic traffic services: this paper presents two design and implementation schemes of traffic services. One is the unified data center service developed for raw traffic data, thus solving the problem of data input. The other is a parallel computing and big data storage service designed to enable existing cloud platforms to gain big data processing power. Traffic application architecture design and case implementation: the traffic application architecture system designed in this paper includes the traffic core application and the traffic intelligence application, whose function is to verify the function of the new generation transportation cloud platform. The main function of the traffic core application is to calculate and analyze the traffic data based on the Map Reduce framework, and the traffic intelligence application is to show the calculation results to the users in a visual way. Through the successful deployment of the application, the feasibility of the platform integration scheme can be verified by running. 4. The integration of Hadoop HBase cluster with the original traffic cloud platform: this paper is based on the Socket communication mode. The integration of Hadoop HBase cluster and the original cloud platform makes the new generation intelligent transportation cloud platform have the ability of big data processing. The user uploads and runs the application through the integrated platform entrance, thus obtains the big data computation and the storage ability. In this paper, we propose a design scheme of traffic information parallel processing cloud platform by integrating Hadoop HBase cluster, so as to solve the problem of big data in traffic field. At the same time, traffic application architecture and service design scheme are given, which contribute to the development of intelligent transportation system.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：U495
本文编号：2190945
[Abstract]:With the development of society and the acceleration of urbanization, the number of motor vehicles is increasing explosively. However, limited geographical space and scarce land resources make traffic conflicts between people, cars and roads increasingly prominent. With the development of computer network and information collection technology, traffic data need to be processed gradually. The traditional platform is overstretched in the face of big data problem, which makes the traffic problem more and more serious. Fortunately, there are many mature big data processing technologies, which bring new ideas for the development of its. The main starting point of this design is to realize a new generation of intelligent transportation cloud platform by integrating Hadoop HBase with the existing transportation cloud platform, so as to increase the big data processing capacity of the platform and improve the existing transportation system. The main contents of this paper are as follows: 1. The multi-node deployment of Hadoop Sphere cluster: this paper builds Hadoop and HBase cluster on the multi-virtual nodes based on v Sphere, which is the base of the next generation traffic cloud platform. Design and implementation of basic traffic services: this paper presents two design and implementation schemes of traffic services. One is the unified data center service developed for raw traffic data, thus solving the problem of data input. The other is a parallel computing and big data storage service designed to enable existing cloud platforms to gain big data processing power. Traffic application architecture design and case implementation: the traffic application architecture system designed in this paper includes the traffic core application and the traffic intelligence application, whose function is to verify the function of the new generation transportation cloud platform. The main function of the traffic core application is to calculate and analyze the traffic data based on the Map Reduce framework, and the traffic intelligence application is to show the calculation results to the users in a visual way. Through the successful deployment of the application, the feasibility of the platform integration scheme can be verified by running. 4. The integration of Hadoop HBase cluster with the original traffic cloud platform: this paper is based on the Socket communication mode. The integration of Hadoop HBase cluster and the original cloud platform makes the new generation intelligent transportation cloud platform have the ability of big data processing. The user uploads and runs the application through the integrated platform entrance, thus obtains the big data computation and the storage ability. In this paper, we propose a design scheme of traffic information parallel processing cloud platform by integrating Hadoop HBase cluster, so as to solve the problem of big data in traffic field. At the same time, traffic application architecture and service design scheme are given, which contribute to the development of intelligent transportation system.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：U495
【引证文献】
相关会议论文 前2条
1 周为钢;杨良怀;龚卫华;郑申俊;沈贝伦;沈俊青;陈彬;陈彬彬;万凯明;罗锋;;大数据处理技术在智能交通中的应用[A];第八届中国智能交通年会优秀论文集——智能交通与安全[C];2013年
2 张滔;凌萍;;智慧交通大数据平台设计开发及应用[A];2014第九届中国智能交通年会大会论文集[C];2014年
，本文编号：2190945
本文链接：https://www.wllwen.com/kejilunwen/daoluqiaoliang/2190945.html
