基于大数据的智能交通运输平台的研究

发布时间：2018-07-11 10:19

本文选题：智能交通 + 数据挖掘　；参考：《成都理工大学》2014年硕士论文

【摘要】：随着交通基础服务设施的完善和数据采集系统的多样化，交通系统中的数据量正处于急剧增长的状态，智能交通系统的大数据时代已经到来。交通系统是一个实时的、多变的系统，随着数据量的增多，传统方式处理交通数据已经满足不了我们的需求，尤其是过高的延时性与较低准确性在实时路况预测和路线最优引导方面表现非常差；而使用云存储可以使海量数据的存储成本变得低廉的同时又增加了数据的安全性，使用云计算处理海量数据可以达到最高效的利用现有资源来快速处理数据，所以使用云平台来存储计算智能交通中的海量数据是最佳的选择之一。在众多的云处理平台中，，Hadoop平台以其体系完整、开源的特点成为主流的云平台之一，Hadoop可以充分利用大量廉价的服务器集群来完成对海量数据的快速并行处理。其核心组件包括分布式文件系统HDFS和并行化计算模型MapReduce，分布式文件系统提供安全可靠的高效存储访问能力，MapReduce提供了对海量数据的并行处理能力，这使得我们可以利用Hadoop平台高效的处理智能交通系统中的海量数据挖掘问题。虽然MapReduce框架有一些缺陷，例如单点故障概率高，资源利用率低，但是其升级版YARN很好的解决了这个问题，所以Hadoop平台在智能交通系统中的应用前景是非常广阔的。本文通过对交通系统的分析，发现比较常用的功能实时路况预测以及路线引导相关性比较大，所以本文使用基于历史数据的方法来对路况进行预测并使用预测结果对路段进行动态的赋值，然后再使用相关的单源最短路径算法来模拟最优路径，虽然这种方式使得计算量大大增加，例如动态赋值操作需要对所有路段进行基于历史数据的实时预测方法，但是，本文使用Hadoop平台很好的解决了计算量的问题。通过对数据挖掘在Hadoop下的应用优势的分析研究，在基于历史数据的实时路况预测系统中，本文使用KNN算法进行数据挖掘，将实时路况与历史数据模型进行匹配来预测路况，而KNN算法则使用MapReduce计算模型来实现并行化，以减少数据处理时间；最优路径选择的问题中，本文使用基于MapReduce的并行单源最短路径算法以达到更高的效率。最后对云平台应用到智能交通系统的框架进行了分析设计，系统分为四层五部分，每一层都独立向上层提供服务，这样对提高数据的利用率有很大帮助，另外为了对数据更加高效的存储处理，在使用云存储的同时还使用传统数据进行数据存储，以达到更高的效率。最后对基于MapReduce的并行KNN算法在路况预测以及并行单源最短路径算法在最优路径的应用进行了模拟测试，通过对比测试证明利用Hadoop集群对提升智能交通的性能有很大帮助。
[Abstract]:With the improvement of transportation infrastructure and the diversification of data acquisition system, the data volume in the transportation system is in a state of rapid growth. The big data era of intelligent transportation system has come. Traffic system is a real-time, changeable system, with the increase of data, the traditional way of handling traffic data has been unable to meet our needs. Especially, too high delay and low accuracy are very poor in real-time road condition prediction and route optimal guidance, and cloud storage can make the storage cost of mass data low and increase the security of data at the same time. Using cloud computing to process large amounts of data can achieve the most efficient use of existing resources to quickly process data, so using cloud platform to store the mass data in intelligent transportation is one of the best options. Among many cloud processing platforms, Hadoop platform is one of the mainstream cloud platforms with its complete architecture and open source. Hadoop can make full use of a large number of cheap server clusters to complete the fast parallel processing of massive data. Its core components include distributed file system (HDFS) and parallel computing model MapReduce.The distributed file system provides secure and reliable storage access capability and MapReduce provides parallel processing capability for massive data. This enables us to use Hadoop platform to efficiently deal with the problem of mass data mining in intelligent transportation systems. Although the MapReduce framework has some defects, such as high single point failure probability and low resource utilization, its upgraded version YARN solves this problem very well, so the Hadoop platform has a very broad application prospect in its. Based on the analysis of traffic system, this paper finds that the function of real-time road condition prediction and route guidance is more relevant. So this paper uses the method based on historical data to predict the road condition and uses the forecast result to assign the dynamic value to the section, then uses the correlation single source shortest path algorithm to simulate the optimal path. Although this method greatly increases the computational complexity, for example, the dynamic assignment operation requires a real-time prediction method based on historical data for all sections, but the Hadoop platform is used in this paper to solve the problem of computational complexity. Based on the analysis of the advantages of data mining in Hadoop, in the real-time road condition prediction system based on historical data, this paper uses the KNN algorithm for data mining, matches the real-time road condition with the historical data model to predict the road condition. The KNN algorithm uses MapReduce computing model to realize parallelization to reduce data processing time. In the problem of optimal path selection, this paper uses MapReduce based parallel single-source shortest path algorithm to achieve higher efficiency. Finally, the framework of cloud platform applied to intelligent transportation system is analyzed and designed. The system is divided into four layers and five parts, each layer provides services to the upper layer independently, which is helpful to improve the utilization rate of data. In addition, in order to store the data more efficiently, the traditional data is stored simultaneously with cloud storage, so as to achieve higher efficiency. Finally, the parallel KNN algorithm based on MapReduce and the application of parallel single source shortest path algorithm in optimal path are simulated and tested. It is proved that using Hadoop cluster is very helpful to improve the performance of intelligent traffic.
【学位授予单位】：成都理工大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：U495;TP311.13

【相似文献】