基于Hadoop的海量交通数据研究与应用

发布时间：2018-08-23 10:45

【摘要】：为了更好的管理城市道路交通系统,人们建立了智能交通系统。通过在城市道路上安装各种检测传感器来采集交通信息、监控道路状态。然而随着路网体系的复杂,城市汽车保有量的迅猛增加,智能交通系统采集了海量的低价值密度信息。如何在这些大规模低价值的数据中快速、准确的挖掘出解决城市道路问题的有用信息是现在研究者孜孜追求的目标。在交通问题中交通拥堵的判别预测是重要的领域,此前判断交通拥堵的方法是通过计算交通参数等数据判断出的常规堵点,然后通过规划道路、控制交通灯来解决拥堵问题,未考虑数据客观反映的道路状态和警力实时指挥的问题。而且随着数据的增加导致计算量的几何式增长,耗费大量时间,丧失了交通预测的实时性。此外,交通小区的划分是研究交通出行规律的中观层面,合理的划分有助于制定有效的交通管理措施。在基于杭州交通系统的实际数据,主要包括微波检测数据、浮动车GPS数据、视频监控数据、路网数据等,结合Hadoop平台对海量数据处理的优势,对交通数据的挖掘应用做以下创新性研究:1.本文首次提出并解决了交通异常堵点的概念和分布式检测算法,他对有限的警力的实时优化投放起着重要的指导作用,它与交通状态分级判别有着本质的不同。通过引入历史拥堵概率,首次定义了“异常”的概念和分布式计算模型,进一步,通过“累积异常”效应,提升实时预警的准确性。提出的算法具有“自学习”性,具体体现在历史拥堵概率持续更新环节,即使通过交通组织和道路基础设施发生变化,方法适用性也不会受较大影响。2.提出一种基于海量交通数据的快速分布式密度聚类算法,避免了一般密度聚类算法输入参数对数据聚类的影响,提高密度聚类算法在面对大数据集时的运算效率。利用分布式方法计算点密度与点域间距离等阈值来快速对点进行聚类。为交通小区的划分提供决策依据。
[Abstract]:In order to better manage the urban road traffic system, people have established the intelligent transportation system. The traffic information is collected and the state of the road is monitored by installing various detection sensors on the city road. However, with the complexity of the road network system and the rapid increase of urban vehicle ownership, the intelligent transportation system has collected a large amount of low-value density information. How to quickly and accurately excavate the useful information to solve the urban road problem in these large-scale low-value data is the goal that the researcher pursues diligently now. In traffic problems, the discrimination and prediction of traffic congestion is an important field. In the past, the method of judging traffic congestion was to determine the normal congestion points by calculating traffic parameters and other data, and then to solve the congestion problem by planning roads and controlling traffic lights. The road state and the real-time command of the police are not taken into account. Moreover, with the increase of data, the geometric increase of computation cost a lot of time and lose the real time of traffic prediction. In addition, the division of traffic district is the middle level of traffic travel law, and reasonable division is helpful to establish effective traffic management measures. Based on the actual data of Hangzhou traffic system, including microwave detection data, floating vehicle GPS data, video surveillance data, road network data and so on, combined with the advantages of Hadoop platform for massive data processing, Do the following innovative research on traffic data mining applications: 1. In this paper, the concept of traffic jam point and the distributed detection algorithm are proposed and solved for the first time. It plays an important role in guiding the real-time optimization of limited police force, and it is essentially different from traffic state classification. By introducing historical congestion probability, the concept of "anomaly" and the distributed computing model are defined for the first time. Furthermore, the accuracy of real-time early warning is improved by "cumulative anomaly" effect. The algorithm has "self-learning" property, which is embodied in the continuous updating of historical congestion probability. Even if traffic organization and road infrastructure change, the applicability of the method will not be greatly affected by .2. A fast distributed density clustering algorithm based on mass traffic data is proposed, which avoids the influence of input parameters of general density clustering algorithm on data clustering, and improves the computational efficiency of density clustering algorithm in the face of big data set. The point density and the distance between points are calculated by the distributed method to cluster the points quickly. It provides the decision basis for the division of traffic district.
【学位授予单位】：浙江工业大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：U495

【相似文献】