移动云平台的ITS大数据调用策略研究

发布时间：2018-02-26 17:30

本文关键词： 移动云计算 Spark框架路网评估优化遗传算法并发服务器　出处：《南京大学》2014年硕士论文　论文类型：学位论文

【摘要】：当前,交通监测数据日趋庞杂,路网监测评估与数据处理发布己逐步采取了一系列的手段,而城市交通问题仍日益突出。其症结之一,在于如何高效地处理海量交通数据,实时分析、挖掘、调用,为驾乘提供必要的信息服务[¨。论文依托省科技厅《公路交通传感网信息检测·挖掘·汇聚·发布与辅助决策示范系统》示范工程项目,介绍了国内外智能交通系统中移动采集技术及云计算技术的应用现状,分析了在移动云环境中传输和处理海量交通数据的难点。提出了移动云环境中的优化负载均衡策略,建立了预测评估移动监测数据模型,及其实时数据处理平台,并予路网仿真验证。就流媒体传输需要占用大量带宽,传统流媒体服务器无法提供高并发量服务的难题,提出了基于Reactor模型的流媒体服务器架构,克服了传统多线程模型面临的线程调度开销及同步死锁问题,并采用基于网络状况的自适应缓存机制,实现了云环境中实时稳定的流媒体服务。就移动云环境中资源利用率不平衡问题提出了基于优化遗传算法的负载均衡策略。通过采用多目标的适应度函数及自适应的交叉变异算子,解决了传统调度策略收敛速度慢、易于陷入局部收敛等问题,契合了移动云环境中的资源异质性和动态性特征。就海量交通数据的实时分析和处理问题,基于Spark框架构建了分布式云平台,通过使用基于内存的分布式系统实现海量数据的实时处理,克服了传统框架无法进行实时任务的问题,采用EM算法分析移动设备采集的GPS数据,对路网状况进行实时评估和预测,并通过web接口进行发布。综上,论文研究成果具有重要的理论意义和应用价值。其创新点在于：1)针对基于多线程模型的服务器所面临的进程调度开销及同步死锁问题,采用了基于Reactor模型的I/O复用方式提供高并发服务。2)针对移动云平台中的资源利用率低的问题,优化了传统遗传算法,实现动态负载均衡,克服了移动云环境中计算资源的异质性和动态性问题。3)基于Spark框架实现了海量交通数据实时处理集群,通过构建基于内存的分布式系统,能够在秒量级迭代执行针对海量交通数据的机器学习算法。
[Abstract]:At present, traffic monitoring data are becoming more and more complicated. A series of methods have been adopted gradually in road network monitoring, evaluation and data processing, while urban traffic problems are still becoming more and more prominent. One of the crux of the problem lies in how to deal with mass traffic data efficiently. Real time analysis, mining, calling, providing necessary information services for driving and riding. The paper relies on the demonstration project of "Highway Traffic Sensor Network Information Detection, Mining, aggregation, Publishing and Assistant Decision-Making demonstration system" of the Department of Science and Technology of the province, This paper introduces the application status of mobile acquisition technology and cloud computing technology in intelligent transportation system at home and abroad, analyzes the difficulties of transferring and processing mass traffic data in mobile cloud environment, and puts forward the optimal load balancing strategy in mobile cloud environment. A mobile monitoring data model for prediction evaluation and its real-time data processing platform are established and verified by road network simulation. The traditional streaming media server cannot provide high concurrent services due to the need of a large amount of bandwidth for streaming media transmission. This paper proposes a streaming media server architecture based on Reactor model, which overcomes the problem of thread scheduling overhead and synchronization deadlock faced by traditional multithread model, and adopts adaptive caching mechanism based on network condition. The real-time and stable streaming media service in cloud environment is realized. A load balancing strategy based on optimization genetic algorithm is proposed to solve the imbalance of resource utilization in mobile cloud environment. By adopting multi-objective fitness function and self-adaptation, a load balancing strategy based on optimization genetic algorithm is proposed. Crossover mutation operator, It solves the problems of slow convergence rate and easy to fall into local convergence of traditional scheduling strategy, which accords with the heterogeneous and dynamic characteristics of resources in mobile cloud environment. In this paper, the real-time analysis and processing of massive traffic data are discussed. The distributed cloud platform is constructed based on Spark framework. By using the memory based distributed system to realize the real-time processing of massive data, it overcomes the problem that the traditional framework can not carry out real-time tasks. The EM algorithm is used to analyze the GPS data collected by mobile devices, to evaluate and predict the road network status in real time, and to publish it through the web interface. The research results of this paper have important theoretical significance and application value. The innovation of this paper lies in: 1) aiming at the problem of process scheduling overhead and synchronization deadlock faced by servers based on multithreaded model. I / O reuse mode based on Reactor model is used to provide high concurrent service. 2) aiming at the problem of low resource utilization in mobile cloud platform, the traditional genetic algorithm is optimized to achieve dynamic load balancing. This paper overcomes the heterogeneity and dynamic problem of computing resources in mobile cloud environment. 3) based on Spark framework, the real-time processing cluster of massive traffic data is realized, and the distributed system based on memory is constructed. Machine learning algorithm for mass traffic data can be iterated in second order.
【学位授予单位】：南京大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：U495

【相似文献】