基于强化学习的自适应城市交通信号控制方法研究

发布时间：2018-03-03 18:07

本文选题：强化学习　切入点：智能交通　出处：《浙江师范大学》2015年硕士论文　论文类型：学位论文

【摘要】：城市道路不断兴建和扩宽,基础设施建设投入也越来越大,然而城市交通拥堵问题却越来越严重,主要原因是现有的城市交通信号控制TSC (Traffic Signal Control)系统不能充分做到对交通流量的最优控制和管理。因此,如何通过交通信号的最优控制来设计和优化城市TSC系统,成为保障交通安全和畅通、增加道路通行效率及其缓解交通拥塞问题的关键所在。本文选择基于Q-learning算法的单Agent控制体系结构,基于分布式Q-learning算法的Multi-Agent系统以及Green Light District(GLD)开源仿真平台进行城市TSC系统优化研究,主要做了如下工作：(1)设计了基于单路口和井字形区域路口的城市TSC系统Agent框架,模拟城市道路控制。对于城市单路口,通过一个智能Agent实时检测每个方向的交通流数据,交通流数据通过模糊逻辑化,输入设计的单路口Q-learning决策器,寻得最优控制策略。对于区域交通控制,提出了分布式Q-learning算法和MAS结合的优化控制方式,给出了相邻路口Agent协调控制模型,实现相邻路口之间信息共享。(2)解决了Q-learning算法和分布式Q-learning算法对交通环境状态集S、动作策略集A、奖惩函数R等关键问题。状态空间的选择,设计用模糊逻辑来计算排队长度；动作策略集A：增加、保持和减少相位绿灯时间：奖惩函数R以路口车辆排队长度作为指标,以车辆排队长度最小为目的。(3)实现了分布式Q-learning算法在区域TSC系统优化上的运用,解决了区域信号协调控制问题。分布式Q-learning算法和MAS的结合,实现对城市TSC系统最优控制。城市区域交通网络是分布式的多Agent网络,建立了基于分布式Q-learning算法的Multi-Agent模型框架,同时给出了分布式Q-learning算法设计的详细步骤。最后分析了基于Q-learning算法的单路口城市TSC优化和基于分布式Q-learning算法的区域TSC优化的算法性能。在GLD中,对随机配时,固定配时,Longest-queue, Traffic-controller 1 (TC1), ACGJ-1、Q-learning算法和分布式Q-learning算法优化性能进行了模拟验证分析,实验结果表明了Q-learning算法和分布式Q-learning算法在城市TSC系统优化上优于其他算法。
[Abstract]:With the construction and widening of urban roads and the increasing investment in infrastructure construction, however, the problem of urban traffic congestion is becoming more and more serious. The main reason is that the existing urban traffic signal control TSC traffic Signal control system can not fully achieve the optimal control and management of traffic flow. Therefore, how to design and optimize the urban TSC system through the optimal control of traffic signals, This paper chooses a single Agent control architecture based on Q-learning algorithm, which is the key to ensure traffic safety and smooth flow, increase road traffic efficiency and alleviate traffic congestion. Multi-Agent system based on distributed Q-learning algorithm and Green Light restricted GLD open source simulation platform are used to optimize urban TSC system. The main work is as follows: design the Agent framework of urban TSC system based on single intersection and well shaped area intersection. Simulation of urban road control. For a single intersection of a city, real-time detection of traffic flow data in each direction through an intelligent Agent, traffic flow data through fuzzy logic, input the design of a single intersection Q-learning decision maker, The optimal control strategy is obtained. For regional traffic control, a distributed Q-learning algorithm combined with MAS is proposed, and a coordinated control model of Agent in adjacent junctions is given. Realizing the information sharing between adjacent junctions.) it solves the key problems of Q-learning algorithm and distributed Q-learning algorithm to traffic environment state set, action strategy set A, reward and punishment function R, etc. The choice of state space is designed to calculate queue length with fuzzy logic. Action strategy set A: increase, maintain and reduce phase green time: the reward and punishment function R takes the queue length of the intersection as the index, and takes the minimum queue length as the goal. It realizes the application of the distributed Q-learning algorithm in the optimization of the regional TSC system. The problem of regional signal coordination control is solved. The combination of distributed Q-learning algorithm and MAS realizes the optimal control of urban TSC system. The urban area traffic network is a distributed multi-#en2# network. A Multi-Agent model framework based on distributed Q-learning algorithm is established. At the same time, the detailed steps of designing distributed Q-learning algorithm are given. Finally, the performance of single-intersection TSC optimization based on Q-learning algorithm and regional TSC optimization based on distributed Q-learning algorithm is analyzed. The optimization performance of fixed time scheduling algorithm, Traffic-controller 1 / TC1, ACGJ-1 and distributed Q-learning algorithm is simulated and analyzed. The experimental results show that Q-learning algorithm and distributed Q-learning algorithm are superior to other algorithms in urban TSC system optimization.
【学位授予单位】：浙江师范大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：U491.54

【参考文献】