基于多智能体Q学习的车载通信MAC层信道接入技术研究

发布时间：2018-07-11 09:23

本文选题：车载自组织网络 + 无线信道接入　；参考：《南京邮电大学》2017年硕士论文

【摘要】：车载自组织网络(VANET,Vehicular ad-hoc network)作为支持ITS(Intelligent Transportation Systems,智能交通系统)应用的重要技术,解决安全相关问题时通常依赖于安全消息的实时可靠传输。VANET因其网络拓扑结构变化快、节点高速移动的特点在车辆密度较高的网络场景中使用传统的IEEE 802.11p协议发送消息很容易发生碰撞,难以保证安全消息低时延和高接收率的传输,且对不同网络负载场景不具可扩展性。因此,本文针对信道接入方法的可扩展性和公平性的问题对传统的接入方法进行改进,通过建立新模型为VANET设计高效的MAC协议,主要工作如下:首先,在车载通信车辆接入无线信道发送数据过程中引入Q学习,为车辆节点建立智能体Q学习模型,提出了基于Q学习的动态调整竞争窗口算法(QL-CWmin),推导了车辆节点在网络环境中进行Q学习的状态——动作映射关系和车辆节点更新累计奖赏值的迭代公式,这样车辆节点每次需要接入无线信道发送数据时,就总选择能使累计奖赏值最大的CW值接入无线信道,在理论分析的基础上通过仿真验证了所提出的算法对提高车辆节点接入无线信道的公平性和对不同网络负载场景的可扩展性。其次,针对单个车辆节点在网络环境中进行Q学习存在仅感知部分网络环境、学习搜索空间大效率低、与其他车辆节点无交互学习的问题,为VANET建立多智能体Q学习系统模型,提出了基于多智能体Q学习的动态调整竞争窗口算法(QL-CWMulti-agent),对多个车辆节点Q学习的联合状态——动作对映射关系和车辆节点根据联合策略更新Q值函数的迭代表达式进行了推导,在前一研究点的基础上提高车辆节点接入无线信道的公平性及算法对不同网络负载的可扩展性。最后,将多智能体学习的最终结果收敛到相关均衡,根据eCEQ(Correlated Equilibrium Q,相关均衡Q学习)算法(即最大化所有智能体奖励的最小值)使车载自组织网络中每个车辆节点接入无线信道成功发送数据的次数最大化,通过仿真验证了算法收敛结果的正确性。
[Abstract]:As an important technology to support the application of Intelligent Transportation Systems (its), the Vehicular ad-hoc network (VANET) usually depends on the real-time and reliable transmission of security messages. The characteristics of high speed mobile nodes are that the traditional IEEE 802.11p protocol is used to send messages easily in the high vehicle density network scenario, so it is difficult to ensure the transmission of security messages with low delay and high reception rate. And it is not extensible for different network load scenarios. Therefore, aiming at the scalability and fairness of channel access methods, this paper improves the traditional access methods and designs an efficient MAC protocol for VANET by establishing a new model. The main work is as follows: first of all, Q learning is introduced in the process of vehicle accessing wireless channel to transmit data, and an agent Q learning model is established for vehicle nodes. A dynamic adjusting competition window algorithm (QL-CWmin) based on Q-learning is proposed. The state-action mapping relation of vehicle nodes for Q-learning in network environment and the iterative formula for updating cumulative reward values of vehicle nodes are derived. So every time a vehicle node needs to access the wireless channel to send data, it always selects the CW value, which can make the maximum cumulative reward value, to access the wireless channel. Based on the theoretical analysis, the proposed algorithm is verified by simulation to improve the fairness of the vehicle node access to the wireless channel and the scalability of different network load scenarios. Secondly, aiming at the problem that a single vehicle node only perceives part of the network environment for Q learning in the network environment, the learning search space is inefficient and there is no interactive learning with other vehicle nodes, a multi-agent Q learning system model is established for VANET. A QL-CW Multi-agent algorithm based on multi-agent Q learning is proposed. The joint state-action pair mapping of Q learning for multiple vehicle nodes and the iterative expression for updating Q value function of vehicle nodes according to joint strategy are derived. On the basis of the former research point, the fairness of the vehicle node access wireless channel and the scalability of the algorithm to different network loads are improved. Finally, the final result of multi-agent learning is converged to the relevant equilibrium. According to the eCEQ (related Equilibrium Q) algorithm (that is, to maximize the minimum value of all the agent awards), the number of successful data transmission by each vehicle node in an ad hoc network is maximized. The correctness of the convergence result is verified by simulation.
【学位授予单位】：南京邮电大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TN929.5;U495

【相似文献】