基于强化学习的蓄电池储能系统的优化控制

发布时间：2019-01-01 19:44

【摘要】：以风能、太阳能等为代表性的分布式发电单元受到气候和天气影响,发电功率难以保证平稳,可能会引起频率和电压不稳,进而引起停电事故。为了解决这一问题,在具有分布式电源的系统中引入了储能装置。但是受环境影响,系统供电与用电负荷会出现不平衡的情况,从而导致蓄电池处于亏电状态或过充电状态,长期运行会降低蓄电池组的使用寿命,增加系统维护成本,因此选择合适的蓄电池控制策略具有重要的实际意义。本文研究了一个由分布式发电源、储能设备、用电负荷以及系统的能量管理中心组成的蓄电池储能系统。该储能系统可以与电网进行交互。发电量不足时可以从电网买电,除了供给负荷需求,系统中多余的电量可以卖给电网或向电网提供频率调节服务。系统中分布式发电功率、负荷需求功率、电价和调频的价格相互独立,具有很大的不确定性,本文将它们分别建模为Markov链来研究。蓄电池储能装置从当前状态转移到空和满两个特殊状态的逗留时间不服从指数分布,所以将该储能系统的优化控制问题建模为半Markov决策过程。本文采用基于模型的Sarsa算法来学习最优策略,从而使系统在满足负荷需求的基础上获得的长期收益最大。随着电动汽车产业的发展,电动汽车入网(vehicle-to-grid, V2G)正在成为研究热点。本文考虑将分布式发电装置引入V2G系统中。当发电量不足时,电动车可以从电网买电；当发电能力比较强时,除了供给电动车的用电需求,多余的发电量直接卖给电网。闲置在充电桩上的电动车可以与电网交互,根据自身电量的情况以及电价和调频价格的高低,决定向电网卖电或是提供频率调节服务。假设系统能够在决策周期的初始时刻获得系统的发电量和电价信息。将系统的优化控制问题建模为动态规划过程。用策略迭代的方法来获取最优策略,从而使该系统能够在满足自身需求的同时获取最大收益。
[Abstract]:The distributed generation units, such as wind energy and solar energy, are affected by climate and weather, so it is difficult to ensure stable power generation, which may cause frequency and voltage instability, and then cause power outages. In order to solve this problem, energy storage device is introduced into the system with distributed power supply. However, under the influence of the environment, there will be imbalance between the power supply and the power load of the system, which will lead to the battery in the state of power deficit or overcharge, and the long-term operation will reduce the service life of the battery group and increase the maintenance cost of the system. Therefore, the selection of appropriate battery control strategy has important practical significance. In this paper, a battery energy storage system composed of distributed power generation, energy storage equipment, power load and energy management center of the system is studied. The energy storage system can interact with the power grid. Electricity can be bought from the power grid when the power generation is insufficient. In addition to supplying the load demand, the excess power in the system can be sold to the power grid or to provide frequency regulation services to the power network. The distributed generation power, load demand power, electricity price and frequency modulation price are independent of each other. In this paper, they are modeled as Markov chains. The time of stay of storage battery energy storage device from current state to empty state and full state is not satisfied with exponential distribution, so the optimal control problem of the energy storage system is modeled as a semi-Markov decision-making process. In this paper, the model-based Sarsa algorithm is used to learn the optimal strategy, so that the long-term benefits of the system can be maximized on the basis of satisfying the load requirements. With the development of electric vehicle industry, vehicle-to-grid, V 2 G is becoming a research hotspot. In this paper, we consider introducing distributed generation equipment into V2G system. Electric vehicles can buy electricity from the power grid when the generation capacity is low; when the generation capacity is relatively strong, in addition to the electricity demand for electric vehicles, the excess power generation is sold directly to the power grid. Electric vehicles idle on charging piles can interact with the power grid and decide to sell power to the grid or provide frequency regulation services according to their own electricity quantity and the price of electricity price and frequency modulation. It is assumed that the system can obtain the power generation and price information of the system at the beginning of the decision cycle. The optimal control problem of the system is modeled as a dynamic programming process. The method of policy iteration is used to obtain the optimal policy, so that the system can obtain the maximum profit while satisfying its own requirements.
【学位授予单位】：合肥工业大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TM912

【相似文献】