基于强化学习的自动交易代理

发布时间：2022-09-29 20:37

　　本文使用强化学习构建了与金融市场进行智能交互的自动交易代理。股票市场交易可以用于评估和开发新的机器学习方法,这些方法需要对金融市场交易问题的特征做出调整,尤其是强化学习。预测股市变化是一项非常艰巨的任务,因为驱动市场行为的基本模式是非静态的,这意味着过去学习到的有用的预测模式可能不适合在将来应用。强化学习尚未在该应用领域中广泛应用,相比于其他技术,强化学习的范式可以使代理具有更大自由度地直接学习交易决策模型,例如,无需预设定义用于购买或出售这些决策信号的特定阈值。价格的变化可以自然地被看作是一种奖励,所以强化学习可以避免在监督学习中标注示例和构建训练数据集所需的成本。在对先前文献的研究中,我们发现现有的应用强化学习算法来生成交易决策的研究通常不能解决非静态环境的问题。先前文献中所提出的方法得到的单一代理不会随着时间的变化而重新校准,同时学到的交易策略有时会陷入局部最优。本文提出的方法通过使用多个代理和一个多阶段学习模型来缓解上述提到的问题,多个代理可以竞争性地推荐最佳决策。我们的方法将在线学习与强化学习相结合。在线学习用于在决策点实时从一组代理中选择推荐的交易策略,还可以基于最近的数据... 

【文章页数】：98 页

【学位级别】：硕士

【文章目录】：
摘要
Abstract
Chapter 1 Introduction
    1.1 Motivation
    1.2 Objective
    1.3 Outline
Chapter 2 Background
    2.1 Machine Learning
        2.1.1 Sample Data
        2.1.2 Supervised Learning
        2.1.3 Unsupervised Learning
        2.1.4 Reinforcement Learning
    2.2 Preprocessing
        2.2.1 Feature Selection and the Curse of Dimensionality
        2.2.2 Feature Scaling
    2.3 Model Evaluation
        2.3.1 Regression Metrics
        2.3.2 Classification Metrics
        2.3.3 Reinforcement Learning Metrics
        2.3.4 Underfitting
        2.3.5 Overfitting
        2.3.6 Cross Validation
        2.3.7 Hyperparameter Tuning
    2.4 Models
        2.4.1 Logistic Regression
        2.4.2 Support Vector Machines
        2.4.3 Neural Networks
        2.4.4 Recurrent Neural Networks
        2.4.5 Monte Carlo Method
        2.4.6 Q-learning and Deep Q-learning
        2.4.7 Neuroevolution of Augmenting Topologies
    2.5 Stocks
        2.5.1 Stock Market
        2.5.2 Stocks Exchanges
        2.5.3 Stocks Trading
        2.5.4 Stocks Price
        2.5.5 Features
        2.5.6 Predictability
        2.5.7 Risk and Reward
        2.5.8 Quantitative Trading
    2.6 Previous Works
        2.6.1 Reinforcement Learning
        2.6.2 Supervised Learning
Chapter 3 Methodology
    3.1 Experiment Environment
    3.2 Data
        3.2.1 Selection of Stock Market
    3.3 Preprocessing and Exploratory Data Analysis
        3.3.1 Missing Data
        3.3.2 Feature Extraction and Domain Knowledge
        3.3.3 Feature Scaling
        3.3.4 Test and Training Data Selection
        3.3.5 Rolling Window of Model Inputs
        3.3.6 Cross Validation
    3.4 Multi-Agent Autonomous Trading
        3.4.1 Single Deep Q-learning Agent
        3.4.2 Online Weighted Selection
        3.4.3 Experience Replay
Chapter 4 Experiment
    4.1 Settings of Experiments
        4.1.1 Simulation of Trading
        4.1.2 Parameters of Agents
    4.2 Performance Measurement
        4.2.1 Cumulative Return
        4.2.2 Sharp Ratio
    4.3 Results
        4.3.1 Agent Experience
        4.3.2 Other Hyperparameters and Model Settings
        4.3.3 Comparison of Agents Decision Models on S&P 500 stocks
        4.3.4 Comparison of Base Agents with Multi-Agent on Different Markets
Conclusion
结论
References
Papers Published in the Period of Master Education
Acknowledgements


【参考文献】：
期刊论文
[1]基于粒子群优化WNN股票预测模型的性能评估[J]. 汪志峰,钱萌.  电脑知识与技术. 2019(09)
[2]变步长BLSTM集成学习股票预测[J]. 王子玥,谢维波,李斌.  华侨大学学报(自然科学版). 2019(02)
[3]基于小波神经网络与ARIMA组合模型在股票预测中的应用[J]. 杨进,陈亮.  经济数学. 2018(02)
[4]基于支持向量机的股票预测[J]. 张晨希,张燕平,张迎春,陈洁,万忠.  计算机技术与发展. 2006(06)

硕士论文
[1]BP神经网络在股票预测中的应用研究[D]. 王莎.中南大学 2008



本文编号：3683269