Q-learning for the speed trajectory optimization of the freight train

Xuan Lin; Zhicheng Liang; Tiesheng Yan; Taiqiang Cao; Hua Cheng; Jian Mao; Rui Deng

doi:10.1117/12.2652584

21 November 2022 Q-learning for the speed trajectory optimization of the freight train

Xuan Lin, Zhicheng Liang, Tiesheng Yan, Taiqiang Cao, Hua Cheng, Jian Mao, Rui Deng

Proceedings Volume 12340, International Conference on Frontiers of Traffic and Transportation Engineering (FTTE 2022); 123401H (2022) https://doi.org/10.1117/12.2652584
Event: International Conference on Frontiers of Traffic and Transportation Engineering (FTTE 2022), 2022, Lanzhou, China

Abstract

The train speed trajectory optimization (TSTO) aims at finding the optimal speed trajectory considering the safety, energy efficiency, punctuality and stopping accuracy. From the perspective of mitigating the greenhouse effect, it’s quite significant to study the TSTO problem. This paper proposed an optimization algorithm based on Reinforcement Learning (RL). Firstly, a global optimization model using RL was established. In the model, the control sequence including the control regimes and their switching points was taken as the state. The optimization objectives were taken as the reward function. The adjustment of the position of the switching points in the control sequence was taken as the decision space of the agent. Secondly, an adjustment method of the control sequence based on the deep Q-learning and embedding matrix was proposed. The training data was sampled using the experience replay. The optimal control sequence was obtained through the iterative training of the neural network. Finally, the optimization algorithm based on RL was compared with the driving strategies based on the Pontryagin’s Maximum Principle (PMP) and the field test data. The results show that the energy consumption of the proposed algorithm is reduced by 0.16% in comparison with that of the PMP, which proves that the proposed method can be applied to the multi-objective optimization of the train operation. Comparing with the field test data, the energy consumption of the optimization algorithm is reduced by 4.89%, which demonstrates that the proposed method can be used to guide the drivers to drive the freight train energy-efficiently.

Citation Download Citation

Xuan Lin, Zhicheng Liang, Tiesheng Yan, Taiqiang Cao, Hua Cheng, Jian Mao, and Rui Deng "Q-learning for the speed trajectory optimization of the freight train", Proc. SPIE 12340, International Conference on Frontiers of Traffic and Transportation Engineering (FTTE 2022), 123401H (21 November 2022); https://doi.org/10.1117/12.2652584

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Neural networks

Optimization (mathematics)

Algorithm development

Control systems

Evolutionary algorithms

Data modeling

Machine learning

Keywords/Phrases

Search In:

Publication Years