CN117521485A - Optimizing method for energy-saving design of longitudinal subway lines based on deep reinforcement learning - Google Patents

Abstract

The present invention relates to the technical field of subway longitudinal section line design, and relates to a method for energy-saving design and optimization of subway longitudinal section lines based on deep reinforcement learning. It includes the following steps: 1. Combine the subway line design specification constraints and actual construction condition constraints to establish A design model for longitudinal subway lines with the goal of minimizing train operation energy consumption; 2. Using deep reinforcement learning algorithms to solve the optimal energy-saving subway longitudinal lines under different operating energy consumption calculation factors. Compared with the actual longitudinal section line design scheme, the present invention can simultaneously reduce the energy consumption cost and time cost of train operation.

Description

Optimization method for energy-saving design of subway longitudinal section based on deep reinforcement learning

Technical Field

The present invention relates to the technical field of subway longitudinal section line design, and in particular to a subway longitudinal section line energy-saving design optimization method based on deep reinforcement learning.

Background Art

Existing studies have shown that train traction energy consumption mainly depends on line conditions and train operation organization (train scheduling, driving strategy, stop plan, etc.) [2, 3][2, 3]{Douglas, 2015 #37;Zhou, 2018 #60}{Douglas, 2015#37;Zhou, 2018 #60}. Train operation organization is restricted by line conditions and has limited energy-saving effects. If we want to further reduce traction energy consumption, we need to consider energy saving more in the line design stage. The core content of line design is horizontal and vertical section design. The horizontal design aims to maximize passenger flow attraction and rarely considers the need for energy saving. The vertical section design is more closely related to traction energy consumption. There are three main laying methods for subway transportation lines: underground lines, elevated lines and ground lines. Underground line sections all run underground, and the slope is generally not limited by the slope. It is only limited by factors such as underground buildings, pile foundations, and underground pipelines along the line. It is most suitable to use energy-saving slopes (high station, low section).

At present, the research on energy-saving slopes is based on the perspective of energy saving. Combining the longitudinal section line design principles and traction simulation calculations, the general principles of longitudinal section energy-saving line design forms are summarized and their energy-saving effects are analyzed. Some scholars have found that the longitudinal section layout of the downhill exit and the uphill entrance helps to reduce the train traction energy consumption. Some scholars have proposed an energy-saving slope design method that changes the station elevation value to change the slope of the energy-saving slope. However, the parameters of the energy-saving slope are often selected based on experience, which may not necessarily achieve the best energy-saving effect.

With the rise of intelligent algorithms, domestic and foreign scholars have combined longitudinal section line design with intelligent algorithms, and the study of using computers to automatically optimize longitudinal section line plans has gradually become a hot topic in the field of longitudinal section line design research. Some scholars have constructed an optimization model for the horizontal and vertical sections of urban rail transit lines based on the train running behavior, and used genetic algorithms to solve the optimal design plan for the line in three-dimensional space. Some scholars have used the distance transformation algorithm to simultaneously optimize the mountain railway line shape and station location based on the coupling constraints of the railway route and the station location. Some scholars have proposed a multi-stage decision-making model to jointly optimize the longitudinal section line, cruising speed and taxiing operation point to obtain the lowest cost solution. Some scholars have used a multi-level augmented differential evolution algorithm based on geographic information to solve the horizontal and vertical section line design of intercity railways.

Although the above studies can meet the requirements of relevant design specifications and be used for energy-saving optimization of longitudinal sections, the methods used in these studies have certain limitations.

(1) The need to avoid actual engineering constraints (underground buildings, pile foundations, drainage pipelines, and poor geology) in longitudinal section line design was ignored.

(2) Algorithms such as PSO, GA or PSO-GA require a predefined number of longitudinal line intersections as input. Therefore, these methods are suitable for "optimizing" lines rather than "designing" lines, and can only find the optimal line under this number of intersections.

(3) Even though GA, PSO, DT and other evolution-based algorithms have been greatly improved, they still cannot learn like humans and have difficulty achieving primary-level optimization.

Summary of the invention

The content of the present invention is to provide a method for optimizing the energy-saving design of subway longitudinal sections based on deep reinforcement learning, which can better obtain the optimal energy-saving line of the subway longitudinal section.

The method for optimizing energy-saving design of subway longitudinal section lines based on deep reinforcement learning according to the present invention comprises the following steps:

1. Combining the constraints of subway line design specifications with the constraints of actual construction conditions, a subway longitudinal section line design model with the goal of minimizing train operation energy consumption is established;

2. Use deep reinforcement learning algorithm to solve the optimal energy-saving route of the subway longitudinal section under different operating energy consumption calculation factors.

As a preferred option, the subway longitudinal section line design model includes:

1) Environment: The avoidance area and the transition curve section of the horizontal line constitute the environment for the optimization of the subway longitudinal section line;

2) Agent: It is defined as an intelligent program that determines the direction of energy-saving design of longitudinal sections;

3) State: The spatial position of the slope change point is defined as the state , spatial position refers to the two-dimensional coordinates of the subway longitudinal section, including the longitudinal mileage coordinates and the vertical depth coordinates;

4) Action: The search direction of the next slope change point selected by the agent is defined as the action ;

5) Reward: The value of the reward depends on the feedback from the environment. The energy consumption of the train is the main component of the reward, and the other components are the survival status reward and the target distance reward;

6) Condition: If the agent cannot find a state that satisfies arrive All constraints actions , that is, the current state change condition Not established condition , and the current state The initialization state is ; Otherwise, the state change condition Establishment condition , continue to select the next action.

As a preferred option, the subway line design specification constraints and actual construction condition constraints include:

(1) Constraints on slope length and slope gradient in the station area

Only one ramp is set up in the station area, that is, the sum of the lengths of the entrance and exit ramps is greater than the platform length. , assuming that the lengths of the entrance and exit slopes are the same, then the slope length of the station area is The constraints are expressed as:

;

Station slope Use the given constant ,Right now:

;

(2) Slope length and slope constraints in non-station areas

The slope length constraint of the non-station area is expressed as:

;

The line has a minimum slope and maximum slope ,Right now:

;

(3) Minimum straight line length

The length of the straight line between two adjacent vertical curves It should meet the design specification requirements, and the calculation formula is as follows:

;

In the formula, Indicates The longitudinal mileage of each slope change point; Indicates The length of the tangent line at the slope change point; Indicates the minimum straight line length in the design specification;

(4) Reverse limit slope

When two slope sections in opposite directions are connected, the slope in one direction is Should not be greater than the reverse limit slope ,Right now:

;

(5) Line burial depth constraints

The line depth constraint is that any point on the line Design elevation of the track top of the underground tunnel , should be less than the ground elevation of the point Subtract tunnel height and minimum cover thickness ,Right now:

;

(6) Avoidance area constraints

All avoidance areas of subway longitudinal sections Indicates that the subway longitudinal section line tunnel area is used express, and The intersection of should be an empty set:

;

(7) Plane transition curve segment constraints

Slope change point mileage coordinates The coordinates of the start and end points of the plane transition curve The distance should not be less than the tangent length of the vertical curve, that is:

.

As a preferred embodiment, in the state State, The state space at the end of each action is defined as , as shown below:

;

in, For the The longitudinal mileage coordinates of the slope change points, For the The vertical elevation coordinates of the slope change points, W and H are the upper limits of the mileage and depth of the target optimization area respectively; in the above formula, the overall position of two consecutive slope change points is taken as a state in the longitudinal section line design optimization model.

As a preference, in the action Action, the action The status Convert to status ,action It is expressed in two parts as follows:

;

in and is the length and slope of the slope; The relationship between the three is:

.

As a preferred reward, the reward The formula is as follows:

;

in They represent the unit operating energy cost, survival cost, and distance cost to the destination respectively; for The weight of .

As a preferred method, in the subway longitudinal section line design model, the approximate integration method is used to solve the train motion equation to solve the train running energy consumption, and the upper and lower limits of the integral in the running time and distance formula are divided into several small speed intervals. , let the initial and final velocities in the integral be , then the formula for calculating the train running time and running distance using the approximate integral method is:

Run time

;

Unit Force Calculation

Under different working conditions, it can be written in the following form:

;

In the formula, For traction; The mass of the motor vehicle; For the trailer quality; is the unit braking force of the train; is the common braking coefficient; is the unit traction force, Unit resistance of the train, including basic resistance of train operation , ramp additional resistance , curve additional resistance , additional tunnel resistance .

Preferably, the operating energy consumption calculation factors include reverse slope limit, speed limit and passenger capacity.

Preferably, the deep reinforcement learning algorithm is an improved D3QN algorithm, which is improved as follows:

a. Use the experience replay mechanism to store the interactive experiences one by one in the experience pool. After accumulating a certain amount, the model randomly extracts a certain batch of data from the experience pool to train the neural network at each step;

b. Construct two neural networks with the same structure, namely the valuation network and target value network , the valuation network is used for a given state , calculate the action to take The expected cumulative reward of , the estimated network parameters Continuously updated; the target network is used to calculate the timing difference target value , target value network parameter Fixed, replaced with the latest estimated network parameters at regular intervals ; Target value The calculation formula is as follows:

;

In the formula, For immediate reward; Table is the discount factor; Indicates that in the next state In Maximizing Action ;

Remaining the same over a period of time leads to a valuation network Convergence Target Relatively fixed;

c. Improve the neural network structure and divide its output into two parts, one of which is the state value function that characterizes the quality of each state. , and the other is the advantage function that distinguishes the good and bad actions in a specific state :

;

In the formula, For strategy, is the parameter of the advantage function;

d. The target value of the D3QN model is as follows:

;

e. Mean square error As the loss function, the network parameters are updated as follows:

.

The present invention constructs a deep reinforcement learning model for subway longitudinal section line design, which senses, searches, judges, and makes decisions on the line selection environment without manual experience, and finds the optimal energy-saving line plan by feedback on different constraints. Compared with the actual longitudinal section line design plan, the present invention can reduce the energy consumption cost and time cost of train operation at the same time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method for optimizing energy-saving design of a subway longitudinal section line based on deep reinforcement learning in an embodiment;

FIG2 is a flow chart of the improved D3QN algorithm in the embodiment.

DETAILED DESCRIPTION

In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

Example:

As shown in FIG1 , this embodiment provides a method for optimizing energy-saving design of a subway longitudinal section line based on deep reinforcement learning, which includes the following steps:

1. Combining the constraints of subway line design specifications with the constraints of actual construction conditions, a subway longitudinal section line design model with the goal of minimizing train operation energy consumption is established;

2. Use deep reinforcement learning algorithm to solve the optimal energy-saving route of the subway longitudinal section under different operating energy consumption calculation factors.

The subway longitudinal section line design model includes:

1) Environment: The avoidance area (underground buildings, pile foundations, underground pipeline routes, adverse geology, etc.) and the plane line transition curve section area constitute the environment for subway longitudinal section line optimization;

2) Agent: It is defined as an intelligent program that determines the direction of energy-saving design of longitudinal sections;

3) State: The spatial position of the slope change point is defined as the state , spatial position refers to the two-dimensional coordinates of the subway longitudinal section, including the longitudinal mileage coordinates and the vertical depth coordinates;

4) Action: The search direction of the next slope change point selected by the agent is defined as the action ;

5) Reward: The value of the reward depends on the feedback from the environment. The energy consumption of the train is the main component of the reward, and the other components are the survival status reward and the target distance reward;

6) Condition: If the agent cannot find a state that satisfies arrive All constraints actions (Slope length With slope ), that is, the current state change condition Not established condition , and the current state The initialization state is ; Otherwise, the state change condition Establishment condition , continue to select the next action.

The constraints of subway line design specifications and actual construction conditions include:

(1) Constraints on slope length and slope gradient in the station area

In the station area, if the train runs across a slope, it will cause the train vibration to be superimposed, which is not conducive to the smooth and safe operation of the train. For this reason, it is best to set up only one slope in the station area, that is, the sum of the length of the entrance and exit slopes is greater than the platform length. , assuming that the lengths of the entrance and exit slopes are the same, then the slope length of the station area is The constraints are expressed as:

;

For the slope of the station area, in order to prevent the train from slipping and the luggage from falling off the platform, a smaller slope is used as much as possible while meeting the drainage requirements. Use the given constants in the Metro Design Code ,Right now:

;

(2) Slope length and slope constraints in non-station areas

The "Metro Design Specifications" require that the length of any slope section in the non-station area Should not be less than the future length of subway trains However, since the distance between subway stations is short and drainage ditches should be set up in the interval, the length of a single slope section should not be too long. The slope length constraint of the non-station area is expressed as:

;

Due to the limitation of train traction capacity, the slope of the main line section should not be too steep. The "Metro Design Specifications" stipulate that the maximum slope of the main line should not exceed the maximum slope In addition, in order to facilitate drainage of the section, the line should have a minimum slope ,Right now:

;

(3) Minimum straight line length

The length of the straight line between two adjacent vertical curves It should meet the design specification requirements, and the calculation formula is as follows:

;

In the formula, Indicates The longitudinal mileage of each slope change point; Indicates The length of the tangent line at the slope change point; Indicates the minimum clip line length in the design specification.

(4) Reverse limit slope

Although the "Metro Design Code" does not clearly stipulate the maximum value of the subway gradient difference, in order to improve passenger comfort, facilitate design and construction, reduce maintenance and operation costs, etc., when two slope sections in opposite directions are connected, the slope in one direction is Should not be greater than the reverse limit slope ,Right now:

;

(5) Line burial depth constraints

The line depth constraint is that any point on the line Design elevation of the track top of the underground tunnel , should be less than the ground elevation of the point Subtract tunnel height (distance from rail top to tunnel outer diameter, fixed value) and minimum cover thickness (fixed value), that is:

;

(6) Avoidance area constraints

The design of the subway longitudinal section line should take into account factors such as underground soil, building pile foundations, pipelines, etc., and avoid areas where construction is impossible or difficult. Indicates that the subway longitudinal section line tunnel area is used express, and The intersection of should be an empty set:

;

(7) Plane transition curve segment constraints

Within the vertical curve range, the track surface elevation changes with a certain curvature; within the plane transition curve range, the track surface elevation changes with a certain superelevation along the slope. If the two overlap, on the one hand, the outer rail elevation is not easy to control during track laying and maintenance; on the other hand, the outer rail's straight superelevation along the slope and the center vertical curve will change shape, affecting the stability of driving. Therefore, based on the known plane line type, when designing the longitudinal section line, the mileage coordinates of the slope change point The coordinates of the start and end points of the plane transition curve The distance should not be less than the tangent length of the vertical curve, that is:

.

In the State, the energy-saving design of the subway longitudinal section line refers to optimizing the number and location of slope change points to achieve the purpose of energy saving in train operation. The state space at the end of each action is defined as , as shown below:

;

in, For the The longitudinal mileage coordinates of the slope change points, For the The vertical elevation coordinates of the slope change points, W and H are the upper limits of the mileage and depth of the target optimization area, respectively.

Note that at least two slope change points must be considered to determine whether the line satisfies the constraints. In the above formula, the overall position of two consecutive slope change points is taken as a state in the longitudinal line design optimization model.

In Action, The status Convert to status ,action It is expressed in two parts as follows:

;

in and is the length and slope of the slope; The relationship between the three is:

.

In the reward Reward, the agent takes , change the state from Convert to How to evaluate the adoption of If an action is "better" than other actions, then a reward from the environment is needed to reflect the quality of the action. The formula is as follows:

;

in They represent the unit operating energy cost, survival cost, and distance cost to the destination respectively; for Since the model optimization goal is to minimize the operating energy consumption, is a negative value; This is to ensure that the selected route meets all the constraints mentioned above; It is for the AGENT to choose Each indicator is discussed further below.

A. Energy consumption cost

A. Energy consumption cost

Energy consumption costs include energy consumption in the traction phase, cruising phase, and braking phase. The train operation strategy is: after leaving the station, accelerate with maximum traction and switch to The train then adopts the cruise mode, i.e. runs at a constant speed. When it reaches the cruise-brake mode conversion point, it decelerates and enters the station at the maximum limited power.

a) Traction energy consumption during traction phase

In the train traction acceleration phase, the traction energy consumption of each step can be calculated based on the train motion equation and the functional conversion relationship, and then the traction energy consumption of all steps in this phase is accumulated. for:

;

Where: For the The traction force of each step can be calculated based on the train running speed of the step. Find the specific value on the traction characteristic curve; is the transmission efficiency constant of the train traction motor.

b) Traction energy consumption during cruising phase

When the train speed reaches the set speed, it will enter the cruising stage. When (the sum of the basic resistance of the train and the additional resistance of the line) is positive, a certain traction force is required to maintain a constant speed. When it is negative, a certain braking force is required to maintain a constant speed, and the traction force is zero. Therefore, the train traction force in the cruising stage is:

;

Where: The basic resistance of the train is related to the current running speed; The slope resistance, curve resistance and tunnel resistance that the train encounters when running on the line are related to the current line conditions.

Traction energy consumption during cruising for:

;

c) From the state arrive The total unit energy cost is calculated as follows:

;

B. Cost of living

In this embodiment, it is very necessary to find a feasible longitudinal section route plan that meets various constraints of the subway longitudinal section environment, so a survival reward is added to the reward function to encourage the agent to meet all constraints when selecting actions.

If the agent cannot find a state that satisfies arrive All constraints , ,but Take the negative value and set the current state The initialization state is , restart line selection; if the agent can find a satisfying state arrive All constraints , ,but Take a positive value and start the next Selection.

C. Distance cost from the destination

To prevent the agent from choosing actions that only satisfy the survival reward instead of moving towards the end point, we use To encourage the agent to choose actions that move towards the end point. Therefore, we define the reward function , from arrive as follows:

In the formula, Status The straight-line distance to the end point, Status The straight-line distance to the end point.

In the subway longitudinal section line design model, the two most important indicators in the train traction calculation are the train's running speed and running time, both of which are related to the running distance S, and are reflected in the graph as the VS curve and TS curve.

There are three main methods for solving the running time t and running distance S. There are three main methods: direct integration method, graphical method and approximate integration method. Among them, the approximate integration method is the most widely used method and is also the method used by traction computer.

The approximate integral method for solving the train motion equation is to divide the upper and lower limits of the integral in the running time and distance formula into several small speed intervals. , Pick The unit force at the average speed within the range is The initial and final velocities in the interval are , then the formula for calculating the train running time and running distance using the approximate integral method is:

Run time

;

Note: The smaller the value, the more accurate the calculation result. .

Unit Force Calculation

Under different working conditions, it can be written in the following form:

;

In the formula, For traction; The mass of the motor vehicle; For the trailer quality; is the unit braking force of the train; is the common braking coefficient; is the unit traction force, Unit resistance of the train, including basic resistance of train operation , ramp additional resistance , curve additional resistance , additional tunnel resistance .

Train operation energy consumption refers to the energy consumed during train operation to overcome train running resistance, increase train kinetic energy and overcome gravitational potential energy difference. The calculation factors of operation energy consumption include reverse slope limit, speed limit and passenger capacity.

Reverse Slope Limit

On non-straight roads, since trains need to constantly accelerate and decelerate in order to reach the predetermined running speed, their energy consumption increases significantly compared to straight roads. When the gradient difference at the slope change point is too large (two adjacent opposite steep slopes), the train slows down and accelerates alternately, which affects the train's running energy consumption, driving stability, and reduces passenger comfort. Actual engineering regulations stipulate that when two opposite slopes are connected, the slope in one direction should not be greater than , in the second phase of the project, it was relaxed to Therefore, when the reverse limit slope of this embodiment affects the energy-saving design of the subway longitudinal section line, the reverse limit slope value is as follows:

;

Where: Limit the slope in the opposite direction.

Design speed

Matching the slope section with the design speed can improve the utilization of line potential energy and reduce energy loss caused by unnecessary braking. The energy consumption during train operation is mainly used to overcome running resistance. When the train maintains the existing speed or accelerates, the energy consumption will increase when overcoming resistance, and air resistance is proportional to the square of the train speed. Train design speed It has a significant impact on the train running behavior. When the design speed of this embodiment affects the energy-saving design of the subway longitudinal section line, the design speed value is as follows:

Passenger Capacity

The impact of passenger capacity on train operation energy consumption is mainly reflected in the impact on the total traction weight of the train. Generally, the greater the traction mass of the train, the greater the starting and braking torque required for the train, and the greater the power consumption of the traction motor required to meet the operation needs, resulting in an increase in energy consumption. When the passenger capacity of this embodiment affects the energy-saving design of the subway longitudinal section line, the passenger capacity is taken as follows:

Where: N is the passenger capacity, For an empty car, For full capacity , Overload .

The Q-Learning algorithm uses the next state at each step of exploration. If this idea is directly applied to DQN, the following problems will occur:

(1) The premise of training neural networks is to assume that the training data is independent and identically distributed. However, there is a strong correlation between the sequential data obtained by the interaction of intelligent agents, which can easily cause instability in network training.

(2) The parameters of the DQN network are continuously updated, and the same network is used to generate This causes the temporal difference target of the neural network to change continuously, which is not conducive to the convergence of the algorithm.

(3) The model is not stable enough in the early stage of training, and the value function estimation has deviations. This will cause the model to overestimate the expected benefit of a certain action, misleading the agent to choose the wrong action and causing the model to be unable to find the optimal strategy.

The deep reinforcement learning algorithm is the improved D3QN algorithm, which is improved as follows:

a. Use the experience replay mechanism to store the interactive experiences one by one in the experience pool. After accumulating a certain amount, the model randomly extracts a certain batch of data from the experience pool to train the neural network at each step;

b. Construct two neural networks with the same structure, namely the valuation network and target value network , the valuation network is used for a given state , calculate the selected action The expected cumulative reward of , the estimated network parameters Continuously updated; the target network is used to calculate the timing difference target value , target value network parameter Fixed, replaced with the latest estimated network parameters at regular intervals ; Target value The calculation formula is as follows:

In the formula, For immediate reward; Table is the discount factor; Indicates that in the next state In Maximizing Action ;

Remaining the same over a period of time leads to a valuation network Convergence Target Relatively fixed; the actions of the maximum function generated by the valuation network and the target value network are not necessarily the same. Produce action, Calculating the target value can prevent the model from selecting overestimated suboptimal actions, effectively solving the overestimation problem of the DQN algorithm.

c. Improve the neural network structure and divide its output into two parts, one of which is the state value function that characterizes the quality of each state. , and the other is the advantage function that distinguishes the good and bad actions in a specific state

In the formula, For strategy, is the parameter of the advantage function;

d. The target value of the D3QN model is as follows:

e. Update the network parameters using mean square error E as the loss function as follows:

.

The improved D3QN algorithm flow is shown in Figure 2.

1. Initialization: Initialize the valuation neural network and the target neural network, both of which are deep neural networks used to estimate the action value function. At the same time, initialize the experience playback buffer to store the experience tuples of the agent's interaction with the environment.

2. Collect experience: The agent interacts with the environment, selects actions based on the current strategy, and observes the next state and immediate reward of the environment feedback. These experience tuples are stored in the experience playback buffer for subsequent training.

3. Train the valuation network: Randomly extract a batch of experience tuples from the experience replay buffer. For each experience tuple, use the valuation neural network to estimate the action value of the current state. Then, use the target neural network to estimate the action value of the next state. Calculate the target value of Q-learning and update the parameters of the valuation neural network to make it close to the target value.

4. Update the target network: Regularly update the parameters of the target neural network by copying the parameters of the estimated neural network to the target neural network. This helps stabilize the training process and reduce jitter in the estimated target.

5. Select action: Select an action based on the current strategy. You can use strategies such as ε-greedy to balance exploration and exploitation.

6. Iterative training: Repeat steps 2 to 5 to continuously collect experience, update the valuation network, update the target network, and optimize the decision-making strategy of the agent.

7. Convergence and evaluation: As training progresses, observe how the algorithm's performance converges. You can periodically evaluate in the environment to test how the trained agent performs in new scenarios.

8. End training: When the predetermined number of training steps is reached or the algorithm converges, the training process ends. The agent's valuation neural network is the final training result and can be used to make decisions in practical applications.

Case

This example first selects a typical section of a subway line in Chengdu as the research object, and solves the optimal energy-saving line of the subway longitudinal section under different operation energy consumption calculation factors. Finally, the optimization design is extended to a subway line section in Chengdu and compared with the solution generated by experienced human designers.

The main constraints are shown in Table 1

Table 1 Main constraints

Results under different parameters

The D3QN model for subway longitudinal section line design, which aims to minimize train operation energy consumption, is used to analyze the impact of different energy consumption calculation factors on the energy-saving design of the longitudinal section line of a certain subway section in Chengdu.

(1) Analysis of the impact of reverse slope limitation

Here we analyze the reverse slope limit For the impact of energy-saving design on longitudinal section lines, other energy consumption calculation factors are fixed at: .

① Model optimization effect: Under different reverse slope limit conditions, the energy consumption of the energy-saving design linear calculation results is reduced by 3.51% to 3.83% compared with the original linear calculation results. Moreover, the energy consumption is reduced more with the increase of reverse slope limit, and the running time of each part is close to the original time.

② Line type change: The acceleration slope of the energy-saving design line type is longer than that of the original line type. Due to the reverse slope limit, in order to better convert potential energy into kinetic energy and enable the train to reach the design speed more quickly, the value of the first acceleration slope of the energy-saving design line type increases as the reverse slope limit decreases. The value of the second acceleration slope increases as the reverse slope limit increases.

(2) Analysis of the impact of design speed

Here we analyze the impact of design speed on energy-saving design of longitudinal section lines, and other energy consumption calculation factors are fixed at: .

① Model optimization effect: Under different design speed conditions, the energy consumption of the energy-saving design line type calculation results is reduced by 1.32% to 14.14% compared with the original line type calculation results. The running time decreases with the increase of speed. The model has obvious optimization effect for the design speed near 80km/h. This is because the energy consumption of the original line type at the design speed near 80km/h is too large, which indirectly proves that the original line type is not an "excellent" energy-saving slope line type.

② Line shape change: The slope value of the first acceleration slope of the line energy-saving design line shape increases with the increase of the design speed. It is speculated that in order to allow the train to reach the design speed faster, the slope value of the first acceleration slope is increased, which can better convert potential energy into kinetic energy. Linear gap ratio The large difference in line type is to bypass the plane transition curve segment.

(3) Analysis of the impact of passenger volume

Here we analyze the passenger volume For the impact of energy-saving design on longitudinal section lines, other energy consumption calculation factors are fixed at: .

① Model optimization effect: Under the passenger capacity condition, the energy consumption of the energy-saving design linear calculation results is reduced by 2.33% to 19.71% compared with the original linear calculation results. The operating time increases with the increase of passenger capacity. The model has obvious optimization effect for no-load, with the operating time reduced by 12.5s and the operating energy consumption reduced by 19.71%.

② Linear changes: Although the slope increases with the increase in passenger capacity, the increase is not large. The main change is that the length of the acceleration slope has been extended, which can effectively increase the cruising time and thus achieve the purpose of reducing energy consumption.

This embodiment builds a deep reinforcement learning model for subway longitudinal section line design, which senses, searches, judges, and makes decisions on the line selection environment without manual experience, and finds the optimal energy-saving line plan by feedback on different constraints. Compared with the actual longitudinal section line design plan, this embodiment can reduce the energy consumption cost and time cost of train operation at the same time.

The above is a schematic description of the present invention and its implementation methods, which is not restrictive. The drawings show only one implementation method of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and does not deviate from the purpose of the invention, he or she can design a structure and an embodiment similar to the technical solution without creativity, which should fall within the protection scope of the present invention.

Claims (9)

1. A method for optimizing energy-saving design of subway longitudinal sections based on deep reinforcement learning, characterized in that it includes the following steps: 1. Combining the constraints of subway line design specifications with the constraints of actual construction conditions, a subway longitudinal section line design model with the goal of minimizing train operation energy consumption is established; 2. Use deep reinforcement learning algorithm to solve the optimal energy-saving route of the subway longitudinal section under different operating energy consumption calculation factors. 2. According to the method for optimizing energy-saving design of subway longitudinal section line based on deep reinforcement learning in claim 1, it is characterized in that: the subway longitudinal section line design model includes: 1) Environment: The avoidance area and the transition curve section of the horizontal line constitute the environment for the optimization of the subway longitudinal section line; 2) Agent: It is defined as an intelligent program that determines the direction of energy-saving design of longitudinal sections; 3) State: The spatial position of the slope change point is defined as the state , spatial position refers to the two-dimensional coordinates of the subway longitudinal section, including the longitudinal mileage coordinates and the vertical depth coordinates; 4) Action: The search direction of the next slope change point selected by the agent is defined as the action ; 5) Reward: The value of the reward depends on the feedback from the environment. The energy consumption of the train is the main component of the reward, and the other components are the survival status reward and the target distance reward; 6) Condition: If the agent cannot find a state that satisfies arrive All constraints actions , that is, the current state change condition Not established condition , and the current state The initialization state is ; Otherwise, the state change condition Establishment condition , continue to select the next action. 3. The method for optimizing energy-saving design of subway longitudinal sections based on deep reinforcement learning according to claim 2 is characterized in that: the subway line design specification constraints and actual construction condition constraints include: (1) Constraints on slope length and slope gradient in the station area Only one ramp is set up in the station area, that is, the sum of the lengths of the entrance and exit ramps is greater than the platform length. , assuming that the lengths of the entrance and exit slopes are the same, then the slope length of the station area is The constraints are expressed as: ; Station slope Use the given constant ,Right now: ; (2) Slope length and slope constraints in non-station areas The slope length constraint of the non-station area is expressed as: ; The line has a minimum slope and maximum slope ,Right now: ; (3) Minimum straight line length The length of the straight line between two adjacent vertical curves It should meet the design specification requirements, and the calculation formula is as follows: ; In the formula, Indicates The longitudinal mileage of each slope change point; Indicates The length of the tangent line at the slope change point; Indicates the minimum straight line length in the design specification; (4) Reverse limit slope When two slope sections in opposite directions are connected, the slope in one direction is Should not be greater than the reverse limit slope ,Right now: ; (5) Line burial depth constraints The line depth constraint is that any point on the line Design elevation of the track top of the underground tunnel , should be less than the ground elevation of the point Subtract tunnel height and minimum cover thickness ,Right now: ; (6) Avoidance area constraints All avoidance areas of subway longitudinal sections Indicates that the subway longitudinal section line tunnel area is used express, and The intersection of should be an empty set: ; (7) Plane transition curve segment constraints Slope change point mileage coordinates The coordinates of the start and end points of the plane transition curve The distance should not be less than the tangent length of the vertical curve, that is: . 4. The method for optimizing energy-saving design of subway longitudinal sections based on deep reinforcement learning according to claim 3 is characterized in that: in the state State, The state space at the end of each action is defined as , as shown below: ; in, For the The longitudinal mileage coordinates of the slope change points, For the The vertical elevation coordinates of the slope change points, W and H are the upper limits of the mileage and depth of the target optimization area respectively; in the above formula, the overall position of two consecutive slope change points is taken as a state in the longitudinal section line design optimization model. 5. The method for optimizing energy-saving design of subway longitudinal sections based on deep reinforcement learning according to claim 4 is characterized in that: in the action Action, the action The status Convert to status ,action It is expressed in two parts as follows: ; in and is the length and slope of the slope; The relationship between the three is: . 6. The method for optimizing energy-saving design of subway longitudinal sections based on deep reinforcement learning according to claim 5 is characterized in that: in the reward Reward, the reward The formula is as follows: ; in They represent the unit operating energy cost, survival cost, and distance cost to the destination respectively; for The weight of . 7. The method for optimizing energy-saving design of subway longitudinal section lines based on deep reinforcement learning according to claim 6 is characterized in that: in the subway longitudinal section line design model, the train motion equation is solved by approximate integration method to solve the train running energy consumption, and the upper and lower limits of the integral in the running time and distance formula are divided into several small speed intervals. , let the initial and final velocities in the integral be , then the formula for calculating the train running time and running distance using the approximate integral method is: Run time ; Unit Force Calculation Under different working conditions, it can be written in the following form: ; In the formula, For traction; The mass of the motor vehicle; For the trailer quality; is the unit braking force of the train; is the common braking coefficient; is the unit traction force, Unit resistance of the train, including basic resistance of train operation , ramp additional resistance , curve additional resistance , additional tunnel resistance . 8. According to the deep reinforcement learning-based optimization method for energy-saving design of subway longitudinal sections of claim 7, it is characterized in that the operating energy consumption calculation factors include reverse slope limit, speed limit and passenger capacity. 9. The method for optimizing energy-saving design of subway longitudinal sections based on deep reinforcement learning according to claim 8 is characterized in that the deep reinforcement learning algorithm is an improved D3QN algorithm, which is improved as follows: a. Use the experience replay mechanism to store the interactive experiences one by one in the experience pool. After accumulating a certain amount, the model randomly extracts a certain batch of data from the experience pool to train the neural network at each step; b. Construct two neural networks with the same structure, namely the valuation network and target value network , the valuation network is used for a given state , calculate the action to take The expected cumulative reward of , the estimated network parameters Continuously updated; the target network is used to calculate the timing difference target value , target value network parameter Fixed, replaced with the latest estimated network parameters at regular intervals ; Target value The calculation formula is as follows: ; In the formula, For immediate reward; Table is the discount factor; Indicates that in the next state In Maximizing Action ; Remaining the same over a period of time leads to a valuation network Convergence Target Relatively fixed; c. Improve the neural network structure and divide its output into two parts, one of which is the state value function that characterizes the quality of each state. , and the other is the advantage function that distinguishes the good and bad actions in a specific state : ; In the formula, For strategy, is the parameter of the advantage function; d. The target value of the D3QN model is as follows: ; e. Mean square error As the loss function, the network parameters are updated as follows: .