EP4082868A1 - Method for optimizing rail traffic of a rail traffic network - Google Patents

Abstract

The invention relates to a method (100) for optimizing rail traffic (317) of a rail traffic network with a plurality of rail vehicles (315), the method (100) comprising: - receiving (101) global status data (GS) of a rail traffic network; - determining (103) a set of local status data (S1, S2, ..., SN) of a plurality of rail vehicles (315) of the rail traffic (317) of the rail traffic network based on the global status data (GS);- determining (105) an optimized set of local control actions (A1, A2, ..., AN) for the plurality of rail vehicles (315) based on the set of local state data (S1, S2, ..., SN) of the plurality of rail vehicles (315) and taking into account at least one optimization goal;- generating (107) global control data (GA) based on the optimized set of local control actions (A1, A2, ..., AN), with a control of the plurality of rail vehicles en (315) of the rail traffic (317) according to the control actions (A1, A2, ..., AN) of the global control data (GA) causes the at least one optimization goal to be met; and providing (109) the global control data (GA).

Description

The invention relates to a method for optimizing rail traffic in a rail traffic network with a plurality of rail vehicles.

For the efficient operation of rail transport networks, preventing train delays and adhering to timetables is not only important with regard to a customer satisfaction unit. Adherence to correspondingly optimized timetables is also of interest with regard to the utilization of the capacities of the rail transport network. In particular, timetables optimized for energy consumption are important not only with regard to economic aspects but also with regard to energy policy objectives for the operation of corresponding rail transport networks.

The object of the invention is to provide an improved method for optimizing rail traffic in a rail traffic network with a plurality of rail vehicles.

This object is achieved by a method for optimizing rail traffic in a rail traffic network with a plurality of rail vehicles according to the independent claim. Advantageous configurations are specified in the dependent claims.

• Empfangen von globalen Zustandsdaten eines Schienenverkehrs, wobei die globalen Zustandsdaten einen globalen Zustand des Schienenverkehrs einer Mehrzahl von Schienenfahrzeugen des Schienenverkehrsnetzes beschreiben;
• Ermitteln eines Satzes von lokalen Zustandsdaten einer Mehrzahl von Schienenfahrzeugen des Schienenverkehrs des Schienenverkehrsnetzes basierend auf den globalen Zustandsdaten, wobei lokale Zustandsdaten eines Schienenfahrzeugs einen Zustand des Schienenfahrzeugs innerhalb des Schienenverkehrs beschreiben;
• Ermitteln eines optimierten Satzes von lokalen Steuerungsaktionen für die Mehrzahl von Schienenfahrzeugen basierend auf dem Satz von lokalen Zustandsdaten der Mehrzahl von Schienenfahrzeugen und unter Berücksichtigung wenigstens eines Optimierungsziels, wobei lokale Steuerungsaktionen eines Schienenfahrzeugs eine individuelle Ansteuerung des jeweiligen Schienenfahrzeugs beschreiben;
• Generieren von globalen Steuerungsdaten basierend auf dem optimierten Satz von lokalen Steuerungsaktionen, wobei die globalen Steuerungsdaten die lokalen Steuerungsaktionen umfassen, und wobei eine Ansteuerung der Mehrzahl von Schienenfahrzeugen des Schienenverkehrs gemäß den Steuerungsaktionen der globalen Steuerungsdaten eine Erfüllung des wenigstens einen Optimierungsziels bewirkt; und
• Bereitstellen der globalen Steuerungsdaten.

According to one aspect of the invention, a method for optimizing rail traffic in a rail traffic network with a plurality of rail vehicles is provided, the method comprising:

• Receiving global status data of a rail traffic, the global status data a global status describe the rail traffic of a plurality of rail vehicles of the rail traffic network;
• determining a set of local status data of a plurality of rail vehicles of the rail traffic of the rail traffic network based on the global status data, local status data of a rail vehicle describing a status of the rail vehicle within the rail traffic;
• Determining an optimized set of local control actions for the plurality of rail vehicles based on the set of local status data of the plurality of rail vehicles and taking into account at least one optimization goal, local control actions of a rail vehicle describing an individual activation of the respective rail vehicle;
• Generating global control data based on the optimized set of local control actions, wherein the global control data includes the local control actions, and wherein a control of the plurality of rail vehicles of the rail traffic according to the control actions of the global control data causes the at least one optimization goal to be met; and
• Providing the global control data.

As a result, the technical advantage can be achieved that an improved method for optimizing rail traffic in a rail traffic network with a plurality of rail vehicles can be provided. For this purpose, local status data of the plurality of rail vehicles are separated from global status data of a rail traffic to be optimized, in which a status of the respective rail traffic including the plurality of rail vehicles is described at a predetermined point in time, each describing the states of the individual rail vehicles individually. Based on this, local control actions are generated for the plurality of local status data, which are each used to control the individual rail vehicles and bring about a change in the status of the rail traffic. The local control actions are generated taking into account an optimization goal of rail traffic. Based on the local control actions, global control data are then generated, which include the local control actions and are set up to achieve the respective optimization goal of the rail traffic when controlling the plurality of rail vehicles of the rail traffic according to the control actions.

By extracting the local status data from the global status data of the rail traffic, the amount of data to be processed in the form of the global status data to generate the local control actions can be reduced to the status data of the individual rail vehicles that is only required for generating the local control actions. As a result, the method according to the invention, in particular the generation of the local control actions required for controlling the rail traffic or the individual rail vehicles, can be accelerated in terms of time. As a result, the method according to the invention can be carried out during the operation of the rail vehicles, so to speak in online operation of the rail traffic. In addition, by separating the local status data from the global status data, appropriate control actions can be individually determined for each individual rail vehicle based on the respective local status data of the rail vehicle. As a result, these can be precisely adapted to the achievement of the respective optimization goal, in order to be able to achieve precise optimization of rail traffic.

Within the meaning of the application, rail transport is described by a total of a plurality of rail vehicles operated within a rail transport network. In rail traffic, information regarding control statuses of the rail vehicles is also taken into account, in particular with regard to a regular timetable. Furthermore, in rail transport Information regarding external events, such as passenger numbers, is taken into account.

In this case, states of rail traffic are characterized at least by control states of the plurality of rail vehicles and can differ with regard to delay values with regard to a predefined target timetable of the plurality of rail vehicles.

According to one embodiment, the optimized set of control actions is determined by an action selection rule, wherein the action selection rule is set up to determine at least one corresponding control action for each of the plurality of rail vehicles based on the respective local status data of the rail vehicle, which is suitable for achieving the optimization goal of the to fulfill rail transport.

As a result, the technical advantage can be achieved that an efficient determination of individual control actions for each of the rail vehicles based on the respective local status data is made possible. Since the local status data of the various rail vehicles have an identical data structure, corresponding control actions can be generated for the plurality of different rail vehicles with just one action selection rule that is set up to be applied to the status data of the corresponding data structure. By applying the action selection rule individually to the local status data of the individual rail vehicles, the action selection rule can be applied to different traffic scenarios in which different numbers of rail vehicles are operated in rail traffic, and for which the data structure of the respective global status data of the respective rail traffic thus varies. This enables the method according to the invention to be widely used for different traffic situations.

In terms of registration, an action selection rule corresponds to a policy known from the field of reinforcement learning.

According to one embodiment, the action selection rule is trained through reinforcement learning.

As a result, the technical advantage can be achieved that an efficient method for optimizing rail traffic can be provided. By using reinforcement learning to train the action selection rule, based on the local status data of the various rail vehicles, taking into account the predefined optimization goals, the action selection rule can be automatically trained for a large number of different traffic situations and different optimization goals to generate corresponding control actions that meet the optimization goals. Efficient training of the action selection rule and an efficiently applicable action selection rule can thus be achieved, which can be trained in an offline state of the rail traffic and can be applied to a large number of different traffic situations after the training is complete.

According to one embodiment, the action selection rule is trained on the basis of simulation data, the simulation data being based on a simulation of rail traffic for a plurality of rail vehicles in the rail traffic network.

In this way, the technical advantage can be achieved that simple and efficient training of the action selection rule is made possible. In addition, numerous different traffic situations can be simulated via the simulation, so that an action selection rule that has been trained as efficiently and comprehensively as possible can be provided, which enables efficient optimization of rail traffic for any traffic situation. In addition, the training does not rely on real status data, so that the time-consuming generation of real status data, for example by recording route data through test drives of the rail vehicles or logging traffic data from real traffic situations, can be avoided.

According to one embodiment, the training of the action selection rule includes maximizing a reward function, the reward function defining the at least one optimization goal.

In this way, the technical advantage can be achieved that efficient training of the action selection rule is made possible.

According to one embodiment, the action selection rule is in the form of a neural network.

In this way, the technical advantage can be achieved that an efficient action selection rule can be provided, which is set up to generate local control actions corresponding to the optimization goal within a predetermined time period based on the local status data of the individual rail vehicles. In this way it can be ensured that the method according to the invention can be executed in an online operation of the rail traffic and can provide appropriate control actions for controlling and optimizing the rail traffic deterministically within a predetermined period of time.

According to one embodiment, generating global control data includes:
- Adjust local control actions of different rail vehicles and/or resolve conflicts between control actions of different rail vehicles.

In this way, the technical advantage can be achieved that despite the calculation of the local control actions individually collisions of the various control actions of the individual rail vehicles can be avoided for each of the plurality of rail vehicles. This can ensure safe operation of rail traffic.

According to one embodiment, the local control actions of the rail vehicles include obtaining an arrival time and/or a standstill time and/or a departure time of the respective rail vehicle at at least one stop on a rail route of the rail traffic network traveled by the rail vehicle.

This can achieve the technical advantage that precise optimization of the rail traffic can be provided. By taking into account arrival times, departure times and/or standing times of the individual rail vehicles at various stops in the rail network through the local or global control actions, the rail vehicles can be controlled with a view to optimizing rail traffic in relation to a predefined rail traffic timetable. By taking into account the arrival times, departure times or standing times of the rail vehicles at the respective stops in the respective control actions, the control actions can be designed to optimize or adapt the corresponding arrival times, departure times or standing times in such a way that rail traffic is operated at a optimized timetable or can be optimized with regard to this. The arrival times, departure times and/or waiting times can be determined according to the optimization goal and in particular can deviate from the arrival times, departure times and/or waiting times that are defined in the predefined timetable.

According to one embodiment, the local status data of a rail vehicle includes a position and/or a speed and/or a delay relative to a predefined timetable of the rail vehicle in the rail traffic network and/or a position and/or a speed and/or a delay of rail vehicles traveling ahead and/or following relative to the rail vehicle on a rail route traveled by the rail vehicle and/or an overall delay of the plurality of rail vehicles.

As a result, the technical advantage can be achieved that a precise determination of the states of the individual rail vehicles is made possible based on the local state data. By taking into account other rail vehicles located on a rail route for assessing a state of a rail vehicle, these can also be taken into account when generating the corresponding control actions. As a result, the local control actions of individual rail vehicles can be precisely adapted to the actual state of the rail vehicle within the rail traffic, which also takes other rail vehicles and delays of the individual vehicles into account. This enables precise optimization of rail traffic, which can reduce or eliminate delays in individual rail vehicles or in a plurality of rail vehicles.

According to one embodiment, the at least one optimization goal includes a reduction in an overall delay of the plurality of rail vehicles relative to a predetermined timetable of the rail transport network and/or a variance in delays of individual rail vehicles relative to the predetermined timetable and/or a minimum period of time until the predetermined timetable is restored and /or minimum energy consumption and/or minimum energy consumption peaks of the plurality of rail vehicles.

In this way, the technical advantage can be achieved that rail traffic is based on various relevant optimization goals, such as, for example, an energy consumption of all rail vehicles or an overall delay of all of the rail vehicles or delays of individual rail vehicles can be optimized.

According to one embodiment, the method is carried out in online operation of the plurality of rail vehicles in the rail traffic of the rail traffic network.

As a result, the technical advantage can be achieved that an optimization of the online operation of the rail traffic is made possible.

• Ausführen einer Simulation eines Schienenverkehrs einer Mehrzahl von Schienenfahrzeugen eines Schienenverkehrsnetzes;
• Empfangen von globalen Zustandsdaten des Schienenverkehrsnetzes der Simulation;
• Ermitteln eines Satzes von lokalen Zustandsdaten der Mehrzahl von Schienenfahrzeugen des Schienenverkehrs basierend auf den globalen Zustandsdaten;
• Ermitteln eines optimierten Satzes von lokalen Steuerungsaktionen für die Mehrzahl von Schienenfahrzeugen basierend auf dem Satz von lokalen Zustandsdaten der Mehrzahl von Schienenfahrzeugen und unter Berücksichtigung wenigstens eines Optimierungsziels durch Ausführen der Aktionsauswahlregel auf die lokalen Zustandsdaten;
• Generieren von globalen Steuerungsdaten basierend auf dem optimierten Satz von lokalen Steuerungsaktionen;
• Bereitstellen der globalen Steuerungsdaten an die Simulation des Schienenverkehrs;
• Ausführen der Steuerungsaktionen der globalen Steuerungsdaten durch die Simulation und Überführen des Schienenverkehrs in einen zweiten globalen Zustand;
• Berechnen eines Werts einer Belohnungsfunktion für den zweiten globalen Zustand des Schienenverkehrs in Bezug auf das wenigstens eine Optimierungsziel unter Berücksichtigung von Techniken des bestärkenden Lernens;
• Modifizieren von Parametern der Aktionsauswahlregel gemäß dem Wert der Belohnungsfunktion unter Berücksichtigung von Techniken des bestärkenden Lernens; und
• Iteratives Ausführen der voranstehenden Verfahrensschritte und Maximieren der Belohnungsfunktion.

According to a second aspect of the invention, there is provided a method for training an action selection rule, the method comprising:

• performing a simulation of rail traffic of a plurality of rail vehicles of a rail traffic network;
• receiving global status data of the railway network of the simulation;
• determining a set of local status data of the plurality of rail vehicles of the rail traffic based on the global status data;
• determining an optimized set of local control actions for the plurality of rail vehicles based on the set of local status data of the plurality of rail vehicles and taking into account at least one optimization goal by executing the action selection rule on the local status data;
• generating global control data based on the optimized set of local control actions;
• providing the global control data to the rail traffic simulation;
• performing the control actions of the global control data through the simulation and transitioning the rail traffic to a second global state;
• calculating a value of a reward function for the second global rail traffic state in relation to the at least one optimization goal considering reinforcement learning techniques;
• modifying parameters of the action selection rule according to the value of the reward function considering reinforcement learning techniques; and
• Iteratively executing the above method steps and maximizing the reward function.

In this way, the technical advantage can be achieved that an efficient method for training an action selection rule can be provided. For this purpose, the action selection rule is trained by reinforcement learning based on simulation data of a simulation of rail traffic. For this purpose, local status data of the individual rail vehicles are first generated from global status data of the rail traffic and corresponding local control actions are generated by applying the action selection rule to the local status data, taking into account the optimization goal to be achieved to transfer state. Efficient training of the action selection rule for optimizing rail traffic can be achieved by considering reward values of a reward function and modifying parameters of the action selection rule for maximizing the respective reward function after multiple transitions of the rail traffic to changed states.

According to one embodiment, the method for training an action selection rule is carried out in an offline operation of the railway network.

This can achieve the technical advantage that efficient training of the action selection rule is made possible by being carried out in an offline state of the rail traffic, or the action selection rule for optimizing the rail traffic is only actually used when the offline training based on the simulation corresponding traffic situations is completed.

According to a third aspect, a system for optimizing rail traffic is provided with a computing unit that is set up to execute the method according to the invention for optimizing rail traffic in a rail traffic network with a plurality of rail vehicles according to one of the preceding embodiments, and/or the method according to the invention for training a Execute action selection rule according to one of the preceding embodiments.

According to a fourth aspect of the invention, a computer program product is provided comprising instructions which, when the program is executed by a data processing unit, cause the latter to execute the method according to the invention for optimizing rail traffic in a rail traffic network with a plurality of rail vehicles according to one of the preceding embodiments, and/or that carry out the inventive method for training an action selection rule according to one of the preceding embodiments.

FIG 1

eine schematische Darstellung eines Systems zum Optimieren eines Schienenverkehrs gemäß einer Ausführungsform;

FIG 2

eine weitere schematische Abbildung des Systems zum Optimieren eines Schienenverkehrs gemäß einer Ausführungsform;

FIG 3

ein Flussdiagramm des Verfahrens zum Optimieren eines Schienenverkehrs gemäß einer Ausführungsform;

FIG 4

eine schematische Darstellung des Systems in Fig. 1 in einem Simulationsmodus;

FIG 5

eine weitere schematische Abbildung des Systems zum Optimieren eines Schienenverkehrs gemäß einer Ausführungsform;

FIG 6

ein Flussdiagramm eines Verfahrens zum Trainieren einer Aktionsauswahlregel gemäß einer Ausführungsform; und

FIG 7

eine schematische Darstellung eines Computerprogrammprodukts.

The above-described features and advantages of this invention, and the manner in which they are attained, will be more clearly and fully understood through the discussion of the following highly simplified schematic representations of preferred embodiments. Each show:

FIG 1

a schematic representation of a system for optimizing a rail traffic according to an embodiment;

FIG 2

a further schematic illustration of the system for optimizing a rail traffic according to an embodiment;

FIG 3

FIG. 12 is a flow chart of the method for optimizing rail traffic according to an embodiment; FIG.

FIG 4

a schematic representation of the system in 1 in a simulation mode;

FIG 5

a further schematic illustration of the system for optimizing a rail traffic according to an embodiment;

6

a flow chart of a method for training an action selection rule according to an embodiment; and

FIG 7

a schematic representation of a computer program product.

FIG 1 FIG. 3 shows a schematic representation of a system 300 for optimizing rail traffic according to an embodiment.

According to the embodiment shown, a system 300 for optimizing rail traffic includes a plurality of modules that can be executed on a computing unit 301 . The system 300 can also be divided into both an offline subsystem 302 and an online subsystem 304, with the offline subsystem 302 offline, i.e. independent of the operation of the rail vehicles 203, while the online subsystem 304 is running during operation of the rail vehicles 203 is executed.

The central component of the offline subsystem 302 is a timetable optimization module 311. The timetable optimization module 311 is used to create an optimized timetable for rail traffic for a plurality of rail vehicles in a rail traffic network for a predetermined period of time. The optimized schedule created in the offline subsystem 302 the timetable optimization module 311 can transmit to the online subsystem 304 via a first interface, so that the timetable for controlling the rail traffic can be executed.

In the actual operation of rail transport, the timetable, henceforth called an online timetable, is managed by a timetable management module 305 . For this purpose, position data of individual rail vehicles in rail traffic can be transmitted via an automatic rail vehicle tracking module 307 via a second interface S2 to the timetable management module 305, so that the latter can carry out a comparison between target movements in the timetable and actual movements actually carried out by the rail vehicles in rail traffic.

The timetable management module 305 can also transmit commands to an automatic route selection module 309 via a third interface S3 for the selection of rail routes to be traveled on. Appropriate rail routes can thus be reserved in good time in order to ensure that the rail vehicles can be operated with minimal delays.

The timetable management module 305 also has a bidirectional interface S2, S4 with an automatic rail vehicle control module 303. In the event that the online timetable needs to be changed, either by a new target timetable from the timetable optimization module 311 or by changes made by a dispatcher, the automatic rail vehicle control module 303 can be informed accordingly in order to effect a change in the online timetable. For this purpose, the automatic rail vehicle control module 303 also requires current position data of the rail vehicles in order to be able to determine current delays in the rail vehicles and to be able to initiate suitable controls in the event of larger deviations.

These regulations can include corresponding control actions to be carried out by the rail vehicles, with which desired departure times and/or arrival times and stopping times of the rail vehicles at stops on the rail routes of the rail transport network are to be achieved or observed by the rail vehicles in order to cancel the delay and to change the timetable. The corresponding regulations and/or control actions can be transmitted to the schedule management module 305 via a fifth interface S5.

Finally, to adapt the timetable, the corresponding regulations and control actions and the corresponding optimized arrival times/departure times/stop times are transmitted from the automatic timetable management module 305 via a sixth interface S6 to automatic rail vehicle control modules 313 of the individual rail vehicles 315 of the rail traffic, so that they can carry out the desired changes or control actions to achieve or maintain the optimized arrival/departure/stop times.

In the case of automatically driving rail vehicles, the automatic rail vehicle control module 313 can determine energy-optimized travel trajectories of the rail vehicle 203 that best match the current online timetable. In the case of manual driving, the driver is advised, but is free to implement the regulations.

In the embodiment shown, the automatic rail vehicle control module 303 is set up to execute the inventive method for optimizing rail traffic of a plurality of rail vehicles of a rail traffic network based on the status data of a rail traffic to be optimized of a plurality of rail vehicles.

FIG 2 FIG. 3 shows another schematic diagram of the system 300 for optimizing a rail traffic according to an embodiment.

figure 2 shows a plurality of different components of the system 300, which can be integrated, for example, in the automatic rail vehicle control module 303 or can be executed by this. Due to the components shown, the system 300 is able to carry out the method according to the invention for optimizing rail traffic.

For this purpose, global status data GS are first received from rail traffic 317 . Rail traffic 317 here describes a plurality of rail vehicles operated within a rail traffic network, taking into account the individual properties of the respective rail traffic network, the individual properties of the rail vehicles, and also the optimized target timetable for operating the majority of rail vehicles.

The global status data GS describe a status of the rail traffic 317, which is described for example by a traffic situation of a plurality of different rail vehicles 315 at a predetermined point in time. In addition to detailed descriptions of the status of the individual rail vehicles 315 involved in the respective traffic situation of the rail traffic 317, the global status data GS can contain further information, for example with regard to the target timetable, with regard to various statuses of individual rail routes in the rail transport network and further relevant information for controlling the plurality of rail vehicles 315 or for Description of the respective traffic situation or the state of rail traffic 317.

For different traffic situations in which, for example, a different number of rail vehicles are involved, for example due to the time of day or year, depending on which according to the optimized target timetable A different number of rail vehicles are required to transport passengers, for example, the data structure of the global status data GS can vary.

To compensate for the varying data structure of the global status data GS, in which, for example, at times of day a large number of rail vehicles that are operated in the rail transport network at high frequency are taken into account via corresponding status data, while at times of the day with only a low density of operated rail vehicles, in the global status data GS corresponding to a small number of rail vehicles with corresponding status descriptions are taken into account, local status data S1, S2, ..., SN of the individual rail vehicles are generated by a generation module for local status data LSB from the global status data GS. The local status data S1, S2, ..., SN individually describe the status of the individual rail vehicles of the rail traffic 317.

The data structure of the individual local status data S1, S2, ..., SN can be designed uniformly for different rail vehicles, so that in the individual local status data S1, S2, ..., SN of the different rail vehicles, the respective states of the individual rail vehicles Considering the same parameters are described. The local status data S1, S2, include vehicle-specific information.

In addition, the local status data of a rail vehicle can contain additional information regarding other rail vehicles arranged on the rail route traveled by the rail vehicle, including their positions and speeds or delays. In addition, the local status data S1, S2, ..., SN can include an overall delay of the plurality of rail vehicles relative to the predetermined optimized timetable.

In a following step, an action selection rule P is applied to the majority of the generated local status data S1, S2, ..., SN, which is set up individually for each rail vehicle based on the respective local status data S1, S2, ..., SN of the generate corresponding local control actions A1, A2, ..., AN of the respective rail vehicle for the respective rail vehicle, taking into account at least one optimization goal. The local control actions A1, A2, is reachable.

The local control actions A1, A2, . By executing the local control actions A1, A2, . The rail vehicle is consequently controlled in such a way that it can comply with the correspondingly defined arrival times and/or departure times and/or stopping times.

The optimization goal of optimizing the rail traffic to be achieved in this way can include, for example, a reduction in an overall delay of the plurality of rail vehicles involved in the respective traffic situation relative to the predetermined optimized target timetable. Alternatively or additionally, the optimization goal can include a variance of a plurality of individual delays of individual rail vehicles of the plurality of rail vehicles of the rail traffic relative to the optimized target timetable. As an alternative or in addition to this, the optimization goal can also include an energy requirement of the plurality of rail vehicles, for example a total energy consumption and/or time-limited maximum values of the energy consumption of the plurality of rail vehicles. As an alternative to this, several different optimization goals for optimizing the rail traffic can also be taken into account.

In a further step, global control data GA is subsequently generated by a generation module for global control data GAB based on the plurality of local control actions A1, A2, . . . , AN of the plurality of rail vehicles. The global control data GA here include the plurality of local control actions A1, A2, ..., AN and are set up by the execution of the local control actions A1, A2, ..., AN contained in the global control data GA by the plurality of rail vehicles To optimize traffic of the majority of rail vehicles in relation to the respective optimization goal.

The generation module for global control data GAB can also be set up to avoid collisions of different local control actions A1, A2, ..., AN of different rail vehicles when generating the global control data GA based on the plurality of local control actions A1, A2, ..., AN and to carry out a corresponding adaptation of the control actions A1, A2, ..., AN. In this way, the actuation of the plurality of rail vehicles can be coordinated based on the individually generated local control actions A1, A2, . . . , AN. In particular, it can be avoided that, for example, different rail vehicles are at the same stopping point at identical times.

In order to optimize the rail traffic, individual, local status data S1, S2, . . . Taking into account various optimization goals, the action selection rule P then generates corresponding local control actions A1, A2, ..., AN individually for each rail vehicle based on the respective local status data S1, S2, . The local control actions A1, A2, . or define departure time and/or waiting time for a rail vehicle and at least one stop to be controlled by the rail vehicle.

The individual local control actions A1, A2, different optimization goals of all rail vehicles can be achieved. For example, the different arrival times, departure times or idle times of the different rail vehicles at the various stops can be adjusted in such a way that the total delay of the majority of rail vehicles is reduced relative to the optimized target timetable. Alternatively or additionally, the delays of the individual rail vehicles can be defined by the correspondingly defined arrival times, departure times or idle times in such a way that a more or less uniform deviation of the individual rail vehicles from the target timetable is achieved. Alternatively or additionally, the arrival times, departure times or downtimes of the individual rail vehicles are adjusted so that the energy requirement of all the rail vehicles is reduced, for example by synchronizing the acceleration and braking processes of individual rail vehicles with one another.

The action selection rule P can be trained according to reinforcement learning techniques, which include, for example, maximizing a reward function in which the respective optimization goals are defined. The action selection rule can be applied to any local status data S1, S2, . . . , SN of the individual rail vehicles and is set up to generate corresponding local control actions individually for each rail vehicle. The action selection rule P can, for example, be in the form of an appropriately trained neural network. To apply the action selection rule to the various local status data S1, S2, ..., SN, the local status data S1, S2, ..., SN have identical data structures, so that the local statuses of the individual rail vehicles have a uniform number of different features within of the local status data are described.

The action selection rule P in combination with the generation module for local status data LSB and the generation module for global control data GAB can be set up within a predetermined time interval, for example in the range of seconds of usually 5 to 10 seconds, based on the global status data GS corresponding global control data GA to generate. Thus, an optimization of the rail traffic can be carried out during the online operation of the rail vehicles and, for example, the method according to the invention for optimizing rail traffic as described above can be carried out in a predetermined cycle according to the predetermined time period, so that the respective rail vehicles are provided with corresponding control actions or control data every second can be based on which the Rail vehicles can be controlled in order to optimize rail traffic according to the corresponding optimization goals and, if necessary, adapt it to the predefined optimized target timetable.

3 10 shows a flow diagram of a method 100 for optimizing rail traffic according to an embodiment.

To execute method 100 according to the invention for optimizing rail traffic 317 of a rail traffic network with a plurality of rail vehicles 315, global status data GS of rail traffic 317 is first received in a method step 101, with the global status data describing a global status of rail traffic 317 including the plurality of rail vehicles 315 .

In a method step 103, based on the global status data GS, a set of local status data S1, S2, 315 describe.

In a method step 105, based on the local status data S1, S2, . . . , SN, an optimized set of local control actions A1, A2, . The local control actions A1, A2, ..., AN describe here individual controls of the individual rail vehicles 315. The determination of the optimized set of control actions A1, A2, ..., AN can be performed, for example, by an action selection rule that is set up for to generate corresponding local control actions for each rail vehicle individually based on the local status data S1, S2, ..., SN. The action selection rule P can be trained using reinforcement learning techniques, for example and be designed as an appropriately trained neural network.

In a method step 107, based on the optimized set of local control actions A1, A2, . . . , AN, global control data GA are generated that include the local control actions A1, A2, of rail vehicles 315 according to the control actions A1, A2, ..., AN to meet the optimization goal.

In the embodiment shown, in a method step 111, the local control actions A1, A2, . . . AN of different rail vehicles are adapted and conflicts between control actions A1, A2, .

In a method step 109, the global control data GA are provided, so that the rail vehicles of the rail traffic can be controlled according to the correspondingly generated local control actions A1, A2, . . . , AN.

FIG 4 shows a schematic representation of the system 300 in FIG 1 in a simulation mode.

This in 4 The system 300 shown is based on the system in 1 and has all the components and functionality described there. A renewed detailed description is therefore dispensed with.

Deviating from 1 becomes the system 300 in 4 however, not operated to control rail traffic. Instead, the system serves in 4 for training an action selection rule P according to the embodiments described above and with the properties described there.

For this purpose, in 4 embodiment shown, a simulation SIM is performed that simulates all modules of the system 300, with the exception of the automatic rail vehicle control module 303, in a corresponding simulation. The simulation SIM also simulates rail traffic with a plurality of rail vehicles of a rail traffic network according to the embodiments described above. For this purpose, the simulation SIM is set up to simulate and display different traffic scenarios for different rail traffic networks with rail vehicles of different numbers and types. The simulated traffic scenarios can be changed, so that realistic simulations of traffic situations that actually occur are made possible, which can be changed over a predetermined period of time and thus represent a time course of rail traffic that can be changed over time.

Furthermore, the simulation is set up to represent the functions of the individual modules described above, with the exception of the automatic rail vehicle control module 303, which react to the respective traffic situations by executing corresponding functions. The simulation SIM can be implemented using simulation software for simulating rail traffic, for example using the simulation software Falko.

In contrast, the automatic rail vehicle control module 303 is not simulated by the simulation SIM, but is set up to operate the method according to the invention for training an action selection rule and to train the action selection rule based on the data of the simulation SIM.

For this purpose, the simulation SIM can provide global status data GS for rail traffic in a specific rail traffic network. In this case, the global status data GS includes, analogous to global status data of an actually monitored rail traffic, position data and/or speed data a plurality of rail vehicles and/or arrival times and/or departure times and/or waiting times of the plurality of rail vehicles at different stops in the rail transport network and/or information on the status of the various rail routes in the rail transport network.

5 FIG. 3 shows another schematic diagram of the system 300 for optimizing a rail traffic according to an embodiment.

figure 5 shows an embodiment of the system 300 in which it is set up to execute a method for training an action selection rule P with the features described above. In figure 5 Various components of the system 300 are shown, which may be executed by the automatic rail vehicle control module 303, for example. The embodiment of the system 300 shown here relates to the embodiment in FIG figure 4 , in which the components of the system 300 described there, with the exception of the automatic rail vehicle control module 303, are simulated by a corresponding simulation SIM.

The components of the system 300, in particular the generation module for local status data LSB, the generation module for global control data GAB and the action selection rule P, correspond to FIG figure 2 described components. A renewed detailed description will therefore be dispensed with in the following.

As already explained above, the action selection rule P is preferably not trained on real status data of a rail traffic to be optimized, but on corresponding simulated status data of a simulation SIM of a corresponding rail traffic. As already explained above, the simulation SIM can be used for this purpose, for example by means of a corresponding simulation program for simulation a rail traffic controller with the features described above.

To carry out the method for training the action selection rule P, first global status data GS of the rail traffic simulated by the simulation SIM is received. The generation module for local status data LSG then generates corresponding local status data for the S1, S2, . . . , SN of the plurality of rail vehicles based on the simulated global status data. Appropriate action selection rules A1, A2, ..., AN are generated by applying the action selection rule P to the individual local status data S1, S2, ..., SN. The generation module for global control data GAB generates corresponding global control data GA based on the local status data A1, A2, . . . AN.

By executing the local control actions A1, A2, . Different statuses of the rail traffic relate to certain status recordings of the rail traffic at different times and describe different traffic situations in which the rail vehicles of the rail traffic are located at different positions within the rail traffic network. Different states of the rail traffic can also differ, among other things, in different delay values of the rail vehicles in relation to the predetermined optimized target timetable.

Based on this, new global status data GS is received and global control data GA is generated again according to what is described above by applying the action selection rule P. By applying or executing the local control actions A1, A2, ..., AN of the newly generated global status data GS by the rail vehicles of the simulated Rail traffic, the simulated rail traffic is again brought into a changed state.

During the repeated execution of the steps described above, a corresponding return of a reward function R is calculated after each transfer of the rail traffic into a changed state by executing the previously generated global state data GS by a reinforcement learning optimization module RLO. Based on the calculated return of the reward function R, parameters of the action selection rule P are varied or adjusted by the reinforcement learning optimization module RLO. Thus, the parameters of the action selection rule P can be adjusted after each run through the generation of new global status data GS and the transfer of the simulated rail traffic to a changed status. Alternatively, an adjustment can take place after a predefined number of executed passes. The parameters of the action selection rule P are adjusted here in accordance with the objective of maximizing the return of the reward function R, in accordance with the principles of reinforcement learning. For this purpose, the reinforcement learning optimization module RLO can be designed, for example, according to an algorithm known from the prior art for reinforcement learning, which is set up to vary the parameters of the action selection rule P in such a way that the return of the reward function R is maximized. The reward function R defines the optimization goals to be achieved, so that by maximizing the return of the reward function R, the parameters of the action selection rule P are varied in such a way that the local control actions A1, A2, ..., AN generated by the action selection rule P are set up by Execution or control of the rail vehicles according to the generated local control actions A1, A2, ..., AN to achieve the fulfillment of the optimization goals and the associated optimization of the rail traffic.

Alternatively, the action selection rule P can also be trained on real status data of actual rail traffic.

6 FIG. 2 shows a flow diagram of a method 200 for training an action selection rule P according to an embodiment.

The method 200 according to the invention for training an action selection rule P is based on the in figure 5 system 300 shown applicable.

In a first method step 201, a SIM simulation of rail traffic 317 involving a plurality of rail vehicles 315 is carried out.

In a method step 203, global status data GS of the rail traffic 317 of the simulation SIM are received.

In a method step 205, a set of local status data S1, S2, . . . , SN of the plurality of rail vehicles is determined based on the global status data GS.

In a method step 207, an optimized set of local control actions A1, A2, ..., AN is determined by executing the action selection rule P on the local status data S1, S2, ..., SN.

In a method step 209, global control data GA are generated based on the local control actions A1, A2, . . . , AN.

In a method step 211, the global control data GA are provided to the rail traffic simulation SIM 317 .

In a method step 213, the control actions A1, A2, . . . , AN of the global control data GA are determined by the simulation SIM executed and the simulated rail traffic 317 transferred to a second global state.

In a method step 215, a value R of a reward function for the second global state of the rail traffic is calculated in relation to the at least one optimization goal, taking reinforcement learning techniques into account.

In a method step 217, parameters of the action selection rule P are modified according to the value R of the reward function, taking reinforcement learning techniques into account, in such a way that the reward function is maximized. In this case, the modification can take place after a completed cycle or after a plurality of completed consecutive cycles.

In a method step 219, the method steps 203 to 217 are carried out iteratively and the reward function is thereby maximized.

FIG 7 shows a schematic representation of a computer program product 400.

figure 7 shows a computer program product 400, comprising instructions which, when the program is executed by a computing unit, cause the latter to execute the method 100 according to one of the above-mentioned embodiments. The computer program product 400 is stored on a storage medium 401 in the embodiment shown. The storage medium 401 can be any storage medium known from the prior art.

Although the invention has been illustrated and described in detail by means of the preferred embodiment, the invention is not limited by the disclosed examples and other variations may occur therefrom by those skilled in the art be derived without departing from the scope of the invention.

Claims (15)

Method (100) for optimizing rail traffic (317) of a rail traffic network with a plurality of rail vehicles (315), the method (100) comprising: - Receiving (101) of global status data (GS) of rail traffic (317), the global status data (GS) describing a global status of rail traffic (317) of a plurality of rail vehicles (315) of the rail traffic network; - Determining (103) a set of local status data (S1, S2, ..., SN) of a plurality of rail vehicles (315) of the rail traffic (317) of the rail traffic network based on the global status data (GS), local status data of a rail vehicle ( 315) describe a state of the rail vehicle (315) within the rail traffic (317); - determining (105) an optimized set of local control actions (A1, A2, ..., AN) for the plurality of rail vehicles (315) based on the set of local state data (S1, S2, ..., SN) of the plurality of rail vehicles (315) and taking into account at least one optimization goal, wherein local control actions (A1, A2, ..., AN) of a rail vehicle (315) describe an individual control of the respective rail vehicle (315); - generating (107) global control data (GA) based on the optimized set of local control actions (A1, A2, ..., AN), the global control data (GA) representing the local control actions (A1, A2, ..., AN) include, and wherein a control of the plurality of rail vehicles (315) of the rail traffic according to the control actions (A1, A2, ..., AN) of the global control data (GA) brings about a fulfillment of the at least one optimization goal; and - Providing (109) the global control data (GA). Method (100) according to claim 1, wherein determining (105) the optimized set of control actions (A1, A2, ..., AN) is performed by an action selection rule (P), and wherein the action selection rule (P) is set up, for each of the plurality of rail vehicles based on the respective local status data (S1, S2, ..., SN) of the rail vehicle (315) at least one corresponding Determine control action (A1, A2, ..., AN) suitable to meet the rail traffic optimization goal (317). The method (100) of claim 2, wherein the action selection rule (P) is trained by reinforcement learning. Method (100) according to claim 3, wherein the training of the action selection rule (P) is carried out based on simulation data, and wherein the simulation data is based on a simulation (SIM) of rail traffic (317) of a plurality of rail vehicles (315) of the rail traffic network. Method (100) according to claim 3 or 4, wherein the training of the action selection rule (P) comprises maximizing a reward function, and wherein the reward function defines the at least one optimization goal. Method (100) according to 3, 4 or 5, wherein the action selection rule (P) is designed as a neural network. Method (100) according to one of the preceding claims, wherein the generation (107) of global control data (GA) comprises:
- Adapting (111) the local control actions (A1, A2, ..., AN) of different rail vehicles and/or resolving conflicts between control actions (A1, A2, ..., AN) of different rail vehicles. Method (100) according to one of the preceding claims, wherein the local control actions (A1, A2, ..., AN) of the rail vehicles (315) achieve an arrival time and/or a standing time and/or a departure time of the respective rail vehicle (315) at least one stop a rail route of the rail transport network traveled by the rail vehicle (315). Method (100) according to one of the preceding claims, wherein the local status data (S1, S2, ..., SN) of a rail vehicle (315) a position and/or a speed and/or a delay relative to a predefined timetable of the rail vehicle ( 315) in the rail traffic network and/or a position and/or a speed and/or a delay of rail vehicles traveling ahead and/or following relative to the rail vehicle on a rail route traveled by the rail vehicle and/or an overall delay of the plurality of rail vehicles. Method (100) according to one of the preceding claims, wherein the optimization goal is a reduction in an overall delay of the plurality of rail vehicles (315) relative to a predetermined timetable of the rail transport network and/or a reduction in a variance of delays of individual rail vehicles relative to the predetermined timetable and/or a reduction in a minimum period of time until the predetermined timetable is restored and/or reduction of a minimum energy consumption and/or reduction of minimum energy consumption peaks of the plurality of rail vehicles (315). Method (100) according to one of the preceding claims, wherein the method (100) is carried out in online operation of the plurality of rail vehicles (315) in the rail traffic (317) of the rail traffic network. Method (200) for training an action selection rule (P), comprising: - Executing (201) a simulation (SIM) of rail traffic (317) of a plurality of rail vehicles (315) of a rail traffic network; - Receiving (203) of global status data (GS) of the rail traffic (317) of the simulation (SIM); - determining (205) a set of local status data (S1, S2, ..., SN) of the plurality of rail vehicles (315) of the rail traffic (317) based on the global status data (GS); - determining (207) an optimized set of local control actions (A1, A2, ..., AN) for the plurality of rail vehicles (315) based on the set of local status data (S1, S2, ..., SN) of the plurality of rail vehicles (315) and taking into account at least one optimization goal by executing the action selection rule (P) on the local status data (S1, S2, ..., SN); - Generating (209) global control data (GA) based on the optimized set of local control actions (A1, A2, ..., AN); - Providing (211) the global control data (GA) to the simulation (SIM) of the rail traffic (317); - executing (213) the control actions (A1, A2, ..., AN) of the global control data (GA) by the simulation (SIM) and transferring the rail traffic (317) to a second global state; - calculating (215) a value (R) of a reward function for the second global rail traffic state (317) in relation to the at least one optimization goal, taking into account reinforcement learning techniques; - modifying (217) parameters of the action selection rule (P) according to the value (R) of the reward function taking into account reinforcement learning techniques; and - Iteratively executing (219) method steps (203) to (217) and maximizing the reward function. Method (200) according to claim 12, wherein the method (200) is carried out in an offline operation of the rail transport network. System (300) for optimizing rail traffic with a computing unit (301) which is set up, the method (100) for optimizing rail traffic (317) of a rail traffic network with a plurality of rail vehicles (315) according to one of the preceding claims 1 to 11, and/or to execute the method (200) for training an action selection rule (P) according to claim 12 or 13. Computer program product (400) comprising instructions which, when the program is executed by a data processing unit, cause the latter to carry out the method (100) for optimizing rail traffic (317) of a rail traffic network with a plurality of rail vehicles (315) according to one of the preceding claims 1 to 11 and/or to execute the method (200) for training an action selection rule (P) according to claim 12 or 13.

Images