KR102346507B1 - Method and apparatus for controlling distributed traffic signal based on reinforcement learning in multi-intersections environment - Google Patents

Abstract

The present invention relates to a state differentiated according to a plurality of traffic directions at an intersection, an action of setting a traffic signal as one of a plurality of signal combinations for a specific state, a vehicle traffic volume at the intersection, and average waiting of vehicles at a plurality of intersections adjacent to the intersection setting a reward determined according to time; performing reinforcement learning using predetermined traffic data based on the state, the action, and the reward; and based on the result of the reinforcement learning and the traffic information of the target intersection corresponding to the intersection, setting traffic signals for a plurality of travel directions of the target intersection. A learning-based distributed traffic signal control method is provided.

Description

Distributed traffic signal control method and apparatus based on reinforcement learning in a multi-intersection environment

The present invention provides a method and apparatus for controlling a traffic signal based on reinforcement learning in a multi-intersection environment.

Traffic congestion occurs in urban areas every day. Traffic congestion is costly because it increases fuel consumption, operating costs, and travel time. Also, this causes environmental pollution. Therefore, traffic congestion is a major problem to be solved in the transportation system. In order to solve the traffic congestion, various studies have been conducted on the traffic management system. Recently, research on efficient, safe and eco-friendly intelligent transportation systems has been conducted.

Smart city is the future trend of recent urban development, and it aims to improve the quality of life efficiently and conveniently with almost everything we use in our lives. A smart traffic management system is one of the important aspects of a smart city to reduce traffic congestion. Among the management systems, traffic signal control is the most effective and important means still in use. The general traffic signal control currently used uses a fixed signal mechanism.

However, in line with the development of smart cities, the smart traffic signal control system must process a large amount of data and adapt to the continuously changing traffic environment. Therefore, many researchers have tried to solve the traffic signal control problem by using an artificial intelligent technique.

However, since the intersections are continuous, they affect each other, so traffic can be controlled more efficiently if the signal sequence is flexibly changed depending on the situation.

Accordingly, the present invention is to provide a method and apparatus for controlling a distributed traffic signal optimized using reinforcement learning in a multi-intersection environment.

The present invention provides a method and apparatus for optimizing distributed traffic signal control using reinforcement learning in a multi-intersection environment.

In addition, the present invention provides an algorithm for efficiently distributing traffic signals by distributing appropriate signals so that as many vehicles as possible pass through the intersection, and adjusting the order of traffic signals in consideration of the average vehicle waiting time of all adjacent intersections.

The present invention relates to a state differentiated according to a plurality of traffic directions at an intersection, an action of setting a traffic signal as one of a plurality of signal combinations for a specific state, a vehicle traffic volume at the intersection, and average waiting of vehicles at a plurality of intersections adjacent to the intersection setting a reward determined according to time; performing reinforcement learning using predetermined traffic data based on the state, the action, and the reward; And based on the result of the reinforcement learning and the traffic information of the target intersection corresponding to the intersection, setting traffic signals for a plurality of travel directions of the target intersection. A traffic signal control method is provided.

According to an embodiment, when the specific state is a current state indicating a passage direction in which the number of waiting vehicles is the maximum among the plurality of passage directions of the intersection, the performing of the reinforcement learning may include whenever the action is applied to the current state. , a travel direction in which the number of waiting vehicles is the largest among a plurality of travel directions of the intersection may be reset to a new current state.

According to an embodiment, the reinforcement learning may be Q-learning.

According to an embodiment, the setting of the traffic signal may include: determining a traffic direction in which the number of waiting vehicles is the largest from among a plurality of passing directions of the target intersection by using the traffic information of the target intersection; calculating a maximum travel signal that is a travel signal that maximizes the value of the Q function among the plurality of signal combinations by using the travel direction in which the number of waiting vehicles is the maximum and the Q function as a result of the reinforcement learning; and setting a traffic signal as the maximum traffic signal for a passage direction in which the number of waiting vehicles is the maximum.

According to an embodiment, the compensation may be proportional to the amount of vehicle traffic at the intersection per unit time, and may be inversely proportional to the average waiting time of vehicles at a plurality of intersections adjacent to the intersection.

According to an embodiment, the compensation may be defined by Equation (1).

[Equation 1]

where r _t is compensation, i is an intersection index, α is a weighting factor, ω is a value greater than 1, tp is vehicle traffic per unit time, L _agent is the number of adjacent intersections, and ξ is It is a value between 0 and 1, and wt is the average waiting time of vehicles at a plurality of adjacent intersections.

In addition, the present invention relates to a state divided according to a plurality of traffic directions at an intersection, an action of setting a traffic signal as one of a plurality of signal combinations for a specific state, a vehicle traffic volume at the intersection, and a vehicle at a plurality of intersections adjacent to the intersection a preparation unit for setting a reward determined according to the average waiting time; a learning unit that performs reinforcement learning using predetermined traffic data based on the state, the action, and the reward; and a setting unit configured to set traffic signals for a plurality of travel directions of the target intersection based on the result of the reinforcement learning and traffic information of the target intersection corresponding to the intersection. Provides traffic signal control devices.

According to an embodiment, when the specific state is a current state indicating a passage direction in which the number of waiting vehicles is the largest among the plurality of passage directions of the intersection, the learning unit receives the plurality of intersections whenever the action is applied to the current state. Among the travel directions of , the travel direction with the largest number of waiting vehicles may be reset to a new current state.

According to an embodiment, the reinforcement learning may be Q-learning.

According to an embodiment, the setting unit uses the traffic information of the target intersection to determine a traveling direction in which the number of waiting vehicles is the largest among a plurality of traveling directions of the target intersection, and the travel direction in which the number of waiting vehicles is the maximum and the reinforcement Using the Q function as a result of learning, a maximum traffic signal that is a traffic signal that maximizes the value of the Q function among the plurality of signal combinations is calculated, and the maximum traffic signal is used as the maximum traffic signal for the traffic direction in which the number of waiting vehicles is the maximum. You can set traffic signals.

According to an embodiment, the compensation may be proportional to the amount of vehicle traffic at the intersection per unit time, and may be inversely proportional to the average waiting time of vehicles at a plurality of intersections adjacent to the intersection.

Reinforcement learning-based traffic signal control method and apparatus according to an embodiment of the present invention distributes signals appropriately so that as many vehicles as possible pass through the intersection, and the order of traffic signals at the intersection in consideration of the average vehicle waiting time of all adjacent intersections It has the effect of efficiently distributing traffic signals in a multi-intersection environment by adjusting the

1 is a flowchart of a method for controlling a distributed traffic signal based on reinforcement learning in a multi-intersection environment according to an embodiment of the present invention.
2 is a flowchart of a method of setting a traffic signal using a reinforcement learning result according to an embodiment of the present invention.
3 is a block diagram of an apparatus for controlling a distributed traffic signal based on reinforcement learning in a multi-intersection environment according to an embodiment of the present invention.
4 is a view showing an intersection according to an embodiment of the present invention.
5 is a diagram illustrating a signal combination according to an embodiment of the present invention.
6A to 6C are diagrams showing results of comparing an embodiment of the present invention with that of the prior art.
7A to 7B are diagrams illustrating a multi-intersection environment.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart of a method for controlling a traffic signal based on reinforcement learning according to an embodiment of the present invention.

Reinforcement learning is an algorithm that learns by itself through the past learning process. Q-learning, a type of reinforcement learning, uses a trial-and-error approach to explore the environment and selects the best action in the current state based on experience. That is, Q-learning has the concepts of state, action, and reward, and in a specific state, the action can be determined in the direction of maximizing the reward. Taking the action (a _t) on the state (s _t) is moved to the next (s _{t + 1)} state. A related formula is the following Equation 1.

[Equation 1]

)을 적용하여 새로운 상태로 업데이트 된다. 이와 관련 공식은 다음 수학식 2와 같다.In addition, in Q-learning, the Q-table that stores the experience is the current state (s _t ), the action (a _t ), the reward (r _t ), and the maximum value of the next state (max _a Q(s _t+1 ,a _{t) +1} )) to the learning rate (

) is applied to update to the new state. The related formula is the following Equation 2.

[Equation 2]

은 기존의 상태값이고,

는 학습률(learning rate)이고,

는 할인팩터(discount factor)이다.here,

is the existing state value,

is the learning rate,

is the discount factor.

In step S110, the traffic signal control device sets a traffic signal as one of a plurality of signal combinations for a state divided according to a plurality of traffic directions at the intersection, a specific state, and the amount of vehicle traffic at the intersection and a plurality of vehicles adjacent to the intersection Set up a reward that is determined by the average waiting time of vehicles at the intersection.

At this time, the traffic signal control device transmits information about the average waiting time of vehicles at each intersection from a plurality of local agents corresponding to each of the plurality of intersections adjacent to the intersection (eg, a traffic signal control device of an adjacent intersection) through wired or wireless communication. can be received through

For example, referring to FIG. 7A , assuming that the traffic signal control device corresponds to the intersection 701, since the traffic signal control device is adjacent to the intersections 702 and 704, from the local agent of the intersections 702 and 704 Information on the average waiting time of vehicles at each intersection can be received.

Here, the state may be classified according to a plurality of passage directions constituting the intersection. For example, referring to FIG. 4 , assuming that the intersection is an intersection, and that straight ahead (401, 403, 405, 407) and left turn (402, 404, 406, 408) are possible in each passing direction, a total of eight traffic directions are may exist. Therefore, finally, a total of eight states can be defined. Similarly, in the case of a five-way intersection, 10 states may be defined, and in the case of a six-way intersection, 12 states may be defined. However, in the present invention, the traveling direction of the vehicle is not limited to going straight and turning left, and various directions such as a U-turn and a right turn may be applied.

In addition, an action is applied to a specific state that is one of the aforementioned states at a specific time point t, and a traffic signal is set as one of a plurality of signal combinations for the specific state. For example, referring to FIG. 4 , when going straight 407 is in a specific state, the action in the present invention may set a traffic signal to one of three signal combinations 501 , 502 , 503 shown in FIG. 5 for that specific state. . However, in the present invention, the signal combination is not limited to the signal combination shown in FIG. 5, and various signal combinations may be applied according to the configuration of the intersection.

In addition, the reward may be determined using, as parameters, the amount of vehicle traffic per unit time (tp) and the average waiting time (wt) of vehicles at a plurality of intersections adjacent to the intersection in order to minimize delay at the intersection. In this case, the vehicle traffic volume tp refers to the number of vehicles that have passed the intersection for a unit time, and the vehicle average waiting time wt refers to the average vehicle waiting time at a plurality of intersections directly connected to the intersection.

In another embodiment, the compensation may be proportional to the amount of vehicle traffic at the intersection per unit time, and may be inversely proportional to the average waiting time of vehicles at a plurality of intersections adjacent to the intersection.

Considering that the present invention aims to minimize the waiting time of vehicles and allow as many vehicles to pass through the intersection, the compensation increases or decreases in proportion to the amount of vehicle traffic at the intersection, and at a plurality of intersections adjacent to the intersection. It will be apparent that the average vehicle waiting time increases or decreases.

In another embodiment, the compensation may be defined by Equation (3).

[Equation 3]

where r _t is compensation, i is an intersection index, α is a weighting factor, ω is a value greater than 1, tp is vehicle traffic per unit time, L _agent is the number of adjacent intersections, and ξ is It is a value between 0 and 1, and wt is the average waiting time of vehicles at a plurality of adjacent intersections.

That is, the compensation r _t is proportional to the amount of vehicle traffic at the intersection and is inversely proportional to the average waiting time of vehicles at a plurality of adjacent intersections.

In step S120, the traffic signal control device performs reinforcement learning using predetermined traffic data based on the state, behavior, and reward.

That is, the traffic signal control apparatus may perform reinforcement learning by using the previously determined state, behavior, and reward, and applying predetermined traffic data. In this case, the traffic data may be actual data collected at the corresponding intersection or data collected at another intersection having a similar structure.

For example, the traffic signal control device may continuously update the Q-table while performing Q-learning, a type of reinforcement learning, using traffic data. In this case, the updated Q function value is a reward generated from experience (traffic data). In addition, it is possible to add a search (ε) to carve out a new path. Exploration with randomness has a value between 0 and 1, allowing us to find a better way.

)은 0에서 1사이의 값으로, 학습할 새로운 정보의 양을 결정한다. 이 값이 1에 가까울수록, 새롭게 취득된 정보가 더 중요해진다. 또한, 할인팩터(

)는 미래 상태의 중요성을 결정한다. 할인팩터가 1에 가까워짐에 따라, 현재의 경험보다 미래의 보상에 대하여 초점이 맞춰지게 된다.On the other hand, referring to Equation 2, the learning rate (

) is a value between 0 and 1, which determines the amount of new information to learn. The closer this value is to 1, the more important the newly acquired information. In addition, the discount factor (

) determines the importance of the future state. As the discount factor approaches 1, the focus is on future rewards rather than on present experiences.

In another embodiment, when the specific state is a current state indicating a passage direction in which the number of waiting vehicles is the largest among a plurality of travel directions of the intersection, the traffic signal control device determines that whenever an action is applied to the current state, the plurality of travel directions of the intersection Reinforcement learning can be performed while resetting the travel direction with the maximum number of waiting vehicles to the new current state.

That is, the traffic signal control apparatus may apply an action to the current state by setting the specific state as a current state indicating a passage direction in which the number of waiting vehicles is the maximum among a plurality of passage directions of an intersection. In addition, whenever an action is applied, the traffic signal control apparatus may update the travel direction in which the number of waiting vehicles is the maximum to a new current state as shown in Equation 4 below.

[Equation 4]

Here, S _t+1 is the new current state, and qt _i is the number of waiting vehicles in the i-th direction.

In other words, the traffic signal control device applies the action a _{t to the current state s t} at a specific time point t, and then at the next time point t+1, a new current state in the direction of travel with the largest number of waiting vehicles. (s _t+1 ) can be reset.

Finally, in step S130, the traffic signal control device sets traffic signals for a plurality of travel directions of the target intersection based on the result of the reinforcement learning and the traffic information of the target intersection corresponding to the intersection.

That is, the traffic signal control apparatus may set traffic signals for a plurality of travel directions of the target intersection by using the result of the completion of reinforcement learning using predetermined traffic data and traffic information of the target intersection.

For example, when Q-learning is used, the traffic signal control apparatus may determine an action to maximize the value of the Q function, and set a traffic signal corresponding to the determined action.

In this case, the target intersection may be the same intersection as the intersection or another intersection having the same or similar structure (eg, number of lanes, direction of passage) as the intersection.

In another embodiment, reinforcement learning may be Q-learning.

On the other hand, in the present invention, simulation was performed in the environment of 9 intersections. Experiments were conducted by specifying the proposed model, D-TCS (distributed), in which traffic volume and waiting time at adjacent intersections were applied as parameters, and F-TCS (fixed-time), which is a fixed method, and A-TCS (adaptive) for the adaptive method. proceeded. The simulation was conducted at 9 consecutive 4-way intersections.

6A to 6C are experimental results of measuring performance according to traffic volume at an intersection. 6a shows the results of measuring the average queue length, when the traffic volume is between 120% and 160%, in general, the average length of the D-TCS was about 35% shorter than that of the F-TCS and about 60% shorter than that of the A-TCS. The short average queue length of D-TCS means that there are not many vehicles waiting at the intersection. This proves that the proposed system, D-TCS, efficiently distributed traffic signals. 6B is a result of measuring the average waiting time. In this case of traffic volume of 120%, D-TCS showed about 32% shorter latency performance than F-TCS and A-TCS. This is interpreted as the better performance of D-TCS using information from nearby intersections because intersections influence each other. 6c is a result of measuring the traffic volume, when the traffic volume is 120%, the D-TCS handles about 15% more vehicles than the F-TCS. It also appears that, on average, D-TCS handles about 25% more vehicles than A-TCS.

2 is a flowchart of a method of setting a traffic signal using a reinforcement learning result according to an embodiment of the present invention.

In step S210 , the traffic signal control device determines a passage direction in which the number of waiting vehicles is the largest from among a plurality of travel directions of the target intersection by using the traffic information of the target intersection.

That is, the traffic signal control apparatus may determine a passage direction in which the number of waiting vehicles is the maximum with respect to a plurality of passage directions existing at the target intersection.

In step S220, the traffic signal control device calculates a maximum traffic signal, which is a traffic signal that maximizes the value of the Q function among a plurality of signal combinations, by using the passage direction in which the number of waiting vehicles is the maximum and the Q function as a result of reinforcement learning. do.

At this time, the traffic signal control device inputs the direction of travel in which the number of waiting vehicles is the maximum and all actions applicable to the direction of travel into the Q function to calculate the action that maximizes the value of the Q function, that is, the maximum travel signal. have.

Finally, in step S230, the traffic signal control device sets the traffic signal as the maximum traffic signal for the traffic direction in which the number of waiting vehicles is the maximum.

That is, the traffic signal control apparatus can reduce the standard deviation of the queue length while increasing the amount of vehicle traffic at the target intersection by setting the traffic signal as the maximum traffic signal for the traffic direction in which the number of waiting vehicles is the maximum.

3 is a block diagram of an apparatus for controlling a traffic signal based on reinforcement learning according to an embodiment of the present invention.

Referring to FIG. 3 , the apparatus 300 for controlling a traffic signal based on reinforcement learning according to an embodiment of the present invention includes a preparation unit 310 , a learning unit 320 , and a setting unit 330 .

The preparation unit 310 performs an action of setting a traffic signal as one of a plurality of signal combinations for a state divided according to a plurality of traffic directions at an intersection, a specific state, and a vehicle traffic volume of the intersection and a plurality of intersections adjacent to the intersection. Set the reward determined according to the average waiting time of the vehicle.

The learning unit 320 performs reinforcement learning using predetermined traffic data based on the state, behavior, and reward.

In another embodiment, when the specific state is a current state indicating a passage direction in which the number of waiting vehicles is the maximum among a plurality of passage directions of an intersection, the learning unit 320 is configured to perform a plurality of passages in the intersection whenever an action is applied to the current state. Among the directions, the travel direction with the largest number of waiting vehicles may be reset to a new current state.

Finally, the setting unit 330 sets traffic signals for a plurality of travel directions of the target intersection based on the result of the reinforcement learning and traffic information of the target intersection corresponding to the intersection.

In another embodiment, reinforcement learning may be Q-learning.

In another embodiment, the setting unit 330 uses the traffic information of the target intersection to determine the passage direction in which the number of waiting vehicles is the largest among a plurality of passage directions of the target intersection, and strengthens the passage direction in which the number of waiting vehicles is the maximum. Using the Q function, which is the result of learning, calculates the maximum traffic signal, which is a traffic signal that maximizes the value of the Q function among a plurality of signal combinations, and sets the traffic signal as the maximum traffic signal for the traffic direction in which the number of waiting vehicles is the maximum. can be set.

In another embodiment, the compensation may be proportional to the amount of vehicle traffic at the intersection per unit time, and may be inversely proportional to the average waiting time of vehicles at a plurality of intersections adjacent to the intersection.

The above description is merely illustrative of the technical spirit of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those of ordinary skill in the art to which the present invention pertains. Accordingly, the embodiments implemented in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (11)

According to a state distinguished according to a plurality of traffic directions at an intersection, an action of setting a traffic signal as one of a plurality of signal combinations for a specific state, and the vehicle traffic volume at the intersection and the average waiting time of vehicles at a plurality of intersections adjacent to the intersection setting a determined reward;
performing reinforcement learning using predetermined traffic data based on the state, the action, and the reward; and
setting traffic signals for a plurality of travel directions of the target intersection based on the result of the reinforcement learning and traffic information of the target intersection corresponding to the intersection
including,
the reward is
Distributed traffic signal control method based on reinforcement learning in a multi-intersection environment, characterized in that it is proportional to the amount of vehicle traffic at the intersection per unit time and is inversely proportional to the average waiting time of vehicles at a plurality of intersections adjacent to the intersection.
According to claim 1,
When the specific state is a current state indicating a passage direction in which the number of waiting vehicles is the largest among a plurality of passage directions of the intersection,
The step of performing the reinforcement learning is
Distributed traffic signal control based on reinforcement learning in a multi-intersection environment, characterized in that whenever the action is applied to the current state, the traffic direction with the largest number of waiting vehicles among the plurality of traffic directions of the intersection is reset to a new current state Way.
According to claim 1,
The reinforcement learning is a distributed traffic signal control method based on reinforcement learning in a multi-intersection environment, characterized in that Q-learning.
4. The method of claim 3,
The step of setting the traffic signal is
determining a traveling direction in which the number of waiting vehicles is the largest among a plurality of traveling directions of the target intersection by using the traffic information of the target intersection;
calculating a maximum travel signal that is a travel signal that maximizes the value of the Q function among the plurality of signal combinations by using the travel direction in which the number of waiting vehicles is the maximum and the Q function as a result of the reinforcement learning; and
Setting a traffic signal as the maximum traffic signal for the traffic direction in which the number of waiting vehicles is the maximum
Distributed traffic signal control method based on reinforcement learning in a multi-intersection environment, comprising:
delete According to claim 1,
the reward is
Distributed traffic signal control method based on reinforcement learning in a multi-intersection environment, characterized in that defined by Equation 1.
[Equation 1]

where r _t is compensation, i is an intersection index, α is a weighting factor, ω is a value greater than 1, tp is vehicle traffic per unit time, L _agent is the number of adjacent intersections, and ξ is It is a value between 0 and 1, and wt is the average waiting time of vehicles at a plurality of adjacent intersections.
According to a state distinguished according to a plurality of traffic directions at an intersection, an action of setting a traffic signal as one of a plurality of signal combinations for a specific state, and the vehicle traffic volume at the intersection and the average waiting time of vehicles at a plurality of intersections adjacent to the intersection a preparation unit for setting a reward to be determined;
a learning unit that performs reinforcement learning using predetermined traffic data based on the state, the action, and the reward; and
Based on the result of the reinforcement learning and the traffic information of the target intersection corresponding to the intersection, a setting unit for setting traffic signals for a plurality of travel directions of the target intersection
including,
the reward is
Distributed traffic signal control apparatus based on reinforcement learning in a multi-intersection environment, characterized in that it is proportional to the amount of vehicle traffic at the intersection per unit time and is inversely proportional to the average waiting time of vehicles at a plurality of intersections adjacent to the intersection.
8. The method of claim 7,
When the specific state is a current state indicating a passage direction in which the number of waiting vehicles is the largest among a plurality of passage directions of the intersection,
the learning unit
Distributed traffic signal control based on reinforcement learning in a multi-intersection environment, characterized in that whenever the action is applied to the current state, the traffic direction with the largest number of waiting vehicles among the plurality of traffic directions of the intersection is reset to a new current state Device.
8. The method of claim 7,
The reinforcement learning is a distributed traffic signal control device based on reinforcement learning in a multi-intersection environment, characterized in that Q-learning.
10. The method of claim 9,
the setting unit
Using the traffic information of the target intersection, determining a traveling direction in which the number of waiting vehicles is the largest among a plurality of traveling directions of the target intersection,
Using the travel direction in which the number of waiting vehicles is the maximum and the Q function as a result of the reinforcement learning, a maximum travel signal that is a travel signal that maximizes the value of the Q function among the plurality of signal combinations is calculated,
Reinforcement learning-based distributed traffic signal control device in a multi-intersection environment, characterized in that the traffic signal is set as the maximum traffic signal for the traffic direction in which the number of waiting vehicles is the maximum.


delete

Images