KR102400833B1 - Method and apparatus for controlling traffic signal based on ai reinforce learning - Google Patents

Abstract

Disclosed is a traffic signal control device, which comprises: an intersection learning unit for deriving a green light time adjustment ratio between barriers and an inconsistent current time green light time adjustment ratio as a first action on the basis of a first state, and outputting an intersection signal control variable; a network learning unit for deriving an offset adjustment ratio for each intersection and an offset for each intersection as a second action on the basis of a second state, and outputting a network signal control variable; a traffic learning model unit for simulating a traffic situation on the basis of an input through-traffic volume for each movement type and an initial queue for each movement type to output the first state to the intersection learning unit and to output the second state to the network learning unit, and simulating the traffic situation again on the basis of the first and second actions to update the first and second states; and a signal control unit for receiving the intersection signal control variable and the network signal control variable, and applying the same to a traffic signal network. Therefore, the urban traffic congestion can be relieved.

Description

AI reinforcement learning-based traffic signal control device and method for performing the same

The present invention relates to an apparatus for controlling a traffic signal based on artificial intelligence (AI) reinforcement learning and a method for performing the same.

Recently, due to the development of AI, there are cases where it is applied to urban engineering, but the technology to control traffic signals using AI has been mainly used for information collection. In other words, information was collected by applying AI, but the development of technology to control traffic signals using it is still in its infancy.

On the other hand, the previously studied AI reinforcement learning-based traffic signal control technologies are a method of determining whether the appearance will be maintained in the next stage or any other manifestation will be made by a display-moving method for each stage of learning. This has a disadvantage in that it is difficult to apply to the actual field because it is difficult to accept in the standard signal controller installed in the current field, which requires restrictions on the display order, period, and barrier.

Therefore, it is necessary to develop a technology that can be applied to actual fields where AI can actively respond to changes in traffic patterns by directly adjusting signal control variables.

Korean Patent Publication No. 10-2171671 (2020.10.23)

An object of the present invention is to provide a traffic signal control device and a method using the same in which AI can actively respond to changes in traffic patterns by directly adjusting signal control variables by AI.

In addition, the problem to be solved by the present invention is to provide a traffic signal control device capable of reflecting a dual-ring-based display system so that it can be operated in an actual field, and a method using the same.

In addition, the problem to be solved by the present invention is to provide a traffic signal control apparatus based on traffic flow model learning for optimizing not only intersections but also network interworking and a method using the same.

In addition, the problem to be solved by the present invention is, in a given traffic situation, based on reinforcement learning for changes in traffic conditions of a traffic flow model according to changes in signal control variables such as appearance, period, and offset. To provide a traffic signal control device and a method using the same.

According to an embodiment of the present invention, a traffic signal control device is provided. The traffic signal control apparatus includes: an intersection learning unit for deriving a green time adjustment ratio between barriers and a green time adjustment ratio for a conflicting current time as a first action based on a first state, and outputting an intersection signal control variable; a network learning unit that derives an offset adjustment ratio for each intersection and an offset for each intersection as a second action based on the second state, and outputs a network signal control variable; The first state is output to the intersection learning unit by simulating the traffic situation based on the input passing traffic volume for each moving flow and the initial waiting matrix for each moving flow, and the second state is output to the network learning unit, and the first action and the first a traffic learning model unit for re-simulating the traffic situation based on 2 actions and updating the first state and the second state; and a signal control unit receiving the intersection signal control variable and the network signal control variable, and applying the intersection signal control variable and the network signal control variable to a traffic signal network.

The first state may include at least one of a space occupancy rate for each current flow, a green time ratio for each current flow, an average intersection control delay, and a saturation level for each current.

The second state may include a section average control delay between control intersections and an offset for each intersection.

The intersection signal control variable may include a green time for each display.

The network signal control variable may include a signal change period, and an intersection offset.

The intersection learning unit and the network learning unit may include a Deep Deterministic Policy Gradient (DDPG) algorithm for performing continuous action space learning.

The traffic learning model unit may compensate for the first action when the intersection delay is reduced due to the first action, and compensate the second action when the network delay is reduced due to the second action.

The traffic learning model unit may derive the latencies of the spatiotemporal cell based on a cell propagation model representing traffic flow shock wave propagation in units of spatiotemporal cells, and may derive the first state and the second state based on the lag.

According to another embodiment of the present invention, there is provided a traffic signal control method by a traffic signal control device including an intersection learning unit and a network learning unit. The traffic signal control method includes: outputting a first state to the intersection learning unit by simulating a traffic situation based on an input amount of passing traffic for each flow and an initial waiting queue for each flow; learning a green time adjustment ratio between barriers and a conflicting current time green time adjustment ratio as a first action based on the first state; updating the first state by re-simulating the traffic situation based on the first action; compensating for the first action when the intersection delay is reduced due to the first action; and deriving an optimal intersection signal control variable based on the first state, the first action, and the reward, and applying the derived intersection signal control variable to a traffic signal network.

The traffic signal control method includes: outputting a second state to the network learning unit by simulating a traffic situation based on an input amount of passing traffic for each flow and an initial waiting queue for each flow; learning an offset adjustment ratio for each intersection and an offset for each intersection as a second action based on the second state; updating the second state by re-simulating the traffic situation based on the second action; compensating for the second action when network delay is reduced due to the second action; The method may further include deriving an optimal network signal control variable based on the second state, the second action, and the reward, and applying the derived network signal control variable to a traffic signal network.

The first state may include at least one of a space occupancy rate for each current flow, a green time ratio for each current flow, an average intersection control delay, and a saturation level for each current. The second state may include a section average control delay between control intersections and an offset for each intersection. The intersection signal control variable may include a green time for each display. The network signal control variable may include a signal change period and an intersection offset.

The step of simulating the traffic situation and the step of simulating the traffic situation again includes deriving the delay of the spatiotemporal cell based on a cell propagation model representing the traffic flow shock wave propagation in units of spatiotemporal cells, and based on the delay, the first deriving a state and the second state.

The intersection signal control variable may be optimized for every predetermined signal period.

The network signal control variable may be optimized every predetermined time.

According to an embodiment of the present invention, there is provided a traffic signal control device and a method using the same, in which AI can actively respond to changes in traffic patterns by directly adjusting signal control variables by AI.

According to an embodiment of the present invention, a traffic signal control device capable of reflecting a dual-ring-based display system so as to be operated in an actual field, and a method using the same are provided.

According to an embodiment of the present invention, there is provided a traffic signal control apparatus based on traffic flow model learning for optimizing network interworking as well as an intersection, and a method using the same.

According to an embodiment of the present invention, there is provided a traffic signal control device and a method using the same, based on reinforcement learning on a change in traffic conditions according to changes in signal control variables such as appearance, period, and offset in a given traffic situation. .

Through this, it is possible to solve the current traffic congestion problem and to alleviate urban traffic congestion by securing the urban traffic flow optimization management technology using AI technology, which is rapidly developing recently.

1 is a control block diagram of a traffic signal control apparatus according to an embodiment of the present invention.
2 is a diagram for explaining the concept of AI reinforcement learning.
3 is a diagram for explaining a model applied to a traffic learning model unit according to an example of the present invention.
4 is a view for explaining a signal control group of an individual intersection according to an embodiment of the present invention.
5 is a control flowchart illustrating a traffic signal control method according to an embodiment of the present invention.
6 is a diagram illustrating a computing device according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

In the present specification, duplicate descriptions of the same components will be omitted.

Also, in this specification, when it is said that a certain element is 'connected' or 'connected' to another element, it may be directly connected or connected to the other element, but other elements in the middle It should be understood that there may be On the other hand, in this specification, when it is mentioned that a certain element is 'directly connected' or 'directly connected' to another element, it should be understood that the other element does not exist in the middle.

In addition, the terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention.

Also, in this specification, the singular expression may include the plural expression unless the context clearly dictates otherwise.

Also, in this specification, terms such as 'include' or 'have' are only intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and one or more It is to be understood that the existence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof, is not precluded in advance.

Also in this specification, the term 'and/or' includes a combination of a plurality of described items or any item of a plurality of described items. In this specification, 'A or B' may include 'A', 'B', or 'both A and B'.

Also, in this specification, detailed descriptions of well-known functions and configurations that may obscure the gist of the present invention will be omitted.

In the case of the existing signal operation, the SA is divided by axis and the signal interlocking plan is carried out in the axis unit, and the connection between SAs is performed based on the operator's experience. Existing traffic flow models for interlocking optimization are also performing axial optimization, and there is no traffic flow model for network interlocking optimization.

Accordingly, the present invention applies AI-based model predictive control (Model Predictive Control) network signal optimization technology to learn changes in traffic conditions according to changes in signal control variables (present, period, offset) in a given traffic situation through AI reinforcement learning. and through this, it aims to perform network interworking optimization close to the optimal value.

In addition, the present invention applies a Kinematic Wave-based MESO model that can simulate the traffic situation in a simple and realistic way, so that it is possible to learn in detail the change of the traffic situation according to the signal control variable change (appearance, period, offset), and the network Through signal operation management, a network between intersections with similar traffic flow characteristics can be constructed.

Hereinafter, the present invention will be described in detail with reference to the drawings.

1 is a control block diagram of a traffic signal control apparatus according to an embodiment of the present invention.

As shown, the traffic signal control device according to this embodiment includes a traffic learning model unit 100 , an intersection learning unit 200 , a network learning unit 300 , and a signal control unit 400 , and the traffic signal control device outputs a signal control variable to the traffic signal network 500 .

As described above, the traffic signal control device directly adjusts traffic signals by AI in response to changes in traffic conditions. , display order), it is possible to operate in the standard traffic signal controller installed in the field, that is, the traffic signal network 500 . To this end, the initial signal control variables (period, green time, offset) and collected/processed traffic information are input into the Kinematic Wave-based MESO model, that is, the traffic learning model unit 100 to simulate the current traffic state, and then the signal When adjusting the control variables (period, green time, offset), the AI learns the change in traffic condition according to the shock wave-based mesoscopic model and derives the optimal signal control variable in response to the traffic condition. AI learning is performed by the intersection learning unit 200 and the network learning unit 300 , and signal control variables by AI learning are output to the signal control unit 400 , and may be applied to the traffic signal network 500 .

Prior to the description of the components of FIG. 1, AI reinforcement learning applied to the present invention will be described as follows. 2 is a diagram for explaining the concept of AI reinforcement learning.

AI reinforcement learning is to learn which action (Action, hereinafter, action) is optimal to take in the current state (State, hereinafter, state), and a reward is given whenever an action is taken. Learning is performed to determine the action in the direction of maximizing the reward given repeatedly over and over again. An action for the maximum reward is determined according to a predetermined period, this action is reflected in the state setting again, and an algorithm for deriving an optimal action is applied in such a way that a reward is given again.

In the present invention, according to an example, DDPG (Deep Deterministic Policy Gradient), which is one of various reinforcement learning algorithms, may be applied. The DDPG algorithm is based on the Deterministic Policy Gradient (DPG) algorithm called off-policy, continuous actor-critic.

In addition, the algorithm according to this embodiment applies soft update and batch learning proposed by deep learning techniques (DQN, Deep Q-Network) to perform more complex learning, and the action space ( Action Space) is a model optimized for AI reinforcement learning in a continuous environment.

In summary, since the action space is suitable for directly learning the continuous signal control variable adjustment value, the traffic signal control device according to the present invention adopts the DDPG algorithm as a reinforcement learning model for learning the signal control variable adjustment to AI.

The AI-based model predictive control network signal optimization technology applied to the traffic signal control device according to the present invention may consist of an individual intersection signal optimization algorithm and a network offset optimization algorithm, wherein each AI agent is Appearance optimization can be performed in parallel. The above-described DDPG algorithm-based AI agent may be implemented by the intersection learning unit 200 and the network learning unit 300 , and the model for simulating the traffic situation may be implemented by the traffic learning model unit 100 .

The intersection learning unit 200 derives the green time adjustment ratio between barriers and the conflicting current time green time adjustment ratio as a first action based on the first state, and outputs the intersection signal control variable, and the network learning unit 300 Based on the second state, an offset adjustment ratio for each intersection and an offset for each intersection may be derived as a second action, and a network signal control variable may be output.

The intersection learning unit 200 for optimizing the intersection signal control variable and the network learning unit 300 for optimizing the network signal control variable operate in parallel with each other and optimize the signal control variable according to different cycles, respectively.

The traffic learning model unit 100 simulates the traffic situation based on the input traffic volume for each movement flow and the initial queue for each movement flow, and outputs the first state to the intersection learning unit 200, and simulates the traffic situation to obtain the second state. It outputs to the network learning unit 300 and performs modeling in which the first state and the second state are updated by re-simulating the traffic situation based on the first action and the second action.

The first state according to this embodiment may include at least one of a space occupancy rate for each current flow, a green time ratio for each current flow, an intersection average control delay, and a saturation level for each current flow, and the second state includes an average control delay between control intersections and It may include an offset for each intersection. In addition, the intersection signal control variable may include a green time for each appearance, and the network signal control variable may include a signal change period and an intersection offset.

That is, the traffic learning model unit 100 can derive the space occupancy rate for each current flow, the green time ratio for each current flow, the intersection average control delay and the saturation for each current as the first state, based on the input traffic information, as the first state. The first state is input to the intersection learning unit 200, and the intersection learning unit 200 outputs the green time adjustment ratio between barriers and the green time adjustment ratio for the conflicting current time as a first action through learning of the input first state. can do. Based on this first action, the green time for each appearance, which is an intersection signal control variable, may be changed, and the traffic learning model unit 100 may update the first state again based on the changed green time for each appearance. As this process is repeated, an optimal intersection signal control variable can be derived.

Similarly, the traffic learning model unit 100 may derive the average control delay between control intersections and the offset for each intersection as a second state based on the input traffic information, and the second state is the network learning unit 300 ), and the network learning unit 300 may output an offset adjustment ratio for each intersection and an offset for each intersection as a second action through learning of the input second state. Based on this second action, the signal change period and intersection offset, which are network signal control variables, may be changed, and the traffic learning model unit 100 updates the second state again based on the changed signal change period and the intersection offset. can do. As this process is repeated, an optimal network signal control variable can be derived.

Meanwhile, information required for AI reinforcement learning for optimizing traffic signals according to the present embodiment may include information on a moving flow traffic volume and a waiting queue for each moving flow, and information about a moving traffic volume and a waiting queue for each moving flow is traffic learning. It may be input to the model unit 100 and used to simulate a traffic situation.

Information on the moving flow traffic volume and the waiting queue for each moving flow is initially set to a predetermined initial value, and when a signal control variable is applied to the traffic signal network 500 due to repeated updates of states and actions, ) can be updated again based on the actually collected traffic information.

The amount of traffic per movement flow can be defined as the amount of traffic passing through the stop line, the time processing unit can be set to a predetermined cycle in which the signal is changed, and the spatial processing unit can be defined by the current movement flow (approach direction, straight/left turn). ) can be set as an intersection space.

In addition, the initial queue for each flow may be defined as the amount of traffic remaining in the detection area at the time of transition from green time to red time, and the time processing unit and space processing unit may be set to be the same as the traffic volume for each movement flow.

Meanwhile, according to an example, the traffic learning model unit 100 may apply a Kinematic Wave-based MESO model that is a traffic flow simulation mesoscopic model for learning the traffic flow change according to the signal control variable change to the AI agent. This is a model developed to be suitable for simulating urban traffic networks based on the Kinematic Wave Model (Daganzo, 1994), which is a model representing the propagation of traffic shock waves in units of space and time cells. Based on the initial density of each cell, it determines the supply and demand traffic according to the density of each cell, determines the transition traffic at each cell boundary, updates the cell density value every second, and so on. By assigning a density value to each cell, the transition of the shock wave can be derived. The smaller of the traffic volume that can be received from the downstream cell (supply) and the amount of traffic that the upstream cell wants to send (demand) can be determined as the traffic volume, and the density value of the cell is determined by the cell density value 1 second before and the traffic volume at the boundary . The degree of delay on the road may be derived from this density value.

The Kinematic Wave-based MESO model according to this embodiment reflects the characteristics of the urban intersection geometry and various display systems, and through this, it is possible to calculate the delay using the space-time cell.

In urban traffic flow, a delay in one flow may affect the traffic situation of another adjacent flow. For example, if a delay occurs in the left-turning flow and the queue crosses the left-turning lane, the straight-through flow may also be affected and delay may occur. When such a situation is simulated in the existing traffic flow model, it is determined that the delay in going straight is simply aggravated, but in the model according to this embodiment, it can be determined that the traffic flow is worsening due to oversaturation of the left turn.

The traffic learning model unit 100 according to this embodiment classifies two situations according to the intersection geometry, which is illustrated in FIG. 3 .

3 is a diagram for explaining a model applied to a traffic learning model unit according to an example of the present invention.

Fig. 3 (a) shows the "First In First Out" structure, and when the number of lanes for the straight-through flow is one, it shows a geometrical structure in which both sides cannot escape if the straight-forward one-lane is blocked, reflecting this geometry In order to do this, the traffic learning model unit 100 used the Daganzo (1993) classification model (Combining Downstream Supply).

In addition, (b) of FIG. 3 shows the "Non First In First Out" structure, and when the straight moving flow is a multi-lane road, when the straight one-lane road is blocked, the capacity of going straight is reduced, but it shows a shape of a geometric structure that can pass In order to reflect this geometric structure, the traffic learning model unit 100 and Lebacque (1996) classification model (Splitting Upstream Demand) can be used.

In addition, the traffic learning model unit 100 can reflect various display systems such as straight line straight, line left turn, simultaneous signal, overlapping display, etc., and derives the total delay by the sum of the density values of cells having a density greater than the initial density. can do. In order to derive the delay, it is possible to reflect the various geometries of various roads. For example, the total delay can be calculated by dividing the straight-only section, the left-turn only section, and the straight-left mixed section, and the mixed section can be reflected by giving weight to the ratio of the straight-line and left-turn traffic. At this time, the delay can be calculated by dividing the total delay in going straight and turning left by the number of vehicles going straight and turning left, respectively, and averaging.

One intersection learning unit 200, which is an AI agent, may be disposed per intersection, and performs green time adjustment for each intersection display. This is by adjusting the barrier boundary and the conflicting display boundary, and individual intersection display optimization can be performed for each predetermined period, for example, three basic periods.

4 is a diagram illustrating signal control parameters of individual intersections according to an embodiment of the present invention.

As shown in FIG. 4 , signals for two barrier groups may be changed during one period. The intersection learning unit 200 serves to adjust the time boundary (I) between the two barriers, and to adjust the time boundary (II) in case of conflict within one ring bayer group.

The states, actions, and rewards for the intersection learning unit 200 to perform AI reinforcement learning are as follows.

The state means an indicator that can represent the traffic condition according to the adjustment of the signal control variable of the control target, and the space occupancy and green time ratio for each state can be set as state indicators. For example, there may be 16 state indicators based on 4 intersections.

The action is an action for adjusting the signal control variable of the control target, and can be set as a green time adjustment ratio for each barrier and a conflicting current time green time adjustment ratio for dual-ring-based appearance adjustment. For example, there may be 5 actions such as 1 between barriers and 4 during conflict based on 4 intersections.

The compensation is a compensation value for evaluating the result according to the action, and may be determined depending on whether the delay at each intersection increases or decreases. For example, compensation of +1 may be performed when intersection delay is reduced, and compensation may be set to 0 when intersection delay increases.

Another AI agent, the network learning unit 300 , may be arranged one per control network, and performs offset adjustment between intersections constituting the network. This means that a signal is adjusted so that a vehicle passing through a plurality of intersections can pass through the intersection without stopping, so that individual network offset optimization can be performed at a predetermined period, for example, a basic period of one hour.

The states, actions, and rewards for the network learning unit 300 to perform AI reinforcement learning are as follows.

1 네트워크 기준으로 4개 스테이트 지표가 존재할 수 있고, 이 때 스테이트는 제어교차로 간 구간 평균제어지체(양방향 평균) 2개 및 제어교차로 간 상대옵셋 2개로 구성될 수 있다. The state means an indicator that can represent the traffic condition according to the adjustment of the signal control variable of the control target, and can be set as the average control delay between control intersections in both directions and the relative offset between the control intersections. For example, 3

There may be four state indicators based on one network, and in this case, the state may consist of two average control delays (two-way average) between control intersections and two relative offsets between control intersections.

2 네트워크 기준으로 5개 액션이 존재할 수 있다. The action is an action for adjusting the signal control variable of the control target and may be set as an offset adjustment ratio for each control intersection. For example, the central intersection of the control network has an offset of 0 and 3

2 There may be 5 actions based on the network.

The reward is a reward value for evaluating the result according to the action, and may be determined depending on whether the network delay increases or decreases. For example, when network delay decreases, compensation of +1 may be performed, and when network delay increases, compensation may be set to 0.

The signal control unit 400 may receive the intersection signal control variable and the network signal control variable, and substantially apply the intersection signal control variable and the network signal control variable to the traffic signal network 500 .

5 is a control flowchart illustrating a traffic signal control method according to an embodiment of the present invention. The traffic signal control method according to the present embodiment is summarized as follows with reference to FIG. 5 .

As described above, the intersection learning unit 200 and the network learning unit 300, which are AI agents, operate in parallel with the traffic learning model unit 100, and since the signal control variables to be optimized are different, the states and actions are different, but , the algorithm for the operation is the same. In FIG. 5 , the first state and the first action related to the intersection learning unit 200 are mainly described, and when the first state is changed to the second state and the first action is changed to the second action, the network learning unit 300 is AI reinforcement learning process can be derived.

First, the traffic signal control device may collect and process traffic state information and convert it into information that the traffic learning model unit 100 can utilize ( S510 ).

The traffic learning model unit 100 may derive the first state (i) and the intersection total delay (i) for compensation calculation based on the amount of passing traffic for each flow and the initial queue for each flow ( S520 ).

As described above, the first state may include a space occupancy rate for each current flow, a green time ratio for each current flow, an average intersection control delay, and a saturation level for each current.

Upon receiving the first state (i), the intersection learning unit 200 may derive a first action (i) based on the first state (i) ( S530 ).

The first action may be an inter-barrier green time adjustment ratio and a conflicting current time green time adjustment ratio.

When the first action (i) is again input to the traffic learning model unit 100, the traffic learning model unit 100 is based on the signal control variable adjustment in which the first action (i) is reflected, the first state (i+1) ) and the intersection total delay (i+1) can be derived (S540). That is, the intersection learning unit 200 may update the first state (i) to the first state (i+1) based on the first action (i). This means that the traffic situation is newly simulated based on the first action (i). The traffic learning model unit 100 compares the existing intersection total occupancy system (i) according to the first state (i) and the intersection general occupancy system (i+1) according to the first state (i+1) to perform a first action ( A reward (i) for i) can be derived. That is, if it is determined that the intersection total zoning system (i+1) updated by the first action (i) is reduced compared to the intersection general zoning system (i), that is, if the intersection delay is reduced, a reward (i) of +1 is given. (S550).

The AI agent updated by storing and learning the first state (i), the first state (i+1), the first action (i), and the reward (i), that is, the intersection learning unit 200, is the first state ( A first action (i+1) corresponding to i+1) may be derived (S560).

The processes of S520 to S560 may be repeated a preset number of times or for a predetermined period, and when one cycle is completed, it may be determined whether the process is repeated for a predetermined period (S570).

If the cycle is not repeated for a predetermined period, i is updated to i+1 and the processes of S520 to S560 are repeated again (S580). If the cycle is repeated for a predetermined period, the intersection learning unit 200 is optimized based on compensation of signal control variables can be output.

These optimal signal control parameters may be applied to the traffic signal network 500 (S590).

The signal control variables applied to the actual traffic signal network 500 are applied for a certain period, and new signal control variables for the application result may be collected again as new traffic state information.

The traffic signal control apparatus may repeat the process of FIG. 5 based on the newly collected data.

As described above, by applying the traffic signal control apparatus and the method using the same according to the present embodiment to the domestic urban traffic signal network with many traffic pattern changes, it is possible to respond to the pattern change, and in the axial unit control system of the existing signal control system. In addition, by performing traffic flow management for a network unit, real-time signal operation that conforms to network characteristics can be performed.

In addition, since AI can perform real-time traffic signal operation by directly adjusting signal control variables (period, green time, offset) in response to changes in traffic patterns, AI technology can be actively used to change traffic patterns. In addition, unlike existing technologies, it can be applied to traffic sites in practice by observing the restrictions (order of appearance, period, barriers) for field application.

6 is a diagram illustrating a computing device according to an embodiment of the present invention. The computing device TN100 of FIG. 6 may be a device described herein (eg, a traffic signal control device, an intersection learning unit, a network learning unit, or a traffic learning model unit, etc.).

In the embodiment of FIG. 6 , the computing device TN100 may include at least one processor TN110 , a transceiver device TN120 , and a memory TN130 . In addition, the computing device TN100 may further include a storage device TN140 , an input interface device TN150 , an output interface device TN160 , and the like. Components included in the computing device TN100 may be connected by a bus TN170 to communicate with each other.

The processor TN110 may execute a program command stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to an embodiment of the present invention are performed. The processor TN110 may be configured to implement procedures, functions, and methods described in connection with an embodiment of the present invention. The processor TN110 may control each component of the computing device TN100.

Each of the memory TN130 and the storage device TN140 may store various information related to the operation of the processor TN110. Each of the memory TN130 and the storage device TN140 may be configured as at least one of a volatile storage medium and a nonvolatile storage medium. For example, the memory TN130 may include at least one of a read only memory (ROM) and a random access memory (RAM).

The transceiver TN120 may transmit or receive a wired signal or a wireless signal. The transceiver TN120 may be connected to a network to perform communication.

On the other hand, the embodiment of the present invention is not implemented only through the apparatus and/or method described so far, and a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded may be implemented. And, such an implementation can be easily implemented by those skilled in the art from the description of the above-described embodiment.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims are also presented. It belongs to the scope of the invention.

Claims (10)

A traffic signal control device comprising:
an intersection learning unit for deriving a green time adjustment ratio between barriers and a green time adjustment ratio for conflicting current time as a first action based on the first state, and outputting an intersection signal control variable;
a network learning unit that derives an offset adjustment ratio for each intersection and an offset for each intersection as a second action based on the second state, and outputs a network signal control variable;
The first state is output to the intersection learning unit by simulating the traffic situation based on the input passing traffic volume for each moving flow and the initial waiting matrix for each moving flow, and the second state is output to the network learning unit, and the first action and the a traffic learning model unit for re-simulating the traffic situation based on a second action and updating the first state and the second state;
and a signal controller for receiving the intersection signal control variable and the network signal control variable, and applying the intersection signal control variable and the network signal control variable to a traffic signal network.
According to claim 1,
The first state includes at least one of a space occupancy rate for each current flow, a green time ratio for each current flow, an intersection average control delay, and a saturation level for each current,
The second state includes a section average control delay between control intersections and an offset for each intersection,
The intersection signal control variable includes a green time for each time,
The network signal control variable includes a signal change period and an intersection offset.
According to claim 1,
The intersection learning unit and the network learning unit traffic signal control device, characterized in that it comprises a DDPG (Deep Deterministic Policy Gradient) algorithm for performing continuous action space learning.
4. The method of claim 3,
The traffic learning model unit,
Compensating for the first action when the intersection delay is reduced due to the first action,
and compensating for the second action when network delay is reduced due to the second action.
3. The method of claim 2,
The traffic learning model unit derives the delay of the spatiotemporal cell based on the cell propagation model representing the traffic flow shock wave propagation in units of spatiotemporal cells,
The traffic signal control apparatus of claim 1, wherein the first state and the second state are derived based on the delay.
According to claim 1,
The intersection signal control variable is optimized for every predetermined signal period,
The traffic signal control device, characterized in that the network signal control variable is optimized every predetermined time.
A traffic signal control method by a traffic signal control device including an intersection learning unit and a network learning unit, the method comprising:
outputting a first state to the intersection learning unit by simulating a traffic situation based on the input traffic volume for each moving flow and an initial waiting matrix for each moving flow;
learning a green time adjustment ratio between barriers and a conflicting current time green time adjustment ratio as a first action based on the first state;
updating the first state by re-simulating the traffic situation based on the first action;
compensating for the first action when the intersection delay is reduced due to the first action;
and deriving an optimal intersection signal control variable based on the first state, the first action, and the reward, and applying the derived intersection signal control variable to a traffic signal network. Way.
8. The method of claim 7,
outputting a second state to the network learning unit by simulating a traffic situation based on the input passing traffic for each flow and an initial waiting matrix for each flow;
learning an offset adjustment ratio for each intersection and an offset for each intersection as a second action based on the second state;
updating the second state by re-simulating the traffic situation based on the second action;
compensating for the second action when network delay is reduced due to the second action;
The method further comprising the step of deriving an optimal network signal control variable based on the second state, the second action, and the reward, and applying the derived network signal control variable to a traffic signal network. control method.
9. The method of claim 8,
The first state includes at least one of a space occupancy rate for each current flow, a green time ratio for each current flow, an average intersection control delay, and a saturation level for each current,
The second state includes a section average control delay between control intersections and an offset for each intersection,
The intersection signal control variable includes a green time for each display,
The traffic signal control method according to claim 1, wherein the network signal control variable includes a signal change period and an intersection offset.
Memory; and
A processor for controlling the memory,
The processor is
The first state is output by simulating the traffic situation based on the input traffic volume for each moving flow and the initial queue for each moving flow, and based on the first state, the green time adjustment ratio between barriers and the green time adjustment ratio for the conflicting current time are calculated based on the first state. Learning with 1 action, re-simulating the traffic situation based on the first action, and updating the first state
controller.

Images