KR102398493B1 - Method and apparatus for urban traffic network modeling with multiple cctv videos - Google Patents

Abstract

Urban traffic network modeling method using multiple CCTV video performed by the device according to an embodiment of the present invention is (a) collecting video from multiple CCTV networks, but sampling the video by adjusting the sampling frequency according to the speed change of the vehicle to do; (b) generating vehicle flow data from the sampled video based on a preset vehicle tracking algorithm; (c) combining the generated vehicle flow data and multiple CCTV networks including each CCTV location information to create an incomplete traffic network in which observed data and non-observed data are identified; and (d) generating missing data for the first non-observed data and the second non-observed data through the missing data prediction model, and then modeling the urban traffic network in which the observed data and the missing data are merged.

Description

A method and apparatus for modeling a city traffic network using multiple CCTV videos

The present invention relates to a method and apparatus for modeling a city traffic network using multiple CCTV videos.

The common goal of existing research in traffic pattern discovery research is to analyze changes in traffic patterns over time and to understand urban traffic ecosystems. Among the visual analysis systems, a method of discovering vehicle movement patterns and clustering similar patterns, identifying homes and workplaces using movement patterns reflected in smart traffic card data, a new visualization method that can analyze large-scale traffic flows at intersections, etc. This has been studied

As situational awareness exploration and predictive research, there are several analytical and predictive systems. There are studies that analyze how traffic conditions at a specific time and place affect the present or the near future, and there are studies that analyze traffic patterns using semantic-based Q&A interfaces. Papers on how to analyze traffic congestion, search for similar patterns, and then measure the degree of traffic congestion propagation, route optimization, route recommendation, and situational awareness analysis in urban planning research have been published.

Existing traffic network modeling technology uses vehicle detector (VD) data and GPS trajectory data. Although VD data and GPS track data are suitable for understanding traffic conditions, they have many limitations in analyzing urban traffic networks. VD data is data that records the number and speed of vehicles passing a specific point, and most of them are installed on highways, so they cannot be used to model urban traffic networks. GPS trajectory data has problems such as statistical representation in modeling urban traffic networks with the movement paths of some sampled vehicles.

On the other hand, since GPS orbit data directly represents the movement patterns of people in a city, it can be applied to various fields along with an appropriate analytical model. For example, there are various studies such as movement pattern search, route recommendation, urban planning, and traffic jam detection. Technology that uses taxi data to analyze people's living patterns in a particular city, technology that recommends routes so that taxi drivers can find passengers quickly, and technology that can use visual query models to quickly explore movement patterns in a city There is research. There is also research on interactive visual analytics systems that can use trajectory data to investigate optimal billboard positions.

To analyze the causes of traffic congestion in a city, it is necessary to understand the traffic situation. Currently, many traffic congestion estimation algorithms are based on calculating the number of vehicles, the average speed of the road, and the spatiotemporal traffic density. Although there are studies that analyze using only one specific parameter, such as average speed, travel time, or vehicle density, to calculate the degree of traffic congestion, in reality, there are many uncertainties in estimating the congestion area because various situations are combined to cause traffic congestion. There is this.

Existing research on visual analysis is focused on detecting traffic congestion areas and predicting congestion levels. However, in order to solve traffic congestion, it is necessary to be able to analyze the causes and effects of congestion on the road.

In this regard, Korean Patent Laid-Open Publication No. 10-2006-0092909 (Title of the invention: traffic prediction using modeling and analysis of probabilistic interdependence relationships and context data) provides Disclosed is a method for predicting traffic flow and congestion based on an abstraction of a traffic system with a set of random variables, including time and variables representing the time until congestion is resolved.

In order to solve the above problems, the present invention is to model an urban traffic network using multiple CCTV videos and to analyze the urban traffic flow. In addition, it is intended to provide a method and apparatus for modeling an urban traffic network using multiple CCTV videos that estimate the traffic flow of unobserved roads and predict the traffic flow of the entire city by learning the diffusion process of the spatiotemporal traffic flow in the traffic network.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems described above, and other technical problems may exist.

As a technical means for achieving the above-mentioned technical problem, the urban traffic network modeling method using multiple CCTV videos performed by the device according to an aspect of the present invention is (a) collecting video from multiple CCTV networks, but the speed of the vehicle sampling the video by adjusting the sampling frequency according to the change; (b) generating vehicle flow data from the sampled video based on a preset vehicle tracking algorithm; (c) generating an incomplete traffic network in which observation nodes and non-observation nodes are identified by combining the generated vehicle flow data and multiple CCTV networks including each CCTV location information; and (d) a city in which missing data including the first unobserved data and the second unobserved data are generated through the missing data prediction model, and the vehicle flow data for the observation node and the missing data for the non-observed node are merged Modeling a traffic network; including, wherein the first non-observed data is vehicle flow data of roads that are not observed by CCTV among roads connected to the intersection, and the second non-observed data is the intersection of roads and roads connected to the intersection without CCTV. It is vehicle flow data, and the missing data prediction model is trained through DCRNN (Diffusion Convolutional Recurrent Neural Network).

In step (a), it is determined whether to sample the real-time video by the adaptive sampling module, wherein the adaptive sampling module includes a sampling time, a sampling period, and a sampling interval according to a change in vehicle speed.

In step (b), a plurality of complementary vehicle tracking algorithms are performed in parallel to extract vehicle flow data, and the vehicle flow data includes vehicle speed, moving direction, and traffic volume according to time.

In step (d), when estimating the missing data for the first non-observation data, (d-1) the observation node corresponding to the CCTV location, the road edge corresponding to the road adjacent to the CCTV location, and the CCTV cannot be observed generating an incomplete transportation network including non-observed nodes corresponding to locations, (d-2) adding a primary virtual node and a primary virtual road edge to the incomplete transportation network to create a first incomplete virtual network, (d) -3) Create an input network for DCRNN learning excluding non-observed nodes and road edges connected to non-observed nodes, (d-4) create a DCRNN output network that has learned missing data for the primary virtual node, ( d-5) A second virtual network is created in which an observation node and a road edge are added based on the primary virtual node adjacent to the non-observed node.

In step (d), when estimating missing data for the second non-observation data, (d-6) adding a non-observing node corresponding to a location where there is no CCTV in the second virtual network, (d-7) 2 Create a third virtual network including the primary virtual node and the secondary virtual road edge, create an input network for DCRNN learning excluding the road edge connected to the non-observation node, (d-8) for the secondary virtual node A DCRNN output network trained on the missing data is generated, and (d-9) the primary virtual node and the secondary virtual node are removed to generate a complete city traffic network in which the missing data is estimated.

In response to a request for modeling a city traffic network, a user interface is provided for displaying a predicted traffic volume, wherein the user interface controls a first area displaying a real-time video sampling state, a second area providing a traffic CCTV real-time screen, and video direction movement. In the third area, the fourth area that displays a vehicle image not detected by the vehicle tracking algorithm, the fifth area that provides map information corresponding to CCTV location information and a predictive traffic volume monitoring map linking the city traffic network, the monitoring map Interlocking with the 6th area where the reflected traffic speed, traffic volume, congestion type, and traffic propagation type identification items are displayed, the 7th area and 5th area providing a time slider to control the predicted traffic volume displayed on the monitoring map, The eighth area provides urban traffic network modeling including observation nodes and non-observation nodes of Includes 9 areas.

Urban traffic network modeling apparatus using multiple CCTV video according to another aspect of the present invention is a memory in which the urban traffic network modeling method program is stored; Comprising a processor executing a program stored in the memory, the processor collects video from multiple CCTV networks by executing the program, samples the video by adjusting the sampling frequency according to the speed change of the vehicle, and a preset vehicle tracking algorithm Generate vehicle flow data from the sampled video based on After generating the missing data including the first unobserved data and the second unobserved data through the predictive model, model the urban traffic network in which the vehicle flow data for the observation node and the missing data for the non-observation node are merged, The first non-observed data is vehicle flow data of roads that are not observed by CCTV among roads connected to the intersection, the second non-observed data is vehicle flow data of intersections and roads connected to the intersection without CCTV, and the missing data prediction model is DCRNN It is learned through (Diffusion Convolutional Recurrent Neural Network).

The processor determines whether to sample the real-time video by the adaptive sampling module, wherein the adaptive sampling module includes a sampling time, a sampling period, and a sampling interval according to a change in vehicle speed.

The processor performs a plurality of complementary vehicle tracking algorithms in parallel to extract vehicle flow data, and the vehicle flow data includes vehicle speed, movement direction, and traffic volume according to time.

When estimating missing data for the first non-observation data, the processor includes an observation node corresponding to the CCTV location, a road edge corresponding to a road adjacent to the CCTV location, and a non-observing node corresponding to a location that cannot be observed by the CCTV. Creates an incomplete traffic network that uses Create an input network for the first virtual node, generate a DCRNN output network that has learned missing data for the first virtual node, and create a second virtual network with observation nodes and road edges added based on the primary virtual nodes adjacent to non-observed nodes. create

When estimating missing data for the second non-observed data, the processor adds a non-observed node corresponding to a location where there is no CCTV in the second virtual network, and a third including a secondary virtual node and a secondary virtual road edge. Create a virtual network, create an input network for DCRNN training excluding road edges connected to non-observation nodes, create a DCRNN output network trained on missing data for a secondary virtual node, create a primary virtual node and a secondary By eliminating virtual nodes, we create a complete city transport network with an estimated missing data.

The processor provides a user interface for displaying a predicted traffic volume in response to a city traffic network modeling request, wherein the user interface includes a first area displaying a real-time video sampling state, a second area providing a real-time traffic CCTV screen, and a direction movement of the video A third area to control the vehicle, a fourth area to display a vehicle image not detected by the vehicle tracking algorithm, a fifth area to provide a map information corresponding to CCTV location information and a predictive traffic volume monitoring map linking the city traffic network, monitoring Interlocks with the 6th area where identification items of traffic speed, traffic volume, congestion type and traffic propagation type reflected on the map are displayed, and the 7th and 5th areas that provide a time slider to control the predicted traffic volume displayed on the monitoring map, , provides a real-time CCTV video screen corresponding to the observation node when clicking on the observation node displayed in the eighth area, and the fifth area and the eighth area, which provides urban traffic network modeling including a plurality of observation nodes and non-observation nodes a ninth area to

The computer-readable recording medium stores a computer program for performing the urban traffic network modeling method according to the urban traffic network modeling method using multiple CCTV videos.

The present invention is to solve the problems of the prior art, and it is possible to provide real-time city traffic flow analysis and future traffic flow prediction results using existing traffic CCTV camera infrastructure.

In addition, it has the advantage of using CCTV video to analyze city traffic networks rather than using GPS trajectory and vehicle detection (VD) data, which are traditional traffic data. In addition, by using CCTV video for urban traffic network analysis, it is possible to explore traffic conditions in real time and overcome the statistical representation problem, which is a limitation of GPS track data.

1 is a block diagram of a city traffic network modeling system using multiple CCTV videos according to an embodiment of the present invention.
2 is a diagram illustrating a city traffic network modeling process using multiple CCTV videos in an embodiment of the present invention.
3A is a diagram for explaining a sampling method of a real-time CCTV video that adapts to a traffic flow change amount according to an embodiment of the present invention.
Figure 3b is a graph showing a sampling result according to the adaptive CCTV video sampling module according to an embodiment of the present invention.
4 is a diagram for explaining that a vehicle direction is determined according to a basic direction and a lane area of a CCTV network according to an embodiment of the present invention.
5 is a diagram for explaining a DCRNN process for estimating a traffic flow for first unobserved data according to an embodiment of the present invention.
6 is a diagram for explaining a DCRNN process for estimating a traffic flow for second unobserved data according to an embodiment of the present invention.
7 is an example of a user interface for displaying urban traffic network modeling and predicted traffic volume according to an embodiment of the present invention.
8 is a flowchart illustrating a method for modeling a city traffic network using multiple CCTV videos according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or to reproduce one or more processors. Accordingly, as an example, '~' indicates components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

The "city traffic network modeling apparatus using multiple CCTV videos" mentioned below may be implemented as a computer or portable terminal that can be connected to a server or other terminal through a network. Here, the computer includes, for example, a laptop, a desktop, and a laptop equipped with a web browser (WEB Browser), and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , various mobile communication-based terminals, smartphones, tablet PCs, and the like, may include all types of handheld-based wireless communication devices. In addition, “network” means a wired network such as a local area network (LAN), a wide area network (WAN) or a value added network (VAN), or a mobile radio communication network or satellite It may be implemented in any kind of wireless network such as a communication network.

Hereinafter, for convenience of description, the urban traffic network modeling apparatus 100 using multiple CCTV videos according to an embodiment of the present invention will be briefly referred to as a 'traffic network modeling apparatus 100'.

1 is a block diagram of a city traffic network modeling system using multiple CCTV videos according to an embodiment of the present invention.

The urban traffic network modeling method using multiple CCTV videos may be implemented in the traffic network modeling apparatus 100 . Multiple CCTV videos can be obtained from an external server 10 that provides CCTV video data collected from multiple CCTV networks built in the city. The missing data prediction model is a model that predicts unobserved data by learning based on the collected CCTV video, and can be trained using an artificial neural network such as a diffusion convolutional recurrent neural network (DCRNN).

1 , specifically, the traffic network modeling apparatus 100 includes a memory 110 , a communication module 120 , a processor 130 , and a display unit 140 .

The memory 110 stores a city traffic network modeling method program. Urban traffic network modeling method program collects irregular spatiotemporal data through a network, generates training data based on the collected irregular spatiotemporal data, and learns a missing data prediction model to predict missing data by inputting the learning data as an input make it

Various types of data generated during the execution of an operating system for driving the traffic network modeling apparatus 100 or an urban traffic network modeling method program are stored in the memory 110 .

In this case, the memory 110 collectively refers to a non-volatile storage device that continuously maintains stored information even when power is not supplied, and a volatile storage device that requires power to maintain the stored information.

In addition, the memory 110 may perform a function of temporarily or permanently storing data processed by the processor 130 . Here, the memory 110 may include magnetic storage media or flash storage media in addition to the volatile storage device requiring power to maintain stored information, but the scope of the present invention is limited thereto. it is not going to be

The communication module 120 may include one or more components that allow communication with the outside, such as an external server 10 that provides CCTV video data. For example, the communication module 120 may be a device including hardware and software necessary for transmitting and receiving signals such as control signals or data signals through wired/wireless connection with other network devices.

The processor 130 executes the program stored in the memory 110 , and controls the entire process according to the execution of the urban traffic network modeling method program. Each operation performed by the processor 130 will be described in more detail later.

The processor 130 may include all kinds of devices capable of processing data. For example, it may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed as code or instructions included in a program. As an example of the data processing device embedded in the hardware as described above, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific (ASIC) An integrated circuit) and a processing device such as a field programmable gate array (FPGA) may be included, but the scope of the present invention is not limited thereto.

The display unit 140 may output a user interface for displaying a predicted traffic amount in response to a user's request for modeling a city traffic network.

Also, the traffic network modeling apparatus 100 may include a database (not shown) that stores or provides data required for the urban traffic network modeling system under the control of the processor 130 . Such a database may be included as a component separate from the memory 110 , or may be built in some area of the memory 110 .

2 is a diagram illustrating a city traffic network modeling process using multiple CCTV videos in an embodiment of the present invention.

2, the processor 130 collects video from the multiple CCTV networks 101, and samples the video by adjusting the sampling frequency according to the speed change of the vehicle (S201), and based on a preset vehicle tracking algorithm Generates vehicle flow data 102 from the sampled video (S202), and combines the generated vehicle flow data 102 and multiple CCTV networks 101 including each CCTV location information 103 to create an observing node and a non-observing node Generates an incomplete traffic network identified by ( S203 ), and generates missing data including the first unobserved data and the second unobserved data through the missing data prediction model 104 , and then the vehicle flow data for the observation node And it is possible to model the urban traffic network in which missing data for non-observation nodes are merged (S204).

In this case, the first non-observed data is vehicle flow data of roads that are not observed by CCTV among roads connected to the intersection, and the second non-observed data is vehicle flow data of intersections and connected roads without CCTV. For example, 95% of CCTV cameras in Seoul only record some areas of intersections. Therefore, the flow of vehicles on some roads connected to the intersection is unknown in the initial traffic network. This means that it is difficult to model spatiotemporal traffic flows with incomplete networks with missing data (non-observational data). On the other hand, one city road is connected to an adjacent road at an intersection. Also, the traffic flow on one road is affected by the traffic flow on the adjacent road. Accordingly, the present invention can estimate the traffic flow of an unobserved road by referring to the traffic flow of an adjacent road. In addition, in order to predict future urban traffic changes, a traffic network corresponding to unobserved data may be trained using the missing data prediction model 104 . In this case, the missing data prediction model 104 may be trained through a diffusion convolutional recurrent neural network (DCRNN).

3A is a diagram for explaining a sampling method of a real-time CCTV video that adapts to a traffic flow change amount according to an embodiment of the present invention.

Figure 3b is a graph showing a sampling result according to the adaptive CCTV video sampling module according to an embodiment of the present invention.

Meanwhile, CCTV videos installed at intersections show a pattern in which vehicle flow regularly changes according to the signal cycle. In the non-vehicle area, changes in vehicle flow are not affected by regular patterns such as signal cycles. In other words, the flow of vehicles also changes very irregularly due to abnormal events such as traffic accidents. Therefore, the sampling frequency of each CCTV video can be dynamically changed with traffic conditions and time. For example, while looking at CCTV images in Seoul, the CCTV camera may irregularly film other areas according to the decision of the traffic monitoring officer. Accordingly, when the CCTV camera rotates, the traffic network modeling apparatus 100 of the present invention may determine a new sampling frequency by monitoring a new traffic condition. That is, while optimizing the sampling frequency, it is possible to determine the sampling frequency according to whether the recording area of the CCTV camera changes. With such an adaptive CCTV video sampling module, the present invention does not need to collect all streaming video, which can prevent overload of the system and is efficient.

Referring to FIG. 3A , the processor 130 determines whether to sample real-time video by the adaptive CCTV video sampling module, but the adaptive sampling module includes a sampling time (STn), a sampling period (SPn) and Includes sampling interval (SIn).

Specifically, the adaptive CCTV video sampling module may determine whether to sample streaming video according to traffic conditions change. For example, the sampling time STn may be determined according to Equation (1).

[Equation 1]

Here, αn is +1 or -1 according to a change in traffic conditions, and is as follows.

En is the entropy of velocity with respect to vehicle speed during SPn (sampling period). Illustratively, if SPmin is set to 1 min and SPmax is set to 7 min, since there is no traffic information when the system starts up, the system generates video from S1 and S2 at 0 min and 2×SPmax for the initial sampling period SPmax, respectively. is sampled twice. Then, the system adjusts the sampling time (STn), the sampling period (SPn) and the sampling interval (SIn) as defined in Equation (1).

As an example, entropy may be calculated as a probability of vehicle speed. For example, a histogram of vehicle speed can be created. The interval (bin) of the histogram is 5 km=h, and the number of vehicles corresponding to the speed of each histogram section may be set to xi. Therefore, P(xi) is the probability that a specific speed is observed, and the probability can be calculated by dividing the number of vehicles in each section by the total number of vehicles observed in the CCTV video. Therefore, a high entropy value can be obtained when the vehicle speed changes more frequently.

Referring to FIG. 3B , the gray line represents the speed of the original unsampled video stream and the yellow dot represents the observed vehicle speed in the sampled video. This indicates that video sampling occurs more frequently during 9-11 AM and 5-8 PM, when vehicle speed changes are more frequent, while the sampling interval is longer between 12 PM and 5 PM, when the speed is relatively uniform. .

That is, in the adaptive CCTV video sampling module of the present invention, when the change in vehicle speed is large, the sampling frequency is shortened, and when the change in vehicle speed is small, the sampling frequency can be longer.

The processor 130 performs a plurality of complementary vehicle tracking algorithms in parallel to extract vehicle flow data, and the vehicle flow data includes vehicle speed, moving direction, and traffic volume according to time.

Specifically, the processor 130 may use the YOLOv3 model to identify the vehicle position in two-dimensional coordinates in each video frame space. In addition, the SORT model can be applied to track the two-dimensional coordinates of the vehicle detected in the video frame and extract the vehicle trajectory. In addition, it is possible to estimate the vehicle speed, the vehicle moving direction, and the amount of traffic (the number of vehicles) from the vehicle track. Because we estimate the data from sampled streaming video, the estimated vehicle speed and traffic volume are not continuous. Therefore, spline interpolation can be applied to approximate the vehicle speed and traffic volume during the unobserved period.

Illustratively, the vehicle tracking algorithm may generate vehicle flow data from CCTV video. As an example, the vehicle tracking algorithm may use YOLOv3, SORT, and FgSegNet models.

For example, YOLOv3 can be used to detect vehicle coordinates per frame in streaming video. The SORT algorithm can extract vehicle trajectories from vehicle coordinates. That is, based on the vehicle trajectory, the vehicle speed, the traffic amount, and the vehicle direction may be calculated. Spline interpolation approximates vehicle speed and traffic volume during unobserved periods due to video sampling. That is, spline interpolation can approximate data values in unobserved time using the observed sampling points from the above-described adaptive CCTV video sampling module.

On the other hand, it is possible to update the learning data to capture sufficient traffic information by extracting the vehicle image not detected by YOLOv3. For example, vehicle coordinates can be detected by YOLO and FgSegNet models in streaming video. SORT algorithm can estimate vehicle trajectory from vehicle coordinates detected by two vehicle detection models. That is, it compares the estimated vehicle trajectories of the two models and extracts the vehicles that the YOLO model cannot detect. The undetected vehicle image may be added to the training data to update the training data.

In addition, in order to detect a vehicle that the YOLOv3 model cannot detect, the processor 130 first detects a vehicle through the YOLOv3 model and extracts a vehicle location set only from the detected vehicle. Then, to identify undetected vehicles, the FgSegNet model can be used to remove the foreground of the video and extract another set of vehicle positions with only the area considered to contain the vehicle. Next, the SORT model can be applied to two sets of locations to track the vehicle trajectory. The two sets of trajectories can then be compared to identify missing trajectories in the trajectories sets detected by YOLOv3 and SORT. And it is possible to update the training data for YOLOv3 by capturing an image of a vehicle with missing tracks and adding the captured image to the undetected vehicle image candidate. Accordingly, the traffic network modeling apparatus 100 of the present invention may provide a module through which the user selects a vehicle image and updates the learning data through the user interface 141 .

In the case of traffic, it can be calculated based on the number of tracks extracted by the SORT algorithm. To estimate vehicle speed, two-dimensional coordinates in video frame space can be transformed into ground coordinates. Orbital distances can be calculated from two or more fixed ground coordinates and a homographic transformation that requires the actual distance between the coordinates. For example, a traffic CCTV camera in Seoul records at 30 fps, which translates to about 0.033 seconds per frame. Accordingly, the vehicle speed can be calculated using the track distance and frame time.

4 is a diagram for explaining that a vehicle direction is determined according to a basic direction and a lane area of a CCTV network according to an embodiment of the present invention.

4, the processor 130 records the peripheral CCTV ID (E/W/S/N) connected to each CCTV according to the basic direction set in the CCTV network of (a). On the two-way road in (b), all vehicles in one lane are moving in the same direction. On the three two-way roads in (C) and four two-way roads in (d), if the vehicle is observed in the green zone, the vehicle track can be oriented.

Illustratively, as shown in (a) of FIG. 7 , the processor 130 may map a default direction to each edge of the CCTV network and define north in each streaming video. By extracting the road from the streaming video and comparing the vehicle trajectory to the two-dimensional coordinates of the road, the direction of the vehicle can be estimated. 7 (b), (c) and (d), assuming that there is no U-turn, the vehicle cannot change the driving direction to the opposite direction. Therefore, all trajectories mapped to one road can travel in the same direction. As illustrated in (c) and (d) of FIG. 7 , when the vehicle cannot change the driving direction, the coordinates of the green area may be extracted. If a vehicle trajectory is observed in one of the green areas, the vehicle direction can be easily determined. However, if the vehicle track is not mapped to the green zone, it means that the vehicle has not yet entered the intersection. Accordingly, the processor 130 cannot determine in which direction the vehicle is moving, and in this case, the vehicle flow data does not include the vehicle direction.

5 is a diagram for explaining a DCRNN process for estimating a traffic flow for first unobserved data according to an embodiment of the present invention.

6 is a diagram for explaining a DCRNN process for estimating a traffic flow for second unobserved data according to an embodiment of the present invention.

Referring to FIG. 5 , when the processor 130 estimates missing data for the first non-observed data, as shown in FIG. It is possible to create an incomplete traffic network comprising corresponding road edges, non-observed nodes corresponding to locations that CCTV cannot observe. Subsequently, as shown in FIG. 5B , the incomplete first virtual network may be created by adding a primary virtual node and a primary virtual road edge to the incomplete transportation network. Next, as shown in (c) of FIG. 5 , an input network for DCRNN learning may be generated excluding the non-observing node and the road edge connected to the non-observing node. And, as shown in (d) of FIG. 5 , it is possible to generate a DCRNN output network that has learned the missing data for the primary virtual node. Next, as shown in FIG. 5E , a second virtual network in which an observation node and a road edge are added may be generated based on the primary virtual node adjacent to the non-observed node.

Then, referring to FIG. 6 , when the processor 130 estimates missing data for the second non-observed data, as shown in FIG. Non-observation nodes can be added. And as shown in (b) of FIG. 6 , a third virtual network including a secondary virtual node and a secondary virtual road edge is created, and an input network for DCRNN learning is generated excluding the road edge connected to the non-observing node. can do. Next, as shown in FIG. 6C , a DCRNN output network that has learned missing data for the secondary virtual node may be generated. Finally, as shown in (d) of FIG. 6 , a complete urban transportation network in which missing data is estimated may be generated by removing the primary virtual node and the secondary virtual node.

For example, the processor 130 may generate a CCTV network including a node (CCTV camera location) and an edge (road) between the CCTV camera and an adjacent node to secure a city traffic network. Thereafter, as shown in (a) of FIG. 5 , an incomplete traffic network may be created by integrating the vehicle flow data and the CCTV network. In an incomplete transport network, each node represents a road connected to an intersection, and an edge represents the distance between two adjacent nodes. Traffic outflow and speed from intersection to road can be used for node data.

Since some CCTV cameras do not record the entire intersection, an incomplete traffic network may consist of observing nodes and non-observing nodes where traffic flows are not observed. For example, in an urban area, one road connects to two or more intersections, and the observed traffic flow on the road may affect the traffic flow of the connected intersection and adjacent roads. Therefore, the traffic volume of the unobserved road can be estimated by referring to the traffic volume observed on the adjacent road.

Illustratively, extended DCRNN modeling a spatiotemporal spreading process with graph data can be applied to estimate unobserved nodes. The extended DCRNN model defines an incomplete transport network as G = (V, E, W). where V is the set of nodes and |V|=N. E is the set of edges. W is the weighted adjacency matrix representing the distance and weight between nodes. X(t) is the input feature matrix at time t, where X(t) ∈RN × M. N is the number of nodes and M is the number of features. Given G and [X(t-T + 1), ..., X(t)], to predict the vehicle speed and traffic in [X(t + 1), ..., X(t + T)] A function f can be trained on the extended DCRNN.

In addition, the processor 130 may generate an incomplete first virtual network as shown in FIG. 5 ( b ) for efficient DCRNN model learning. The incomplete first virtual network may be composed of an incomplete transportation network, virtual nodes, and virtual edges. The virtual node and the virtual edge can be used to determine the traffic volume (first unobserved data) of the road that is not observed at the intersection during DCRNN model training. Each virtual node virtually represents the traffic flow on two adjacent roads at the intersection. The feature matrix of the virtual node is calculated by convolution of the feature matrices of two adjacent nodes. For the weighted adjacency matrix W, the weight is set to 1 if the two observation nodes are directly connected, and 0 otherwise. In addition, the weight between the virtual node and the observation node can be set as follows. The weight between the virtual node and two adjacent observation nodes may be set to 1.5. The weight between virtual nodes within the intersection (intersection) may be set to 1.2. A weight between virtual nodes directly connected to adjacent virtual nodes outside the intersection may be set to 1. The weight between virtual nodes and unconnected nodes may be set to zero. It is not limited to the above-described weights, and the weight set may be changed.

And, referring to FIG. 5C , the processor 130 may generate (exclude) non-observed nodes and edges connected to observation nodes after generating the incomplete first virtual network. Then, the sorted incomplete first virtual network can be used as the input network of the DCRNN model. The DCRNN model learns the input network shown in Fig. 5(c) using the aforementioned T sequence [X(t-T + 1), ..., X(t)], and As, we can predict the output network for the future T sequence [X(t-T + 1), ..., X(t)]. In this case, in (b) of FIG. 5 , the non-observing node may be learned through two adjacent virtual nodes. Accordingly, the second virtual network shown in FIG. 5E may be generated by replacing the feature matrix of the unobserved node with the average of two feature matrices corresponding to two adjacent virtual nodes.

As described above, the processor 130 may estimate the missing data (vehicle flow data) for the first non-observation data, and then estimate the missing data for the second non-observation data.

Exemplarily, referring to (a) of FIG. 6 , the processor 130 may estimate the road of four non-observing nodes and edges in the second virtual network shown in FIG. 5(e). As shown in FIG. 6B , virtual nodes and edges are re-created, and each virtual node may be connected to an adjacent virtual node connected to the intersection. After the third virtual network is created by setting the feature matrix and the neighbor weight matrix of the virtual node in the same manner as in the process shown in FIG. 5 , the DCRNN input network may be created. Thereafter, as shown in FIG. 6C , a complete third virtual network (output network) may be created by creating a virtual node and an edge and then applying DCRNN. Finally, as shown in Fig. 6(d), in order to create a complete urban traffic network, the virtual node created in the process shown in Figs. 5 and 6 may be removed.

7 is an example of a user interface for displaying urban traffic network modeling and predicted traffic volume according to an embodiment of the present invention.

Referring to FIG. 7 , the traffic network modeling apparatus 100 may provide a user interface 141 displaying a predicted traffic amount through the display unit 140 in response to a user's request for urban traffic network modeling. Exemplarily, the user interface 141 includes a first area (a) for displaying a real-time video sampling state, a second area (b) for providing a real-time traffic CCTV screen, and a third area (c) for controlling the direction movement of the video. ), a fourth area (d) that displays a vehicle image that is not detected by the vehicle tracking algorithm, a fifth area (e) that provides a predictive traffic volume monitoring map linking map information corresponding to CCTV location information and a city traffic network; The sixth area (f) where identification items of traffic speed, traffic volume, congestion type, and traffic propagation type reflected in the monitoring map are displayed, and the seventh area (g) which provides a time slider to control the predicted traffic volume displayed on the monitoring map , interlocking with the fifth area (e) and providing urban traffic network modeling including a plurality of observation nodes and non-observation nodes in the eighth area (h), and the fifth area (e) and eighth area (h) When the displayed observation node is clicked, it may include a ninth area (i) that provides a real-time CCTV video screen corresponding to the observation node.

Illustratively, the user interface 141 may include first to fourth areas (a)-(d) supporting vehicle flow data generation in multiple CCTV videos, and fifth to fourth areas supporting modeling and analysis of traffic networks. It includes 9 areas (e)-(i).

Referring to FIG. 7 , the first region (a) may provide a sampling state of a real-time streaming video. As an example, the processor 130 may display the real-time state of CCTV sampling in a table so that it can be identified by color. The sampling state consists of four states, blue indicates a video sampling state, green indicates video pending, and the processor 130 may wait until the next sampling. Red indicates that the recording direction of the CCTV has been changed, and when the recording direction of the CCTV is changed, the default direction of the CCTV can be reset. After selecting the CCTV ID from the table of the first area (a), you can check the CCTV video in the second area (b) and define the north side of the video in the third area (c). The fourth region d may provide a vehicle candidate image that cannot be detected through a vehicle detection algorithm. A vehicle image may be selected to update learning data for improving vehicle detection through the fourth region (d). The fifth area (e) provides a predicted traffic volume monitoring map, and may display identification items including traffic speed, traffic volume (volume), congestion type, and traffic propagation type. The seventh area (f) may provide an identification item visualized through the monitoring map. For example, in a traffic network, a node represents an intersection and an edge represents a road. Nodes may be displayed in white if there are CCTV cameras at the intersection and gray if there are no CCTV cameras at the intersection. If the CCTV camera is only recording part of the intersection, the system can only display the road captured by the CCTV camera in white. The color of the edge represents vehicle flow as shown in area 7 (f), and the traffic network can be encoded by vehicle speed, traffic volume, congestion type and radio waves. The eighth area (g) is a time slider, and may be provided by monitoring traffic conditions for the past 30 minutes with respect to the current time and analyzing expected traffic conditions for the next 30 minutes.

In the fifth area (e), there is no vehicle flow on some roads of the traffic network. In the eighth area (h), it is possible to identify potentially predictable roads in an incomplete traffic network. A gray node in the eighth area (h) indicates an intersection without a CCTV camera, and a white node indicates an intersection with a CCTV camera. A gray edge means that all vehicle flow can be observed in both directions on the road. A blue edge means that vehicle flow in one direction cannot be observed on the road, and a red edge indicates that vehicle flow cannot be observed in both directions on the road. In the eighth area (h), if you brush the network including the node and edge you want to estimate and click the estimate button (Estimation), the processor 130 displays the vehicle flow (first and second unobserved data). In the process of estimating the vehicle flow, the processor 130 needs to train the DCRNN model, and when the DCRNN model needs to be trained, it may take time depending on the size and complexity of the generated traffic network. When the unobserved vehicle flow on the road is estimated, the processor 130 may update the traffic network predicted in the fifth area (e). When a node is clicked in the fifth area (e) and the eighth area (h), the processor 130 may play a real-time CCTV streaming video in the ninth area (i). That is, the user can repeat the modules of the fifth to ninth areas (e)-(i) through the user interface 141 to estimate a road that is not observed in the traffic network and analyze the urban traffic situation. .

Hereinafter, a description of the configuration performing the same function among the configurations shown in FIGS. 1 to 7 will be omitted.

8 is a flowchart illustrating a method for modeling a city traffic network using multiple CCTV videos according to an embodiment of the present invention.

Referring to FIG. 8, the urban traffic network modeling method using multiple CCTV video collects video from the multiple CCTV network 101, and controls the sampling frequency according to the speed change of the vehicle to sample the video (S110), Generating vehicle flow data from the sampled video based on the set vehicle tracking algorithm (S120), combining the generated vehicle flow data and multiple CCTV networks including each CCTV location information to identify an observing node and a non-observing node. After generating the traffic network (S130) and missing data including the first unobserved data and the second unobserved data through the missing data prediction model, the vehicle flow data for the observation node and the vehicle flow data for the unobserved node Modeling the city traffic network in which the missing data is merged (S140).

The first non-observed data is vehicle flow data of roads that are not observed by CCTV among roads connected to the intersection, the second non-observed data is vehicle flow data of intersections and roads connected to the intersection without CCTV, and the missing data prediction model is DCRNN It can be learned through (Diffusion Convolutional Recurrent Neural Network).

In step S110, the adaptive sampling module determines whether to sample the real-time video, but the adaptive sampling module may include a sampling time, a sampling period, and a sampling interval according to a change in vehicle speed.

In step S120, a plurality of complementary vehicle tracking algorithms are performed in parallel to extract vehicle flow data, and the vehicle flow data may include vehicle speed, movement direction, and traffic volume according to time.

In step S130, when estimating the missing data for the first non-observation data, the observation node corresponding to the CCTV location, the road edge corresponding to the road adjacent to the CCTV location, and the non-observation node corresponding to the location that the CCTV cannot observe Create an incomplete transportation network including Create an input network for training, create a DCRNN output network trained on missing data for a primary virtual node, and add an observation node and a road edge based on a primary virtual node adjacent to an unobserved node. You can create a network.

Then, when estimating missing data for the second non-observation data, a non-observation node corresponding to a location where there is no CCTV is added to the second virtual network, and a third virtual node including a secondary virtual node and a secondary virtual road edge is added. Create a virtual network, create an input network for DCRNN training excluding road edges connected to non-observation nodes, create a DCRNN output network trained on missing data for a secondary virtual node, and create a primary virtual node and secondary By eliminating virtual nodes, it is possible to create a complete city transit network with an estimated missing data.

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module to be executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: traffic network modeling device
110: memory
120: communication module
130: processor
140: display unit
141: user interface

Claims (13)

In the urban traffic network modeling method using multiple CCTV videos performed by the device,
(a) collecting video from multiple CCTV networks, sampling the video by adjusting the sampling frequency according to the speed change of the vehicle;
(b) generating vehicle flow data from the sampled video based on a preset vehicle tracking algorithm;
(c) generating an incomplete traffic network in which observation nodes and non-observation nodes are identified by combining the generated vehicle flow data and the multiple CCTV networks including each CCTV location information; and
(d) After generating missing data including the first unobserved data and the second unobserved data through the missing data prediction model, the vehicle flow data for the observation node and the missing data for the unobserved node are merged modeling an urban transport network; including,
The first non-observed data is vehicle flow data of roads that are not observed by CCTV among roads connected to the intersection,
The second non-observation data is vehicle flow data of an intersection without CCTV and a road connected to the intersection,
The missing data prediction model will be learned through DCRNN (Diffusion Convolutional Recurrent Neural Network),
A method of modeling a city traffic network using multiple CCTV videos.
According to claim 1,
In step (a) above
determine whether real-time video is to be sampled by the adaptive sampling module,
wherein the adaptive sampling module includes a sampling time, a sampling period, and a sampling interval according to a change in vehicle speed,
A method of modeling a city traffic network using multiple CCTV videos.
According to claim 1,
In step (b)
A plurality of complementary vehicle tracking algorithms are performed in parallel to extract the vehicle flow data,
The vehicle flow data will include vehicle speed, moving direction, and traffic volume over time,
A method of modeling a city traffic network using multiple CCTV videos.
According to claim 1,
In step (d),
When estimating missing data for the first non-observation data,
(d-1) generating the incomplete traffic network comprising an observation node corresponding to a CCTV location, a road edge corresponding to a road adjacent to the CCTV location, and a non-observing node corresponding to a location that the CCTV cannot observe,
(d-2) creating an incomplete first virtual network by adding a first virtual node and a first virtual road edge to the incomplete transportation network;
(d-3) generating an input network for learning the DCRNN except for the non-observing node and a road edge connected to the non-observing node,
(d-4) generating a DCRNN output network that has learned the missing data for the primary virtual node,
(d-5) generating a second virtual network to which an observation node and a road edge are added based on the primary virtual node adjacent to the non-observed node;
A method of modeling a city traffic network using multiple CCTV videos.
5. The method of claim 4,
In step (d),
When estimating missing data for the second non-observation data,
(d-6) adding a non-observing node corresponding to a location where there is no CCTV in the second virtual network,
(d-7) creating a third virtual network including a secondary virtual node and a secondary virtual road edge, and generating an input network for learning the DCRNN excluding the road edge connected to the non-observing node,
(d-8) generating a DCRNN output network that has learned the missing data for the secondary virtual node,
(d-9) removing the primary virtual node and the secondary virtual node to generate a complete city traffic network in which the missing data is estimated;
A method of modeling a city traffic network using multiple CCTV videos.
6. The method of claim 5,
In accordance with the urban traffic network modeling request, providing a user interface for displaying the predicted traffic volume,
The user interface is
a first area displaying the real-time video sampling status;
The second area that provides real-time traffic CCTV screens,
a third area to control the direction movement of the video;
a fourth area displaying a vehicle image not detected by the vehicle tracking algorithm;
A fifth area that provides map information corresponding to the CCTV location information and a predicted traffic volume monitoring map linking the city traffic network;
A sixth area in which identification items of traffic speed, traffic volume, congestion type, and traffic propagation type reflected in the monitoring map are displayed;
A seventh area providing a time slider for controlling the predicted traffic amount displayed on the monitoring map;
an eighth area interworking with the fifth area and providing the urban traffic network modeling including a plurality of observation nodes and non-observation nodes; and
When the observation node displayed in the fifth area and the eighth area is clicked, a ninth area that provides a real-time CCTV video screen corresponding to the observation node is included,
A method of modeling a city traffic network using multiple CCTV videos.
In the urban traffic network modeling device using multiple CCTV video,
a memory in which a city traffic network modeling method program is stored;
A processor for executing the program stored in the memory;
The processor collects video from multiple CCTV networks by executing the program, but samples the video by adjusting the sampling frequency according to the speed change of the vehicle,
generate vehicle flow data from the sampled video based on a preset vehicle tracking algorithm;
Combining the generated vehicle flow data and the multiple CCTV networks including each CCTV location information to create an incomplete traffic network in which observation nodes and non-observation nodes are identified,
After generating missing data including the first unobserved data and the second unobserved data through the missing data prediction model, the vehicle flow data for the observation node and the missing data for the unobserved node are merged in an urban transportation network model, but
The first non-observed data is vehicle flow data of roads that are not observed by CCTV among roads connected to the intersection,
The second non-observation data is vehicle flow data of an intersection without CCTV and a road connected to the intersection,
The missing data prediction model will be learned through DCRNN (Diffusion Convolutional Recurrent Neural Network),
Urban traffic network modeling device using multiple CCTV videos.
8. The method of claim 7,
the processor is
determine whether real-time video is to be sampled by the adaptive sampling module,
wherein the adaptive sampling module includes a sampling time, a sampling period, and a sampling interval according to a change in vehicle speed,
Urban traffic network modeling device using multiple CCTV videos.
8. The method of claim 7,
the processor is
A plurality of complementary vehicle tracking algorithms are performed in parallel to extract the vehicle flow data,
The vehicle flow data will include vehicle speed, moving direction, and traffic volume over time,
Urban traffic network modeling device using multiple CCTV videos.
8. The method of claim 7,
the processor is
When estimating missing data for the first non-observation data,
generating the incomplete traffic network comprising an observing node corresponding to a CCTV location, a road edge corresponding to a road adjacent to the CCTV location, and a non-observing node corresponding to a location not observable by the CCTV;
creating an incomplete first virtual network by adding a first virtual node and a first virtual road edge to the incomplete transportation network;
generating an input network for learning the DCRNN except for the non-observing node and a road edge connected to the non-observing node,
generating a DCRNN output network that has learned the missing data for the primary virtual node,
generating a second virtual network to which an observation node and a road edge are added based on the primary virtual node adjacent to the non-observation node;
Urban traffic network modeling device using multiple CCTV videos.
11. The method of claim 10,
the processor is
When estimating missing data for the second non-observation data,
adding a non-observing node corresponding to a location where there is no CCTV in the second virtual network;
Create a third virtual network including a secondary virtual node and a secondary virtual road edge, and create an input network for DCRNN learning excluding road edges connected to the non-observing node,
generating a DCRNN output network that has learned the missing data for the secondary virtual node,
removing the primary virtual node and the secondary virtual node to create a complete city traffic network in which the missing data is estimated;
Urban traffic network modeling device using multiple CCTV videos.
12. The method of claim 11,
the processor
In accordance with the urban traffic network modeling request, providing a user interface for displaying the predicted traffic volume,
The user interface is
a first area displaying the real-time video sampling status;
The second area that provides real-time traffic CCTV screens,
a third area to control the direction movement of the video;
a fourth area displaying a vehicle image not detected by the vehicle tracking algorithm;
A fifth area that provides map information corresponding to the CCTV location information and a predicted traffic volume monitoring map linking the city traffic network;
A sixth area in which identification items of traffic speed, traffic volume, congestion type, and traffic propagation type reflected in the monitoring map are displayed;
A seventh area providing a time slider for controlling the predicted traffic amount displayed on the monitoring map;
an eighth area interworking with the fifth area and providing the urban traffic network modeling including a plurality of observation nodes and non-observation nodes; and
When the observation node displayed in the fifth area and the eighth area is clicked, a ninth area that provides a real-time CCTV video screen corresponding to the observation node is included,
Urban traffic network modeling device using multiple CCTV videos.
A non-transitory computer-readable recording medium in which a computer program for performing the urban traffic network modeling method according to claim 1 is recorded.

Images