KR102561656B1 - Control system of traffic flow with sensing of vehicle based on deep learning - Google Patents

Abstract

The present invention is configured to analyze images captured by cameras to generate object information and traffic information, create an optimal display system according to the generated traffic information, and then control traffic signals according to the generated optimal display system. As a result, not only can the vehicle congestion rate and waiting time be significantly reduced, but also can reduce social costs by effectively reducing fuel consumption and pollutant gas emissions accordingly, and by analyzing input images using a deep-learning algorithm, objects Deep learning-based object detection that can dramatically increase the accuracy and reliability of analysis by improving the recognition rate, and can further increase the object recognition rate by applying the YOLO model based on the Convolution Neural Network (CNN) as a deep-learning algorithm It relates to a vehicle flow control system using

Description

Vehicle flow control system using deep-learning based object sensing {Control system of traffic flow with sensing of vehicle based on deep learning}

The present invention relates to a vehicle flow control system using deep-learning-based object detection, and more particularly, by analyzing images obtained by camera shooting using a deep-learning basis to detect vehicles, pedestrians, etc. After detecting object analysis information, it is configured to establish a signal system based on the detected object analysis information, thereby increasing the accuracy and reliability of object detection and response, significantly reducing vehicle waiting time in the intersection, and cracking down on violating vehicles. It is about a vehicle flow control system using deep-learning based object sensing that can increase the rate.

In recent years, as the vehicle penetration rate has increased and the city center has been expanded, traffic congestion has become a daily routine, causing enormous economic losses. Accordingly, various efforts are being made, such as constructing or expanding roads, to solve traffic congestion. , this method not only requires enormous financial resources, but also has limitations that are difficult to solve in the short term.

Therefore, various studies are being conducted to reduce vehicle waiting time by optimizing the signal system of intersections.

As a conventional representative intersection signal system, a fixed periodic signal operation method that outputs a fixed signal period and present time according to TOD (Time Of Day) using accumulated data detected by vehicle detection means, and collected traffic information It is classified as an active signal operation method that controls signals according to traffic conditions using

In particular, the active signal operation method has the advantage of being able to effectively reduce vehicle congestion by responding flexibly according to the current traffic situation, and thus various studies are being conducted on this.

On the other hand, recently, research on image analysis technology for performing object detection, type, tracking, etc. using deep learning techniques has been actively conducted, and this image analysis technology using deep learning is Since the error rate is reduced by itself through learning, the application field is increasing exponentially due to the advantage of significantly increasing the accuracy and precision of detection.

1 is a block diagram showing an image-based traffic signal control device disclosed in Korean Patent Publication No. 10-2020-0141834 (Title of Invention: Image-based Traffic Signal Control Apparatus and Method).

The image-based traffic signal control device (hereinafter referred to as the prior art) 100 of FIG. 1 acquires an image by photographing a road, generates traffic object analysis data, and uses the generated data to determine the response signal request edge. When receiving traffic object analysis data and response request data from the camera 110 and the edge camera 110, the traffic signal controller 120 controls the traffic lights 131 and 132 on the main road and secondary roads, and the edge It consists of a traffic control center server 140 that stores images collected by the camera transmitted from the camera 110 and traffic object detection results.

In addition, the edge camera 110 has an image sensing unit 111 that photographs an entering vehicle and transmits the captured image to the image recognition/sensitive signal determination unit 112, and traffic objects such as vehicles and pedestrians that are detected and tracked. After generating the object analysis data, a response signal determination unit 112 for determining a response signal request using the generated data, a data storage unit 113 for storing and managing traffic object analysis data, and a traffic signal controller ( 120) and the control center server 140 and the communication unit 114 for transmitting and receiving data.

At this time, the response signal determination unit 112 uses an existing machine learning technique (SVM, Decision Tree, Random Forest, etc.) or a deep neural network object detection algorithm (Faster RCNN, YOLO, SSD, etc.) ), analyzes the video received from ) to detect an object, and generates traffic object analysis data through the detected object information.

The prior art 100 configured as described above has an advantage of increasing the accuracy and precision of object recognition because the sensing signal determining unit 112 recognizes and analyzes image object information through a machine learning algorithm.

However, in the prior art 100, since the edge camera 110 transmits images, which are large amounts of data, to the control center server 140 in real time, assuming that the edge cameras 110 are installed all over the road, the communication load increases. In addition, there is a problem in that data storage efficiency in the control center server 140 is lowered.

In addition, in the prior art 100, machine learning-based image analysis is used only for detecting traffic object analysis data such as traffic volume, queue, point speed, section speed, etc. Failure to do so has a problem in that the control rate is lowered.

In general, when learning a machine learning algorithm, the more the type of recognition target (object type) used for learning is limited, the more the learning effect and object recognition rate increase, but the prior art 100 has no such learning characteristics Since it is not taken into account, learning efficiency and object recognition rate are reduced due to learning of object types that cannot be seen on the road during learning.

In addition, the prior art 100 has a structural limitation in preventing traffic accidents in advance because no technology and method for predicting a collision between a detected vehicle and a vehicle or a collision between a vehicle and a pedestrian is not described.

In addition, the prior art 100 can provide an optimal signal cycle in response to traffic conditions because a specific signal is extended or reduced according to the response signal based on the length of a fixed signal cycle. It has incalculable drawbacks.

In general, image analysis using machine learning algorithms requires complex calculation and image processing, so when image analysis is performed using a single CPU, a load is generated due to excessive calculation processing. There is a trend to use the GPU (Graphic Processing Unit) to perform artificial intelligence-based image analysis and processing.

In the prior art 100, since image analysis and processing are performed in the edge camera 110, assuming that a single CPU is configured, a problem in that overload due to image analysis and processing frequently occurs, and a separate Even if additional GPUs are installed, installation and operation costs increase because expensive GPUs are installed for each camera.

The present invention is to solve this problem, and the problem of the present invention is to analyze images taken by cameras to generate object information and traffic information, and then create an optimal display system according to the generated traffic information. By configuring traffic signals to be controlled according to the generated optimal display system, vehicle congestion rate and waiting time can be remarkably reduced, and fuel consumption and pollutant emissions can be effectively reduced accordingly to reduce social cost consumption. It is to provide a vehicle flow control system using running-based object sensing.

In addition, another problem of the present invention is to improve the object recognition rate by analyzing the input images using a deep-learning algorithm, which can dramatically increase the accuracy and reliability of the analysis. Vehicle flow control using deep-learning-based object detection to provide the system.

In addition, another problem of the present invention is deep-learning based object detection that can further increase the object recognition rate by applying a YOLO model based on a convolutional neural network (CNN) with a deep-learning algorithm Vehicle flow control using object detection to provide the system.

In addition, another problem of the present invention is to further improve the object recognition rate by limiting the recognition target from the conventional 80 types to 5 types that are often seen on the road when learning the deep-learning algorithm. It is to provide a vehicle flow control system using deep-learning based object detection.

In addition, another problem of the present invention is to use deep-learning-based object detection capable of real-time processing and analysis of high-capacity images obtained by shooting cameras by analyzing images using a plurality of GPUs (Graphic Processing Units). It is to provide a vehicle flow control system.

The solution of the present invention for solving the above problems is cameras for acquiring images by photographing at least one road that is a sensing area, a traffic signal controller for controlling the operation of traffic lights installed in the sensing area, and a plurality (M ) In a vehicle flow control system using object sensing including a controller having GPUs (Graphic Processing Units): the controller uses a deep learning-based object analysis algorithm to take pictures of the cameras After analyzing the acquired images and outputting object information including at least one of the location, type, and size of the detected object, traffic information is generated by tracking the outputted object, and based on the generated traffic information After detecting the optimal signal system, a response signal according to the detected optimal signal system is transmitted to the traffic signal controller, and the traffic signal controller controls the traffic lights in response to the response signal received from the controller. a memory storing an image classification table matched with a GPU, which is a target to process images input from each camera; an artificial intelligence learning unit that is executed every predetermined period (T) to learn the object analysis algorithm and stores the learned object analysis algorithm in the memory; an artificial intelligence-based image analysis unit that analyzes images transmitted from the cameras using an object analysis algorithm stored in the memory and outputs object information; an object analysis information generation unit that tracks the object detected by the artificial intelligence-based image analysis unit and generates object analysis information including a lane type, a lane location, a moving speed, and a direction in which the detected object is located; a traffic information generating unit generating traffic information including at least one of queue, waiting time, and traffic volume of each road by using the object analysis information generated by the object analysis information generating unit; a signal system establishing unit that analyzes the traffic information generated by the traffic information generating unit using a preset optimal signal detection algorithm and establishes an optimal display system; A control unit for transmitting a response signal according to the optimal display system established by the signal system establishment unit to the traffic signal controller, and the artificial intelligence-based image analysis unit includes artificial intelligence-based image analysis modules processed by each GPU, , An image classification unit, which is executed when the controller receives images from at least one of the cameras, and classifies images received from the cameras into corresponding GPUs according to matching information stored in the image classification table; a usage monitoring unit that is executed when at least one of the AI-based image analysis modules is operating, detects usage (load) of the AI-based video analysis module in operation, and then stores the detected usage amount in the memory; A data collection module that further includes an image classification table generation/update unit executed every preset first period (T1), wherein the image classification table generation/update unit collects usage data for each GPU during the first period (T1) ; Using the usage data for each GPU collected by the data collection module, the average usage for each GPU ( ) Each GPU average usage calculation module that calculates; The average usage for each GPU calculated by the average usage calculation module for each GPU ( ) And an upper limit setting value (TH) comparison module for comparing a predetermined upper limit setting value (TH), which means the minimum amount of usage that can be determined as overload in the GPU; In the upper limit setting value (TH) comparison module, 1) average usage amount less than the upper limit setting value (TH) ( ) is removed from the inspection candidates, but 2) the average usage above the upper limit setting value (TH) ( ) is detected, an inspection candidate determination module for determining the GPU as an inspection candidate; It is executed when the inspection candidate is determined in the inspection candidate determination module, and the average usage ( ) a change target determination module for determining the lowest GPU as a change target; After determining which one of the plurality of images input to the GPU determined as an inspection candidate in the inspection candidate determination module is input to the GPU determined as a change target in the change target determination module, an image classification table is created so that the determined matching information is reflected. and an image classification table updating module for updating, wherein the image classification unit classifies images received from at least one of the cameras according to the image classification table updated every first period (T1), and the image classification unit an image input module that receives images (including camera identification information) transmitted from the cameras; a comparison and determination module for comparing the total number (N) of the cameras and the total number (M) of the GPUs (Graphic Processing Units); A one-to-one based classification module that is executed when the total number of cameras (N) in the comparison and determination module is less than (N ≤ M) of the total number of GPUs (M), and inputs the input images to each of the GPUs in a one-to-one manner; In the comparison and determination module, it is executed when the total number of cameras (N) exceeds the total number of GPUs (M) (N > M), and an image classification table-based classification module that classifies images according to the image classification table stored in the memory. is to include

delete

In addition, in the present invention, it is preferable that the object analysis algorithm is applied with a Convolution Neural Network (CNN)-based YOLO model.

In the present invention, it is preferable that the artificial intelligence learning unit sets a car, a bus, a motorcycle, a truck, and a human body as objects to be recognized when learning the object analysis algorithm.

In addition, in the present invention, the vehicle flow control system using the object response further includes a control center server, and the controller includes object analysis information generated by the object analysis information, location information of a preset detection area, and current signal system. Violation vehicle enforcement unit for detecting the violation vehicle by utilizing; A number recognition unit for recognizing a vehicle number from an image of a vehicle entering a preset section of the detection area is further included, and when the violation vehicle is detected by the violation vehicle control unit, the control unit controls an image including the violation vehicle. It is preferable to match the license plate number of the violating vehicle and the violating information and transmit the information to the control center server.

delete

delete

delete

According to the present invention having the above problems and solutions, images captured by cameras are analyzed to generate object information and traffic information, and an optimal display system is generated according to the generated traffic information. By being configured to control the traffic signal according to the traffic signal, it is possible to significantly reduce vehicle congestion rate and waiting time, as well as reduce social costs by effectively reducing fuel consumption and pollutant gas emissions.

In addition, according to the present invention, the accuracy and reliability of the analysis can be dramatically increased by improving the object recognition rate by analyzing the input images using a deep-learning algorithm.

In addition, according to the present invention, the object recognition rate can be further increased by applying a YOLO model based on a Convolution Neural Network (CNN) as a deep-learning algorithm.

In addition, according to the present invention, when learning the deep-learning algorithm, the object recognition rate can be further improved by limiting the object to be recognized from the conventional 80 types to 5 types frequently seen on the road.

In addition, according to the present invention, real-time processing and analysis of high-capacity images acquired by shooting of cameras are possible by analyzing images using a plurality of GPUs.

1 is a block diagram showing an image-based traffic signal control device disclosed in Korean Patent Publication No. 10-2020-0141834 (Title of Invention: Image-based Traffic Signal Control Apparatus and Method).
2 is a configuration diagram illustrating a vehicle flow control system using deep learning-based object sensing, which is an embodiment of the present invention.
FIG. 3 is a conceptual diagram of FIG. 2 .
FIG. 4 is an exemplary view for explaining FIG. 2 .
5 is another exemplary view of FIG. 4 .
6 is a block diagram illustrating the controller of FIG. 2 .
7 is a conceptual diagram illustrating a YOLOv4 model applied to the object analysis algorithm of the artificial intelligence learning unit of FIG. 6 .
9 is an exemplary diagram for explaining a CNN applied to the object analysis algorithm of FIG. 7 .
10 is a conceptual diagram of FIG. 9 .
FIG. 11 is an exemplary diagram illustrating tail-biting violation vehicles detected by the violation vehicle enforcement unit of FIG. 6 .
12 is an exemplary diagram for explaining the number recognition unit of FIG. 6 .
FIG. 13 is an exemplary diagram illustrating the image classification table generation/update unit of FIG. 6 .

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

2 is a configuration diagram illustrating a vehicle flow control system using deep learning-based object sensing, which is an embodiment of the present invention, FIG. 3 is a conceptual diagram of FIG. 2, FIG. 4 is an exemplary view for explaining FIG. 2, and FIG. 5 is another exemplary view of FIG. 4 .

The vehicle flow control system 1 using object sensing, which is an embodiment of the present invention of FIGS. 2 to 4, analyzes images obtained by camera shooting using a deep learning basis to detect vehicles, pedestrians, etc. After detecting object analysis information, it is configured to establish a signal system based on the detected object analysis information, thereby increasing the accuracy and reliability of object detection and response, significantly reducing vehicle waiting time in the intersection, and cracking down on violating vehicles. to increase the rate.

In addition, as shown in FIGS. 2 to 4, the vehicle flow control system 1 using object sensing according to the present invention includes cameras 5-1, ..., (5-N) that capture each lane group of an intersection. ), and object analysis information is detected by analyzing the images acquired by the shooting of each camera 5 using a deep learning-based object analysis algorithm, and then a corresponding object analysis information is detected. A controller 3 that generates an optimal signal system and transmits a response signal according to the optimal signal system to a traffic signal controller 7 to be described later, and a traffic light according to the response signal according to the optimal signal system received from the controller 3 A traffic signal controller 7 that controls the operation of (8), display means 11 installed on a road shoulder or a traffic light support to display a preset warning text under the control of the controller 3, and the controller 3 A communication network ( 10).

At this time, in FIGS. 2 to 5, for convenience of explanation, it has been described that the road to which the signal control target is applied is an intersection (S) as an example, but the target to which the present invention is applied is not limited to the intersection (S), and is a single lane It is natural that it can be applied to various roads such as

In addition, although not shown in the present invention for convenience of description, the controller 3 includes the cameras 5-1, ..., 5-N, the traffic signal controller 7 and the display unit 11. It is composed of a wired cable or network and can transmit and receive data.

The communication network 10 supports data communication between the controller 3 and the control center server 9, and in detail, a wide area network (WAN), a LAN (local area network) network, a VAN (Value Added Network) network, It may be implemented in a wired communication network or the like.

The traffic lights 8 are installed at the intersection S to guide the movement direction of vehicles or pedestrians, and can be generally classified into traffic lights for vehicles and traffic lights for pedestrians.

In addition, the traffic lights 8 receive control signals according to the signal system of the traffic signal controller 7, and turn on and off.

The display unit 11 outputs a preset warning text or warning light under the control of the controller 3 . For example, the display means 11 may be configured to be installed on the shoulder of a right turn lane and output a preset warning message when a pedestrian is detected when turning right.

The traffic signal controller 7 manages and controls the operation of the traffic lights 8 of the intersection S according to the current signal system.

In addition, when the traffic signal controller 7 receives a response signal according to an optimal signal system from the controller 3, it controls the traffic lights 8 to operate according to the received response signal.

The cameras 5-1, ..., and 5-N are installed to take pictures of each connecting lane S1, S2, S3, and S4 connected to the intersection S. At this time, the cameras (5-1), ..., (5-N) are installed in the direction from the top to the bottom as the cameras (5-1), ..., (5-N) are coupled through a separate auxiliary arm to the upper end of the previously installed traffic light support it is desirable At this time, the cameras 5-1, ..., 5-N are preferably implemented as normal PTZ cameras capable of PTZ (Pan-Tilt_Zoom) control.

In addition, the cameras 5-1, ..., and 5-N transmit the acquired images to the controller 3 by photographing the preset connection lanes.

6 is a block diagram illustrating the controller of FIG. 2 .

As shown in FIG. 6, the controller 3 includes a control unit 30, a memory 31, a communication interface unit 32, an artificial intelligence learning unit 33, an image classification unit 34, and an artificial intelligence based image. Analysis unit 35, usage monitoring unit 36, object analysis information generation unit 37, traffic information generation unit 38, signal system establishment unit 39, violation vehicle control unit 40, number recognition unit ( 41), a warning message output unit 42, and an image classification table generation/update module 43.

At this time, the artificial intelligence-based image analysis unit 35 is composed of a plurality of artificial intelligence-based image analysis modules 35-1, ..., (35-N), and each artificial intelligence-based image analysis module is each GPU ( It is installed in the Graphic Processing Unit) to perform image analysis.

The control unit 30 is an O.S (Operating System) of the controller 3, and the control target 31, (32), (33), (34), (35), (36), (37), (38) , (39), (40), (41), (42), (43) to manage and control.

In addition, the control unit 30 executes the artificial intelligence learning unit 33 at each preset period T, and executes the image classification table generation/update module 43 at each preset first period T1.

In addition, the control unit 30 inputs the images transmitted from the cameras 5-1, ..., (5-N) to the image classification unit 34 through the communication interface unit 32, and the image classification unit ( According to the classification by 34), the images are input to the corresponding AI-based image analysis modules 35-1, ..., 35-M, respectively.

In addition, the control unit 30 inputs the object analysis information generated by the object analysis information generation unit 37 to the traffic information generation unit 38, the violating vehicle control unit 40, and the warning message output unit 42.

In addition, when the optimal signal system is established by the signal system establishment unit 39, the control unit 30 generates a response signal according to the established optimal signal system, and then converts the response signal according to the generated optimal signal system to the traffic signal controller ( 7) to control the communication interface unit 32 to be transmitted.

In addition, when the violating vehicle is detected by the violating vehicle enforcement unit 40, the control unit 30 matches the image of the violating vehicle with the license plate number, and then transmits the matched data to the control center server 9 through the communication interface unit ( 32) to control.

That is, the present invention does not transmit images to the control center server 9 in normal times when no violating vehicle is detected, but transmits images to the control center server 9 only when an event in which a violating vehicle is detected occurs. As the images of are transmitted to the control center server 9 at the same time, it is possible to dramatically solve the problems of communication load, rapid increase in computational throughput, and low data storage efficiency.

The location and identification information of each camera 5 are preset and stored in the memory 31 .

In addition, the location and identification information of each display unit 11 are preset and stored in the memory 31 .

In addition, in the memory 31, the location information of the detection area of each camera 5 and the location information of the intersection S and each connecting lane S1, S2, S3, and S4 are preset and stored. do.

In addition, the object analysis algorithm learned by the artificial intelligence learning unit 33 is stored in the memory 31 .

In addition, the image classification table generated/updated by the image classification table generation/update module 43 is generated in the memory 31 . At this time, the image classification table refers to data matched by a GPU, which is a target to process images input from each camera.

Also, images transmitted from the cameras 5 are temporarily stored in the memory 31 .

In addition, object analysis information and traffic information generated by the object analysis information generation unit 37 and the traffic information generation unit 38 are temporarily stored in the memory 31 .

In addition, the vehicle number recognized by the number recognition unit 41 is temporarily stored in the memory 31 .

7 is a conceptual diagram illustrating a YOLOv4 model applied to the object analysis algorithm of the artificial intelligence learning unit of FIG. 6, and FIG. 9 is an exemplary diagram for explaining a CNN applied to the object analysis algorithm of FIG.

The artificial intelligence learning unit 33 is executed every predetermined period (T) under the control of the control unit 30, and learns an object analysis algorithm based on deep learning, and in detail, the period (T ), using the collected images and preset object types, create learning data that can learn the correlation between images and object types, and use the generated learning data to create parameter values for the correlation between object images and object types. Derive an extraction model, which is a set of

At this time, as shown in FIGS. 7 and 8, the object analysis algorithm takes the input image as input data, but outputs object information (recognition, location, type, size, etc.) of vehicles and pedestrians set in advance from the input image. - It is a learning algorithm.

In addition, as an object analysis algorithm, YOLO based on Convolution Neural Network (CNN), one of the deep neural network object detection algorithms, was applied. 1) YOLO divides each image into S X S grids and calculates the reliability of the grid. By adjusting the position of the bounding box, not only the object recognition rate is excellent, but also has the advantage of real-time processing. 2) CNN is a method typically used in image recognition. Up to C5, the 3D matrix of the segmented image area is analyzed for data processing, and in FC6 and FC7, it is summarized/organized into 2D and used, and the organized matrix is configured to be used for deep learning or image analysis.

As a result of testing in the NVIDIA RTX 2080 Super 2-way environment to evaluate the recognition rate of YOLO, Pretrained YOLOv4 showed the highest average recognition rate (75.67%) for 80 types of objects frequently seen in real life, among which cars, For three types of objects, buses and trucks, the average recognition rate was 80.39%. At this time, the recognition rate was calculated based on the recognition results obtained by forward propagating the model evaluation dataset (MSCOCO 2017) to each AI model, (the number of objects recognized by the AI model / the number of objects present in the image). .

Based on the recognition rate of this YOLO, in the present invention, the object analysis algorithm does not set the recognition target to 80 types as in the prior art, but five types of objects that can be seen on the road, that is, cars, buses, motorcycles, trucks, and the human body. By reducing the setting to , the recognition rate was improved by more than 5%.

In this way, the object analysis algorithm learned by the artificial intelligence learning unit 33 of the present invention is learned by applying CNN-based YOLO, but object recognition and classification by limiting the recognition target to 5 types commonly seen on the road Accuracy and precision can be significantly increased, and accordingly, the accuracy and reliability of establishing an optimal signal system are increased together, thereby dramatically reducing vehicle waiting time.

9 is a block diagram illustrating the image classification unit of FIG. 6 , and FIG. 10 is a conceptual diagram of FIG. 9 .

As shown in FIG. 9, the image classification unit 34 includes an image input module 341, a comparison and judgment module 342, a one-to-one classification module 343, and an image classification table-based classification module 344. .

As shown in FIG. 9, the image classification unit 34 includes an image input module 341, a comparison and judgment module 342, a one-to-one based classification module 343, and an image classification table based classification module 344. .

The image input module 341 receives images (including camera identification information) transmitted from the cameras 5-1, ..., and 5-N.

The comparison and determination module 342 compares the total number N of cameras 5 installed at the intersection S with the total number M of GPUs (Graphic Processing Units) provided in the controller 3, Specifically, it is compared whether the total number of cameras (N) is less than or equal to (N ≤ M) the total number of GPUs (M).

At this time, each GPU is provided with artificial intelligence-based image analysis modules 35-1, ..., (35-M), respectively, and uses the deep-learning-based object analysis algorithm learned by the artificial intelligence learning unit 33. Analyze the input image using

In addition, the comparison and determination module 342 executes the one-to-one based classification module 343 if 1) the total number of cameras (N) is less than or equal to the total number of GPUs (M), and 2) if the total number of cameras (N) is If the number of GPUs (M) is exceeded, the classification module 344 based on the image classification table is executed.

The one-to-one based classification module 343 is executed when the total number of cameras (N) in the comparison and determination module 342 is less than or equal to (N ≤ M) the total number of GPUs (M). That is, the one-to-one based classification module 343 is executed when the image of each camera 5 can be processed by a single GPU 35 .

In addition, the one-to-one based classification module 343 classifies the images by inputting the input images to the GPU (artificial intelligence based image analysis unit) 35, respectively.

That is, the one-to-one based classification module 343 inputs the images acquired by the camera 5 to each GPU 35 on a one-to-one basis.

The image classification table-based classification module 344 is executed when the total number of cameras (N) exceeds the total number of GPUs (M) (N>M) in the comparison and determination module 342. That is, the image classification table-based classification module 344 is executed when at least one of the GPUs 35 has to process two or more images.

In addition, the image classification table-based classification module 344 extracts the image classification table generated/updated by the image classification table generation/update module 43 and stored in the memory 31. At this time, the image classification table refers to data matched by a GPU, which is a target to process images input from each camera.

Also, the image classification table-based classification module 344 classifies the images by inputting the input images to the corresponding GPU 35 according to the extracted image classification table.

The artificial intelligence-based image analysis modules 35-1, ..., (35-M) use the object analysis algorithm learned by the artificial intelligence learning unit 35 to analyze the input image to obtain object information (recognition). , location, type, size, etc.) are output.

The usage monitoring unit 36 is executed when at least one of the AI-based image analysis modules 35-1, ..., and 35-M operates.

In addition, the usage monitoring unit 36 detects usage (load) for image analysis of the artificial intelligence-based image analysis modules 35-1, ..., (35-M) in operation, and stores the detected usage information in memory. Save to (31).

That is, the usage monitoring unit 36 detects in real time the usage amount of each GPU due to operation processing for image analysis and stores it in the memory 31 .

The object analysis information generating unit 37 includes the object information detected by the AI-based image analysis modules 35-1, ..., and 35-M, and the location of the detection area of each camera 5. Information and the location information of the intersection (S) and each connection lane (S1), (S2), (S3), and (S4) are utilized, and the detected objects are tracked at the same time, so that the type of lane and lane where each object is located, Creates object analysis information including movement speed and direction.

At this time, the control unit 30 stores the object analysis information generated by the object analysis information generation unit 37 in the memory 31 and inputs it to the traffic information generation unit 38 and the violation vehicle control unit 40 at the same time.

The traffic information generation unit 38 utilizes and refers to the object analysis information input from the object analysis information generation unit 37 to generate Kyoto information including queue, waiting time and traffic volume of each connecting lane.

At this time, the traffic information generated by the traffic information generation unit 38 is stored in the memory 31 under the control of the control unit 30 and inputted to the signal system establishment unit 39 at the same time.

The signal system establishment unit 39 establishes an optimal signal system based on the traffic information input from the traffic information generation unit 38 using a preset optimal signal detection algorithm.

At this time, since the technique and method for the optimal signal detection algorithm for establishing the optimal signal system based on traffic information is a technique and method commonly used in signal control systems, detailed descriptions thereof will be omitted, and various well-known techniques and methods will be omitted. this may apply.

FIG. 11 is an exemplary diagram illustrating tail-biting violation vehicles detected by the violation vehicle enforcement unit of FIG. 6 .

The violating vehicle control unit 40 includes the object analysis information generated by the object analysis information generation unit 37, the current present information input from the traffic signal controller 7, the intersection S and each connecting lane S1, Using the location information of (S2), (S3), and (S4), as shown in FIG. 11, violating vehicles such as biting tails are detected.

At this time, when a violation vehicle is detected by the violation vehicle enforcement unit 40, the control unit 30 provides the detected violation vehicle information, the license plate number of the violation vehicle recognized by the number recognition unit 41, and the photographed image of the violation vehicle. The images are matched, and the communication interface unit 32 is controlled so that the matched data is transmitted to the control center server 9.

12 is an exemplary view for explaining the number recognition unit of FIG. 6 .

As shown in FIG. 12 , the number recognition unit 41 recognizes the vehicle number by analyzing an image of a vehicle passing through a section where a vehicle on a connecting lane enters an intersection S.

The warning message output unit 42 uses the object analysis information generated by the object analysis information generation unit 37 to output a warning message to the corresponding display means 11 when a collision between a vehicle and a pedestrian is predicted.

FIG. 13 is an exemplary diagram illustrating the image classification table generation/update unit of FIG. 6 .

The image classification table generation/update unit 43 of FIG. 13 is executed every preset first period T1 according to the control of the control unit 30 .

In addition, as shown in FIG. 13, the image classification table generation/update unit 43 includes a data collection module 431, a total usage calculation module 432 for each GPU, an average usage calculation module 433 for each GPU, and an upper limit setting. It consists of a value (TH) comparison module 434, an inspection candidate determination module 435, a GPU alignment module 346, a change target determination module 347, and an image classification table update module 348.

The data collection module 431 collects usage data for each GPU during the first preset period T1.

The total usage amount calculation module 432 for each GPU calculates the total usage amount for each GPU by using usage data for each GPU collected by the data collection module 431 .

Each GPU average usage calculation module 433 utilizes the total usage amount and the first cycle T1 for each GPU calculated by each GPU total usage calculation module 432 to calculate the average usage for each GPU ( ) is calculated.

The upper limit setting value (TH) comparison module 434 is the average usage for each GPU calculated by the average usage calculation module 433 for each GPU ( ) and the preset upper limit value (TH), and in detail, the average usage for each GPU ( ) is higher than the upper limit setting value (TH) ( ) to compare cognition.

At this time, the upper limit setting value (TH) is defined as the minimum usage value that can be determined as overload in the GPU.

In the inspection candidate determination module 435, in the upper limit set value (TH) comparison module 434, 1) the average amount of use less than the upper limit set value (TH) ( ), the GPU 35 having is removed from the inspection candidates, but 2) the average usage over the upper limit setting value (TH) ( ) is detected, the GPU is determined as an inspection candidate.

The GPU alignment module 436 is executed when an inspection candidate is determined in the inspection candidate determination module 435 .

In addition, the GPU sorting module 436 calculates the average usage (calculated by each GPU average usage calculation module 433). ), the average usage of the GPUs 35 ( ) is sorted in descending order.

The change target determination module 437 uses the GPU sorting module 436 to determine the average usage ( ) determines the GPU with the lowest value as the change target.

The image classification table updating module 438 inputs one of the plurality of images matched to the GPU determined as an inspection candidate in the inspection candidate determination module 435 to the GPU determined as a change target in the change target determination module 347. After determining to do so, the image classification table is updated to reflect the determined matching information.

At this time, the image classification table updated by the image classification table updating module 438 is stored in the memory 31 under the control of the controller 30 .

The vehicle flow control system 1 using object sensing, which is an embodiment of the present invention configured as described above, analyzes images captured by cameras to generate object information and traffic information, and then optimizes the optimum according to the generated traffic information. After creating the display system, traffic signals are controlled according to the generated optimal display system, which not only significantly reduces vehicle congestion and waiting time, but also effectively reduces fuel consumption and pollutant emissions, thereby reducing social costs. savings can be made.

In addition, the vehicle flow control system 1 using object sensing of the present invention can dramatically increase the accuracy and reliability of analysis by improving the object recognition rate by analyzing input images using a deep-learning algorithm.

In addition, the vehicle flow control system 1 using the object response of the present invention can further increase the object recognition rate by applying a YOLO model based on a Convolution Neural Network (CNN) as a deep-learning algorithm.

In addition, when learning the deep-learning algorithm, the vehicle flow control system 1 using object sensing of the present invention is configured so that learning is performed by limiting recognition objects from the conventional 80 types to 5 types frequently seen on the road. As a result, the object recognition rate can be further improved.

In addition, the vehicle flow control system 1 using object sensing of the present invention enables real-time processing and analysis of high-capacity images obtained by shooting cameras by analyzing images using a plurality of GPUs.

1: Vehicle flow control system using object sensing
3: controller 5-1, ..., 5-N: cameras
7: traffic signal controller 8: traffic light
9: control center server 10: communication network
11: display means 30: control unit
31: memory 32: communication interface unit
33: artificial intelligence learning unit 34: image classification unit
35-1, ..., 35-M: AI-based image analysis module
36: usage monitoring unit 37: object analysis information generation unit
38: traffic information generation unit 39: signal system establishment unit
40: violation vehicle control unit 41: number recognition unit
42: warning message output unit 43: image classification table generation / update unit
341: image input module 342: comparison and determination module
343: one-to-one based classification module 344: image classification table based classification module
431: Data collection module 432: Each GPU total usage calculation module
433: each GPU average usage calculation module 434: upper limit setting value (TH) comparison module
435: check candidate determination module 436: GPU alignment module
437: Change target determination module 438: Image classification table update module

Claims (8)

A controller equipped with cameras that acquire images by taking pictures of at least one road that is a sensing area, a traffic signal controller that controls the operation of traffic lights installed in the sensing area, and multiple (M) GPUs (Graphic Processing Units). In the vehicle flow control system using object sensing that includes:
The controller
Using a deep learning-based object analysis algorithm, images obtained by the cameras are analyzed to output object information including at least one of the location, type, and size of the detected object. , Tracks the output object to generate traffic information, detects an optimal signal system based on the generated traffic information, and transmits a response signal according to the detected optimal signal system to the traffic signal controller,
The traffic signal controller controls the traffic lights in response to the sensitive signal transmitted from the controller,
The controller
a memory storing an image classification table matched with a GPU, which is a target to process images input from each camera;
an artificial intelligence learning unit that is executed every predetermined period (T) to learn the object analysis algorithm and stores the learned object analysis algorithm in the memory;
an artificial intelligence-based image analysis unit that analyzes images transmitted from the cameras using an object analysis algorithm stored in the memory and outputs object information;
an object analysis information generation unit that tracks the object detected by the artificial intelligence-based image analysis unit and generates object analysis information including a lane type, a lane location, a moving speed, and a direction in which the detected object is located;
a traffic information generating unit generating traffic information including at least one of queue, waiting time, and traffic volume of each road by using the object analysis information generated by the object analysis information generating unit;
a signal system establishing unit that analyzes the traffic information generated by the traffic information generating unit using a preset optimal signal detection algorithm and establishes an optimal display system;
A control unit for transmitting a response signal according to the optimal display system established by the signal system establishment unit to the traffic signal controller;
The artificial intelligence-based image analysis unit includes artificial intelligence-based image analysis modules processed by each GPU,
The controller
an image classification unit that is executed when images are received from at least one of the cameras and classifies images received from the cameras into corresponding GPUs according to matching information stored in the image classification table;
a usage monitoring unit that is executed when at least one of the AI-based image analysis modules is operating, detects usage (load) of the AI-based video analysis module in operation, and then stores the detected usage amount in the memory;
Further comprising an image classification table generating/updating unit that is executed every preset first period (T1);
The image classification table generation/update unit
a data collection module for collecting usage data for each GPU during the first period T1;
Using the usage data for each GPU collected by the data collection module, the average usage for each GPU ( ) Each GPU average usage calculation module that calculates;
The average usage for each GPU calculated by the average usage calculation module for each GPU ( ) And an upper limit setting value (TH) comparison module for comparing a predetermined upper limit setting value (TH), which means the minimum amount of usage that can be determined as overload in the GPU;
In the upper limit setting value (TH) comparison module, 1) average usage amount less than the upper limit setting value (TH) ( ) is removed from the inspection candidates, but 2) the average usage above the upper limit setting value (TH) ( ) is detected, an inspection candidate determination module for determining the GPU as an inspection candidate;
It is executed when the inspection candidate is determined in the inspection candidate determination module, and the average usage ( ) is the change target determination module for determining the lowest GPU as the change target;
After determining which one of the plurality of images input to the GPU determined as an inspection candidate in the inspection candidate determination module is input to the GPU determined as a change target in the change target determination module, an image classification table is created so that the determined matching information is reflected. An image classification table update module for updating,
The video classification unit
According to the image classification table updated every first period (T1), the image received from at least one of the cameras is classified,
The video classification unit
an image input module that receives images (including camera identification information) transmitted from the cameras;
a comparison and determination module for comparing the total number (N) of the cameras and the total number (M) of the GPUs (Graphic Processing Units);
A one-to-one based classification module that is executed when the total number of cameras (N) in the comparison and determination module is less than (N ≤ M) of the total number of GPUs (M), and inputs the input images to each of the GPUs in a one-to-one manner;
In the comparison and determination module, it is executed when the total number of cameras (N) exceeds the total number of GPUs (M) (N > M), and an image classification table-based classification module that classifies images according to the image classification table stored in the memory. Vehicle flow control system using object sensing, characterized in that it comprises a. delete The vehicle flow control system using object sensing according to claim 1, wherein the object analysis algorithm is applied with a YOLO model based on a Convolution Neural Network (CNN). The vehicle flow control system using object sensing according to claim 3, wherein the artificial intelligence learning unit sets cars, buses, motorcycles, trucks, and human bodies as objects to be recognized when learning the object analysis algorithm. The method of claim 4, wherein the vehicle flow control system using the object sensing further comprises a control center server,
The controller
a violation vehicle enforcement unit that detects a violation vehicle by utilizing object analysis information generated by the object analysis information, location information of a preset detection area, and current signal system;
Further comprising a number recognition unit for recognizing a vehicle number from an image of a vehicle entering a preset section of the detection area,
When a violating vehicle is detected by the violating vehicle enforcement unit, the control unit matches an image including the violating vehicle with a license plate number of the violating vehicle and violation information, and transmits the result to the control center server. Vehicle flow control system using delete delete delete