KR20220107875A - Traffic monitoring system of multi lane based on deep learning using camera - Google Patents

Abstract

The present invention relates to a deep learning-based multi-lane unmanned speed enforcement system using a camera, which includes: a lighting unit; an image capturing unit; a control unit; and a communication unit, wherein the control unit includes a vehicle object recognition unit, a speed detection unit, a license plate recognition unit, and a traffic information analysis unit. The present invention provides the system which operates without a complicated structure and additional manpower regardless of installation location through communication with a server without the need for a separate controller through real-time image processing (Real-Time) in the camera.

Description

{Traffic monitoring system of multi lane based on deep learning using camera}

The present invention relates to an image processing technology that detects a vehicle object using deep learning, classifies a region, and measures speed.

The present invention relates to a deep learning-based multi-lane unmanned traffic speed enforcement system using a camera.

More specifically, it relates to a system that can control signals according to enforcement and traffic volume by installing a high-resolution single camera on a multi-lane road and using an SSD deep learning-based vehicle object detection method.

Unmanned enforcement cameras are being operated across the country to prevent traffic accidents caused by speeding. There are three types of unmanned traffic enforcement equipment in operation in Korea: fixed enforcement cameras, mobile enforcement cameras, and section enforcement cameras. In general, buried loop detectors, radars, and lasers are used to detect vehicles.

Since the buried roof detection method cuts the pavement surface for the installation of the loop detector, it can be one of the causes of road damage. have.

In the case of speed cameras installed in Gyoha-dong, Paju-si, Pocheon-si, and Geumhyeon-ri, Gasan-myeon, Pocheon-si, the roof was disconnected due to ground subsidence and lost its function. In addition, when the detector is constructed, it is necessary to control the vehicle, which is a cause of traffic accidents and traffic congestion.

Compared to other detection methods, the accuracy is lower and the vehicle type is not distinguished. In addition, there is a limitation in that it cannot be installed on a bridge with poor pavement or a steel structure due to a decrease in sensitivity and the inability to secure a depth of burial.

The radar-based traffic enforcement camera is a radar sensor detection method that allows simultaneous secondary measurement and is advantageous in terms of enforcement efficiency and maintenance because it is not buried. However, scattering, reflection, and distortion of radio waves occur, and there is a possibility of misregistration due to errors. And since it is measured from a long distance, it is difficult to measure the speed at an accurate point.

The laser-based traffic enforcement camera is an unburied type that detects using laser light. The laser detection method has the advantage of having a long measurement distance, but multi-lane detection is not possible because it is affected by vehicles traveling in different lanes.

Each district office has one or two units, so equipment is aging severely. It is said that the number of crackdowns using mobile unmanned surveillance cameras installed in Jiyeon, northern Gyeonggi Province in 2019 due to equipment failure decreased by 10% compared to 2018 in the first quarter.

In addition, there are many cases where the camera is left unattended as an empty box booth without a camera due to a problem such as a shortage of police personnel to manage the mobile enforcement cameras.

According to the recent enforcement of the Civil Code Act, it is mandatory to install unmanned surveillance cameras in all school zones across the country. Currently, a total of 820 unmanned speed cameras are installed among 16,789 school zones across the country, accounting for only 4.9% of the installation rate. For this reason, the government announced that it would install 8,800 additional unmanned enforcement cameras in school zones across the country by 2022, but the cost of enforcement cameras is about 30 million won, which is a heavy burden on the government and local governments.

The non-buried method includes a laser sensor and a radar sensor method. The laser sensor detection method has the advantage of a long measurement distance, but there is a limitation in that it is possible to crack down on only a single lane due to the influence of large vehicles or vehicles traveling in other lanes.

In the case of the laser sensor detection method, it has the advantage of being detected in multiple lanes with a wide distance. However, there is a limit in detecting up to a maximum of two lanes simultaneously, and there is a problem of detection rate due to an error due to scattering, reflection, distortion, etc. of radio waves. Therefore, there is a need for a traffic enforcement camera that is improved from the existing radar and laser methods to detect more than three lanes and reduces misguided speed.

In addition, in preparation for a smart city, smart unmanned transportation equipment capable of handling ITS and various traffic information and traffic operation as well as the function of speeding enforcement should be developed. And because there is no radar or specific sensor, the price or maintenance cost is low, and the overall cost can be reduced, which is advantageous in terms of price.

Republic of Korea Patent Registration No. 101967610000

Although some similar studies are being conducted on the multi-lane video surveillance and enforcement equipment based on artificial intelligence and deep learning of the present invention, vehicle object recognition and speed measurement software using a vehicle number recognition algorithm and a single camera and real-time computer vision are installed. It is completely different from this commercialization task, which is to develop multi-purpose unmanned traffic enforcement equipment that can be installed using existing structures by increasing the accuracy and configuring it as a one-body system capable of real-time image processing without a controller.

The system according to the present invention comprises:

lighting unit;

video recording unit;

control unit; and

including a communications department;

The control unit is a deep learning-based multi-lane unmanned traffic speed enforcement system using a camera including a vehicle object recognition unit, a speed detection unit, a vehicle number recognition unit, and a traffic information analysis unit.

The present invention provides a system that operates without a complicated structure and without additional manpower regardless of an installation location through communication with a server without the need for a separate controller through real-time image processing (Real-Time) within the camera.

1 is a schematic diagram of an unmanned traffic enforcement device based on deep learning.
2 is an exemplary view of a vehicle number recognition method.
3 is an exemplary diagram of sensitive signal information.

The present invention includes the following technical matters.

Develop a vehicle detection algorithm using a single camera, and implement and develop a multi-lane license plate number recognition and vehicle speed measurement algorithm within a certain range through deep learning based on this vehicle object location information (distance)

- Existing vehicle detection system technology (product) research

- Analyze and derive requirements

- A study on the practical application of a deep learning-based vehicle detection system

- Development of vehicle object recognition algorithm on multi-lane roads

- Development of vehicle number recognition algorithm on multi-lane roads

- Development of vehicle speed measurement algorithm on multi-lane roads

- Traffic information extraction and situation detection SW development

- Development of control and management SW for road managers

The developed SW and algorithm are transplanted into the high-resolution camera housing to produce a smart camera prototype, the prototype is tested on the road, and performance evaluation and certification are obtained through pilot operation and supplementation, and finally, deep learning-based unmanned transportation using a single camera. Developing speed control equipment and commercializing it

- Single camera-based module HW design and prototype production

- Vehicle object, number, and speed algorithm ported in smart camera

- Road site self-test

- Complementing other functions and performance in road field tests

- Final prototype design and production

- Evaluation and supplementation of test operation results

- ITS performance evaluation and road traffic authority test certification

- Product commercialization

Table 1 shows the main algorithm and system design and differentiation points of the present invention.

algorithm

딥러닝 알고리즘 기법의 하나인 YOLO(You O nly Look Once) 이용으로 이미지를 매우 빠르게 판단

PPM(Pixel Per Meter)을 사용하여 차량의 속 도를 감지하고 속도 계산은 추적된 차량이 이동 한 거리를 픽셀 단위로 0.2 초 단위로 계 산하여 구함

Judging images very quickly by using YOLO (You Only Look Once), one of the deep learning algorithm techniques

PPM (Pixel Per Meter) is used to detect the vehicle's speed, and the speed calculation is obtained by calculating the distance traveled by the tracked vehicle in pixels in units of 0.2 seconds. system design part

Use of ultra-high-resolution camera lenses

Use only ultra-high-resolution cameras without a separate sensor

All-in-one integrated into one without a separate control device

Sensational design compared to competitors Differentiating points

It is an integrated surveillance camera and can be installed without a separate control device.

Traffic enforcement and traffic information analysis are performed using deep learning.

Sensual design compared to domestic competitors

Table 2 summarizes the main performance indicators of the present invention.

Hereinafter, it will be described in more detail with reference to the drawings. According to this, the deep learning-based unmanned traffic enforcement equipment has a structure as shown in FIG. 1, and the deep learning-based module is divided into four parts. < Overview of major performance indicators > Key performance indicators1) unit Final development goal2) Before technology development
level The world's highest level or the level required by the customer3)
(Applicable company) Proportion of total items4)(%) Evaluation method 5) Simultaneous interdiction lane by car 5 lanes 2 lanes National Police Agency 25 official transcript Violation vehicle enforcement rate % over 90 80% or more National Police Agency 5 official transcript speed error rate % 3% or less 5% National Police Agency 25 official transcript speeding error rate % less than 1% 2% National Police Agency 2 official transcript simultaneous recognition lane 2 5 lanes 2 lanes National Police Agency 25 Reports and Videos recognition rate % 99% or more 95% or more National Police Agency 10 official transcript < Sample definition and measurement method > Key performance indicators sample definition Number of measurement samples6)
(n≥5 pieces) Measurement method 7) (standard, environment, calculation of results, etc.) Anti-vehicle enforcement rate 1 meal of finished product 5 Number of vehicles intercepted (number of output vehicles) / (Number of vehicles subject to enforcement - Number of abnormal vehicles)ㅧ100 speed error rate 1 meal of finished product 5 Less than 60km/h: Less than ㅁ3km/h
50km/h or more ~ 80km/h or less: ㅁ Less than 4km/h
80km/h or more ~ 100km/h or less: ㅁ Less than 5km/h
100km/h or more: ㅁ less than 5% speeding error rate 1 meal of finished product 5 Number of overspeeding and erroneous speeding vehicles / Number of intercepted vehicles (number of output vehicles) ㅧ 100 recognition rate 1 meal of finished product 5

(오차백분율 'PE' 식)
E = 분석단위시간 동안 해당 장비가 번호판을 오인식 또는 미인식(미검지 포함)한 차량 대수
Y = 분석단위시간동안 검지 한 유효차량 대수
분석단위시간 : 60분Recognition rate = "100 (%) - Percent error (PE)"

(Error percentage 'PE' formula)
E = The number of vehicles that the equipment misrecognized or did not recognize (including non-detection) the license plate during the analysis unit time
Y = number of valid vehicles detected during analysis unit time
Analysis unit time: 60 minutes

The vehicle object recognition unit detects, recognizes, and learns vehicle objects through deep learning when the camera (video camera) of the 10 MP or larger image capturing unit records real-time high-resolution images. It is implemented so that it is possible to accurately classify objects (car model, people, and other objects).

In the case of the lighting unit, the illuminance is adjusted so that there is no problem when shooting at night or in a specific environment.

The vehicle number recognition unit recognizes the letters and numbers of the vehicle license plate as shown in FIG. 2 to extract information, classifies the vehicle, and integrates and provides the license plate information of each vehicle.

The speed detector designates a specific point of the camera's field of view and detects the vehicle's speed using PPM (Pixer Per Meter). The speed calculation is obtained by calculating the distance traveled by the tracked vehicle in units of pixels (pixel distance traveled by the vehicle in one frame) in units of 1 second. Since the width is different for each road, the value is assigned manually. It is strong against noise, distortion, and illuminance changes, so it can greatly reduce the speed error rate compared to the existing ones.

In case an enforcement target is caught during video shooting, the data (frame) is separately saved to the storage unit (local server) through the control unit. The stored image (frame) is transmitted to the communication unit through the control unit through the deep learning module and is output to the server transmission or the manager's output device.

The traffic information analysis unit is the part that analyzes the recognized vehicle and traffic data to determine the sensitive signal, and learns to build an appropriate signal system through deep learning (Fig. 3).

After the deep learning section, the learned data is stored in the storage unit, sent to the communication unit through the control unit, and transmitted to an external server (National Police Agency) through the communication network.

As shown in FIG. 4 , many deep learning algorithm techniques have been developed so far. Before R-CNN or YOLO (You Only Look Once), object detection algorithms required considerable time and resources for image or video detection, but now, like Fast R-CNN, SSD, and YOLO, the process is streamlined. Algorithms that increase accuracy and speed have emerged.

Algorithms such as Fast R-CNN and SSD have high accuracy in image processing, but are slow, so it is difficult to guarantee real time. However, in the case of YOLO, it is very fast thanks to the method of processing image detection object detection only with a single CNN and judging by looking at the entire image, which is different from the existing object recognition algorithm. It has speed and object detection accuracy. Therefore, this development is developed through the YOLO v5 algorithm.

Claims (1)

lighting unit;
video recording unit;
control unit; and
including a communications department;
The control unit is a deep learning-based multi-lane unmanned traffic speed enforcement system using a camera including a vehicle object recognition unit, a speed detection unit, a vehicle number recognition unit, and a traffic information analysis unit.

Images