KR102637924B1 - System and method for cracking down on non-wearing helmets of two-wheeled vehicles based on artificial intelligence - Google Patents

Abstract

The present invention uses deep learning-based computer vision technology to extract video signals from cameras installed on the road when cracking down on illegal driving of two-wheeled vehicles, extracts the area of the two-wheeled vehicle, and requires the driver and passengers to wear helmets (hard hats) in the area of the two-wheeled vehicle. This relates to an artificial intelligence-based two-wheeled vehicle enforcement system and method for not wearing a helmet, which detects and reads the license plate number of a two-wheeled vehicle and transmits it to enforcement agencies when a helmet is not worn. The present invention detects a two-wheeled vehicle from an image signal captured through a camera installed to photograph the rear of each two-wheeled vehicle, detects whether the driver and passengers of the detected two-wheeled vehicle are wearing helmets, and, if not wearing a helmet, plates the vehicle license plate. A plurality of first to Nth non-wearing helmet enforcement devices (101, 102, 103) that extract the license plate number from the vehicle and transmit it to the traffic violation enforcement server (200), which is a server of the enforcement agency; And a traffic violation enforcement server (200) that notifies enforcement information on violation of helmets for vehicles transmitted from a plurality of first to Nth traffic violation enforcement devices (101, 102, 103). Provides an artificial intelligence-based two-wheeled vehicle enforcement system for not wearing helmets.

Description

{System and method for cracking down on non-wearing helmets of two-wheeled vehicles based on artificial intelligence}

The present invention relates to the crackdown on non-wearing of helmets on two-wheeled vehicles. More specifically, when cracking down on illegal driving of two-wheeled vehicles using deep learning-based computer vision technology, video signals are received from cameras installed on the road to extract the area of the two-wheeled vehicle. This relates to an artificial intelligence-based two-wheeled vehicle enforcement system and method that detects and reads the license plate number of a two-wheeled vehicle when the driver and passenger are not wearing a helmet (hard hat) and transmits this to the enforcement agency.

Two-wheeled vehicles have been used by many people even before the widespread use of automobiles. In addition, although the spread of automobiles has become more active in the modern era, two-wheeled vehicles are still used for leisure or as a means of transportation, following the unique charm of two-wheeled vehicles.

In particular, with the delivery culture revitalized due to the coronavirus, the amount of two-wheeled vehicles driven on the road is increasing.

However, unlike regular cars, two-wheeled vehicles have a somewhat freer driving range on the road. This is not legally free, and since it is smaller than a car and driven by two wheels, it is possible to move freely like a bicycle.

Therefore, the number of drivers who do not follow road laws and drive recklessly on the road is increasing. There are cases where ordinary cars driving safely on the road or pedestrians walking safely are harmed by these two-wheeled vehicles.

In this way, in general, when trying to crack down on traffic law violations by two-wheeled vehicles (hereinafter referred to as two-wheeled vehicles), drivers of two-wheeled vehicles do not comply with the instructions of the police, it is difficult to identify license plates when running away, and unmanned cameras are used to control two-wheeled vehicles. Even if you tried to crack down on cars by automatically filming them, there was no license plate on the front, so even if you filmed them, you couldn't tell who owned them, so the crackdown was ineffective.

In particular, although riders of two-wheeled vehicles are required to wear helmets to prevent safety accidents, they often do not wear helmets for reasons such as discomfort. However, in order to crack down on this, there is a difficulty in that the police must crack down on it themselves.

Various technologies are being developed to solve these problems.

For example, Registered Patent Publication No. 10-1942491 describes a CCTV integrated control center system with an artificial intelligence CCTV integrated control intermediary control module consisting of road traffic situation monitoring control, real-time traffic information analysis, and traffic signal control.

In the above technology, on-site CCTC images taken from existing CCTV cameras are directly transmitted to the integrated control center system without DSP processing and integrated control interface processing, so the image quality of on-site CCTV images is shaken or blurred, making it difficult to detect specific objects with the naked eye. In the conventional integrated control center system, immediate response is difficult because the controller monitors the system 24 hours a day, and it is difficult to identify and track actual risk factors through object recognition, and various abnormalities. The problem of real-time object detection and tracking of risk factors when a situation occurs is impossible, and the cost of system construction is that the road traffic situation monitoring and control system, traffic information analysis system, and traffic light control system are built separately in the integrated control center system. In order to improve the problems of increased cost, slow processing speed due to system overload, and frequent breakdowns, on-site CCTV module (100), CCTV integrated control management server (200), and artificial intelligence CCTV integrated control brokerage control module (300) ), through the artificial intelligence CCTV integrated control mediation control module formed between multiple on-site CCTV modules and the CCTV integrated control management server, through DSP processing and control interface processing, quality on-site CCTV video improved by 80% compared to the existing one. Data can be displayed on the monitoring screen of the integrated control center, and the road traffic situation monitoring and control functions required by the CCTV integrated control management server, specific mobile vehicle detection function, specific mobile vehicle tracking function, traffic information analysis function, and traffic signal control function are performed. The artificial intelligence CCTV integrated control intermediary control module can be implemented, so the cost of building an updated system of the CCTV integrated control management server can be reduced by 70% compared to the existing one, and the monitoring screen display speed and analysis of the CCTV integrated control management server can be improved. The processing speed can be improved by 2 to 4 times compared to the existing one, and the failure rate of the CCTV integrated control management server can be reduced to less than 60%, and the grid cell-based convolutional neural network structure and RNN (Recurrent Neural Network) neural network structure. Through self-learning, based on the learned self-learning data, only specific moving vehicles can be detected by comparing them with moving vehicles in field CCTV video data for each frame, increasing the detection speed of specific moving vehicles by 80% compared to the existing method. It can be improved, and by reducing the time or number of failures, the blank time can be reduced by 80% compared to the existing one, and when detecting a moving vehicle, an ID is given to each moving vehicle, unlike the existing vehicle detection implemented in a stationary environment. Therefore, even if it cannot be recognized in the current frame screen, it is possible to establish that it is the same moving vehicle by comparing the characteristic pattern, vector, and color information of the moving vehicle in the next frame screen captured by another neighboring site CCTV module. The purpose is to provide a CCTV integrated control center system with an artificial intelligence CCTV integrated control brokerage control module consisting of road traffic situation monitoring control/real-time traffic information analysis/traffic signal control that can improve vehicle detection rate by 80%.

Of course, the conventional technology for monitoring automobiles along with road conditions is also designed to monitor two-wheeled vehicles, but there is a loophole in applying this conventional automobile monitoring technology in the case of two-wheeled vehicles.

For example, when a two-wheeled vehicle enters the roadway through a crosswalk and violates traffic laws, the conventional technology configured to photograph the vehicle's license plate may miss the two-wheeled vehicle's license plate due to turning.

In addition, Registered Patent Publication No. 10-2166784 describes a bicycle lane regulation and management system.

The above technology relates to a bicycle lane CCTV enforcement management system, which is installed on the upper part of a support fixed on the bicycle lane, uses a camera to acquire images of a specific area designated on the bicycle lane, and preset An image acquisition device that transmits an image of a specific area acquired along with the latitude and longitude of the support, is connected to the image acquisition device and a communication network, and acquires the image with the latitude and longitude of the support preset from the image acquisition device through the communication network. By receiving images of a specific area and processing and analyzing images using preset image recognition technology based on this, illegal vehicles on bicycle lanes are recognized, the license plates of the illegal vehicles are detected, and the latitude and longitude of the preset support are determined. The current location information of the illegal vehicle is detected by calculating the latitude and longitude of the current location of the license plate of the illegal vehicle, and the manager obtains the license plate image of the illegal vehicle as well as the license plate number, latitude and longitude of the illegal vehicle. By including a road management server that provides a service that allows real-time monitoring of the current location and time visually or audibly, it is possible to not only provide sufficient warning to cyclists in advance but also effectively reduce the occurrence of collision accidents. there is.

Additionally, Registered Patent Publication No. 10-0464619 describes a data transmission and reception device for regulating and managing two-wheeled vehicles.

However, the above-described technologies had a problem in that they were specialized for two-wheeled vehicles and did not apply technology to detect license plates.

Registered Patent Publication No. 10-2349113 (2022.01.11.) Registered Patent Publication No. 10-2368127 (2022.02.23.) Registered Patent Publication No. 10-2166784 (2020.10.16.) Registered Patent Publication No. 10-0464619 (2005.01.06.)

Therefore, the present invention is intended to solve all the shortcomings and problems of the prior art as described above. By utilizing deep learning-based computer vision technology, video signals are received from cameras installed on the road when cracking down on illegal driving of two-wheeled vehicles, and the two-wheeled vehicles are An artificial intelligence-based two-wheeled vehicle enforcement system that extracts the area and detects and reads the license plate of the two-wheeled vehicle when the driver and passenger are not wearing a helmet (hard hat) in the area of the two-wheeled vehicle and transmits this to the enforcement agency. The purpose is to provide methods and methods.

In order to achieve the above object, the present invention detects a two-wheeled vehicle from an image signal captured through a camera installed to photograph the rear of the two-wheeled vehicle, and detects whether the driver and passenger of the detected two-wheeled vehicle are wearing helmets. In case of not wearing a helmet, a plurality of first to Nth non-wearing helmet enforcement devices (101, 102, 103) extract the license plate number from the vehicle license plate and transmit it to the traffic violation enforcement server (200), which is a server of the enforcement agency; And a traffic violation enforcement server (200) that notifies enforcement information on violation of helmets for vehicles transmitted from a plurality of first to Nth traffic violation enforcement devices (101, 102, 103). Provides an artificial intelligence-based two-wheeled vehicle enforcement system for not wearing helmets.

Here, each of the plurality of first to Nth helmet non-wearing control devices 101, 102, and 103 secures data of the two-wheeled vehicle taken between 0 degrees and 60 degrees in order to detect the object of the two-wheeled vehicle from the rear in the direction of travel. A deep learning network with a high FPS (Frame per second) is used, and transfer learning is performed to add data of a two-wheeled vehicle captured from the top view to the deep learning network learned with the MS COCO data set.

In order to achieve the above object, the present invention provides a recognition neural network that includes general and two-wheeled vehicle recognition, boundary box creation, license plate area detection, license plate area reading, and learning data on rear-view images when training a general automobile and two-wheeled vehicle recognition neural network. Installing the learned first to Nth non-wearing helmet control devices (101, 102, 103) (S100); When an image is received from the image receiver 120 of the plurality of first to Nth helmet non-wearing enforcement devices 101, 102, and 103 (S110), the two-wheeled vehicle detection unit 130 of the helmet non-wearing enforcement devices 101, 102, and 103 Detecting a two-wheeled vehicle in an image through (S120); Detecting that the driver or passenger is not wearing a helmet for the detected two-wheeled vehicle (S150); And if the helmet is not worn, tracking the two-wheeled vehicle object, assigning an object ID, extracting the vehicle number from the license plate, and transmitting traffic violation enforcement information to the traffic violation enforcement server 200 through the communication network 300 (S170); Provides an artificial intelligence-based method of cracking down on non-wearing of helmets for two-wheeled vehicles, comprising:

The present invention achieved in this way has the following effects.

First, when cracking down on illegal driving of two-wheeled vehicles using deep learning-based computer vision technology, the area of the two-wheeled vehicle is extracted by receiving video signals from cameras installed on the road, and whether the driver and passengers are wearing helmets (hard hats) in the two-wheeled vehicle area. Accordingly, the convenience of enforcement can be increased by detecting and reading the license plate of a two-wheeled vehicle and transmitting it to the enforcement agency when a helmet is not worn.

Second, it can be used particularly effectively in crackdown on two-wheeled vehicles by filming the rear of a two-wheeled vehicle with a license plate only on the back.

Third, the license plate of a two-wheeled vehicle is extracted through a license plate recognition device that receives video signals from a camera installed on the road, and a deep learning-based object recognition task is performed on the extracted license plate to classify and detect regular cars and two-wheeled vehicles. It has the effect of enabling highly reliable license plate recognition through image noise and deep learning-based rear-shot image correction algorithms that enlarge the image of the license plate of the detected object and enlarge the license plate image.

1 is a diagram illustrating an embodiment of an artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention;
FIG. 2 is a block diagram illustrating an embodiment of the non-wearing helmet enforcement device shown in FIG. 1;
Figure 3 is a diagram illustrating an example of a neural network used in the artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention;
Figure 4 is a diagram illustrating an embodiment of the flow of enforcement for not wearing a helmet in the artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention;
Figure 5 is a diagram illustrating an example of transfer learning in an artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention;
Figures 6 and 7 are diagrams for explaining the IOU used in the artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention;
Figure 8 is a diagram illustrating an embodiment of helmet and license plate discrimination in the artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention;
Figure 9 is a flowchart for explaining an embodiment of an artificial intelligence-based method of cracking down on not wearing a helmet on a two-wheeled vehicle according to the present invention.

Hereinafter, preferred embodiments according to the present invention will be described with reference to the attached drawings.

Embodiments of the present invention may be modified in various forms, and the scope of the present invention should not be construed as being limited to the embodiments described in detail below. This example is provided to more completely explain the present invention to those with average knowledge in the art. Therefore, the shapes of elements in the drawings may be exaggerated to emphasize a clearer description. It should be noted that identical members in each drawing may be indicated by the same reference numerals. Detailed descriptions of well-known functions and configurations that are judged to unnecessarily obscure the gist of the present invention are omitted.

FIG. 1 is a diagram illustrating an embodiment of an artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention, and FIG. 2 is a block diagram illustrating an embodiment of the license plate recognition device shown in FIG. 1. Figure 3 is a diagram illustrating an example of a neural network used in the artificial intelligence-based two-wheeled vehicle's non-wearing helmet enforcement system according to the present invention, and Figure 4 is an artificial intelligence-based two-wheeled vehicle's helmet not wearing enforcement system according to the present invention. This is a diagram to explain an embodiment of the flow of enforcement for not wearing a helmet.

As shown in FIG. 1, the artificial intelligence-based two-wheeled vehicle's non-wearing enforcement system according to the present invention includes a plurality of first to Nth helmet non-wearing enforcement devices (101, 102, 103), a traffic violation enforcement server (200), and Includes a communication network 300.

Here, the plurality of first to Nth helmet non-wearing control devices 101, 102, and 103 each detect a two-wheeled vehicle from an image signal captured through a camera installed to photograph the rear of the two-wheeled vehicle, and detect the driver of the detected two-wheeled vehicle and the driver of the two-wheeled vehicle. After detecting whether the passenger is wearing a helmet, if the helmet is not worn, the vehicle number is extracted from the vehicle license plate and transmitted to the traffic violation enforcement server 200, which is a server of the enforcement agency.

Here, each of the plurality of first to Nth helmet non-wearing control devices 101, 102, and 103 includes a communication unit 110, an image reception unit 120, a two-wheeled vehicle detection unit 130, and a helmet detection unit 140, as shown in FIG. 2. , It consists of an object tracking unit 150, an object ID granting unit 160, a license plate determining unit 170, a vehicle number extracting unit 180, a memory unit 190, and a control unit 195.

The traffic violation enforcement server 200 is a server that notifies enforcement information (fine fines, penalty points, etc.) on traffic violation enforcement vehicles transmitted from a plurality of first to Nth traffic violation enforcement devices 101, 102, and 103. It could be the National Police Agency server.

The communication network 300 is composed of a wired and wireless communication network that performs communication between a plurality of first to Nth traffic violation enforcement devices 101, 102, and 103 and the traffic violation enforcement server 200.

Here, the communication unit 110 communicates with the traffic violation enforcement server 200 through the communication network 300.

The video receiving unit 120 receives video of a road or sidewalk captured through a camera (CCTV). At this time, the camera is used to take pictures not only from the front but also from the rear of the road in order to check the license plate of a two-wheeled vehicle. To explain this in more detail, in order to detect the object of a two-wheeled vehicle from an enforcement camera installed at a certain distance from the ground, data on the two-wheeled vehicle captured at various angles (between 0 degrees and 60 degrees) is secured. At this time, the enforcement system of the present invention uses a deep learning network with high accuracy and high FPS (Frame per second) (YOLO v4, etc.). In addition, transfer learning is performed to add data of a two-wheeled vehicle captured from the top view to the deep learning network trained with the MS COCO data set. In addition, in order to detect Indian driving, video data from cameras installed in India are set to ROI (Region of Interest) to indicate India's only area. When a two-wheeled vehicle is detected in the ROI area, the operation of detecting and recognizing the license plate is performed regardless of whether or not a helmet is worn.

The two-wheeled vehicle detection unit 130 detects only two-wheeled vehicles (two-wheeled vehicles) from the image received through the image receiving unit 120.

At this time, the two-wheeled vehicle is detected through recognition neural network learning data for general cars and two-wheeled vehicles (motorcycles) according to the neural network learning data as shown in FIG. 3.

When a two-wheeled vehicle is detected by the two-wheeled vehicle detector 130, the helmet detection unit 140 detects whether the driver and passenger are wearing helmets. Whether or not a helmet is worn is detected through IOU (Area of Overlap / Area of Union) used in the artificial intelligence-based two-wheeled vehicle's non-wearing enforcement system as shown in FIGS. 6 and 7.

The object tracking unit 150 tracks an object detected by the helmet detection unit 140 as not wearing a helmet, that is, a motorcycle object.

The object ID granting unit 160 assigns an identifier (ID) to the object tracked by the object tracking unit 150. At this time, a unique ID is assigned by creating a bounding box for the two-wheeled vehicle. This will be explained in more detail with reference to FIG. 3 .

The license plate determination unit 170 determines the license plate of the two-wheeled vehicle not wearing a helmet assigned by the object ID granting unit 160. This will be explained in more detail with reference to FIG. 4 .

The license plate extraction unit 180 extracts the license plate number from the license plate determined by the license plate determination unit 170. At this time, the vehicle number is extracted by detecting the two-wheeled vehicle in the two-wheeled vehicle detection unit 130, and in the case of no helmet in the helmet detection unit 140, the license plate determination unit 170 among the objects tracked by the object tracking unit 150. The vehicle number of the license plate determined in is extracted, and when viewed on the road, it can be seen that there are cases where multiple two-wheeled vehicles are adjacent to one lane. Accordingly, in order to prevent errors in detecting the vehicle number of a two-wheeled vehicle other than a two-wheeled vehicle not wearing a helmet, the present invention uses a vehicle number extractor ( 180) is set to recognize the nearest license plate.

The memory unit 190 is received from the image receiving unit 120, and includes an image of a case where either the driver or a passenger is not wearing a helmet among the two-wheeled vehicles detected by the two-wheeled vehicle detection unit 1300, and the vehicle number extractor. The vehicle number extracted from (180) is matched and stored.

The control unit 195 includes a communication unit 110, an image receiver 120, a two-wheeled vehicle detection unit 130, a helmet detection unit 140, an object tracking unit 150, an object ID granting unit 160, a license plate determination unit 170, The vehicle number extraction unit 180 and the memory unit 190 are controlled, and the vehicle number of the two-wheeled vehicle not wearing a helmet stored in the memory unit 190 is controlled to be transmitted to the traffic violation enforcement server 200. In addition, the control unit 195 detects a two-wheeled vehicle in the two-wheeled vehicle detection unit 130 with respect to the image received from the image receiving unit 120, but the image receiving unit 120 detects a two-wheeled vehicle (ROI) to indicate the area of the sidewalk only, not the roadway. Interest) In the case of an image from the set image receiver 120, the license plate is detected and recognized regardless of whether a helmet is detected, and the license plate number is controlled to be transmitted to the traffic violation enforcement server 200.

Meanwhile, the neural network used in the artificial intelligence-based two-wheeled vehicle's helmet failure enforcement system according to the present invention performs recognition neural network learning for general cars and two-wheeled vehicles (motorcycles) through cameras, as shown in Figures 3 and 4. do. To achieve this, first, after vehicle recognition, a unique ID is assigned through the creation of a bounding box for the vehicle recognized as a two-wheeled vehicle, and vector value tracking (+ or -) for the two-wheeled vehicle assigned a unique ID is learned. Perform data completion and illegal U-turn enforcement neural network learning.

And the neural network determines whether a helmet is worn in real-time images through a camera and recognizes the license plate area. In other words, the license plate number for a two-wheeled vehicle with a driver and passenger not wearing a helmet is detected according to the learning data learned in the neural network learning process.

At this time, image frame data is extracted in real time from the image data received from the camera, and the image data is used as input to a CNN (Convolutional Neural Network) for object detection of two-wheeled vehicles. A unique ID is assigned to the bounding box obtained as a result of two-wheeled vehicle object detection, and object tracking is performed by comparing the current frame with the previous frame (using Deep SORT; Simple Online and Realtime Tracking algorithm, etc.).

Figure 5 is a diagram illustrating an example of transfer learning in an artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention.

In transfer learning, two-wheeled vehicle data is acquired, and inference results are confirmed in new images through pre-trained CNN learning and MS COCO Dataset. In the case of pre-trained CNN, there is no need for separate learning, and the advantage is that you can find a pre-trained model and proceed with inference right away.

And the license plate detection and recognition process is carried out through CRNN (Convolution Recurrent Neural Network), a complex structure model that combines CNN and RNN (Recurrent Neural Network). CRNN extracts feature sequences from input images through CNN, and they predict the text sequence of the image as input to RNN. The predicted text sequence is converted to text.

Figures 6 and 7 are diagrams to explain the IOU used in the artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention.

When learning the IOU (Area of Overlap / Area of Union) used to detect two-wheeled vehicles in the artificial intelligence-based two-wheeled vehicle enforcement system for driving violations according to the present invention, the reliability of each class is obtained by multiplying the individual reliability prediction and the conditional class probability, and the existing Both probability and fit between predicted entities are considered.

Each cell predicts the bounding box and the probability of the classified class.

The bounding box has X, Y coordinates, horizontal and vertical size information, and Confidence Score (Score) values. Score indicates whether B (Bonnding Box) holds the object in the area and predicts the class well. Score is generally defined as Pr(Object) * IOU, where Pr(Object) is the probability that an object exists within the bounding box. If the bounding box holds only the background as the area, the value of Pr(Object) is 0, so the Score is 0. IOU stands for intersection over time and indicates the degree to which the bounding box of the training data matches the predicted bounding box.

0 ≤ IOU ≤ 1

At this time, it can be expressed as IOU < 0.7 = Poor, 0.7 < IOU < 0.89 = Good, IOU > 0.9 = Excellent.

Here, the class probability represents the classification probability of the picture in the grid cell. It is expressed symbolically as Pr(Class_i | Object), and is the conditional probability of each class when B contains an object that is not the background. If B predicts the background, the probability becomes 0. Finally, by multiplying the class conditional probability C and the Confidence prediction value of each bounding box, the Confidence Score value for each class of each box can be obtained.

In Figure 7, refers to the center coordinates, width, and height of the area detected through the deep learning network. is the top left coordinate of the grid represents the dimensions of the grid.

At this time, image frame data is extracted in real time from the video data received from the CCTV camera, and the image data is used as input to a CNN (Convolutional Neural Network) for object detection of two-wheeled vehicles. A unique ID is assigned to the bounding box obtained as a result of two-wheeled vehicle object detection, and object tracking is performed by comparing the current frame with the previous frame (using Deep SORT; Simple Online and Realtime Tracking algorithm, etc.).

And as previously explained, the license plate detection and recognition process is carried out through CRNN (Convolution Recurrent Neural Network), a complex structure model that combines CNN and RNN (Recurrent Neural Network). CRNN extracts feature sequences from input images through CNN, and they predict the text sequence of the image as input to RNN. The predicted text sequence is converted to text.

Figure 8 is a diagram illustrating an embodiment of helmet and license plate discrimination in the artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet according to the present invention.

An embodiment of helmet and license plate discrimination in the artificial intelligence-based two-wheeled vehicle's non-wearing helmet enforcement system according to the present invention is, for example, the two-wheeled vehicle detection unit 120 detects the two-wheeled vehicle in the image received from the image receiving unit 120 regarding a motorcycle on the roadway. The helmet detection unit 140 detects whether the driver of the two-wheeled vehicle is wearing a helmet, and a plurality of object IDs (for example, bike 1 to bike 10) assigned by the object ID tracking unit 160 are detected. ) This is a diagram showing that in two two-wheeled vehicles, bike 5 and bike 6, it was detected that the driver or passenger was not wearing a helmet (no_helmet 2, no_helmet 3). At this time, in the present invention, when detecting that no helmet is worn (No Helmet), the object recognizes the license plate whose center point is closest to the vehicle number extraction error for the license plate of the two-wheeled vehicle without a helmet that may occur due to the proximity of the two-wheeled vehicle on the road. was prevented or minimized.

Figure 9 is a flowchart illustrating an embodiment of an artificial intelligence-based method of cracking down on not wearing a helmet on a two-wheeled vehicle according to the present invention.

An embodiment of the artificial intelligence-based two-wheeled vehicle enforcement method for not wearing a helmet according to the present invention is shown in Figure 9, when learning a general car and two-wheeled vehicle recognition neural network, general and two-wheeled vehicle recognition, boundary box creation, and license plate recognition are performed on rear-view images. The first to Nth non-wearing helmet enforcement devices (101, 102, 103) trained with a recognition neural network including area detection, license plate area reading, and learning data are installed to control two-wheeled vehicles (two-wheeled vehicles) on roads and sidewalks (S100) ). At this time, in order for the enforcement devices installed in India to detect Indian driving, the video data from cameras installed in India is set as ROI (Region of Interest) to indicate India's only area.

And when an image is received from the image receiver 120 of the plurality of first to Nth helmet non-wearing enforcement devices 101, 102, and 103 (S110), the two-wheeled vehicle detection unit 130 of the helmet non-wearing enforcement devices 101, 102, and 103 ) to detect a two-wheeled vehicle (two-wheeled vehicle) in the image (S120).

At this time, it is determined whether the images of the non-helmet enforcement devices 101, 102, and 103 have an ROI set to indicate the delivery area (S130).

As a result of the judgment (S130), even though it is a video from India and the enforcement devices 101, 102, and 103 do not wear helmets, the object is tracked through the object tracking unit 150 to crack down on two-wheeled vehicles in India regardless of helmet wearing, and the object is tracked through the object tracking unit 150. An object ID is assigned through the ID granting unit 160, the license plate is determined through the license plate determination unit 170, and the license plate is extracted through the license plate extraction unit 180 and stored in the storage unit 190. Traffic violation enforcement information (two-wheeled vehicle video, license plate number, installation location information of helmet non-wearing enforcement devices (101, 102, 103), etc.) is transmitted to the traffic violation enforcement server (200) through the communication network (300) and communication department (110). (S140).

However, if the image is not from a sidewalk, that is, from a roadway, the helmet detection unit 140 detects whether the driver and/or passenger is not wearing a helmet (S150).

And as a result of the detection, it is determined whether the helmet is not worn (S160).

As a result of the determination (S160), if the helmet is not worn, the object is tracked through the object tracking unit 150 and the object ID granting unit 160 is used to control the two-wheeled vehicle while driving on the road or stopped at a traffic light while not wearing a helmet. ), the object ID is assigned through the license plate determination unit 170, the license plate is determined through the license plate determination unit 170, the license plate is extracted through the license plate extraction unit 180, and stored in the storage unit 190, and at the same time, the communication network 300 and Traffic violation enforcement information (two-wheeled vehicle video, license plate number, installation location (enforcement location) information, etc. of the non-helmet enforcement devices 101, 102, 103) is transmitted to the traffic violation enforcement server 200 through the communication unit 110. (S170). At this time, in order to prevent errors in cracking down on two-wheeled vehicles without helmets when two-wheeled vehicles are nearby, the license plate whose center point is closest to the two-wheeled vehicle (object) with the no helmet factor is determined, and the vehicle license plate is extracted. do.

One embodiment of the present invention described above is merely illustrative, and those skilled in the art will be able to appreciate that various modifications and other equivalent embodiments are possible. . Therefore, it will be understood that the present invention is not limited to the forms mentioned in the detailed description above. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the attached patent claims. In addition, the present invention should be understood to include all modifications, equivalents and substitutes within the spirit and scope of the present invention as defined by the appended claims.

101, 102, 103: 1st to Nth helmet non-wearing enforcement devices
110: Communication unit 120: Video receiving unit
130: Two-wheeled vehicle detection unit 140: Helmet detection unit
150: object tracking unit 160: object ID granting unit
170: License plate determination unit 180: Vehicle number extraction unit
190: memory unit 195: control unit
200: Traffic violation enforcement server 300: Department of Communications

Claims (3)

Two-wheeled vehicles are detected from video signals captured through cameras installed to capture the rear of each two-wheeled vehicle, and whether the driver and passengers of the detected two-wheeled vehicle are wearing helmets is detected. If helmets are not worn, the vehicle number is retrieved from the vehicle license plate. A plurality of first to Nth non-helmet enforcement devices (101, 102, 103) that extract and transmit the extracted information to the traffic violation enforcement server (200), which is a server of a enforcement agency; and
It is configured to include a traffic violation enforcement server (200) that notifies information on enforcement of non-wearing of helmets for vehicles violating driving violations transmitted from a plurality of first to Nth traffic violation enforcement devices (101, 102, 103),
Each of the plurality of first to Nth helmet non-wearing control devices (101, 102, 103),
The communication unit 110 communicates with the traffic violation enforcement server 200 through the communication network 300, and receives images of the road or sidewalk captured through a camera (CCTV), and the camera checks the license plate of a two-wheeled vehicle as well as the front. For this purpose, filming is used from the rear of the road, and in order to detect Indian driving, the camera image data installed on the sidewalk is set to ROI (Region of Interest) to indicate India's only area, and two-wheeled vehicles are placed in the ROI area. When detected, an image receiver 120 that detects and recognizes the license plate regardless of whether or not a helmet is worn, and a two-wheeled vehicle detector 130 that detects only two-wheeled vehicles in the image received through the image receiver 120. and a helmet detection unit 140 that detects whether the driver and passenger are wearing helmets when a two-wheeled vehicle is detected in the two-wheeled vehicle detection unit 130, and tracking a two-wheeled vehicle object detected by the helmet detection unit 140 as not wearing a helmet. An object tracking unit 150 that performs, an object ID granting unit 160 that grants an identifier (ID) to the object tracked by the object tracking unit 150, and an object ID granting unit 160 assigns an identifier (ID) to the object tracked by the object tracking unit 150. A license plate determination unit 170 that determines the license plate of a two-wheeled vehicle not wearing a helmet, a license plate extractor 180 that extracts a license plate number from the license plate determined by the license plate determination unit 170, and a video reception unit 120 that receives the license plate number. Among the two-wheeled vehicles detected by the two-wheeled vehicle detection unit 130, the image of a case where the driver or passenger is not wearing a helmet in the helmet detection unit 140 and the vehicle number extracted from the vehicle number extraction unit 180 match. The memory unit 190 and the communication unit 110, the image receiver 120, the two-wheeled vehicle detection unit 130, the helmet detection unit 140, the object tracking unit 150, the object ID granting unit 160, and the license plate determination unit are stored. Controls the unit 170, the license plate extraction unit 180, and the memory unit 190, and controls the license plate number of the two-wheeled vehicle not wearing a helmet stored in the memory unit 190 to be transmitted to the traffic violation enforcement server 200. , The two-wheeled vehicle detection unit 130 detects a two-wheeled vehicle with respect to the image received from the image receiving unit 120, but the image receiving unit 120 is set as an ROI (Region of Interest) to indicate the area of the sidewalk only, not the roadway. In the case of an image from the image receiver 120, it is composed of a control unit 195 that detects and recognizes the license plate number regardless of whether or not a helmet is detected and controls the license plate number to be transmitted to the traffic violation enforcement server 200,
When the vehicle number extraction unit 180 detects that the driver or passenger is not wearing a helmet from the plurality of object IDs assigned by the object ID tracking unit 160 to prevent errors in vehicle number extraction for license plates of two-wheeled vehicles not wearing helmets, the vehicle number extraction unit 180 detects that the driver or passenger is not wearing a helmet. The object is configured to recognize the license plate whose center point is closest to prevent license plate extraction errors that may occur due to the proximity of two-wheeled vehicles on the road.
Each of the plurality of first to Nth non-wearing helmet enforcement devices (101, 102, 103) secures data of the two-wheeled vehicle taken between 0 degrees and 60 degrees in order to detect the object of the two-wheeled vehicle from the rear of the road in the direction of travel. And, an artificial intelligence-based two-wheeled vehicle enforcement system for not wearing a helmet, characterized by transfer learning to add data of a two-wheeled vehicle captured from the top view to a deep learning network learned with the MS COCO data set.
delete When learning a general car or two-wheeled vehicle recognition neural network, a recognition neural network including general and two-wheeled vehicle recognition, boundary box creation, license plate area detection, license plate area reading, and learning data is trained on rear-view images, respectively, through the communication network 300. The communication unit 110 communicates with the traffic violation enforcement server 200 and receives images of the road or sidewalk captured through a camera (CCTV), and the camera is directed not only to the front but also to the rear of the road in order to check the license plate of a two-wheeled vehicle. In order to detect Indian driving, video data from cameras installed in India are set to ROI (Region of Interest) to indicate India's only area. If a two-wheeled vehicle is detected in the ROI area, helmets are worn. An image receiver 120 that detects and recognizes a license plate regardless of whether it has a license plate, a two-wheeled vehicle detection unit 130 that detects only two-wheeled vehicles in the image received through the image receiver 120, and a two-wheeled vehicle detection unit 130 ), when a two-wheeled vehicle is detected, a helmet detection unit 140 that detects whether the driver and passengers are wearing helmets, and an object tracking unit that tracks the two-wheeled vehicle object detected as not wearing a helmet in the helmet detection unit 140 ( 150), an object ID granting unit 160 that assigns an identifier (ID) to the object tracked by the object tracking unit 150, and a license plate of a two-wheeled vehicle without a helmet assigned by the object ID granting unit 160. The license plate determination unit 170 determines the vehicle number, and the object ID tracking unit 160 extracts the vehicle number from the license plate determined by the license plate judgment unit 170 to prevent an error in extracting the vehicle number for the license plate of a two-wheeled vehicle without a helmet. If it is detected that the driver or passenger is not wearing a helmet from the multiple object IDs provided, the object recognizes the license plate whose center point is closest to prevent license plate extraction errors that may occur due to the proximity of two-wheeled vehicles on the road. Among the two-wheeled vehicles received by the license plate extraction unit 180 and the image receiving unit 120 and detected by the two-wheeled vehicle detection unit 130, the helmet detection unit 140 detects cases where either the driver or the passenger is not wearing a helmet. A memory unit 190 and the communication unit 110, an image receiver 120, a two-wheeled vehicle detection unit 130, a helmet detection unit 140, in which the image and the vehicle number extracted from the vehicle number extraction unit 180 are matched and stored, Controls the object tracking unit 150, object ID granting unit 160, license plate determining unit 170, vehicle number extracting unit 180, and memory unit 190, and stores a two-wheeled vehicle without a helmet stored in the memory unit 190. The vehicle number of the vehicle is controlled to be transmitted to the traffic violation enforcement server 200, and the two-wheeled vehicle detection unit 130 detects a two-wheeled vehicle with respect to the image received from the image receiving unit 120, and the image receiving unit 120 detects a two-wheeled vehicle on the roadway. In the case of an image from the image receiver 120 with an ROI (Region of Interest) set to indicate an area only in India, the license plate is detected and recognized regardless of whether a helmet is detected, and the vehicle number is sent to the traffic violation enforcement server ( Step (S100) of installing the first to Nth non-wearing helmet enforcement devices (101, 102, 103) consisting of a control unit (195) that controls the transmission to 200);
When an image is received from the image receiver 120 of a plurality of first to Nth helmet non-wearing enforcement devices 101, 102, and 103 (S110), the two-wheeled vehicle detection unit of the helmet non-wearing enforcement devices 101, 102, and 103 ( 130) detecting a two-wheeled vehicle in the image (S120);
If the images of the first to Nth helmet non-wearing enforcement devices (101, 102, 103) have an ROI set to indicate the Indian area (S130), the object is tracked to crack down on two-wheeled vehicles in India regardless of helmet wearing. Tracks an object through the unit 150, assigns an object ID through the object ID granting unit 160, determines the license plate through the license plate determination unit 170, and determines the license plate through the license plate extractor 180. The vehicle license plate is extracted and stored in the memory unit 190, and at the same time, traffic violation enforcement information such as two-wheeled vehicle image, license plate number, and helmet is sent to the traffic violation enforcement server 200 through the communication network 300 and the communication unit 110. A step (S140) in which installation location information of the unworn control devices (101, 102, 103) is transmitted;
If the image is not from the sidewalk but from the roadway, detecting that the driver or passenger is not wearing a helmet for the detected two-wheeled vehicle (S150); and
If a helmet is not worn, the two-wheeled vehicle object is tracked, an object ID is assigned, the vehicle number is extracted from the license plate, and traffic violation enforcement information is transmitted (S170) to the traffic violation enforcement server (200) through the communication network (300). When extracting the vehicle number, in order to prevent errors in cracking down on two-wheeled vehicles without helmets in cases where two-wheeled vehicles are nearby, the license plate whose center point is closest to the two-wheeled vehicle (object) with the no helmet factor is determined, and the vehicle An artificial intelligence-based method of cracking down on non-wearing of helmets for two-wheeled vehicles, characterized by extracting license plates.