KR102506971B1 - Method and system for recognizing license plates of two-wheeled vehicles through deep-learning-based rear shooting - Google Patents

Abstract

The present invention relates to a deep learning-based method for recognizing a two-wheeled vehicle license plate through rear-side photographing and a system thereof, which can solve an overload problem of reading a license plate. The system comprises: a plurality of first to N^th vehicle number recognition devices (101, 102, 103); and a road crime prevention server (200) for transmitting crime prevention-related vehicle information.

Description

Method and system for recognizing license plates of two-wheeled vehicles through deep-learning-based rear shooting}

The present invention relates to license plate recognition of a two-wheeled vehicle, and more particularly, by extracting a license plate of a two-wheeled vehicle through a license plate recognition device installed on a road, and recognizing a license plate of a two-wheeled vehicle through deep learning-based object recognition. After classifying and detecting cars, deep learning-based intelligent crime prevention including vehicle crime prevention and living crime prevention is possible through image enlargement of the license plate of the detected object and image noise according to license plate image enlargement and deep learning-based image correction algorithm. It relates to a method and system for recognizing a license plate of a two-wheeled vehicle through rear-viewing.

In general, as an image processing system for collecting image information, a CCTV security system collects image information through a fixed or mobile surveillance camera installed in a certain area, an object recognition system that classifies individual objects by applying an intelligent image analysis technique, An image processing system that can collect various types of image information in conjunction with various surveillance cameras, such as a vehicle license plate recognition system that detects speed violations, signal violations, and parking violations of vehicles in motion, is widely used.

In order to apply this image processing system as a system for identifying and tracking target objects for various purposes, a surveillance camera having a pan/tilt/zoom function is installed at the target surveillance point, The surveillance point is configured by electrically and electronically connecting it to a device for recording and reproducing images taken by a surveillance camera.

As a prior art that has been disclosed in relation to the image processing system as described above, Republic of Korea Patent Registration No. 1060528 (August 24, 2011) is installed at a specific point in order to photograph an accident occurrence situation, and is a general purpose for always photographing the target point. with a camera; an original video extracting unit for extracting the original video captured by the general-purpose camera in the form of a stream when the accident occurs; an image processing unit that divides the image of the video extracted by the original image extraction unit into a target object and surrounding objects, recognizes the target object and surrounding objects, and processes these information into data types that can be respectively processed; a microcomputer for recognizing a target object based on the data transmitted through the image processing unit, generating and outputting data for identifying and tracking the target object, and controlling each component; a data server for pre-stored data for identifying and recognizing target objects and surrounding objects in the image processing unit, and providing the previously stored data through the microcomputer when distinguishing and recognizing the target object and surrounding objects; Applied as a security system capable of obtaining a more improved recognition rate for a target object as configured by including; Image processing systems are known.

In addition, Patent Registration No. 1053939 (July 28, 2011) includes a camera that simultaneously photographs a road wide enough for at least two vehicles to run side by side, and an image controller that processes the original image captured by the camera. And, the image controller includes a storage medium; an image receiving unit receiving the original image from the camera; a vehicle recognition unit extracting and recognizing an image region of a vehicle traveling on the road from the original image; and a video segmentation unit for cutting out a partial image including the recognized vehicle image area from the original image and storing the partial image in the storage medium, thereby efficiently storing and managing a vehicle image captured by a high-resolution camera. A CCTV system for vehicle security using a high-resolution camera capable of performing is known.

However, in both of the prior art Patent Registration Nos. 1060528 and 1053939, in order to detect a specific object such as an object or a vehicle in the surveillance area, the distance value according to the zoom function of the surveillance camera must be set close, so that it can be monitored. The area range is very limited, and if the distance value according to the zoom function of the surveillance camera is set far, the surveillance area can be enlarged, but there is a problem that the recognition rate of object recognition is significantly lowered.

In addition, in the above prior art, in order to increase the object recognition efficiency, in the case of No. 1060528, a plurality of surveillance cameras equipped with a general-purpose camera and a dedicated camera are applied, or the range of the usual surveillance area is very limited, as well as detailed information about objects other than object tracking. There is a problem that the detection efficiency is insufficient, and in the case of No. 1053939, a high-resolution camera was applied, but there was a problem that the recognition efficiency was limited because it simply depended on the resolution of the camera.

That is, in the case of the prior art, real-time video is generally processed through a fixed surveillance camera or a mobile surveillance camera, and in the case of a general vehicle, the size of an object occupying a lane is about the width of the lane, and the resolution of letters or numbers on a license plate If the size is 30 pixels or more, there is no problem with the current technology in recognizing, but on the contrary, if it is remarkably small, it is difficult to recognize the vehicle number.

Because of this problem, the resolution of fixed and mobile surveillance cameras is increased very much, or the lens is zoomed in so that the license plate number can be recognized. There is a problem that tracking is not easy and it becomes difficult to extract information of a corresponding object at a specific location.

In particular, in the case of a two-wheeled vehicle, it is difficult to recognize the license plate of a two-wheeled vehicle due to the small license plate size, so it is necessary to enlarge the license plate size of a general vehicle in order to recognize a small license plate.

However, when the original image is simply enlarged, the outline of the image is blurred and feature points are reduced, which greatly reduces the accuracy of number recognition. The image processing method of correcting video image enlargement takes a lot of processing speed and has a problem that its accuracy is low.

Republic of Korea Patent No. 10-1060528 (2011.08.24.) Republic of Korea Patent Registration No. 10-1053939 (2011.07.28.)

Therefore, the present invention is to solve the various disadvantages and problems of the prior art as described above, and extracts the license plate of a two-wheeled vehicle through a license plate recognition device installed on the road, and deep-learning-based object recognition work for the extracted license plate After classifying and detecting general cars and two-wheeled vehicles through image enlargement of the license plate of the detected object and image noise according to license plate image enlargement and deep learning-based image correction algorithm, intelligent including vehicle crime prevention and life prevention Its purpose is to provide a method and system for recognizing two-wheeled vehicle license plates through deep learning-based rear-viewing capable of crime prevention.

The present invention for achieving the above object is installed on the road to photograph the rear of the vehicle, and classifies general cars and two-wheeled vehicles on the road through deep learning-based object recognition for images captured by a built-in surveillance camera. It tracks the movement of classified objects, classifies and detects the object at a specific location, determines whether or not the detected object has a license plate, and then enlarges the image of the license plate and image noise and deep learning based on the image of the license plate. Recognizes the license plate of a two-wheeled vehicle through rear-viewing based on deep learning that recognizes the license plate recognition of a two-wheeled vehicle through an image correction algorithm of A plurality of first to N-th license plate recognition devices (101, 102, 103) for transmitting identification information and enforcement-related information of the device; And transmitting crime prevention-related vehicle information including license plate information of a crime prevention target vehicle to a plurality of the first to Nth vehicle number recognition devices 101, 102, and 103, and a plurality of first to Nth vehicle number recognition devices ( 101, 102, 103) the road security server 200 that notifies the information transmitted from the National Police Agency to related organizations including the National Police Agency; to provide.

Here, the two-wheeled vehicle license plate recognition is characterized by using a Super Resolution Conveoltion Neural Network (SRCNN).

And in order to achieve the above object, the present invention is a recognition neural network including general and two-wheeled vehicle recognition, boundary box generation, license plate area detection, license plate area reading, and learning data for rear view images when learning a general car and two-wheeled vehicle recognition neural network. Learning step (S100); learning a reading neural network when the learning data is completed (S110); installing the learned reading neural network data as a program in a storage unit of each of the plurality of first to Nth license plate recognition devices 101, 102, and 103 (S120); Periodically updating the crime prevention-related vehicle number from the road security server 200 through the communication network and the communication unit 110 in the storage unit 170 of each of the first to Nth vehicle number recognition devices 101, 102, and 103 (S130); The monitoring camera unit 120 of each of the first to Nth license plate recognizing devices 101, 102, and 103 receives an image by photographing the rear of a vehicle running on the road, and the received image is classified into each preliminary object. Classifying in unit 130 into general vehicles and two-wheeled vehicles (S140); The object tracking unit 140 of the first to Nth license plate recognizing devices 101, 102, and 103 tracks the motion of a two-wheeled vehicle (two-wheeled vehicle) among the objects classified in the preliminary object classification unit 130 ( S150); The number extraction unit 160 of each of the first to Nth vehicle number recognition devices 101, 102, and 103 extracts a number from a license plate of a two-wheeled vehicle (two-wheeled vehicle) (S170); characterized in that it comprises a It provides a two-wheeled vehicle license plate recognition method through rear-viewing based on deep learning.

Here, the step of extracting the number (S170) is characterized by using a Super Resolution Conveoltion Neural Network (SRCNN).

The present invention made in this way has the following effects.

First, the license plate of the two-wheeled vehicle is extracted through the license plate recognition device installed on the road, and the extracted license plate is classified and detected through deep learning-based object recognition. There is an effect of recognizing the license plate of a two-wheeled vehicle capable of intelligent crime prevention including vehicle crime prevention and living crime prevention through image noise and deep learning-based rear image correction algorithm according to image enlargement and license plate image enlargement.

Second, by taking a picture of the rear of a two-wheeled vehicle (two-wheeled vehicle) where the license plate is located only at the back, it can be effectively used for cracking down on two-wheeled vehicles.

Third, by performing the license plate reading necessary for crime prevention-related crackdowns in each license plate recognition device, unnecessary video data transmission and reception to read the number from all video data in the server that has received the video information previously taken by multiple cameras is innovatively As it can be reduced, it is possible to reduce unnecessary data communication as well as to solve the overload problem of license plate reading related to crime prevention caused by performing vehicle license plate reading for all transmitted image data in the server.

1 is a diagram for explaining an embodiment of a two-wheeled vehicle license plate recognition system through deep learning-based rearward shooting according to the present invention;
Figure 2 is a block diagram for explaining an embodiment of the license plate recognition device shown in Figure 1,
3 is a view for explaining an embodiment of a neural network used in a two-wheeled vehicle license plate recognition system through deep learning-based rear-viewing according to the present invention;
4 is a diagram showing the network structure of SRCNN used in the two-wheeled vehicle license plate recognition system through deep learning-based rear imaging according to the present invention;
5 is a diagram showing an example of a license plate recognition result of a two-wheeled vehicle through deep learning-based rearward shooting according to the present invention;
6 is a flowchart for explaining an embodiment of a method for recognizing a license plate of a two-wheeled vehicle through rear imaging based on deep learning according to the present invention.

Hereinafter, preferred embodiments according to the present invention will be described with reference to the accompanying drawings.

Embodiments of the present invention may be modified in various forms, and the scope of the present invention should not be construed as being limited to the examples described in detail below. This embodiment is provided to more completely explain the present invention to those skilled in the art. Therefore, the shapes of elements in the drawings may be exaggerated to emphasize a clearer explanation. It should be noted that in each drawing, the same members are sometimes indicated by the same reference numerals. Detailed descriptions of well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention are omitted.

1 is a diagram for explaining an embodiment of a two-wheeled vehicle license plate recognition system through deep learning-based rear imaging according to the present invention, and FIG. 2 is a block configuration for explaining an embodiment of the license plate recognition device shown in FIG. 1 FIG. 3 is a diagram for explaining an embodiment of a neural network used in a two-wheeled vehicle license plate recognition system through deep learning-based rear imaging according to the present invention, and FIG. 4 is a deep learning-based rear imaging according to the present invention. Figure 5 is a diagram showing the network structure of SRCNN used in a two-wheeled vehicle license plate recognition system through , and FIG.

An embodiment of the two-wheeled vehicle license plate recognition system through deep learning-based rear imaging according to the present invention is as shown in FIGS. 1 to 4. First, as shown in FIG. 1, the embodiment of the system is It is configured to include a plurality of first to Nth vehicle number recognition devices (101, 102, 103), a road security server 200, and a communication network installed on a passing road.

Each of the plurality of first to Nth license plate recognizing devices 101, 102, and 103 receives an image signal from a camera installed on the road, and recognizes a general vehicle and two-wheeled vehicle on the road through deep learning-based object recognition from the received image. Cars are classified, the motions of the classified objects are tracked, the object is classified and detected at a specific location, and after determining whether or not the detected object has a license plate, the image of the extracted license plate is enlarged and image noise and deep learning based on license plate enlargement are used. Recognizes the license plate of a two-wheeled vehicle through rear-viewing based on deep learning that recognizes the license plate recognition of a two-wheeled vehicle through an image correction algorithm of Transmits preset information such as device identification information and enforcement-related information (photographed image information, recording time, recording location, license plate, etc.).

The road security server 200 transmits vehicle information related to crime prevention, such as license plate information of a vehicle subject to crime prevention, to a plurality of first to Nth vehicle number recognition devices 101, 102, and 103, and a plurality of first to Nth vehicle numbers. The information transmitted from the recognition devices 101, 102, and 103 is notified to the National Police Agency, which is a related institution.

As shown in FIG. 2, each of the plurality of first to Nth license plate recognition devices 101, 102, and 103 includes a communication unit 110, a surveillance camera unit 120, a preliminary object classification unit 130, and an object tracking unit. It is configured to include a unit 140, a license plate determination unit 150, a number extraction unit 160, a storage unit 170, a control unit 180 and a power supply unit 190, and the storage unit 170 is a first storage unit. It may be composed of a part 170a and a second storage part 170b.

The communication unit 110 communicates with the road security server 200 through a communication network.

The surveillance camera unit 120 receives a video signal on the road with a camera installed on the road. This surveillance camera unit 120 is generally a surveillance camera 120 that photographs the rear rather than the front of the vehicle, because a license plate is attached only to the rear in the case of a two-wheeled vehicle.

The preliminary object classification unit 130 classifies vehicles on the road into general vehicles (trucks, general vehicles, etc.) and two-wheeled vehicles through deep learning-based object recognition from the images received from the surveillance camera unit 120. At this time, image frame data is extracted in real time from the video data received from the monitoring camera unit 120 . Image data is used as an input for CNN (Convolutional Neural Network) for object detection of general cars and motorcycles.

The object tracking unit 140 tracks the movement of a two-wheeled vehicle (two-wheeled vehicle) among the objects classified by the preliminary object classification unit 130 . In other words, a unique ID is assigned to the bounding box obtained as a result of object detection for general vehicles and motorcycles, and object tracking is performed by comparing the current frame with the previous frame (using Deep SORT; Simple Online and Realtime Tracking algorithm). Then, the corresponding object (license plate) of a specific location set in advance is classified.

The license plate determining unit 150 determines whether the object detected by the object tracking unit 140 has a license plate.

The number extractor 160 extracts a number from the license plate when it is determined that the license plate determination unit 150 has a license plate. At this time, in the present invention, vehicle license plate image data obtained by photographing the rear of a two-wheeled vehicle often has a lot of noise or 2) a low resolution image. Therefore, for license plate identification, the license plate of the two-wheeled vehicle is detected from the image (image), and the detected area is cropped and used as an input of two noise removal and high-resolution restoration deep learning networks.

That is, in the method of recognizing the license plate of a two-wheeled vehicle, a motorcycle, which is a two-wheeled vehicle, is obtained by cropping an area through object detection of a deep neural network (CNN) from an image (image), and thus the motorcycle license plate is obtained. The OCR of the license plate obtained here is composed of a small size, and misrecognition occurs when a number is recognized based on a general existing deep neural network. Therefore, after converting the blurry video image into a high-resolution image using a high-resolution restoration deep learning network algorithm for the license plate image, number recognition is performed. At this time, the present invention uses Single Image Super Resolution as an algorithm for restoring a resolution image to high resolution, and among them, SRCNN as shown in FIG. 4 is used.

For reference, first, a low-resolution image is enlarged by Bicubic interpolation. Next, in the case of SRCNN (Super Resolution Conveoltion Neural Network), multi-dimensional patches having characteristics are extracted with the stretched image as input, and high-resolution images are restored by performing non-linear mapping with patches of different dimensions. Then, the difference between the high-resolution image and the actual high-resolution image is back-propagated to learn the weights of the neural network.

At this time, the SRCNN network of attached FIG. 4 uses a low resolution (LR) image as an input, and increases the image to a desired size using the Bicublc interpolation method, which is an image processing method, to enlarge the image (High Resolution: HR), here By applying a convolutional layer to , noise removal operation is performed, so the recognition rate for the license plate of a two-wheeled vehicle photographed from the rear is improved. At this time, FIG. 5 is a diagram showing an example of a two-wheeled vehicle license plate recognition result through deep learning-based rearward shooting according to the present invention, showing that there is a clear difference in the sharpness of an image before SRCNN network processing and an image after SRCNN network processing there is.

Referring to FIG. 3, which is an embodiment of the neural network used in the system for recognizing two-wheeled vehicles through rear-viewing based on deep learning according to the present invention, general, two-wheeled vehicle recognition, and boundary boxes for rear-viewed images when learning a neural network for recognizing general vehicles and two-wheeled vehicles After generation, license plate area detection, license plate area reading, and training data are completed, the reading neural network is learned. Vehicle license plate reading is performed according to two-wheeled vehicle recognition, vehicle license plate area recognition, and noise removal by deep learning (super resolution application).

The storage unit 170 stores the number extracted by the number extraction unit 160. At this time, in addition to the extracted number, the storage unit 170 stores the license plate information of the crime prevention-related vehicle transmitted from the road security server 200 in advance. To this end, the storage unit 170 includes a first storage unit 170a for storing the license plate information of the crime prevention-related vehicle transmitted from the road security server 200 and a second storage unit 170a for storing the number extracted from the number extraction unit 160. It may be configured as a storage unit (170b).

The controller 180 includes a communication unit 110, a surveillance camera unit 120, a dictionary object classification unit 130, an object tracking unit 140, a license plate determination unit 150, a number extraction unit 160, and a storage unit 170. ), and if a number related to crime prevention is extracted by comparing the number information of the first storage unit 170a and the second storage unit 170b, the identification information of the license plate recognition device and enforcement related information (photographed image information, photographing time, photographing location, license plate, etc.) are transmitted to the road security server 200 through the communication unit 110.

6 is a flowchart for explaining an embodiment of a method for recognizing a license plate of a two-wheeled vehicle through rear imaging based on deep learning according to the present invention.

As shown in FIG. 6, the deep learning-based method for recognizing two-wheeled vehicle license plates through rear-viewing according to the present invention is general, two-wheeled vehicle recognition, and boundary box for rear-photographed images when learning a neural network for recognizing general vehicles and two-wheeled vehicles. Generation, license plate area detection, license plate area reading, and recognition neural network training including learning data are performed (S100).

When the learning data is completed, the reading neural network is learned (S110).

The reading neural network data is installed as a program in the storage unit of each of the plurality of first to Nth license plate recognition devices 101, 102, and 103 (S120).

And, through the communication network and the communication unit 110, the vehicle number related to crime prevention is periodically stored in the storage unit 170 of each of the first to Nth vehicle number recognition devices 101, 102, and 103 from the road security server 200 ( updated) (S130).

The monitoring camera unit 120 of each of the first to Nth license plate recognizing devices 101, 102, and 103 receives an image by photographing the rear of a vehicle running on the road, and the received image is a preliminary object classification unit 130 ), general vehicles (trucks, general cars, etc.) and two-wheeled vehicles are classified (S140).

In addition, the object tracking unit 140 of each of the first to Nth vehicle number recognition devices 101, 102, and 103 tracks the movement of a two-wheeled vehicle (two-wheeled vehicle) among the objects classified in the preliminary object classification unit 130, A unique ID is given to the bounding box obtained as a result of the two-wheeled vehicle object detection, and object tracking is performed (using Deep SORT; Simple Online and Realtime Tracking algorithm, etc.) by comparing the current frame with the previous frame (S150).

Meanwhile, the license plate determining unit 150 of the license plate recognizing device determines whether the object (two-wheeled vehicle) detected by the object tracking unit 140 has a license plate (S160).

In addition, when the number extraction unit 160 of the license plate recognizing device determines that there is a license plate in the license plate determining unit 150, the number is extracted from the corresponding license plate (S170). At this time, after converting the blurry video image into a high-resolution image by using a high-resolution restoration deep learning network algorithm for the license plate image, number recognition is performed. learning proceeds. At this time, SRCNN (Super Resolution Conveoltion Neural Network) is used.

The control unit 180 of the license plate recognizing device stores the number extracted by the number extraction unit 160 in the storage unit 170. At this time, since the license plate information of the crime prevention-related vehicle transmitted from the road security server 200 is stored in the storage unit 170 in advance in addition to the extracted number, the control unit 180 stores the storage unit 170 (first storage unit 170a and By comparing the number information of the second storage unit (170b) (S180), when a number related to crime prevention is extracted, the identification information of the vehicle number recognition device and enforcement-related information (photographed image information, recording time, recording location, license plate number) etc.) is transmitted to the road security server 200 through the communication unit 110 (S190).

One embodiment of the present invention described above is only exemplary, and those skilled in the art will appreciate that various modifications and equivalent other embodiments are possible therefrom. . Therefore, it will be well understood that the present invention is not limited to the forms mentioned in the detailed description above. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims. It is also to be understood that the present invention includes all modifications, equivalents and alternatives within the spirit and scope of the present invention as defined by the appended claims.

101, 102, 103: a plurality of first to Nth vehicle number recognition devices
110: communication unit 120: monitoring camera unit
130: dictionary object classification unit 140: object tracking unit
150: license plate determination unit 160: number extraction unit
170: storage unit 170a: first storage unit
170b: second storage unit 180: control unit
190: power supply unit 200: road security server
210: communication unit 220: database
230: method information storage unit 240: control unit

Claims (4)

It is installed on the road to photograph the rear of the vehicle, and classifies general cars and two-wheeled vehicles on the road through deep learning-based object recognition on the images captured by the built-in surveillance camera, tracks the movements of the classified objects, and The object is classified and detected at the location, and after determining whether the detected object has a license plate, the image of the extracted license plate is enlarged, image noise is removed according to the image of the license plate, and the rear is photographed through a deep learning-based image correction algorithm. A plurality of 1st to Nth vehicle numbers for recognizing a license plate of a two-wheeled vehicle and transmitting identification information of the vehicle number recognition device and control-related information to the road security server 200 for vehicle information recognized as a crime prevention target among them. Recognition devices 101, 102, 103; and
The plurality of first to Nth vehicle number recognition devices 101, 102, and 103 transmit crime prevention-related vehicle information including license plate information of a crime prevention target vehicle, and a plurality of first to Nth vehicle number recognition devices 101 , 102, 103) the road security server 200 for notifying the information transmitted from) to related organizations including the National Police Agency;
The two-wheeled vehicle license plate recognition uses SRCNN (Super Resolution Conveoltion Neural Network), and low resolution ( The Low Resolution: LR) image is increased to a High Resolution (HR) image of the desired size using Bicubic interpolation, and a convolutional layer is applied to remove noise to shoot the rear Improve the recognition rate of the license plate of a two-wheeled vehicle,
Each of the plurality of first to Nth vehicle number recognition devices 101, 102, 103,
A communication unit 110 communicating with the road security server 200 through a communication network;
A monitoring camera unit 120 for photographing a license plate attached to the rear of a two-wheeled vehicle;
A preliminary object classification unit 130 that classifies trucks, general vehicles, and two-wheeled vehicles with respect to vehicles on the road through deep learning-based object recognition from the video received from the monitoring camera unit 120;
Among the objects classified by the preliminary object classification unit 130, the movement of the two-wheeled vehicle is tracked, but a unique ID is assigned to the bounding box obtained as a result of the object detection of the two-wheeled vehicle, and the current frame and the previous frame are compared to deep (Deep) Object tracking unit 140 for classifying the license plate of a two-wheeled vehicle, which is a corresponding object at a specific location set in advance, by tracking the object through SORT (Simple Online and Realtime Tracking) algorithm;
A license plate determination unit 150 for determining whether or not the object detected by the object tracking unit 140 has a license plate;
Acquire motorcycles, which are two-wheeled vehicles, by cropping the area through object detection of a deep neural network (CNN) from license plate video images of motorcycles, obtain license plates of acquired motorcycles, and reconstruct the license plate images in high resolution Deep Learning Network A number extraction unit 160 that performs number recognition after converting a blurry video image into a high-resolution image through SRCNN (Super Resolution Conveoltion Neural Network) among SISR (Single Image Super Resolution) using an algorithm,
A first storage unit 170a for storing the number plate information of the crime prevention-related vehicle transmitted from the road security server 200 and a second storage unit 170b for storing the number extracted from the number extraction unit 160 The configured storage unit 170,
Controls the communication unit 110, surveillance camera unit 120, dictionary object classification unit 130, object tracking unit 140, license plate determination unit 150, number extraction unit 160 and storage unit 170, , When a number related to crime prevention is extracted by comparing the number information of the first storage unit 170a and the second storage unit 170b, identification information of the vehicle number recognition device and photographed image information, photographing time, photographing location, vehicle A control unit 180 for controlling transmission of enforcement-related information including license plates to the road security server 200 through the communication unit 110 recognition system.
delete A communication unit 110 that communicates with the road security server 200 through a communication network, a monitoring camera unit 120 that photographs a license plate attached to the rear of a two-wheeled vehicle, and a deep video from the video received from the monitoring camera unit 120. A preliminary object classification unit 130 that classifies vehicles on the road into trucks, general vehicles, and two-wheeled vehicles through a learning-based object recognition task, and the movement of two-wheeled vehicles among the objects classified in the preliminary object classification unit 130 While tracking, object tracking is performed through Deep SORT (Simple Online and Realtime Tracking) algorithm by assigning a unique ID to the bounding box obtained as a result of object detection of the two-wheeled vehicle and comparing the current frame with the previous frame. an object tracking unit 140 that classifies the license plate of a two-wheeled vehicle, which is a corresponding object at a specific location set in advance, and a license plate determination unit 150 that determines whether or not there is a license plate of the object detected by the object tracking unit 140; A motorcycle, a two-wheeled vehicle, is obtained by cropping an area through object detection of a deep neural network (CNN) from a license plate video image of a car, and a license plate image of the obtained motorcycle is obtained, and the license plate image is restored in high resolution Deep Learning Network Using an algorithm, a number extraction unit 160 that performs number recognition after converting a blurry video image into a high-resolution image through SRCNN (Super Resolution Conveoltion Neural Network) among SISR (Single Image Super Resolution), and the road security Storage consisting of a first storage unit 170a for storing the number plate information of the crime prevention-related vehicle transmitted from the server 200 and a second storage unit 170b for storing the number extracted from the number extraction unit 160 unit 170, the communication unit 110, monitoring camera unit 120, dictionary object classification unit 130, object tracking unit 140, license plate determination unit 150, number extraction unit 160, and storage unit 170, and the first storage unit 170a and When a number related to crime prevention is extracted by comparing the number information of the second storage unit 170b, enforcement-related information including identification information of the license plate recognition device, photographed image information, photographed time, photographed location, and vehicle license plate is transmitted to the communication unit. Each configuration includes a control unit 180 that controls transmission to the road security server 200 through 110, and is installed on the road to photograph the rear of the vehicle, and the video captured by the built-in monitoring camera 120 Classify general cars and two-wheeled vehicles on the road through deep learning-based object recognition, track the movement of the classified objects, classify and detect the object at a specific location, determine whether or not the detected object has a license plate, and then extract Recognize the license plate of a two-wheeled vehicle photographed from the rear through image enlargement of the license plate, image noise removal according to image enlargement of the license plate, and image correction algorithm based on deep learning, and among them, the vehicle information recognized as a crime prevention target is described above. A plurality of first through Nth license plate recognition devices 101, 102, and 103 transmitting identification information and enforcement-related information of the license plate recognition device to the road security server 200, and a plurality of the first through Nth license plate recognition devices The vehicle number recognition device (101, 102, 103) transmits crime prevention related vehicle information including license plate information of the vehicle to be crime prevention, and transmitted from a plurality of first to Nth license plate recognition devices (101, 102, 103) The road security server 200 that notifies information to related organizations including the National Police Agency; license plate recognition of a two-wheeled vehicle through deep learning-based rearward imaging of a two-wheeled vehicle license plate recognition system through deep learning-based rearviewing in a way,
Learning a recognition neural network including general and two-wheeled vehicle recognition, boundary box generation, license plate area detection, license plate area reading, and learning data for rear-viewed images when learning a general car and two-wheeled vehicle recognition neural network (S100);
learning a reading neural network when the learning data is completed (S110);
installing the learned reading neural network data as a program in the storage unit 170 of each of the plurality of first to Nth license plate recognition devices 101, 102, and 103 (S120);
The first to Nth vehicle number recognition devices 101, 102, 103, respectively, in the storage unit 170, through the communication network and the communication unit 110, the crime prevention-related vehicle number is periodically updated from the road security server 200 Step (S130);
The monitoring camera unit 120 of each of the first to Nth license plate recognizing devices 101, 102, and 103 receives an image by photographing the rear of a vehicle running on the road, and the received image corresponds to each of the dictionary objects. Classifying in the classification unit 130 into general vehicles and two-wheeled vehicles (S140);
The object tracking unit 140 of each of the first to Nth license plate recognizing devices 101, 102, and 103 tracks the movement of the two-wheeled vehicle among the objects classified in the preliminary object classification unit 130 (S150). );
The license plate determination unit 150 of the first to Nth license plate recognition devices 101, 102, and 103 that tracked the movement of the two-wheeled vehicle determines whether or not the license plate of the two-wheeled vehicle, which is the object detected by the object tracking unit 140, is present. Determining step (S160);
The number extraction unit 160 of the first to Nth vehicle number recognition devices 101, 102, and 103 extracts the number from the license plate of the two-wheeled vehicle (S170); and
The controller 180 of each of the first to N-th vehicle number recognition devices 101, 102, and 103 includes a first storage unit 170a of each of the first to N-th vehicle number recognition devices 101, 102, and 103. By comparing the number information of the crime prevention-related vehicle stored in and the number information of the second storage unit 170b of each of the first to Nth vehicle number recognition devices 101, 102, and 103 (S180), the crime prevention-related number is If extracted, transmitting enforcement-related information including identification information of the license plate recognition device, photographed image information, photographing time, photographing location, and license plate to the road security server 200 through the communication unit 110 ( S190); A method for recognizing a two-wheeled vehicle license plate through deep learning-based rear imaging, characterized in that it comprises. delete

Images