KR102234936B1 - Apparatus and method for recognizing license plates in real time - Google Patents

Abstract

The present invention relates to a technical idea of real-time recognition of a vehicle number through a process of detecting a license plate, extracting a character string, and recognizing a character string using image data collected from a CCTV, and the vehicle license plate recognition apparatus according to an embodiment is generated through a CCTV. A license plate detection unit that sets a license plate detection area in consideration of the ratio of the license plate from the obtained image data, and crops the license plate at a predicted position by applying convolution to the set license plate detection area, and the cropping in consideration of the ratio Included in the extracted character string according to a character string extraction unit for extracting the character string displayed on the license plate and modifying the network based on the size of the character included in the character string, and a text classifier classified based on the modified network. It may include a character recognition unit for real-time recognition of each character.

Description

Apparatus and method for recognizing license plates in real time}

The present invention relates to a technology for recognizing a vehicle license plate in real time, and to a technical idea of recognizing a vehicle number in real time through a license plate detection, character string extraction, and character string recognition process using image data collected from CCTV.

ALPR (Automatic License Plate Recognition) is a technology for recognizing the license plate of a vehicle, and is used in various fields such as vehicle access control, parking management, and illegal vehicle enforcement.

In general, in order to recognize the license plate of a vehicle, it can be implemented through three steps of license plate detection, character string extraction, and character string recognition. For license plate detection, a method of detecting a change in contrast or geometric structure based on a vertical edge or based on a color or structural feature of a license plate was used.

Meanwhile, in order to extract the character string from the license plate, it was possible to extract it through a technology based on binarized blob labeling.

In addition, template-based matching, machine learning and neural network-based recognition technologies were used to recognize character strings in license plates.

Nevertheless, the existing method had a disadvantage in that the environment in which the license plate can be recognized is largely limited.

Specifically, in order to accurately recognize the license plate, it is possible only in an environment where there are constraints such as a certain amount of light and a vehicle entry direction.

In the case of an environment where CCTV for crime prevention is installed, various external variables may exist.In particular, there are many errors in license plate recognition due to variables caused by weather phenomena such as inconsistent amount of light, different amounts of light for each day, night, and dawn time. There was a problem that occurred.

In addition, geometric distortion occurred according to the CCTV installation location and vehicle entry direction, which contributed to lowering the license plate recognition rate.

In order to solve these problems, technology is being developed in the direction of increasing the license plate recognition rate in recent years.

Korean Patent Registration No. 10-1949765 "Vehicle location information recognition apparatus and method using license plate recognition" Korean Patent Laid-Open Patent No. 10-2016-0057356 "Vehicle License Plate Recognition Method and System"

An object of the present invention is to increase the recognition rate of vehicle license plates in CCTV for crime prevention.

An object of the present invention is to increase the recognition rate of a license plate even for geometric distortion according to the amount of light, the CCTV installation location, and the vehicle entry direction.

An object of the present invention is to provide a deep learning-based license plate detection and recognition technology that can improve performance through learning in case of various variables.

The vehicle license plate recognition apparatus according to an embodiment sets a license plate detection area in consideration of the ratio of the license plate from image data generated through CCTV, and crops the license plate at a predicted position by applying convolution to the set license plate detection area ( A license plate detection unit to crop), a character string extraction unit that extracts the character string displayed on the cropped license plate in consideration of the ratio, and modifies the network based on the size of the character included in the character string, and the modified network According to the classified text classifier, a character recognition unit for real-time recognition of each character included in the extracted character string may be included.

The license plate detection unit according to an embodiment sets common regions of regions in which characters of the license plate are identified in the previously learned image data as regions of interest, and detects the license plate within a region masked with the set region of interest. You can set the area.

The license plate detection unit according to an embodiment applies convolution to the set license plate detection area, but does not separate and calculate the image data layer, and performs one type of filter by varying the stride at which the filter is applied. By using the convolution multiple times, the predicted position can be determined.

The license plate detection unit according to an embodiment learns by applying an initialized weight instead of a pre-learned weight to the generated image data, but simultaneously learning all layers of the image data to an anchor box corresponding to the learning set. You can learn by recalculating the (anchor box).

The character string extractor according to an embodiment learns using the cropped license plate, but applies an initialized weight to the cropped license plate instead of a pre-learned weight, and simultaneously learns all layers of the cropped license plate. Thus, the anchor box corresponding to the training set can be recalculated.

The character string extractor according to an embodiment configures the learning set using training data, test data, and positive data in which an object exists in an image, wherein at least one of the training data, the test data, and the positive data is It may include an object label, a bounding box midpoint x coordinate, a bounding box midpoint y coordinate, a bounding box width, and a bounding box height.

The character recognition unit according to an embodiment recognizes the character using a previously learned character string, but resizes each character included in the previously collected character string, and then convolves to recognize the character. You can learn.

The character recognition unit according to an embodiment may extract positive data by applying an early termination technique to the previously collected character string, and learn the character string using the extracted positive data.

The vehicle license plate recognition method according to an embodiment comprises the steps of setting a license plate detection area in consideration of the ratio of the license plate from image data generated through CCTV, and cropping the license plate at a predicted position by applying convolution to the set license plate detection area. (crop), extracting a character string displayed on the cropped license plate in consideration of the ratio, modifying a network based on the size of a character included in the character string, and classifying it based on the modified network According to the generated text classifier, real-time recognition of each character included in the extracted character string may be included.

The setting of the license plate detection area according to an embodiment includes: setting common areas of areas in which characters of the license plate are identified from previously learned image data as areas of interest, and masking the image data to the set areas of interest. (masking) setting the license plate detection area within one area.

In the step of setting the license plate detection area according to an embodiment, convolution is applied to the set license plate detection area, but the image data layer is not separated and calculated, and a filter is applied by varying the stride. It may include the step of determining the predicted position by applying convolution a plurality of times using one type of filter.

Extracting the character string according to an embodiment includes learning using the cropped license plate, but applying an initialized weight to the cropped license plate instead of the previously learned weight, and It may include the step of simultaneously learning all the layers and recalculating an anchor box corresponding to the learning set.

In the real-time recognition of the character according to an embodiment, the character is recognized using a previously learned character string, but the process of resizing each character included in the previously collected character string and recognizing it by convolving it is repeated. Thus, it may include the step of learning the character string.

Learning the character string according to an embodiment includes extracting positive data by applying an early termination technique to the previously collected character string, and learning the character string using the extracted positive data. can do.

According to one embodiment, it is possible to increase the recognition rate of the vehicle license plate in the security CCTV.

According to an embodiment, it is possible to increase the recognition rate of the license plate even for geometric distortion according to the amount of light, the CCTV installation location, and the vehicle entry direction.

According to an embodiment, it is possible to provide a deep learning-based license plate detection and recognition technology that can improve performance through learning in case of various variables.

1 is a diagram illustrating an apparatus for recognizing a vehicle license plate according to an exemplary embodiment.
FIG. 2 is a diagram illustrating a deep learning-based Automatic License Plate Recognition (ALPR) using a vehicle license plate recognition apparatus according to an exemplary embodiment.
3 is a diagram illustrating an embodiment of recognizing a character by extracting an ROI from image data.
4 is a diagram for explaining an embodiment of setting the size of an input in consideration of the characteristics (ratio) of the license plate in detecting the license plate.
5 to 7 are diagrams for explaining a structure for reducing an amount of convolution calculation due to an increased input image size in detecting a license plate.
8 is a diagram illustrating a model for extracting a character string of a vehicle license plate.
9 is a diagram illustrating test data for learning to extract a character string.
10 is a diagram illustrating a structure for recognizing a character string without using a pre-trained weight.
11 is a diagram illustrating a character string recognition data set according to an embodiment.
12 is a diagram illustrating a method of recognizing a vehicle license plate according to an exemplary embodiment.

Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed in the present specification are exemplified only for the purpose of describing embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention They may be implemented in various forms and are not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can apply various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in the present specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by the terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of the rights according to the concept of the present invention, the first component may be referred to as the second component, Similarly, the second component may also be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle. Expressions describing the relationship between components, for example, "between" and "just between" or "directly adjacent to" should be interpreted as well.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate that the specified features, numbers, steps, actions, components, parts, or combinations thereof exist, but one or more other features or numbers, It is to be understood that the presence or addition of steps, actions, components, parts, or combinations thereof does not preclude the possibility of preliminary exclusion.

Unless otherwise defined, all terms used herein including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be construed as having a meaning consistent with the meaning of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present specification. Does not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. The same reference numerals shown in each drawing indicate the same members.

1 is a diagram illustrating an apparatus 100 for recognizing a vehicle license plate according to an exemplary embodiment.

The vehicle license plate recognition apparatus 100 according to an embodiment may increase the recognition rate of the vehicle license plate in a security CCTV. In addition, it provides a deep learning-based license plate detection and recognition technology that can increase the recognition rate of the license plate for geometric distortions according to the CCTV installation location and the vehicle entry direction, and improve performance through learning when there are various variables. can do.

To this end, the vehicle license plate recognition apparatus 100 according to an embodiment may include a license plate detection unit 110, a character string extraction unit 120, and a character recognition unit 130.

First, the license plate detection unit 110 according to an embodiment may set the license plate detection area in consideration of the ratio of the license plate from image data generated through CCTV. In addition, the license plate may be cropped at a predicted position by applying convolution to the set license plate detection area.

To this end, the license plate detection unit 110 sets the common areas of areas in which the characters of the license plate are identified in the previously learned image data as the ROI, and sets the license plate detection area within the area masked with the set ROI. I can.

In addition, the license plate detection unit 110 may determine a predicted position by applying convolution to the set license plate detection area.

In particular, the license plate detection unit 110 does not separate and calculate the layers of the image data, and determines the predicted position by applying convolution multiple times using one type of filter by varying the stride to which the filter is applied. I can. That is, the license plate detection unit 110 may determine a predicted position by applying different strides as 1 or 2 without applying Max Pooling to the image.

The license plate detection unit 110 according to an embodiment may learn by applying an initialized weight to the generated image data instead of a previously learned weight. In addition, during the learning process, an anchor box corresponding to the learning set may be recalculated and learned by simultaneously learning all layers of image data.

The processing according to the learning may be performed by the learning processing unit 140 under the control of the license plate detection unit 110.

Next, the character string extraction unit 120 may extract the character string displayed on the cropped license plate in consideration of the ratio, and modify the network based on the size of the character included in the character string.

For example, the character string extraction unit 120 may learn using a cropped license plate. Similarly, the character string extraction unit 120 may also control the learning processing unit 140 to learn a character string extraction process for the license plate.

The string extraction unit 120 according to an embodiment applies an initialized weight to a license plate cropped for learning, instead of a weight learned in advance, and simultaneously learns all layers of the cropped license plate to obtain an anchor corresponding to the learning set. It can be processed to recalculate the anchor box.

The character string extraction unit 120 according to an embodiment may construct a learning set using training data, test data, and positive data in which an object exists in an image. In particular, at least one of training data, test data, and positive data constituting the training set may include an object label, a bounding box midpoint x coordinate, a bounding box midpoint y coordinate, a bounding box width, and a bounding box height.

Next, the character recognition unit 130 may recognize each character included in the extracted character string in real time according to the text classifier classified based on the modified network.

The character recognition unit 130 according to an embodiment recognizes the character using a previously learned character string, but resizes each character included in the previously collected character string, and then convolves to recognize the character. It is possible to control the learning processing unit 140 to learn.

In particular, the character recognition unit 130 may extract positive data by applying an early termination technique to a previously collected character string, and learn the character string using the extracted positive data.

FIG. 2 is a diagram illustrating a deep learning-based automatic license plate recognition (ALPR) procedure 200 using a vehicle license plate recognition apparatus according to an exemplary embodiment.

According to the present invention, when there are various variables, it is possible to detect and recognize a license plate based on deep learning that can improve performance through learning.

Reference numeral 210 denotes an input image, for example, since an image collected from a CCTV is input, the resolution of the image resolution may be 1920 x 1080 or 1280 x 720.

In addition, the image input method mainly uses RTSP communication, but there may be variations.

Since the hardware must operate at a relatively low performance compared to the desktop, it must be implemented with an algorithm with a small amount of computation.

In particular, even if a blur phenomenon occurs in the image due to the movement of the vehicle and text information in the license plate is lost, the license plate must be detected. For example, even if it is not recognized well, it can be of great help in the actual field if it is detected well.

To this end, in the present invention, a license plate detection region may be set as shown by reference numeral 220 so that only the region of interest 221 may be cropped.

For example, in order to apply a deep learning model robust to various variables to each ALPR module, the license plate detection process 230 may use an open source YOLOv3-tiny.

The YOLOv3-tiny model can be implemented through high detection performance and relatively low computational load, and new image inputs that can be operated in various CCTV environments can be received.

In the present invention, based on the features analyzed through convolution, the license plate region 241 may be extracted from the region of interest as shown by reference numeral 240.

In addition, in the extracted license plate area 250, a character string 270 may be extracted from the vehicle license plate through the process of reference numeral 260.

The extracted character string 270 may be recognized as a character in the license plate of the vehicle based on deep learning through the process of reference numeral 280. In this process, characters can be recognized based on CNN.

The recognized character may be identified by the recognized vehicle number as shown by reference numeral 290.

3 is a diagram illustrating an embodiment of recognizing a character by extracting a region of interest from image data.

For license plate detection, a region of interest (ROI) can be set. That is, the area in which the characters of the license plate are identified must be set, and the characters of the license plate must be identified in the image in the recognition step.

In general, there may be a problem that occurs in the conventional case in which a region other than the region of interest is not excluded together with the region of interest as shown by reference numeral 310.

Specifically, the size of the input image of the deep learning model for object detection may be mostly fixed.

For example, in the case of a fully convolution structure, the size of the input image may vary, but the output size of the detection layer may be uneven. In addition, all images must be input after being resized to fit the size of the input image.

In addition, when a cropped image is used as an input image without worrying about the ratio of the region of interest, when the aspect ratio and size of the region of interest are different, a difference in structural information may occur.

Accordingly, in the present invention, an image 321 in which the region of interest is masked as shown by reference numeral 320 may be used as an input image. Accordingly, it is always possible to resize at a constant ratio regardless of the size and ratio of the region of interest. In addition, as structural information due to down-sampling and up-sampling, distortion with respect to the ratio and size can be minimized.

4 is a diagram for explaining an embodiment in which the size of an input is set in consideration of a characteristic (ratio) of a license plate in detecting a license plate.

Reference numeral 410 corresponds to an input image of 1920x1080 resolution. In addition, reference numeral 420 denotes a resolution of 864 x480 as an input extracted by applying masking in an ROI as the ratio of the input image.

Since the input to which the masking is applied follows the ratio of the input image, distortion to the original is not large. In the past, since resizing was performed using a ratio that was not related to the ratio of the input image, distortion of the input occurred, which directly affected the decrease in the license plate recognition rate. For example, in the case of using YOLOv3 tiny, since the input was resized to 416 x 416, distortion of the license plate occurred, which caused the recognition rate to deteriorate.

That is, the present invention can process the input by modifying the input image size and Aspect Ratio in consideration of the general CCTV image size (FHD, HD) and Aspect Ratio (16:9).

In other words, if a 1920x1080 image is processed as it is, the required convolution operation amount and memory size may be too large, and the present invention can minimize loss of structural information (high frequency component) of an object due to down-sampling and up-sampling.

5 to 7 are diagrams for explaining a structure for reducing an amount of convolution calculation due to an increased input image size in detecting a license plate.

First, referring to FIG. 5, reference numeral 510 denotes an example of a layer (0 to 3) structure for detecting a license plate, and reference numeral 520 denotes an example of a layer (0 to 3) structure according to the present invention. .

As shown in reference numeral 510, convolution was applied to the previously set license plate detection area, but the image data layer was separated and calculated in consideration of the scale of Max pooling. This has a significant amount of convolutional computation due to the increased input image size.

Meanwhile, as shown by reference numeral 520, the license plate detection unit 110 according to the present invention applies convolution to the set license plate detection area, but does not separate and calculate the image data layer. That is, the predicted position is determined by applying convolution multiple times using one type of filter while varying the stride to which the filter is applied. For example, in the present invention, a convolution operation by filter 16 and filter 32 is performed while changing the interval to 1 and 2.

The amount of computation in this process can be seen through FIG. 6.

First, reference numeral 610 denotes the operation of separating and calculating layers of image data, and it can be seen that the amount of operation is not reduced according to the Max Pooling process of separating and calculating the image data.

On the other hand, reference numeral 620 does not separate and calculate the layers of the image data, but differs in the stride for applying the filter, and thus it can be seen that the amount of calculation is remarkably reduced.

7 may show the structure 700 of a network for learning according to the present invention.

For example, reference numeral 710 denotes a feature extracted from each input through Feature Extractor, and reference numeral 720 denotes results of up-sampling through Bounding Box & Class Regressor.

That is, a model considering the size of the license plate detected in the CCTV image can be implemented through the structure 700 of the LPD Net (License Plate Detection Net) in consideration of the characteristics of the license plate appearing in the CCTV image.

In the case of the previously proposed object detection-related deep learning models, they are designed based on the detection performance of a data set in which objects of small, medium, and large sizes appear. However, the width and height of most license plates detected in security CCTV images (FHD (1920x1080)) are less than 10% of the total width and height of the image, which is very small compared to the total image size. Not only is the contrast efficiency low, but also the accuracy is low.

A model suitable for detecting objects having a small size was needed, and this model can be implemented through the structure according to the present invention.

For learning, an image masked in the region of interest is required. In addition, training data and test data exist, and the format of the data may include <object label> <bounding box midpoint x coordinate> <bounding box midpoint y coordinate> <bounding box width> <bounding box height>. In addition, x and y coordinates and width and height information have a range normalized by the image width and height.

As a learning method, it is possible to learn from initialized values without using pre-learned weights. In addition, all layers of input data (feature extractor + object detector) can be learned at the same time, and model parameters can be changed by recalculating an anchor box suitable for the training set.

8 is an embodiment 800 illustrating a model for extracting a character string of a vehicle license plate.

As shown in FIG. 8, the present invention proposes a model for extracting a license plate string.

In particular, the input image 810 may be extracted as a character string by applying the YOLO-T open source 830 to the resized image 820 that is masked at the same ratio and then resized.

9 is an embodiment 900 illustrating test data for learning to extract a character string.

As shown in FIG. 9, the extracted character string may be learned by 1,000 pieces of training data, 167 pieces of test data, and positive data in which an object exists in the image. The data format of the extracted string can also include <object label> <bounding box midpoint x coordinate> <bounding box midpoint y coordinate> <bounding box width> <bounding box height>, and information about x, y coordinates, width, and height. Can have a range normalized by the image width and height.

In order to extract a character string, the present invention can process learning from an initialized value without using a pre-learned weight. In addition, the entire layer (feature extractor + object detector) can be learned at the same time, and the anchor box suitable for the training set can be recalculated and used for learning.

10 is an embodiment 1000 illustrating a structure for recognizing a character string without using a pre-trained weight.

The input image 1010 may be converted to a size of 64X64X3 through the resizing process 1020. In addition, the resized input image may be processed into a structure similar to Vgg19 at 1030 to be recognized as a character string.

11 is an embodiment 1100 illustrating a character string recognition data set according to an embodiment.

According to the embodiment 1100, a data set recognized as a character string may be used for learning.

For example, the training data may be 1,000 pieces for each number and 50 pieces for each letter, and the test data at this time may use the training data as it is. In this process, an early termination technique can be used. In order to be recognized as text, it can be mostly positive data.

According to an embodiment, in the present invention, an input image may be processed as an input by randomly changing an angle or size for learning. As a result, it is possible to maintain the recognition rate of the license plate even by various variables.

In addition, in the present invention, it is possible to provide learning data (or learning results) located in area A to area B. For example, the shape of license plates photographed in area A is highly likely to be uniform according to the location and angle of CCTV installations and lanes. On the other hand, in area B, the location and angle of CCTV installations, and the relative position of the lanes are different from those in area A, so it is highly likely that the shape of the license plates being photographed differs from the shape in area A. Although it is unlikely that the license plate will be photographed in area B in area A, in exceptional cases, the license plate may be photographed in area B in area A as well. In this case, there is a high possibility that the license plate is not recognized in area A, and in consideration of this case, the present invention may contribute to improvement of the recognition rate by collecting and sharing the results of learning in different areas.

12 is a diagram illustrating a method of recognizing a vehicle license plate according to an exemplary embodiment.

In the vehicle license plate recognition method according to an embodiment, the license plate detection area may be set in consideration of the ratio of the license plate from image data generated through CCTV (step 1201).

For example, in order to set the license plate detection area, common areas of areas in which characters of the license plate are identified in the previously learned image data may be set as the ROI. In addition, a license plate detection area may be set within an area in which image data is masked as a set region of interest.

In addition, in order to set the license plate detection area, convolution can be applied to the set license plate detection area. To this end, the predicted position may be determined by applying convolution multiple times using one type of filter by varying the stride for applying the filter without separating and calculating the image data layers.

Next, in the vehicle license plate recognition method according to an embodiment, a license plate may be cropped at a predicted position by applying convolution to a set license plate detection area (step 1202).

In the vehicle license plate recognition method according to an embodiment, a character string displayed on the cropped license plate may be extracted in consideration of the ratio of the license plate (step 1203), and the network may be modified based on the size of characters included in the character string (step 1204). .

For example, in order to extract a character string, it is learned using a cropped license plate, but an initialized weight may be applied to the cropped license plate instead of the previously learned weight. In addition, it is possible to recalculate the anchor box corresponding to the learning set by learning all the layers of the cropped license plate at the same time.

According to the text classifier classified based on the modified network, each character included in the extracted character string may be recognized in real time (step 1205).

For example, in order to recognize a character in real time, a character may be recognized using a previously learned character string. In addition, the character string may be learned by repeating a process of resizing and convolving to recognize each character included in the previously collected character string.

For example, in order to learn a string, positive data can be extracted by applying an early termination technique to a previously collected string, and the string can be learned using the extracted positive data.

Consequently, by using the present invention, it is possible to increase the recognition rate of the vehicle license plate in the security CCTV. In addition, according to an embodiment, the recognition rate of the license plate can be increased even for geometric distortions according to the amount of light, the CCTV installation location, and the vehicle entry direction, and deep learning that can improve performance through learning about various variables. It can provide a license plate detection and recognition technology based.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. Further, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or, to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

As described above, although the embodiments have been described by the limited drawings, various modifications and variations are possible from the above description to those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and those equivalent to the claims also fall within the scope of the claims to be described later.

Claims (14)

A license plate detection unit configured to set a license plate detection area in consideration of the ratio of the license plate from the image data generated through CCTV, and crop the license plate at a predicted position by applying convolution to the set license plate detection area;
A character string extracting unit for extracting the character string displayed on the cropped license plate in consideration of the ratio and modifying the network for learning based on the size of the character included in the character string; And
A character recognition unit for real-time recognition of each character included in the character string extracted by the character string extraction unit based on the modified network,
The license plate detection unit,
Applying convolution to the set license plate detection area, but without separating and calculating the image data layers, and applying a filter without applying maximum pooling to the image data is set to 1 or A vehicle license plate recognition device that determines the predicted position by applying convolution a plurality of times using filter 16 and filter 32, which are one type of filter differently to 2. The method of claim 1,
The license plate detection unit,
A vehicle license plate recognition apparatus configured to set common regions of regions in which characters of a license plate are identified in previously learned image data as regions of interest, and to set the license plate detection region within a region masked with the set region of interest. delete The method of claim 1,
The license plate detection unit,
The generated image data is learned by applying an initialized weight instead of a previously learned weight, but learning by recalculating an anchor box corresponding to the training set by simultaneously learning all layers of the image data. Vehicle license plate recognition device. The method of claim 1,
The character string extraction unit,
However, learning using the cropped license plate,
For the cropped license plate, an initialized weight is applied instead of the previously learned weight, and a feature extractor and an object detector, which are all layers of the cropped license plate, are simultaneously learned to correspond to a learning set. A vehicle license plate recognition device that recalculates the anchor box. The method of claim 5,
The character string extraction unit,
The training set is constructed using training data, test data, and positive data in which an object exists in an image, and at least one of the training data, the test data, and the positive data is an object label, a bounding box center point x coordinate, and a bounding. A vehicle license plate recognition device including the box weight y coordinate, the bounding box width, and the bounding box height. The method of claim 1,
The character recognition unit,
Recognizing the character using the previously learned character string,
A vehicle license plate recognition device that learns the character string by repeating a process of resizing and convoluting and recognizing each character included in the previously collected character string. The method of claim 7,
The character recognition unit,
A vehicle license plate recognition device that extracts positive data by applying an early termination technique to the previously collected character string, and learns the character string using the extracted positive data. Setting a license plate detection area in consideration of the ratio of the license plate from the image data generated through CCTV;
Cropping the license plate at the predicted position by applying convolution to the set license plate detection area;
Extracting a character string displayed on the cropped license plate in consideration of the ratio;
Modifying a network for learning based on the size of characters included in the character string; And
Including the step of real-time recognition of each character included in the character string extracted by the character string extraction unit based on the modified network,
The step of setting the license plate detection area,
Applying convolution to the set license plate detection area, but without separating and calculating the image data layers, and applying a filter without applying maximum pooling to the image data is set to 1 or A vehicle license plate recognition method for determining the predicted position by applying convolution a plurality of times by using a filter 16 and a filter 32, which are one type of filter differently to 2. The method of claim 9,
The step of setting the license plate detection area,
Setting common areas of areas in which characters on the license plate are identified as areas of interest in the previously learned image data; And
Setting the license plate detection area within an area in which image data is masked as the set area of interest
Vehicle license plate recognition method comprising a. delete The method of claim 9,
The step of extracting the character string,
Learning using the cropped license plate, but applying an initialized weight to the cropped license plate instead of the previously learned weight; And
And recalculating an anchor box corresponding to a learning set by simultaneously learning a feature extractor and an object detector, which are all layers of the cropped license plate. The method of claim 9,
The step of real-time recognition of the character,
Recognizing the character by using the previously learned character string, resizing each character included in the previously collected character string, and re-recognizing the character by convolution to learn the character string
Vehicle license plate recognition method comprising a. The method of claim 13,
The step of learning the character string,
Extracting positive data by applying an early termination technique to the previously collected character string; And
Learning a character string using the extracted positive data
Vehicle license plate recognition method comprising a.

Images