WO2022065547A1

WO2022065547A1 - Method for recognizing license plate by using hybrid technique, and system therefor

Info

Publication number: WO2022065547A1
Application number: PCT/KR2020/012962
Authority: WO
Inventors: 김상훈; 최진욱; 김희원
Original assignee: 주식회사 키센스
Priority date: 2020-09-24
Filing date: 2020-09-24
Publication date: 2022-03-31

Abstract

Disclosed are a method for recognizing a license plate by using a hybrid technique, and a system therefor. The method for recognizing a license plate according to an embodiment of the present invention may comprise the steps of: when an input image is received, extracting a license plate area from the input image by using a neural network of a pre-learned learning model; and recognizing identification information on a license plate of a corresponding object from the extracted license plate area by using a preconfigured recognition technique, wherein the recognition technique includes an optical character recognition (OCR).

Description

License plate recognition method and system using hybrid technique

The present invention relates to license plate recognition technology, and more particularly, to a method and system capable of improving recognition accuracy for a license plate attached to an object using a hybrid technique combining two different techniques.

In recent years, by leaps and bounds in biomedical technology, the LPR (License plate recognition) system is a device that extracts license plate number data from images captured by a camera, and has previously been used to detect speeding vehicles on highways. It was used to automatically crack down or detect illegal parking on the street.

In addition, in recent years, it has been introduced and widely used in an intelligent parking management system that enables quick and convenient entry and exit of a parking lot.

Recently, a license plate recognition system is widely used in various fields such as parking, crime prevention, signal violation, speed violation, and the like.

However, the conventional vehicle license plate recognition system generally has many difficulties in extracting the license plate due to a difference in light and shade due to natural phenomena, a lot of noise in rainy weather, or reflection of vehicle light light at night.

Therefore, in order to more accurately extract the license plate area, many pre-processing steps such as image contrast change, contour extraction, noise removal, and license plate candidate area extraction are required.

In particular, in the pre-processing process, the contour detection algorithm among the existing algorithms uses a Sobel mask pattern algorithm a lot, and as the noise removal algorithm, noise removal using an erosion and dilation algorithm is widely used.

The present invention proposes a method and system capable of improving license plate recognition accuracy using a hybrid technique that combines a deep learning neural network for extracting a license plate region and an optical character recognition technique (OCR) for recognizing identification information in the license plate region.

Embodiments of the present invention provide a method and system capable of improving the recognition accuracy for a license plate attached to an object using a hybrid technique combining two different techniques and a system.

Specifically, embodiments of the present invention improve license plate recognition accuracy by using a hybrid technique that combines a deep learning neural network for extracting a license plate region from an image of an object and an optical character recognition technique (OCR) for recognizing identification information in the license plate region A method and a system for doing so are provided.

The license plate recognition method according to an embodiment of the present invention comprises the steps of extracting a license plate area from the input image using a neural network of a pre-trained learning model when an input image is received; and recognizing identification information on the license plate of the corresponding object from the extracted license plate area using a preset recognition technique.

The recognition technique may include optical character recognition (OCR).

The recognizing may perform image correction on the extracted license plate area, and recognize the identification information from the image corrected license plate area using the recognition technique.

In the recognizing step, when the contrast of the extracted license plate area is lower than the preset reference contrast, the contrast of the extracted license plate area is adjusted to the reference contrast and then the identification information is obtained using the recognition technique. can recognize

Furthermore. The license plate recognition method according to an embodiment of the present invention further includes generating a learning model for extracting a license plate area attached to the object through learning using image data including a preset object, the step of extracting may extract the license plate area from the input image using the neural network of the generated learning model.

The license plate recognition system according to an embodiment of the present invention includes: an extractor for extracting a license plate area from the input image using a neural network of a pre-trained learning model when an input image is received; and a recognition unit for recognizing identification information on the license plate of the corresponding object from the extracted license plate area using a preset recognition technique.

The recognition technique may include optical character recognition (OCR).

The recognition unit may perform image correction for the extracted license plate region, and recognize the identification information from the image corrected license plate region using the recognition technique.

When the contrast of the extracted license plate area is lower than the preset reference contrast, the recognition unit adjusts the contrast of the extracted license plate area to the reference contrast and then uses the recognition technique to recognize the identification information can

Furthermore, the license plate recognition system according to an embodiment of the present invention further comprises a generator for generating a learning model for extracting a license plate area attached to the object through learning using image data including a preset object, the extraction The unit may extract the license plate area from the input image by using the neural network of the generated learning model.

According to embodiments of the present invention, an object image, for example, a license plate area attached to an object from a car image is extracted using a neural network of a pre-trained learning model, for example, CNN, and another technique example For example, by using an optical character recognition technique (OCR) to recognize the identification information contained in the license plate region from the extracted license plate region, for example, characters, it is possible to improve the license plate recognition rate and improve recognition accuracy.

According to embodiments of the present invention, by using a hybrid technique that combines a neural network and OCR technique using deep learning, the extraction accuracy for the license plate area attached to the object is improved, and the OCR technique after image correction for the extracted license plate area By recognizing the identification information of the license plate by using it, it is possible to overcome the disadvantages of the OCR method and the disadvantages of deep learning, thereby reducing the cost and improving the recognition accuracy.

1 shows an exemplary diagram of a system for explaining the present invention.

Figure 2 shows an operation flowchart for a license plate recognition method using a hybrid technique according to an embodiment of the present invention.

3 shows an exemplary view for explaining the method of the present invention.

Figure 4 shows the configuration of a license plate recognition system using a hybrid technique according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be embodied in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments, and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

The OCR method has the advantage of being very fast and inexpensive compared to the deep learning method, but the accuracy of locating the license plate from the image using computer vision technology is lower than that of deep learning, and it is too bright, dark, shadow, backlight, etc. It is weak against the noise of the image, it can mistake the continuous patterns in the image as the character area, and because it tries to extract the number from a non-car, it can be misjudged by something similar to the license plate. There is a problem in that it is difficult to extract the license plate area.

The deep learning method has a problem in that the accuracy of finding license plates is relatively higher than the OCR method and the recognition accuracy is also high, but the cost is higher than the OCR method.

Embodiments of the present invention improve the recognition accuracy for a license plate attached to an object by using a hybrid technique that combines the advantages of two different methods, for example, the advantages of the deep learning method and the advantages of the OCR method. make that the gist of it.

Here, the present invention extracts a license plate area attached to an object from an object image, for example, a car image, using a neural network of a pre-trained learning model, for example, CNN, and another technique, for example, OCR By recognizing the identification information contained in the license plate area from the extracted license plate area using the method, for example, by recognizing characters, the disadvantages of the OCR method and the disadvantages of deep learning are overcome to reduce the cost while reducing the extraction accuracy of the license plate area and the license plate can improve the recognition accuracy of

Furthermore, in order to extract a license plate area attached to an object using a neural network, for example, a convolutional neural network (CNN), the present invention learns a neural network using image data including preset objects, for example, automobiles. By doing so, a learning model or extraction model for extracting the license plate region is created, and the license plate region can be accurately extracted from the input image captured through a shooting means, for example, CCTV, etc. using a neural network having the generated learning model. .

Furthermore, the present invention provides image correction for the license plate region because, when the license plate region is extracted from the object, the license plate region may not be clear due to noise included in the extracted license plate region, for example, too bright, dark, shadow, backlight, etc. After performing the OCR technique for the image-corrected license plate region, it is possible to recognize the identification information included in the license plate region.

The object in the present invention may include all kinds of objects including automobiles, motorcycles, etc. to which a license plate in which identification information is written can be attached.

Hereinafter, in the detailed description of the present invention, an object is limited to a vehicle, but the object in the present invention is not limited to a vehicle and can include all kinds of objects that can identify the object by attaching a license plate. It is apparent to those skilled in the art.

1 is a diagram illustrating a system according to an embodiment of the present invention.

As shown in Fig. 1, the system in the present invention receives the input image input from the image photographing means 100 for photographing the image, for example, CCTV and the image photographing means 100, and is included in the input image. Extracts the license plate area of the object, and includes a license plate recognition system 200 for recognizing identification information included in the extracted license plate area.

The image photographing means 100 is disposed in a certain place to photograph an image of an object, and may be provided in a place where a car is driven, such as an apartment, a road, a parking lot, etc. Or, you can change directions at the location, so you can shoot in multiple directions.

At this time, the image capturing means 100 may provide the image captured in real time to the license plate recognition system through the network, and when providing the image to the license plate recognition system, identification information for the image capturing means, image shooting time, etc. information can be provided together.

The image photographing means 100 may be connected to the license plate recognition system through a network, and the communication method between the image photographing means and the distributed in-time system is not limited, and the network (eg, mobile communication network, wired Internet) that the network may include , wireless Internet, broadcasting network) as well as short-distance wireless communication between devices may be included. For example, the network includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. may include any one or more of the networks of Further, the network may include, but is not limited to, any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, and the like. .

The license plate recognition system 200 receives an input image received through the image capturing means 100, for example, an input image including a car, and learns in advance the license plate localization of the vehicle included in the received input image. The neural network of the trained learning model, for example, is extracted using a convolutional neural network, and the identification information (recognition) included in the license plate is recognized from the extracted license plate region using the OCR method.

Here, the license plate recognition system 200 may generate a learning model of a neural network for extracting the license plate region of a vehicle through training or learning using the learning data, and the learning data for generating the learning model is a clear image In addition, the image may include an image to which a preset noise pattern, for example, a noise pattern by light or a beam, or a noise pattern by a shadow or darkness, is applied to a clear image.

That is, the license plate recognition system 200 of the present invention generates a learning model of a neural network through training using training data including an image containing various noises as well as an image in which the object and the license plate area are clear, when the input image is clear In addition, even when noise is included, the license plate area attached to the object can be accurately extracted from the input image.

Of course, the license plate recognition system 200 of the present invention has been described as identification information recognition using the license plate region extraction and OCR technique by the neural network based on the learning model, but is not limited thereto, and the license plate is combined with the deep learning technique rather than the OCR technique It is also possible to apply various techniques that can reduce the cost while improving the recognition accuracy of the identification information.

A number plate recognition method in such a system will be described with reference to FIGS. 2 and 3 as follows.

Figure 2 shows an operation flowchart for a license plate recognition method using a hybrid technique according to an embodiment of the present invention, Figure 3 shows an exemplary diagram for explaining the method of the present invention.

2 and 3, the license plate recognition method according to an embodiment of the present invention is image data, for example, learning data for extracting a license plate area included in an object including a clear image and an image containing noise Creates a learning model of the neural network that extracts the license plate area attached to the object through learning (S210).

Here, in step S210, the predetermined learning data may be used, but image data including noise is generated using the predetermined learning data and predetermined noise patterns, and training using the training data to which the generated image data is added is performed. Through this, it is also possible to create a learning model or an extraction model for extracting the license plate area from the object. Of course, noise patterns for additionally generating image data used in the learning model include all kinds of noise that may be generated in extracting the license plate area, for example, various kinds of noise patterns such as blur, shadow, darkness, backlight, etc. may include

When a learning model for extracting an object, for example, a license plate area of a car is generated by step S210, the input image received in real time from the image capturing means using a neural network based on the learning model or data is stored without being received in real time Extracts an object, that is, the license plate area of the vehicle from the input image received from the storage means (S220).

For example, as shown in FIG. 3 , a neural network based on a learning model, for example, a vehicle included in the input image from the input image 310 received using a deep-learning AI 311 . The license plate area 320 is extracted (plate area detection) 350 .

Here, since the license plate area extracted by step S220 may contain noise, image compensation for removing such noise or correcting a blurred image into a clear image because the license plate area may be a blurred image (image compensation 360) process can be performed.

For example, when the contrast of the license plate area extracted by step S220 is lower than the preset reference contrast, by adjusting the contrast of the extracted license plate area using the image correction function to a certain contrast, for example, the reference contrast, The extracted license plate area can be processed clearly. Of course, the image correction function is not limited to adjusting the contrast, and various types of image correction functions for sharpening the image may be included.

That is, the method according to an embodiment of the present invention may generate an image 330 of the image corrected license plate area by performing image correction on the image of the extracted license plate area as shown in FIG. 3 .

By performing an image correction function on the license plate area extracted by step S220, when an image corrected license plate area is created, a recognition technique preset for the image corrected license plate area, for example, an image corrected license plate area using an OCR technique Identification information on the license plate of the object included in the input image, for example, recognizes the vehicle number (S230)

That is, as shown in Figure 3, by performing the recognition (character recognition) 370 the characters included in the license plate area from the image 330 of the image corrected license plate area using the OCR technique 341, the license plate area identification information of, for example, a vehicle number "13838B2" (340) is recognized.

As such, the method according to embodiments of the present invention extracts a license plate area attached to an object from an object image, for example, a car image, using a neural network based on a pre-trained learning model, for example, CNN, and another technique For example, by recognizing identification information, for example, characters included in the license plate region from the license plate region extracted using the OCR technique, it is possible to improve the license plate recognition rate and improve recognition accuracy.

In addition, the method according to the embodiments of the present invention improves the extraction accuracy for the license plate area attached to the object using a deep learning neural network, and uses the OCR technique after image correction for the extracted license plate area identification information of the license plate By recognizing , it is possible to improve the recognition accuracy while reducing the cost by overcoming the disadvantages of the OCR method and the disadvantages of the deep learning using a hybrid technique that combines the deep learning technique and the OCR technique.

4 shows a configuration for a license plate recognition system using a hybrid technique according to an embodiment of the present invention, and shows a conceptual configuration of the system performing the above-described FIGS. 1 to 3 .

Referring to FIG. 4 , the license plate recognition system 400 according to an embodiment of the present invention includes a generating unit 410 , an extracting unit 420 and a recognition unit 430 .

The generator 410 is image data, for example, a neural network for extracting a license plate region attached to an object through learning using learning data for extracting a license plate region included in an object including a clear image and an image including noise. create a learning model of

In this case, the generator 410 may use predetermined training data, but generates image data including noise by using the predetermined training data and predetermined noise patterns, and adds the generated image data to the training data. It is also possible to create a learning model or an extraction model for extracting the license plate area from the object through training using. Of course, noise patterns for additionally generating image data used in the learning model include all kinds of noise that may be generated in extracting the license plate area, for example, various kinds of noise patterns such as blur, shadow, darkness, backlight, etc. may include

The extraction unit 420 extracts an object, that is, a license plate area of a vehicle, from the input image received by using the neural network based on the learning model generated by the generation unit 410 .

The recognition unit 430 recognizes the identification information included in the license plate of the object from the license plate area extracted using a preset recognition technique, for example, OCR technique.

At this time, the recognition unit 430 removes such noise because the license plate area extracted by the extraction unit 420 may contain noise, or the license plate area may be a blurred image. An image correction process may be performed.

For example, when the contrast of the extracted license plate region is lower than the preset reference contrast, the recognition unit 430 adjusts the contrast of the extracted license plate region to a certain contrast, for example, the reference contrast by using the image correction function, the extracted license plate Areas can be processed clearly.

That is, the recognition unit 430 generates an image of the image corrected license plate region by performing image correction on the image of the license plate region extracted by the extraction unit 420, and OCR technique for the image corrected license plate region. Identification information on the license plate of the object included in the input image, for example, the vehicle number, is recognized from the image-corrected license plate area using the image.

Although the description of the system of FIG. 4 is omitted, it is obvious to those skilled in the art that the system according to the present invention may include all the contents described in the method of FIGS. 1 to 3 .

The system or apparatus described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, the systems, devices, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA). ), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiments may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims

When an input image is received, extracting a license plate area from the input image using a neural network of a pre-trained learning model; and

Recognizing identification information on the license plate of the object from the extracted license plate area using a preset recognition technique

A license plate recognition method comprising a.
According to claim 1,

The recognition technique is

License plate recognition method comprising an optical character recognition technique (OCR; optical character recognition).
According to claim 1,

The recognizing step

A license plate recognition method, characterized in that performing image correction for the extracted license plate region, and recognizing the identification information from the image corrected license plate region using the recognition technique.
According to claim 1,

The recognizing step

When the contrast of the extracted license plate area is lower than the preset reference contrast, the identification information is recognized using the recognition technique after adjusting the contrast of the extracted license plate area to the reference contrast How to recognize a license plate.
According to claim 1,

Creating a learning model for extracting a license plate area attached to the object through learning using image data including a preset object

further comprising,

The extraction step

License plate recognition method, characterized in that extracting the license plate area from the input image by using the neural network of the generated learning model.
When an input image is received, an extractor for extracting a license plate area from the input image using a neural network of a pre-trained learning model; and

Recognition unit for recognizing identification information on the license plate of the object from the extracted license plate area using a preset recognition technique

A license plate recognition system comprising a.
7. The method of claim 6,

The recognition technique is

License plate recognition system, characterized in that it comprises an optical character recognition technique (OCR; optical character recognition).
7. The method of claim 6,

the recognition unit

A license plate recognition system, characterized in that performing image correction on the extracted license plate region, and recognizing the identification information from the image corrected license plate region using the recognition technique.
7. The method of claim 6,

the recognition unit

When the contrast of the extracted license plate area is lower than the preset reference contrast, the identification information is recognized using the recognition technique after adjusting the contrast of the extracted license plate area to the reference contrast license plate recognition system.
7. The method of claim 6,

A generation unit that generates a learning model that extracts a license plate area attached to the object through learning using image data including a preset object

further comprising,

The extraction unit

License plate recognition system, characterized in that extracting the license plate area from the input image using the neural network of the generated learning model.