KR102509313B1 - License plate recognition method and license plate recognition system - Google Patents

Abstract

The present invention may provide a vehicle license plate recognition method, which is characterized by comprising the steps of: extracting a detection area from a received image and detecting a vehicle in the detection area; and extracting, when the vehicle is detected, the recognition area and recognizing a vehicle number of the vehicle in the recognition area. The method of detecting a vehicle includes a first resolution change phase of reducing the image by changing the resolution of the image to a set value to increase the detection and recognition speed, a detection area extraction phase of extracting a detection area by cutting a random location from the image the size of which has been reduced due to the resolution change, an image change phase of reducing noise in the extracted detection area, and a vehicle detection phase of detecting a vehicle by determining the presence of a vehicle based on changes in pixel image values in the detection area. According to the present invention, improved detection of edges can be achieved by eliminating noise as much as possible by resizing an initial image and resizing an extracted detection area, and the processing speed can be improved by reducing the processing capacity.

Description

License plate recognition method and license plate recognition system {License plate recognition method and license plate recognition system}

The present invention relates to a vehicle license plate recognition method and license plate recognition system, and more particularly, to a vehicle license plate recognition method and license plate recognition that can be applied in various ways such as road crime prevention, parking control, and speed enforcement and can improve server processing speed It's about the system.

In general, a license plate is a plate attached to the front or rear of a vehicle and written with standardized letters (including letters, numbers, and symbols) in a specific pattern. It is possible to check basic information, and it is also possible to check important information such as accident history, maintenance history, and vehicle registration status while driving.

On the other hand, license plate recognition acquires the license plate of the vehicle from the image and recognizes the number through an inherent algorithm. Through this license plate recognition, it is used in connection with road traffic safety and billing, such as speeding enforcement, parking violations, and parking lot access. .

Furthermore, this vehicle license plate recognition technology is a basic technology of social safety net, and its application range is diversifying, and vehicle license plate recognition technology includes a character-based detection method and an edge-based detection method.

However, this conventional vehicle license plate recognition technology has a problem in that the camera is sensitive to camera shaking due to wind or rain as the camera is installed outside, and a lot of noise is generated due to this, and also has a problem of slow processing speed.

Republic of Korea Patent Registration No. 10-2272279

An object of the present invention is to provide a vehicle license plate recognition method and license plate recognition system capable of effectively reducing noise, improving edge search effects, and preventing a decrease in processing speed.

According to one aspect of the present invention, the present invention includes the steps of extracting a detection area from a received image and detecting a vehicle in the detection area; When a vehicle is detected, extracting a recognition area from the image and recognizing the license plate number in the recognition area; wherein, in the step of detecting the vehicle, the resolution of the image is set to a first set value in order to increase the detection speed. A first resolution change step of reducing the image by changing the resolution; a detection region extraction step of extracting a detection region by cutting the image whose size is reduced due to the change in resolution in the form of a band of the total width and set height; and the extracted detection region. It is possible to provide a vehicle license plate recognition method comprising an image change step of reducing noise and a vehicle detection step of detecting a vehicle by determining that there is a vehicle based on a change in the pixel image value of the detection area. .

Here, the image changing step includes a color conversion step of converting the color of the detection area to black and white, and a second resolution change of reducing the image by changing the resolution of the detection area whose color is converted to black and white to a second set value. steps may be included.

In addition, the vehicle detection step may include calculating a pixel image value change amount of the detection area, calculating an average pixel image value change amount of the image reception set number of frames, and determining the pixel image value change amount of the detection area and the Comparing average pixel image value variation and detecting the vehicle by determining that there is a vehicle when the pixel image value variation of the detection area is greater than a set multiple of the average pixel image value variation.

In addition, the recognizing the vehicle number may include extracting a recognition area by cutting a set position in the image, recognizing a license plate in the extracted recognition area using a deep learning algorithm, and a license plate in the recognition area. A step of extracting a license plate area by cutting the area where this location is located, and recognizing the vehicle number in the license plate area may be included.

Recognizing the license plate number may include cutting and extracting the license plate area from the original image, recognizing characters in the license plate area from the original image using a deep learning algorithm, and recognizing the recognized characters based on set coordinates. It may include arranging from left to right to complete the license plate number.

According to another aspect of the present invention, the present invention includes a vehicle detecting unit extracting a detection area from a received image and detecting a vehicle in the detection area; When a vehicle is detected by the vehicle detection unit, a vehicle number recognition unit extracts a recognition area from the image and recognizes a license plate in the recognition area; A resolution changing unit that reduces the image by changing the resolution to a set value, a detection area extraction unit that extracts a detection area by cutting an arbitrary position in the image whose size has been reduced due to the change in resolution, and noise from the extracted detection area. It is possible to provide a vehicle license plate recognition system comprising: an image changing unit that reduces and a detecting unit that detects a vehicle by determining that a vehicle exists based on a change in pixel image value of the detection area.

Here, the license plate recognition unit includes a recognition area extraction unit for extracting a recognition area by cutting a set position in the image, a license plate recognition unit for recognizing a license plate in the recognition area extracted using a deep learning algorithm, and the recognition area It may include a number recognition unit that cuts an area where a license plate is located, extracts a license plate area, and recognizes a vehicle number in the license plate area.

The license plate recognition method and license plate recognition system according to the present invention can provide the following effects.

First, by resizing the extracted detection area as well as the resizing of the initial image, noise can be removed as much as possible and edge detection can be improved.

Second, unlike the conventional method of extracting a specific area from an image, the detection area is extracted by cutting the reduced image in the form of a thin strip in pixel units, and then the vehicle is detected by calculation based on the extracted detection area image, so the throughput is greatly reduced. can be reduced, thereby improving the processing speed.

Third, the vehicle detection rate can be improved by comparing the amount of change in the image value of the detection area and the amount of change in the average image value of previous frames to each other to determine vehicle detection, and the width of not only vehicles but also people can be expanded as targets for detection. there is.

1 is a procedure diagram showing a license plate recognition method according to an embodiment of the present invention.
Figure 2 is a procedure diagram showing a vehicle detection process in the license plate recognition method according to an embodiment of the present invention.
3 to 6 are diagrams illustratively showing a vehicle detection process in the license plate recognition method according to an embodiment of the present invention.
7 is a procedure diagram showing a license plate recognition process in the vehicle license plate recognition method according to an embodiment of the present invention.
8 to 11 are diagrams illustratively illustrating a license plate recognition process in a vehicle license plate recognition method according to an embodiment of the present invention.
12 is a block diagram showing the configuration of a vehicle detection unit in the license plate recognition system according to an embodiment of the present invention.
13 is a block diagram showing the configuration of a license plate recognition unit in the license plate recognition system according to an embodiment of the present invention.

Hereinafter, a preferred embodiment will be described with reference to the accompanying drawings so that the present invention can be more specifically understood.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described in more detail. In order to facilitate overall understanding in the description of the present invention, the same reference numerals are used for the same components in the drawings, and redundant descriptions of the same components are omitted.

First of all, the vehicle license plate recognition method according to an embodiment of the present invention can be widely applied to various fields such as road crime prevention (driving vehicles), parking control, and speeding enforcement, and in the video (image) received from the server through CCTV, etc. It may be configured to recognize the license plate after detecting the vehicle.

Referring to FIG. 1 , a license plate recognition method according to an embodiment of the present invention includes the steps of a server extracting a detection area from a received image and detecting a vehicle in the detection area (S100), and when a vehicle is detected, the image It may be configured to include a step (S200) of extracting a recognition area from and recognizing a license plate number in the recognition area.

Here, a method of detecting a vehicle will be described in detail with reference to FIG. 2 .

Referring to FIG. 2, when a video (image) is received from a CCTV or the like (S110), a first resolution changing step of reducing the image by changing the resolution of the received image to a first set value (S120) to increase the detection speed. goes through

Then, through the detection region extraction step, the resolution is changed and the size-reduced image is cut at an arbitrary location in the form of a band of the total width and set height to extract the detection region (S130).

Here, in the extraction of the detection area, the extraction location can be set to an arbitrary location, and when the location is set in this way, the detection area can be extracted by cutting it in a strip shape at the same setting location.

Then, when the detection area is extracted, the detection area goes through an image change step (S140).

In the image changing step (S140), the noise of the detection area is reduced, and in detail, the color conversion step (S141) of converting the color of the detection area to black and white, and the resolution of the detection area whose color is converted to black and white is set to a second setting. A second resolution changing step (S142) of reducing the image by changing to a value may be included.

Here, in the image changing step (S140), before changing the second resolution, the color of the detection area is first converted to black and white, thereby obtaining an effect of reducing noise.

Furthermore, the reason why the detection area is converted to black and white in the color conversion step is that color is unnecessary and the detection effect can be enhanced, considering that the pixel image value is calculated based on brightness (brightness) in the later step. am.

Meanwhile, at this time, the second set value may be set to the entire width (horizontal) and the height (vertical) of one pixel.

FIG. 3 is a diagram showing an image change process according to the first resolution change step, detection region extraction step, and color conversion step in the above-described image receiving step by way of example.

Referring to the drawings, (a) of FIG. 3 is a photograph showing a received image, and the resolution at this time is 1920×1080. 3(b) is a photograph showing a state in which the resolution of the received image is changed, which is the first resolution change step, and the resolution at this time is reduced to 480×270.

Figure 3 (c) is a photograph of a detection area extracted by cutting an area at an arbitrary position from a reduced image with a changed resolution. Here, it is preferable to cut the detection area in a strip shape to have a resolution of 480×5. This is because noise can be attenuated without deterioration of sensitivity as it is a size capable of preventing noise attenuation and sensitivity deterioration.

3(d) is a photograph showing a state in which the color of the detection area is converted to black and white (RGB -> GRAY) in the color conversion step.

According to the above, in the present invention, the resolution is changed twice through the first resolution changing step (S120) of changing the resolution of the received image and the second resolution changing step (S1420) of changing the resolution of the extracted detection area. configured to change over time. This is to increase the processing speed by reducing the amount of processing, and to find edges more effectively by attenuating noise without deteriorating sensitivity.

After reducing the noise in the detection area according to the above process, a vehicle detection step of detecting a vehicle in the detection area is performed, and the presence of a vehicle is determined based on a change in pixel image value of the detection area, so that the vehicle can be detected. .

Here, the pixel image value is a value including characteristic values of a corresponding pixel, and may include a brightness value converted to black and white and a color value (saturation value).

In detail, in the vehicle detection step, the change amount of the pixel image value (pixel change amount, CPCA, Current pixel change amount; change amount of pixels in the detection area in the current frame) of the detection area is calculated (S151), and the frame before the image reception A step of detecting a vehicle by calculating an average pixel image value change amount (S152) and comparing the pixel image value change amount and the average pixel image value change amount of the detection area (S153).

In detail, the detailed process of detecting a vehicle by comparing the amount of change in the pixel image value of the detection area with the amount of change in the average pixel image value of the detection area is the average of the amount of change in the pixel image value of the detection area in the current frame and the amount of change in the pixel image value of the detection area in set frames. The vehicle can be detected by comparing the averages of .

Here, the average pixel image value change amount can be calculated from a table (PCAT; Pixel change amount table, circular queue) that collects pixel image value change amounts of 30 frames before image reception (FIG. 5). reference).

In the step of detecting the vehicle (S153), if the amount of change in the pixel image value of the detection area is greater than twice the amount of change in the average pixel image value, it is determined that there is a vehicle and the count is increased by 1 (S154), and this accumulated count is 2 If so (S155), it is determined that the vehicle has been detected, and then the detection count is initialized to 0, and then proceeds to the license plate recognition step.

Here, in increasing the count 1, the reason for setting the condition that the amount of change in the pixel image value of the detection area is greater than twice the amount of change in the average pixel image value is that the amount of change in the edge due to shaking is 2 times or less as an experimental value. , and the amount of change of license plates with many edges occurs more than twice.

On the other hand, unlike the above, if the amount of change in the pixel image value of the detection area is equal to or smaller than twice the amount of change in the average pixel image value, the count is initialized to 0 (Detect count 0), and the value of the amount of change in the pixel image value of the detection area is converted to PCAT Add.

4 is a diagram illustratively illustrating a process of changing the resolution of the detection area and calculating the pixel image value change amount of the detection area in the second resolution changing step.

Referring to FIG. 4, the present invention can change the size before measuring the amount of change in the image (video) by converting and resizing the previous 480×5 detection area image to 30×1.

Here, the reason for resizing the detection area image to 30 × 1 is to increase the operation speed without deteriorating the detection rate because the noise is greatly attenuated, but the vertical edge for license plate recognition does not disappear.

If the resolution of the detection area is changed and resized in this way, the amount of change in the pixel image value of the detection area is calculated for the edge detection operation. It is possible to calculate the overall average, and at this time, if there is no change in the image (video), the average is small, and the more edges are observed, the larger the value.

5 is a view showing a process of calculating a pixel image value change amount and comparing a detection area pixel image value change amount and an average pixel image value change amount. It shows the process of calculating the average of the average image value change of 30 frames.

Next, FIG. 6 is a diagram showing a process of updating the PCAT when the variation obtained above is small. Referring to the figure, the index value of the PCAT is moved forward by one space, and at the end, a new value of the variation currently obtained is added. (The index value shown in the drawing is an arbitrary value for example.)

Thereafter, when a vehicle is detected by the above process, a process of recognizing the vehicle number of the detected vehicle is performed, which will be described with reference to FIG. 7 .

Referring to FIG. 7 , recognizing the license plate number includes extracting a recognition area by cutting a set position from the detected image when the vehicle is detected in the above process (S210). FIG. 8 is a diagram showing the process of extracting the recognition area, and shows that the recognition area is extracted by cutting a set position where the license plate is located in the detected image.

When the recognition area is extracted in this way, as shown in FIG. 9, a license plate is recognized in the extracted recognition area using a deep learning algorithm (S220).

In this process, if it is determined that the recognition result is a license plate (S221), as shown in FIG. 10, the license plate area is extracted by cutting the area where the license plate is located in the recognition area, and the vehicle number is recognized in the license plate area (S230). Here, in the extraction of the license plate area, the license plate area found in the low-resolution image (image) is cut out from the original image, because more characters can be found in the original image than in the low-resolution license plate image.

Recognizing the license plate number (S230) includes the step of cutting and extracting the license plate area from the original image (S231), and the license plate from the original image using a deep learning algorithm as shown in (a) of FIG. Recognizing characters in the area (S232) and, as shown in (b) of FIG. 11, after recognizing characters, arranging the recognized characters from left to right based on set coordinates to complete the vehicle number (S234). can include

Hereinafter, a license plate recognition system according to an embodiment of the present invention will be described with reference to FIGS. 12 and 13 .

First of all, the vehicle license plate recognition system according to an embodiment of the present invention is a system that implements the above-described vehicle license plate recognition method, and the function and role of each component thereafter is as shown in the vehicle license plate recognition method described above. Description will be omitted.

A license plate recognition system according to an embodiment of the present invention may include a vehicle detection unit 100 and a license plate recognition unit 200 .

Referring to FIG. 12, the vehicle detection unit 140 (100) extracts a detection area from a received image and serves to detect a vehicle in the detection area, and includes a resolution change unit 110 and a detection area extraction unit. 120, an image change unit 130, and a detection unit 140 may be included.

The resolution change unit 110 may change the resolution of the image to a first set value to increase detection and recognition speed by reducing the image.

The detection region extraction unit 120 may extract a detection region by cutting an arbitrary position from the image whose size is reduced due to a change in resolution.

The image changing unit 130 serves to reduce noise in the extracted detection area, converts the color of the detection area to black and white, and reduces the image by changing the resolution to a second set value.

The detection unit 140 determines that there is a vehicle based on the change in the pixel image value of the detection area, detects the vehicle, calculates the amount of change in the pixel image value of the detection area and the amount of change in the average pixel image value, and compares them to the pixel image value. If the image value change amount is greater than a set multiple of the average pixel image value change amount, it may be determined that there is a vehicle.

Referring to FIG. 13 , when a vehicle is detected by the vehicle detection unit 100, the license plate recognition unit 200 extracts a recognition area from the image and recognizes a license plate in the recognition area.

The vehicle number recognition unit 200 may include a license plate recognition unit 210, a number recognition unit 220 and a number sorting unit 230.

The recognition area extraction unit may extract a recognition area by cutting a set position from an image in which it is determined that the vehicle is detected.

The license plate recognition unit 210 may be configured to recognize a license plate in the recognition area extracted using a deep learning algorithm.

The number recognition unit 220 cuts an area where a license plate is located in the recognition area, extracts a license plate area, and recognizes a vehicle number in the license plate area.

The license plate arranging unit 230 may be configured to complete the character by arranging the characters from left to right when characters representing vehicle numbers are recognized in the license plate area.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is only exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: vehicle detection unit
110: resolution change unit
120: detection area extraction unit
130: image change unit
140: detection unit
200: vehicle number recognition unit
210: license plate recognition unit
220: number recognition unit
230: number sorting unit

Claims (7)

extracting a detection area from the received image and detecting a vehicle in the detection area;
When a vehicle is detected, extracting a recognition area from the image and recognizing a vehicle number in the recognition area; including,
The step of detecting the vehicle is,
A first resolution changing step of reducing the image by changing the resolution of the image to a first set value in order to increase the detection speed;
A detection area extraction step of extracting a detection area by cutting the image whose size has been reduced by changing the resolution so as to attenuate noise without deterioration of sensitivity at an arbitrary position in the form of a band of the entire width of the image and a set height;
A color conversion step of converting the color of the detection area extracted in the form of a band into black and white in order to reduce noise and increase the detection effect, increase processing speed, reduce noise without deteriorating sensitivity, and effectively find edges for license plate recognition. An image change step including a second resolution change step of reducing the image by changing the resolution of the detection area extracted in the form of a band whose color is converted to black and white to a second set value so as to be able to do so;
A vehicle detection step of detecting a vehicle by determining that there is a vehicle based on a change in pixel image values of the detection area extracted in the form of a belt;
The vehicle detection step,
Calculating a pixel image value change amount of the detection area extracted in the form of a band;
Calculating an average pixel image value change amount of a detection area extracted in a band shape for a set number of frames before receiving the image;
The amount of change in pixel image values of the detection area extracted in the form of a strip is compared with the amount of change in the pixel image value of the detection area extracted in the form of a strip and the amount of change in the average pixel image value of the detection area extracted in the form of a strip. and determining that there is a vehicle and detecting the vehicle when the change amount of the average pixel image value of the detection area is greater than a set multiple.
delete delete According to claim 1,
Recognizing the license plate number is
extracting a recognition area by cutting a set position in the image;
Recognizing a license plate in the recognition area extracted using a deep learning algorithm;
and extracting a license plate area by cutting an area where the license plate is located in the recognition area, and recognizing the vehicle number in the license plate area.
According to claim 4,
Recognizing the license plate number is
Cutting and extracting the license plate area from the original image;
Recognizing characters in the license plate area in the original image using a deep learning algorithm;
A vehicle license plate recognition method comprising the step of arranging the recognized characters from left to right based on set coordinates to complete the license plate number.
a vehicle detection unit extracting a detection area from the received image and detecting a vehicle in the detection area;
When a vehicle is detected by the vehicle detection unit, a vehicle number recognition unit extracts a recognition area from the image and recognizes a license plate in the recognition area;
The vehicle detection unit,
A resolution change unit for reducing the image by changing the resolution of the image to a first set value in order to increase detection and recognition speed;
A detection area extraction unit for extracting a detection area by cutting the image whose size has been reduced by changing the resolution so as to attenuate noise without deterioration of sensitivity at an arbitrary position in the form of a band of the entire width of the image and a set height;
In order to reduce noise and increase the detection effect, the color of the detection area extracted in the form of a band is converted to black and white, it is possible to increase the processing speed and attenuate noise without deteriorating sensitivity, and the edges for license plate recognition are converted to black and white so that they can be found effectively. an image changing unit for reducing the image by changing the resolution of the detection area extracted in the form of a color-converted band to a second set value;
Calculate the pixel image value change amount of the detection area extracted in the form of a strip and the average pixel image value change amount of the detection area extracted in the form of a strip for a set number of frames before the image is received, and the detection area extracted in the form of a strip By comparing the amount of change in the pixel image value of the region with the amount of change in the average pixel image value of the detection area extracted in the form of a band, if the amount of change in the pixel image value of the detection area is greater than a set multiple of the average pixel image value change in the detection area, it is determined that there is a vehicle. Including a detection unit for determining and detecting the vehicle,
The vehicle number recognition unit,
a recognition area extraction unit for extracting a recognition area by cutting a set position from the image;
A license plate recognition unit for recognizing license plates in the recognition area extracted using a deep learning algorithm;
and a number recognition unit for extracting a license plate area by cutting an area where a license plate is located in the recognition area and recognizing a vehicle number in the license plate area. delete

Images