KR102008283B1 - License Plate Recognition method and apparatus performing the same - Google Patents

Abstract

The present invention relates to a method for recognizing a license plate and a device for recognizing a license plate performing the same. More specifically, provided are the method for recognizing a license plate which can recognize a license plate of a vehicle included in an image and the device for recognizing a license plate performing the same. The method for recognizing a license plate comprises: an image input step for receiving an original image including a vehicle license plate; an image preprocessing step for preprocessing the original image; and a license plate analysis and recognition step for analyzing and recognizing the vehicle license plate from the image preprocessed in the image preprocessing step. The image preprocessing step includes an image binarization step for binarizing the original image and generating a binarized image. The image binarization step includes: a down-sampling step for down-sampling the original image and generating a down-sampling image; a pixel value average calculation step for calculating a pixel value average on the down-sampling image; a threshold value setting step for setting the pixel value average as a threshold value; and a binarization calculation step for applying the threshold value to the original image and binarizing the original image.

Description

License Plate Recognition method and apparatus performing the same

The present invention relates to a license plate recognition method and a license plate recognition device for performing the same, and more particularly, to a license plate recognition method for recognizing a license plate of a vehicle included in an image and a license plate recognition device for performing the same.

License Plate Recognition System (LPR) aims to detect the license plate area and recognize the license plate number for images acquired by imaging sensors such as CCTV (Closed Circuit Television) or cameras.

Recently, as the use of mobile devices such as smartphones and tablet PCs increases, programs such as character recognition using cameras of mobile devices have been introduced. However, a camera of a mobile camera or tablet PC generally has a large number of pixels, but the lens of the camera does not have enough light to obtain sufficient light, and there is a problem of easily deteriorated image depending on the shooting environment, the shooting technology, and the performance of the camera.

Also, recorded data such as CCTVs installed for reasons of public safety and personal security may be used as evidence for crime prevention and crime occurrences. There is a hassle and human / material / time constraints to watch the vehicle in real time or to play the recorded contents one by one.

In order to solve this problem, an image of a vehicle is acquired from an image sensor such as CCTV, and the importance of a text and number recognition technology for automatically recognizing a vehicle number in the corresponding image is increasing.

In particular, the preprocessing module that performs the preprocessing process to improve and correct the image degraded by various environmental / mechanical factors among these license plate recognition systems is an essential part that determines the success or failure of the license plate recognition system. In the preprocessing step, the speed is slow and the preprocessed result is not good, which causes a problem of lowering the license plate recognition success rate.

SUMMARY OF THE INVENTION An object of the present invention is to recognize a problem as described above, and to provide a license plate recognition method and a license plate recognition device that can improve the speed of license plate recognition by pre-sampling, in particular binarizing the original image including the vehicle license plate. It is.

In addition, an object of the present invention, by generating a binarized image by applying the adaptive binarization method in the preprocessing process for the original image by weighting each in the horizontal and vertical directions, the recognition rate for the characters including the horizontal and vertical strokes It is to provide a license plate recognition method and a license plate recognition device for performing the same.

The present invention is created to achieve the object of the present invention as described above, the image input step of receiving an original image including a license plate; An image preprocessing step of preprocessing the original image; And a license plate analysis and recognition step of analyzing and recognizing a vehicle license plate from the preprocessed image in the image preprocessing step, wherein the image preprocessing step comprises: an image binarization step of generating a binary image by binarizing the original image; The image binarization step includes: a downsampling step of generating a downsampling image by downsampling the original image; a pixel value average calculation step of calculating a pixel value average on the downsampling image; and the pixel value average And a binarization operation step of binarizing the original image by applying the threshold value to the original image.

In the pixel value averaging operation, the pixel value average may be calculated for a mask area having a preset size.

The pixel value averaging operation includes an integrated image generation step of generating a downsampling integral image for the downsampling image to calculate the pixel value average.

The pixel value averaging step includes: a horizontal mean averaging step of calculating a horizontal pixel value average of a 1 × ω1 horizontal mask (ω1 is a natural number) in the downsampling image; In the down-sampling image, a vertical average calculation step of calculating a vertical pixel value average of the ω2 × 1 vertical mask (ω2 is a natural number) may be included.

The threshold value setting step includes a horizontal threshold value setting step of setting a horizontal threshold value by adding a preset horizontal weight value α _h to the horizontal pixel value average, and presetting the vertical pixel value average. And a vertical threshold value setting step of setting a vertical threshold value by adding a vertical weight value α _v .

The binarization operation may include generating a horizontal binarized image by applying the horizontal threshold to the original image, and generating a vertical binarized image by applying the vertical threshold to the original image. And generating a final binarized image by combining the generated vertical binarized image and combining the horizontal binarized image and the vertical binarized image.

In the generating of the final binarized image, the final binarized image may be generated by calculating a logical sum of the horizontal binarized image and the vertical binarized image in pixel units.

The horizontal weight value α _h may be greater than the vertical weight value α _v .

The license plate analysis and recognition step includes a license plate area detection step of detecting a license plate area in the preprocessed image, a license plate analysis step of analyzing individual letters in the license plate area, and a number recognition step of recognizing and outputting the individual letters. It may include.

In the license plate area detecting step, the license plate area may be designated by detecting a boundary line in the preprocessed image.

The license plate recognition method may further include a geometric distortion correction step of correcting a geometric distortion of the license plate area.

In another aspect, the present invention provides an image input unit 100 for receiving an original image including a license plate; An image preprocessor 200 for preprocessing the original image; And a license plate analysis / recognition unit 300 that analyzes and recognizes a vehicle license plate from the image preprocessed by the image preprocessor 200, wherein the image preprocessor 200 generates a binary image by binarizing the original image. In order to downsample the original image, a downsampling image is generated, a pixel value average is calculated on the downsampling image, the pixel value average is set as a threshold, and the threshold is applied to the original image. Disclosed is a license plate recognition device 10, characterized in that for binarizing.

The image preprocessor 200 may generate a downsampling integral image for the downsampling image to calculate the pixel value average.

The license plate analysis / recognition unit 300 includes a license plate area detection unit 310 for detecting a license plate area corresponding to a vehicle license plate from the image preprocessed by the image preprocessing unit 200, and an individual character in the detected license plate area. It may include a license plate analysis unit 320, and a number recognition unit 330 for recognizing the analyzed individual character.

The license plate recognition device 10 may further include an output unit 400 for outputting a vehicle license plate finally recognized by the license plate analysis / recognition unit 300.

The license plate recognition method and the license plate recognition device performing the same according to the present invention have an advantage of improving the speed of license plate recognition by down-sampling an original image including a vehicle license plate, in particular by binarization.

In addition, the license plate recognition method according to the present invention and the license plate recognition device for performing the same, by applying a weighted to the horizontal and vertical directions, respectively, when the binary image is generated by applying the adaptive binarization method in the preprocessing process for the original image, There is an advantage that can greatly improve the recognition rate for characters including horizontal strokes and vertical strokes.

1 is a block diagram showing a license plate recognition device according to an embodiment of the present invention.
2 is a flowchart illustrating a license plate recognition method performed in the license plate recognition device of FIG. 1.
3 is a flowchart illustrating an image preprocessing step in the license plate recognition method of FIG. 2.
Figure 4a is a view showing a vehicle license plate image included in the original image input to the license plate recognition device, Figure 4b is a view showing a binarization image for the original image of Figure 4a, Figure 4c is derived through the binarization image of Figure 4b 4D is a geometric correction image obtained by performing geometric correction on the boundary image of FIG. 4C, and FIG. 4E is a diagram showing a result of performing histogram analysis on individual characters included in the license plate area. Is a view showing a license plate character finally recognized through the license plate recognition device of FIG.
FIG. 5 is a conceptual diagram illustrating an example of spatial transformation relation and geometric distortion correction used in the process of performing the license plate recognition method of FIG. 2.
6A is an original image according to an example, FIG. 6B is a binarized image according to a conventional Otsu algorithm, FIG. 6C is a binarized image according to a conventional Sauvola and Bradley algorithm, and FIG. 6D is a binarization according to a license plate recognition method according to the present invention. It is a video.

Hereinafter, a license plate recognition device according to the present invention will be described with reference to the accompanying drawings.

The license plate recognition device 10 according to the present invention is a license plate recognition device 10 for performing the license plate recognition method, as shown in Figs. 1 to 6d, an image input unit for receiving an original image including a license plate 100; An image preprocessor 200 for preprocessing the original image; And a license plate analysis / recognition unit 300 that analyzes and recognizes the license plate from the preprocessed image in the image preprocessor 200.

The image input unit 100 is an input unit for receiving an original image including a license plate, and various configurations are possible.

The original image is an image acquired by an image sensor such as a CCTV or a camera, and may include a license plate image.

The image input unit 100 may be connected to an image sensor and a wired / wireless network to capture / capture the original image.

The original image input through the image input unit 100 may be stored in a memory (not shown) or transmitted externally through a wired or wireless network.

In this case, the license plate recognition device 10 includes an image acquisition means (not shown) such as a camera or a CCTV for acquiring the original image to be input to the image input unit 100, or external image acquisition means and wired or wireless It can be connected via a network to obtain the original image.

The image preprocessing unit 200 is configured to perform preprocessing on the original image.

The preprocessing of the original image may include various image preprocessing techniques as data processing such as binarization and noise removal of the original image.

The license plate analysis / recognition unit 300 may be configured in various ways as a component for analyzing and recognizing the license plate in the preprocessed image.

For example, as illustrated in FIG. 1, the license plate analysis / recognition unit 300 may include a license plate area detection unit 310 and a license plate area detection unit 310 for detecting a license plate area corresponding to a vehicle license plate from a preprocessed image. It may include a license plate analyzer 320 for analyzing the character, and a number recognition unit 330 for recognizing the analyzed individual character.

At this time, the license plate recognition device 10 according to the present invention may further include an output unit 400 for outputting the individual characters of the license plate recognized through the number recognition unit 330.

The output unit 400 is a configuration for outputting a recognized license plate, and various configurations are possible. For example, the output unit 400 may be configured as a display unit for outputting individual characters of the license plate using various methods such as visual or auditory.

The final plate output through the output unit 400 may be transmitted to the outside through a wired or wireless network.

On the other hand, the license plate recognition device 10, in addition to the above-described configuration, the wireless communication unit for wireless communication, the user input unit for controlling the operation of the license plate recognition device 10 through the user, the sensing unit for sensing the surrounding situation or operation state The device may further include an A / V input unit for acquiring an ambient sound or an image.

2 to 6D, the license plate recognition method performed by the license plate recognition device 10 will be described in detail.

The license plate recognition method, as shown in Figure 2, the image input step of receiving the original image including the license plate (S201); An image preprocessing step of preprocessing the original image (S202); And a license plate analysis and recognition step S207 for analyzing and recognizing the vehicle license plate in the preprocessed image in the image preprocessing step S202.

The image input step S201 may be performed by the image input unit 100 as a step of receiving an original image including a license plate.

The image preprocessing step S202 may be performed by the image preprocessing unit 200 as a step of preprocessing the original image input by the image input unit.

The image preprocessing step (S202) may be performed by adjusting the brightness to easily detect the license plate area in the original image or inversely calculating a specific type of distortion so that no distortion occurs in the license plate area in the portion where the license plate area is located in the original image. Procedures may be included.

For example, the image preprocessing step (S202) may include an image binarization step of generating a binarized image by binarizing the original image.

In this case, the image binarization step includes a downsampling step (S301) of generating a downsampling image by downsampling an original image, a pixel value average calculation step (S303) of calculating a pixel value average on the downsampling image, and a pixel value A threshold setting step of setting an average as a threshold value and a binarization operation step of binarizing the original image by applying a threshold value to the original image may include.

The downsampling step (S301) is a step of generating a downsampling image by reducing an image size (pixel count) by downsampling an original image, and various downsampling techniques such as Gaussian pyramids. This can be utilized.

The pixel value averaging step (S303) is a step of calculating a pixel value average for a downsampling image, and using various techniques for a global average for the entire downsampling image or a local average for a specific region of the downsampling integral image. Can be used to compute.

For example, the pixel value averaging step (S303) may include an integrated image generation step (S302) of generating an integrated image (hereinafter, referred to as a downsampling integral image) for the downsampling image to calculate a pixel value average for the downsampling image. ) May be included.

The integrated image generation step (S302) is a step of generating a downsampling integrated image for the downsampling image. The downsampling integrated image may be generated by calculating an integral value of a pixel as shown in Equation (1) below.

는 원본영상의 픽셀값이고,

는 원본영상에 대한 다운샘플링영상의 픽셀값이며,

은 (0,0)에서 (w-1, h-1)까지의 픽셀값을 누적하여 합한 적분영상의 (w-1, h-1)에서의 픽셀값을 의미한다.here,

Is the pixel value of the original image,

Is the pixel value of the downsampled image from the original image.

Denotes a pixel value of (w-1, h-1) of the integrated image by accumulating and summating pixel values from (0,0) to (w-1, h-1).

In this case, in the pixel value average calculation step S303, the pixel value average may be calculated for a mask area having a preset size.

More specifically, the pixel value averaging step (S303) comprises: a horizontal average averaging step of calculating a horizontal pixel value average of a 1 × ω1 horizontal mask (ω1 is a natural number) in the downsampling image. The downsampling image may include a vertical average calculation step of calculating a vertical pixel value average of the ω2 × 1 vertical mask (ω2 is a natural number).

The horizontal average calculation step is a step of calculating a horizontal pixel value average of a 1 × ω1 horizontal mask (ω1 is a natural number) in the downsampling image. When using the downsampling integrated image, the horizontal pixel value average is It can be calculated through Equation (2).

는 1×ω1 수평마스크(ω1은 자연수) 영역에서의 수평방향픽셀값평균을 의미한다.here,

Denotes an average of horizontal pixel values in a 1 × ω1 horizontal mask (ω1 is a natural number) region.

Similarly, the vertical average calculation step is a step of calculating a vertical pixel value average of ω2 × 1 vertical masks (ω2 is a natural number) in the downsampling image. The average may be calculated through Equation (3) below.

는 ω2×1 수직마스크(ω2은 자연수) 영역에서의 수직방향픽셀값평균을 의미한다.here,

Denotes the average of the vertical pixel values in the ω2 × 1 vertical mask (ω2 is a natural number) region.

The threshold value setting step is a step of setting the pixel value average calculated in the pixel value averaging step as a threshold value, and the threshold value may be utilized in a binarization operation step to be described later.

및 수직방향픽셀값평균

은 다운샘플링영상을 다시 원본영상의 크기로 업샘플링(up-sampling)한 확대영상의 수평방향픽셀값평균

및 수직방향픽셀값평균

로 적용될 수 있다.The horizontal pixel value average

And vertical pixel values average

Is the average horizontal pixel value of the up-sampling downsampled image back to the size of the original image.

And vertical pixel values average

Can be applied as

In this case, in the threshold setting step, weights may be assigned to horizontal pixel value averages and vertical pixel value averages to set final threshold values.

More specifically, the threshold value setting step includes a horizontal threshold value setting step of setting a horizontal threshold value by adding a preset horizontal weight value α _h to the horizontal pixel value average and a vertical pixel value average. And a vertical threshold value setting step of setting a vertical threshold value by adding a preset vertical weight value α _v .

The binarization operation is a step of binarizing the original image by applying the threshold value derived in the threshold setting step to the original image, and if the original image can be converted into a black and white 1bit image (0: black, 1: white) Various image binarization techniques can be utilized.

According to an embodiment, when the horizontal threshold value and the vertical threshold value are set in the threshold setting step, the binarization operation may be performed by applying a horizontal threshold value to the original image to generate a horizontal binarized image. Generation step (S304), a vertical binarization image generation step (S304) of generating a vertical binarization image by applying the vertical threshold value to an original image, and final binarization by combining a vertical binarization image and a horizontal binarization image A final binarization image generation step (S305) of generating an image may be included.

The horizontal binarized image generation step (S304) is a step of binarizing the original image using a horizontal threshold value, and the horizontal binarized image may be generated through Equation (4) below.

Similarly, the vertically binarized image generation step (S304) is a step of binarizing the original image using a vertical threshold value, and the vertically binarized image may be generated through Equation (5) below.

및

는 각각 수평방향이진화영상의 이진화값 및 수직방향이진화영상의 이진화값이며,

및

는 다운샘플링영상을 통해 연산된 수직방향픽셀값평균

및 수직방향픽셀값평균

으로부터 원본영상크기로 다운샘플링영상을 확대(upsampling)한 영상의 픽셀값평균이며,

및

는 각각 수평방향가중치 및 수직방향가중치이다.here,

And

Are the binarization value of the horizontal binarization image and the binarization value of the vertical binarization image, respectively.

And

Is the average of the vertical pixel values computed from the downsampling image.

And vertical pixel values average

Is the average pixel value of the upsampling image from the downsampling image to the original image size from

And

Are horizontal weights and vertical weights, respectively.

)는 상기 수직방향가중치(

) 보다 크게 설정됨이 바람직하나, 인식대상이 되는 문자의 특성에 따라 수평방향가중치(

) 및 수직방향가중치(

)는 다양하게 조정될 수 있음은 물론이다.In this case, in the case of Hangul or Hanja, the horizontal weight value (

Is the vertical weight value (

It is preferable to set larger than), but the horizontal weighting value (

) And vertical weights (

) May be adjusted in various ways.

Specifically, in the case of a character that has a difference in the thickness of the horizontal stroke (horizontal stroke) and the vertical stroke (vertical stroke) such as Korean or Chinese characters (the thickness of the horizontal stroke is thinner than the thickness of the vertical stroke), the horizontal direction and the vertical direction Characters can be effectively detected by varying the weights. For this purpose, if the horizontal weight value is applied to be larger than the vertical weight value, there is an advantage that the identification of the character portion can be greatly improved.

In the final binarized video generation step (S305), a final binarized image may be generated by combining a horizontal binarized image and a vertical binarized image.

For example, the final binarized video generation step (S305) may generate a final binarized image by calculating a logical sum of pixels in a horizontal binarized image and a vertical binarized image in units of pixels. Can be derived.

Here, the operator | means a bitwise OR of the pixel unit.

In summary, the present invention generates an integrated image after downsampling the original image for the binarization of the original image, but a part of the pixels is lost in the downsampling process of the original image, but the purpose is to obtain a pixel value average. Even though some of the pixels are lost in the, the average value is not statistically significant. That is, since the present invention directly applies the pixel value average obtained through the downsampling image with reduced size to the original image, it is possible to greatly reduce the computation amount for setting the binarization threshold value as shown in Table 1 below, thereby greatly improving the computation speed. have.

In addition, the final binarization image generation in the present invention is very fast because the horizontal binarization image and the vertical binarization image are calculated in the form of a bitwise or pixelwise pixel.

Binarization Method
Image size Sauvola
algorithm Bradley
algorithm Gatos
algorithm The present invention One
(230 Х 116) 1.27 0.004 2.03 0.03 2 (600 Х 338) 9.76 0.03 11.34 0.04 3
(1961 Х 1493) 141.03 2.69 176.71 0.15 4
(2592 Х 1936) 252.74 5.91 311.87 0.5 5 (2448 Х 3264) 450.92 17.56 539.94 1.0 6
(4160 Х 3120) 668.87 21.22 849.92 2.17 Average (seconds) 254.10 7.90 315.30 0.65

Referring to Table 1, it can be seen that the processing speed of the present invention is improved by 10 times or more compared with other binarization algorithms. In addition, in the case of the conventional Otsu algorithm, the Sauvola algorithm, or the Bradley algorithm, the boundary is as shown in the image of FIG. 6A. In the case of an ambiguous image as an original image, since the horizontal and vertical strokes of the characters are not considered in the image binarization as in the present invention, as shown in FIGS. 6B to 6C, the binarized image has a specific direction edge of the image (particularly, the horizontal stroke of the characters). ) Is not detected. 6B is a binarized image of the image of FIG. 6A using the Otsu algorithm, and FIG. 6C is a binarized image of the image of FIG. 6A using the Sauvola algorithm or the Bradley algorithm.

On the other hand, since the binarization method according to the present invention weights the horizontal direction and the vertical direction, respectively, and binarizes the image in consideration of this, even when the boundary image is blurred as in the image of FIG. There is an advantage in that the final binarized image in which the character part is effectively detected can be obtained.

The license plate analysis and recognition step may be performed through the license plate analysis / recognition unit 300 as a step of analyzing and recognizing the vehicle license plate from the preprocessed image in the image preprocessing step (S202).

The license plate analysis and recognition step includes a license plate area detection step (S203) for detecting a license plate area in a preprocessed image, a license plate analysis step (S205) for analyzing individual letters in a license plate area, and a number for recognizing and outputting individual letters. It may include a recognition step (S207).

The license plate area detecting step (S203) is a step of detecting (detecting) a license plate area having a license plate image from a preprocessed image, wherein the preprocessed image may be a final binarized image of the original image.

In detail, in the license plate area detecting step S203, as illustrated in FIG. 4C, the license plate area may be designated by detecting a boundary line in the preprocessed image.

Meanwhile, FIG. 4A is an original image including a license plate image, FIG. 4B is a binarized image of the original image, and FIG. 4C is a borderline detection image in which a boundary line is detected through the binarized image.

In the license plate area detection step (S203), the boundary line detection may be derived through various boundary line detection algorithms, and for example, may be derived based on contour-based detection or machine learning using text analysis.

Based on machine learning, license plate area detection can detect license plate areas through trained models based on multiple samples.

In this case, the license plate area detection may be divided into single-line and multi-line detection by the horizontal / vertical ratio, and may distinguish between the image to be detected and the image that does not have a shape, thereby obtaining a plurality of samples. Training can then be done to create a training model (supervision-based learning model).

In the system based on the trained model, when the original image is input, it can indicate that there is an object to be detected by giving a position in the form of a box compared to the possessed model.

In the case of detection using text analysis, the license plate area of the license plate can be found by expanding the character group having the minimum size.

Unlike the method of machine learning, the detection by text analysis is based on the boundary images through binarization of the original image, and the continuous characters exist in consideration of the aspect ratio of the closed surface, the width, and the connection between the boundary lines. After analyzing the data, the candidate area can be designated as a bounding box of the maximum size that satisfies all given conditions.

On the other hand, it is necessary to analyze the character attribute considering the characteristics of the letters (size, aspect ratio, number of letters) that make up the license plate for each country. Considering the ratio, you can expand the region of interest to find the boundaries of the license plate.

On the other hand, the license plate area detection step (S203), as shown in Figure 2, may further include a license plate area detection confirmation step (S204) for checking whether the license plate area detection is successful.

If it is determined that the license plate area detection has failed through the license plate area detection confirmation step (S204), the license plate area detection step (S203) may be performed again.

For example, in case of license plate area detection by machine learning, a case that fails to detect an object may occur when the characteristics of the trained model and the input image are different due to various external factors such as irregular brightness change.

After the license plate area detection step (S203) by machine learning (trained model) through the step of checking whether the license plate detection is successful (S204), and if it fails, by using a text-based analysis (contour-based detection), The step of detecting the area again may be performed.

On the other hand, the license plate area detected through the license plate area detection, as shown in Figure 4c, most have a geometric deformation, and this may need to be corrected because it may cause errors in character recognition.

Accordingly, the license plate recognition method according to the present invention may further include a geometric distortion correction step of correcting the geometric distortion of the license plate area.

The geometric distortion correction step may estimate and correct the geometric deformation from the group of detected consecutive characters.

To this end, the geometric distortion correction step, using the hough transform from the binarized image (object edge and boundary image) to find the license plate boundary line and the degree of geometric distortion (skew) through the horizontal / vertical line Calculation and correction (deskew) using the inverse of the calculation.

는 x-y 평면 상 직선을 찾는데 활용될 수 있으므로, 허프변환으로 찾아진 물체의 테두리와 왜곡되지 않은 물체의 정보를 안다면 아래 수학식 (7)을 통해 기하학적왜곡을 보정할 수 있다.A point in huff space via a hough transform

Since can be used to find a straight line on the xy plane, if you know the information of the edge of the object and the non-distorted object by the Hough transform can be corrected geometric distortion through the following equation (7).

는 왜곡이 없는 위치(reference)들에 대해 변환함수(또는 왜곡함수

)를 통해 변환이 가해진 위치들

의 관계를 나타낼 수 있다. 그러므로 역변환함수(Inverse Transform) 를 구할 수 있으면 왜곡된 사각형에 속한 모든 위치 값들은 왜곡되지 않은 위치로 변환할 수 있다.For example, as shown in Figure 5, consisting of four points that are the vertices of a rectangle

Is a transform function (or distortion function) for references that are free of distortion.

Where translations were applied via

It can represent the relationship of Therefore, if the inverse transform function is available, all position values belonging to the distorted rectangle can be transformed to the non-distorted position.

When the license plate boundary is found and the geometric correction is completed, as shown in FIG. 4D, the license plate may be corrected according to the ratio of the license plate for each country and region so that the license plate can be always seen both vertically and vertically.

The license plate analysis step S205 is a step of analyzing individual characters in the detected license plate area, and various analysis methods may be applied.

For example, the license plate analysis step (S205), as shown in Figure 4e, to determine the location of the license plate type and letter group in order to distinguish the individual characters in the vehicle license plate, using the information as a mask noise around the individual characters This can be done using histogram projection after removing.

That is, after performing the bit-wise-and operation with the character area mask on the binarized image of each line whose geometric distortion is corrected (only for the character area), the vertical direction (x coordinate axis in the xy plane system) Accumulated by the character area, the histogram has a relatively large value and the letter and the letter are distributed at a very low value. Therefore, the letter area can be usefully used to find the space between letters as well as the area of the letter.

Similarly, in the case of the license plate having the upper and lower multiple lines, the histogram can be projected in the horizontal direction to know the size of the letters of each line and the spacing between the lines.

The number recognition step S207 is a step of recognizing and outputting individual characters, and various techniques such as optical character recognition (OCR) and deep learning (deep learning) may be utilized.

For example, the number recognition step (S207), as shown in Figure 4f, to input each individual character analyzed through the license plate analysis step (S205) to the OCR (Optical Character Recognition) engine to output each result Can be.

At this time, the number recognition step (S207) shows the final recognition result by combining the recognition results for each individual character in the order of the letters of each line of the license plate, in particular, the character string of each country and region license plate for the result If defined, it can show possible combinations of results.

Since the above has been described only with respect to some of the preferred embodiments that can be implemented by the present invention, the scope of the present invention, as is well known, should not be construed as limited to the above embodiments, the present invention described above It will be said that both the technical idea and the technical idea which together with the base are included in the scope of the present invention.

10: license plate recognition device

Claims (14)

An image input step of receiving an original image including a license plate; An image preprocessing step of preprocessing the original image; And a license plate analysis and recognition step of analyzing and recognizing a vehicle license plate from the preprocessed image in the image preprocessing step.
The image preprocessing step includes an image binarization step of generating a binarized image by binarizing the original image,
The image binarization step may include: a downsampling step of generating a downsampling image by downsampling the original image; A threshold value setting step of setting and a binarization operation step of binarizing the original image by applying the threshold value to the original image,
The pixel value averaging step includes: a horizontal mean averaging step of calculating a horizontal pixel value average of a 1 × ω1 horizontal mask (ω1 is a natural number) in the downsampling image; And a vertical average calculation step of calculating a vertical pixel value average of the ω2 × 1 vertical mask (ω2 is a natural number) in the downsampling image.
The threshold value setting step includes a horizontal threshold value setting step of setting a horizontal threshold value by adding a preset horizontal weight value α _h to the horizontal pixel value average, and presetting the vertical pixel value average. A vertical threshold value setting step of setting a vertical threshold value by adding a vertical weight value α _v ,
The binarization operation may include generating a horizontal binarized image by applying the horizontal threshold to the original image, and generating a vertical binarized image by applying the vertical threshold to the original image. And a final binarized image generating step of generating a final binarized image by combining the horizontal binarized image and the vertical binarized image. The method according to claim 1,
In the pixel value averaging operation, the pixel value average is calculated for a mask area having a predetermined size. The method according to claim 1,
The pixel value averaging operation includes an integrated image generation step of generating a downsampling integral image for the downsampling image to calculate the pixel value average. delete delete The method according to claim 1,
The final binarized video generation step,
And calculating the logical sum of the horizontal binarized image and the vertical binarized image in pixel units to generate the final binarized image. The method according to claim 1,
The horizontal weight value α _h is greater than the vertical weight value α _v . The method according to any one of claims 1 to 3 and 6 to 7,
The license plate analysis and recognition step,
License plate recognition, comprising: a license plate area detection step of detecting a license plate area in the preprocessed image, a license plate analysis step of analyzing individual characters in the license plate area, and a number recognition step of recognizing and outputting the individual characters. Way. The method according to claim 8,
The license plate area detection step,
The license plate recognition method, characterized in that for designating the license plate area by detecting a boundary line in the pre-processed image. The method according to claim 8,
The license plate recognition method,
And a geometric distortion correction step of correcting the geometric distortion of the license plate area. An image input unit 100 for receiving an original image including a license plate; An image preprocessor 200 for preprocessing the original image; And a license plate analysis / recognition unit 300 which analyzes and recognizes a vehicle license plate from the pre-processed image by the image preprocessor 200.
The image preprocessor 200 generates a downsampling image by downsampling the original image to generate a binarized image by binarizing the original image, calculates a pixel value average to the downsampled image, and calculates the pixel value average. Setting a threshold value and applying the threshold value to the original image to binarize the original image,
The image preprocessing unit 200 calculates a horizontal pixel value average of a 1 × ω1 horizontal mask (ω1 is a natural number) in the downsampling image. Calculate a vertical pixel value average of the ω2 × 1 vertical mask (ω2 is a natural number) in the downsampling image, and set a horizontal threshold by adding a preset horizontal weight value α _h to the horizontal pixel value average. The vertical threshold value is set by adding a preset vertical weight value α _v to the vertical pixel value average, and generates a horizontal binarized image by applying the horizontal threshold value to the original image. The license plate recognition device (10), wherein the vertically generated binarized image is generated by applying the vertical threshold to an original image, and a final binarized image is generated by combining the horizontally binarized image and the vertically binarized image. The method according to claim 11,
The image preprocessing unit 200,
And a downsampling integral image for the downsampling image to calculate the pixel value average. The method according to claim 11,
The license plate analysis / recognition unit 300,
The license plate area detection unit 310 for detecting a license plate area corresponding to the vehicle license plate in the image preprocessed by the image preprocessing unit 200, a license plate analyzer 320 for analyzing individual characters in the detected license plate area, and the analysis License plate recognition device 10, characterized in that it comprises a number recognition unit 330 for recognizing the individual characters. The method according to any one of claims 11 to 13,
The license plate recognition device 10,
The license plate recognition device (10), further comprising an output unit (400) for outputting the vehicle license plate finally recognized by the license plate analysis / recognition unit (300).

Images