KR101403876B1 - Method and Apparatus for Vehicle License Plate Recognition - Google Patents

Abstract

The present invention provides a method and apparatus for recognizing a vehicle license plate for CCTV surveillance capable of recognizing high-performance vehicle images that can be distorted and input without using a trigger signal.

Description

TECHNICAL FIELD The present invention relates to a method and apparatus for recognizing a license plate,

The present invention relates to an apparatus and method for recognizing a license plate, and more particularly, to an apparatus and method for recognizing an accurate license plate using license plate image information.

Recently, as interest in smart city construction has increased nationwide, studies on the construction of intelligent transportation system are actively being carried out. Especially, the technology to recognize and manage the license plates of vehicles running at constant speed on the road in real time is a key element of building intelligent transportation system.

Generally, when a vehicle image is acquired from a CCTV installed on the road, the image is distorted due to the inclination of the license plate, the influence of sunlight or illumination, and thus it is difficult to recognize the license plate accurately. Particularly, in order to simplify the installation of CCTV, when the image of the vehicle traveling at a constant speed without using the trigger signal is acquired, the geometric distortion of the image of the license plate or the degradation of the image quality is deepened, May occur.

When the driving position of the vehicle is changed in a state where the CCTV camera is fixed, the license plate image obtained may be variously deformed in its geometric shape, and the characters and the edges of the license plate may come into contact with each other depending on the vertical photographing angle . In addition, a smear phenomenon may occur due to the influence of sunlight, or a distorted image may be input where blurring occurs due to the influence of night illumination. If the distortion of the license plate image occurs, the detection of the license plate candidate region becomes relatively difficult, which can be a decisive factor for lowering the license plate recognition rate as a whole. A method of using the color information of the license plate, a method of using the outline frame information, and a method of using the license plate character arrangement information are mainly used in order to detect the license plate candidate area. Among them, it is known that the method of detecting the license plate candidate region by analyzing the unique arrangement information of license plate characters is relatively strong, such as geometrical distortion and deterioration of image quality.

When the image of the license plate is distorted, the characters of the license plate are deformed or the characters of the license plate are in contact with each other. Therefore, interpretation of the character configuration information for determining the license plate area becomes very difficult. A high-performance license plate recognition system can be constructed only by extracting character stroke information accurately from license plate images having such distortion. If only character stroke information can be accurately extracted, a rule for detecting a license plate candidate region can be clearly defined.

In case of distortion of the image of the license plate, it is known that the method of detecting the license plate candidate region by interpreting the unique arrangement information of the license plate characters is relatively strong, such as geometric distortion and image quality degradation.

In order to analyze the character information recorded on the license plate for detecting the license plate candidate area, it is essential to have a technique of finding frame information in which character strokes are well preserved. However, there is a need for methods for extracting frame information with well preserved character strokes from distorted images of license plates that may appear when acquiring vehicle images from CCTVs that do not use trigger signals.

Korean Patent Laid-Open Publication No. 10-2005-0075989 ("Automatic Vehicle Number Identification System "; hereinafter referred to as Prior Art 2) discloses a vehicle number recognition system.

Korean Patent Publication [10-2005-0075989]

SUMMARY OF THE INVENTION Accordingly, the present invention has been made in order to solve the above-mentioned problems, and it is an object of the present invention to provide a frame-based character segment extraction algorithm in order to enable precise analysis of character strokes even in an image in which geometric distortion or image quality deteriorates, And to provide an apparatus and a method that can increase the number of users.

According to another aspect of the present invention, there is provided a license plate recognition method of a vehicle license plate recognition method, comprising: a license plate region extraction step of recognizing license plates from an inputted vehicle image to extract a license plate area; A license plate image correcting step of removing noise of the license plate image obtained in the license plate area extracting step to perform a filtering and correcting operation; An individual character segmentation step of segmenting recognition object individual character blobs in the image corrected in the image correction step; And recognizing the individual character blobs segmented in the individual character segmentation step using a Hangul and a numeral recognizer.

Also, the number plate region extracting step may include: partitioning and labeling a license plate area, which performs border-based partitioning and labeling on the input vehicle image; A license plate area scanning step of selectively scanning a portion determined as a license plate area in the license plate area dividing and labeling step; And a license plate area detecting step of detecting a license plate position and a color component using the image scanned in the license plate area scanning step.

Also, the number plate region dividing and labeling step may include a brightness image extracting step of extracting vehicle luminance image information by applying a DoG filter to the input vehicle image; A second differential image extracting step of extracting image information by performing second derivative filtering on the entire vehicle image of the image obtained in the brightness image extracting step; A horizontal direction zero crossing extracting step of extracting a zero crossing point in a horizontal direction with respect to an image filtered in the second differential image extracting step; A character coloring image generation step of generating a character segment coloring image in such a manner that the same generation is allocated to the generated character stroke division by accumulating line frame partitions connecting the zero crossings extracted in the horizontal direction zero crossing point extraction step in the vertical direction ; A character information extraction step of extracting character information from the frame-based segment coloring image generated in the character coloring image generation step; And a license plate area determination step of determining a license plate area based on the numerical blob matching the predetermined pattern of the image information extracted in the character information extraction step using the predetermined number plate drawing information, .

In addition, the numeric blob corresponding to the predetermined pattern may be characterized by being four consecutive blobs of the same size, which are relatively large, in the license plate area.

In addition, the license plate area determination step may include a left search step of searching the left side based on the four consecutive blobs and judging the presence of remaining numbers and Hangul area, the number of characters and the number of digits; An upper search step of searching upwards on the basis of the four consecutive blobs and judging the presence or absence of remaining numbers and Hangeul regions, the number of characters and the number of numbers; A geometric distortion information extraction step of extracting geometric distortion information in an array (D6-D1) of six numbers in case of TYPE1c arranged in one line; And a normalizing step of correcting the distortion and correcting the distortion by geometric transformation according to the character array information in the license plate extracted in the step of extracting the geometric distortion information, . ≪ / RTI >

Also, the license plate area determination step may include a left search step of searching the left side based on the four consecutive blobs and judging the presence or absence of remaining numbers and Hangul areas, the number of characters and the number of numbers; A right search step of searching upwardly based on the four consecutive blobs and judging the presence or absence of remaining numbers and Hangeul regions, the number of characters and the number of numbers; A geometric distortion information extraction step of extracting geometric distortion information from four numerical arrays (D4-D1) in case of TYPE2c arranged in two lines; And a normalizing step of correcting distortion and correcting the distortion by geometric transformation in accordance with the character array information in the license plate extracted in the step of extracting the geometric distortion information.

The license plate image correcting step may include: a filtering process step of performing a filtering process for noise removal of the license plate image obtained in the license plate area extracting step; And a filtering image improving step of performing an improving operation on the filtered image in the filtering processing step.

The filtering process may include: a perspective correction step of performing a perspective correction using the license plate image obtained in the license plate area extraction step; And a noise removal step of removing the noise by successively performing Gaussian smoothing and median filtering on the license plate image corrected in the perspective correction step.

The filtering image enhancement step may include a distortion correction step of performing a de-skew operation on the geometric distortion; And a brightness image enhancement step of applying an image enhancement technique in the brightness image to emphasize the stroke of the license plate background contrast characters.

Further, the distortion correction step

The following equation

에 대입, 대응되는 원본 영상의 S1(x,y), S2(x,y), S3(x,y), 및 S4(x,y) 좌표를 x,y에 대입)(Four corner coordinates T1, T2, T3 and T4 in the license plate image in the license plate image to be normalized, respectively

(X, y), S2 (x, y), and S4 (x, y) coordinates of the corresponding original image to x, y)

, The license plate source images extracted in various shapes and sizes are normalized, and the geometric distortion is corrected by introducing a matching and an inverse matching method.

The individual character segmentation step may include a binary image conversion step of converting the image corrected in the license plate image correction step into a binary image; A character-based segmentation step of segmenting the individual characters of the license plate by character segmentation based on the character string arrangement information according to license plate type in the binary image conversion step; And an individual character blob extracting step of extracting individual character blobs to be recognized as the respective divided characters in the character unit division extracting step.

In addition, the license plate character recognizing step may include a character division normalization step of normalizing characters divided in the individual character division step; And an individual character recognizing step of recognizing individual numbers and Korean characters using a neural network using vehicle image training, character training, and neural network training.

In order to solve the above-described problems, the vehicle license plate recognition apparatus of the present invention comprises: an image collection unit for acquiring a vehicle image; A license plate region extracting unit for extracting a license plate area of the image acquired by the image collecting unit; A license plate image correcting unit for correcting the image acquired by the license plate area extracting unit; A character division unit for dividing individual characters of the image corrected by the image correction unit; And a character recognition unit for recognizing individual characters divided by the character division unit.

The number plate region extracting unit may include a number plate region dividing unit for dividing and labeling the number plate region of the image acquired by the image collecting unit; A license plate area scanning unit for selectively scanning a portion determined as a license plate area by using information obtained from the license plate area division unit using at least one license plate drawing information determined in advance; And a license plate area detector for detecting a license plate position and a color component using the image scanned by the license plate area scan unit.

Also, the license plate area dividing unit may include a brightness image extracting unit for extracting vehicle luminance image information by applying a DoG filter to the input vehicle image; A second-order differential image extracting unit for performing second-order differential filtering on the entire vehicle image 201 to extract image information; A horizontal direction zero crossing point extracting unit for extracting a zero crossing point in a horizontal direction with respect to the image filtered by the second differential image extracting unit; A character coloring image generating unit for generating a character segment coloring image in such a manner that the same segmentation is allocated to the generated character stroke segmentation by accumulating line segment segments connecting the zero crossings extracted by the horizontal direction zero crossing point extracting unit in the vertical direction, ; A character information extracting unit for extracting character information from the frame-based segment coloring image generated by the character coloring image generating unit; And a license plate area determination unit for finding a numeric blob corresponding to a predetermined pattern from the image information extracted by the character information extraction unit and determining a license plate area based on the numerical blob.

The license plate image correcting unit may include: a filtering processing unit for performing a filtering process for noise removal of the license plate image obtained by the license plate region extracting unit; And a filtering image improving unit for performing an improving operation on the filtered image in the filtering processing unit.

In addition, the filtering processing unit may include a perspective sensing unit that performs perspective sensing using the license plate image 104 obtained from the license plate region extracting unit; And noise eliminator for removing noise by sequentially performing Gaussian smoothing and median filtering on the license plate image calibrated in the perspective sensing part.

According to the present invention, in the method of recognizing a license plate, even when the image is distorted due to inclination of the license plate, influence of sunlight or illumination when acquiring the vehicle image from the CCTV installed on the road, Can be guaranteed. In order to enable precise analysis of character strokes even in images with geometric distortion or deteriorated image quality, it is possible to improve recognition performance of license plate by providing a frame-based character segment extraction algorithm.

In addition, by providing a rule for searching character strokes based on the frame-based segment filtering result, it is possible to detect the license plate area even in various environments.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a flow chart of a license plate position detection and recognition algorithm in a vehicle image.
2 is a detailed flowchart of a license plate position detection and recognition algorithm in a vehicle image.
Figure 3 is an example of two types of license plate estimation rules.
Figure 4 is an example geometric transformation for normalizing an image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a vehicle license plate recognition method and apparatus according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 2 is a detailed flowchart of a license plate position detection and recognition algorithm in a vehicle image. FIG. 3 is an example of two types of license plate estimation rules, and FIG. 4 is an example of a geometric transformation for normalizing an image.

The vehicle license plate recognizing apparatus according to an embodiment of the present invention may include an image collecting unit, a license plate region extracting unit, a license plate image correcting unit, a separate character dividing unit, and a license plate character recognizing unit. A DVR, a network camera, a CCD camera, and an infrared camera.

The license plate area extracting unit may include a license plate area dividing unit, a license plate area scanning unit, and a license plate area detecting unit. The license plate area dividing unit may include a brightness image extracting unit, a second differential image extracting unit, a horizontal direction zero crossing extracting unit, a character coloring image generating unit, a character information extracting unit, and a license plate area deciding unit. The license plate image correcting unit may include a filtering processing unit and a filtering image improving unit. The filtering processing unit may include a perspective sensitivity correcting unit and a noise removing unit. The filtered image improving unit may include a distortion correcting unit and a brightness image improving unit . The individual character dividing unit may include a binary image converting unit, a character unit division extracting unit, and an individual character blob extracting unit. The license plate character recognition unit may include a character division normalization unit and an individual character recognition unit.

As shown in FIG. 1, the method for recognizing license plates according to the present invention includes a license plate region extraction step 100, a license plate image correction step 200, an individual character segmentation step 300; And a license plate character recognition step (400).

The position of the license plate in the vehicle image acquired from the CCTV is located in the lower part of the vehicle. After performing the differential filtering on the entire vehicle image, the position of the license plate can be estimated in the form of interpreting whether or not the characters constituting the license plate are present while generating an image in which segment-based segment information is searched for the frame- have.

The vehicle license plate recognition method and apparatus according to an embodiment of the present invention can accurately analyze character strokes even in an image in which geometric distortion or image quality deteriorates with respect to vehicle images that can be distorted and input without using a trigger signal In this paper, we propose a frame - based character segment extraction algorithm.

The license plate area extracting step 100 may include a license plate area dividing and labeling step 110, a license plate area scanning step 120 and a license plate area detecting step 130. In the inputted vehicle image, It can recognize and extract the license plate area.

The plate region segmentation and labeling step 110 includes a brightness image extraction step 111, a second derivative image extraction step 112, a horizontal direction zero crossing point extraction step 113, a character coloring image generation step 114, Step 115, and license plate area determination step 116, and may perform edge-based partitioning and labeling on the input vehicle image.

The luminance image extracting step 111 may extract the luminance image information by applying a DoG filter to the input vehicle image. When using a general binarization algorithm and edge extraction algorithm to detect license plate area in a vehicle image where geometric distortion or image quality deteriorates, the character stroke itself may be cut off or the character segment may touch the peripheral segment, It can be difficult. Even when the character information of a license plate is distorted, information of primitive stroke elements can be extracted to correctly interpret the information. The license plate background and character color may be interpreted in all cases, because dark color characters are recorded on a white or yellow background, or bright color characters are recorded on a green background. DoG filters can be used to extract accurate frame information even in distorted or degraded images.

The second differential image extracting step 112 extracts image information by performing second differential filtering on the entire vehicle image of the image obtained in the brightness image extracting step 111. [ By performing the second-order differential filtering, character extraction for detecting a license plate candidate region can be accurately performed even in an image distorted or deteriorated in image quality. In the second derivative result, the zero crossing method of the border part varies depending on the character and the background color, so that the starting point and the ending point of the character stroke can be accurately estimated.

The horizontal direction zero intersection extraction step 113 may extract the zero intersection in the horizontal direction of the filtered image in the second derivative image extraction step 112.

The character coloring image generation step 114 is a method of assigning the same creature to the character stroke division generated by vertically accumulating line frame segments connecting the zero crossings extracted in the horizontal direction zero crossing point extracting step 112 A character segment coloring image can be generated.

The character information extraction step 115 may extract character information from the frame-based segment coloring image generated in the character coloring image generation step 114. [

The license plate area determination step 116 may determine a license plate area based on the numerical blob matching the predetermined pattern of the image information extracted in the character information extraction step 115 using at least one license plate drawing information determined in advance . The numeric blobs corresponding to the predetermined pattern may be four consecutive blobs of the same size, which are relatively large in the plate area.

Although we used the edge information of the license plate when converting the geometric distorted license plate image into the normalized image size, the image input at night in the CCTV installed on the road, or the image of the license plate that is physically distorted or damaged, It may be difficult to extract. According to the present invention, the degree of distortion can be estimated using the positional information of individual characters in order to improve the distortion, and the geometric distortion correction processing can be performed.

In detecting the license plate area in the method and apparatus for recognizing license plate according to the embodiment of the present invention, two types of license plates as shown in FIG. 3 are handled according to the character arrangement type, a) shows a two-row license plate and Fig. 3 (b) shows a rule for a one-row license plate. Even if the character information of the license plate is distorted, it is possible to more precisely search the license plate string if the information of the basic character stroke elements can be extracted accurately. The type of the license plate image can be expressed as TYPEbc according to the number of arrays b (= 1,2) and the number of characters c (= 7, 8, 9). The result of the second derivative result frame can be searched for the presence of a numerical sequence for two types of plates, two lines of TYPE2c and one line of TYPE1c, while completing character strokes by accumulating line segments. Since there are four large digits (D4, D3, D2, D1) on the right side in both types of license plates, when searching the area estimated to be 4 digits, the search area is expanded to the left, And the position of the license plate can be estimated. The geometric distortion information is extracted from four numeric arrays (D4-D1) for TYPE2c arranged in two lines, and six numerical arrays (D6-D1) for TYPE1c arranged in one line. Can be normalized.

The license plate area determination step 116 according to the method and apparatus for recognizing license plate according to the first embodiment of the present invention may include a left search step, an upper search step, a geometric tone information extraction step, and a normalization step.

The left searching step searches the left side based on the four consecutive blobs and judges the presence or absence of the remaining numbers and the Hangul area, and the number of the characters and the number of the numbers.

The upper search step may search upwards with respect to the four consecutive blobs and judge the presence or absence of remaining numbers and Hangeul regions, and the number of characters and the number of numbers.

In the geometric distortion information extraction step, geometric distortion information can be extracted from the array (D6-D1) of six numbers in case of TYPE1c arranged in one line.

The normalizing step may correct the distortion by correcting the size by correcting the distortion through the geometric transformation according to the character array information in the license plate extracted in the geometric distortion information extracting step.

The license plate area determination step 116 according to the method and apparatus for recognizing license plates according to the second embodiment of the present invention may include a left search step, an upper search step, a geometric tone information extraction step, and a normalization step.

The left searching step searches the left side based on the four consecutive blobs and judges the presence or absence of the remaining numbers and the Hangul area, and the number of the characters and the number of the numbers.

The upper search step may search upwards with respect to the four consecutive blobs and judge the presence or absence of remaining numbers and Hangeul regions, and the number of characters and the number of numbers.

In the step of extracting the geometric distortion information, geometric distortion information can be extracted from the four-digit array (D4-D1) in case of TYPE2c arranged in two lines.

The normalizing step may correct the distortion by correcting the size by correcting the distortion through the geometric transformation according to the character array information in the license plate extracted in the geometric distortion information extracting step.

The license plate area scanning step 120 may selectively scan a portion determined as a license plate area in the license plate area division and labeling step 110. [

The license plate area detecting step 130 may detect the license plate position and the color component using the image scanned in the license plate area scanning step 120. [

The license plate image correcting step 200 may include a filtering process step 210 and a filtering image improving step 220. The license plate image correcting step 200 may remove the noise of the license plate image obtained in the license plate area extracting step 100, Can be performed.

The filtering processing step 210 may include a perspective correction step 211 and a noise elimination step 212. The filtering processing step 210 may perform filtering processing for noise removal of the license plate image obtained in the license plate region extraction step 100 have.

The perspective correction step 211 may perform perspective correction using the license plate image obtained in the license plate area extraction step.

In the noise removing step (212), the license plate image corrected in the perspective correction step may be sequentially subjected to Gaussian smoothing and median filtering to remove noise. In the median filtering (median filtering), the media filtering obtains brightness values of the pixels within the mask size, arranges the brightness values located in the middle of them, and outputs the brightness value. In the case of small noise, media filtering can easily remove median filtering. Smoothing or blurring is a very simple and frequently used image processing technique. The smoothing technique is mainly used to mitigate the noise or damage of the image. It removes the detail of the image and makes the edge of the character stroke smooth. Smoothing can also play an important role in reducing the size of the image.

The filtered image enhancement step 220 may include a distortion correction step 221 and a brightness enhancement step 222 and may perform an enhancement operation on the filtered image in the filtering processing step 210.

As shown in FIG. 4, the distortion correction step 221 may perform a de-skew operation on the geometric distortion. At this time, the distortion correction step 221 extracts four vertex coordinates (S1, S2, S3, S4) from the license plate images extracted in various shapes and sizes and transforms them into normalized images T1, T2, T3, . ≪ / RTI > To convert the extracted license plate source image into a normalized target image

The following equation

에 대입하고 각 좌표에 대응되는 원본 영상의 S1(x,y), S2(x,y), S3(x,y), 및 S4(x,y) 좌표를 x,y에 대입하여 방정식을 풀면 매개변수 k1,k2,...k8의 값을 구할 수 있다. Here, the four vertex coordinates T1, T2, T3 and T4 in the license plate image to be normalized through correction of the geometric distortion are

(X, y), S2 (x, y), and S4 (x, y) of the original image corresponding to each coordinate are substituted into x and y to solve the equation The values of the parameters k1, k2, ... k8 can be obtained.

에 대입, 대응되는 원본 영상의 S1(x,y), S2(x,y), S3(x,y), 및 S4(x,y) 좌표를 x,y에 대입)(Four corner coordinates T1, T2, T3 and T4 in the license plate image in the license plate image to be normalized, respectively

(X, y), S2 (x, y), and S4 (x, y) coordinates of the corresponding original image to x, y)

, The license plate source images extracted in various shapes and sizes are normalized, and the geometric distortion is corrected by introducing a matching and an inverse matching method. Reverse-matching may be performed in order to prevent a hole effect from occurring in the geometric transformation result. It is possible to calculate which pixel value is located at which coordinates of the extracted license plate image in which the pixel value to be substituted in the T (x, y) coordinates of the normalized license plate image is extracted. By finding and filling the pixel values by the inverse matching method at every pixel position of the normalized plate, it is possible to make an image that compensates for the geometric distortion. By using the interpolation method in the geometric transformation process, the image quality of the transformed result image can be increased.

The brightness enhancement step 222 may apply an image enhancement technique in the brightness image to emphasize the stroke of the characters relative to the license plate background. In this case, the brightness image enhancement step 222 may precede the Gaussian smoothing operation and the image enhancement operation for removing noise before converting the geometrically normalized brightness level plate image into a binary image and segmenting the individual character regions. The histogram stretching method using the sigmoid transform function performs the image enhancement work. The center and the gradient of the sigmoid curve for image enhancement can be inferred by using the histogram distribution of the character region. The sigmoid transform function can be calculated using two exponential functions as follows.

In other words, the image enhancement step before segmentation is performed by the histogram stretching method using the sigmoid transform function, and the center and the gradient of the sigmoid curve are inferred by using the histogram distribution of the character area. The sigmoid transform function has two exponential functions as follows:

Where a represents the slope of the sigmoid function with respect to the curve. x represents the input range value for stretching to which the sigmoid is applied, and y represents the output value by the sigmoid transform. In fact, since this conversion takes time, it is possible to perform a Gaussian smoothing operation for noise elimination and a high-speed conversion of an image improvement operation by generating a look-up table.

The individual character segmentation step 300 may include a binary image transformation step 310, a character unit segmentation step 320 and an individual character blob extraction step 330. In the image correction step 200, It is possible to segment individual character blobs to be recognized in the captured image.

The binary image conversion step 310 may convert the corrected image in the license plate image correction step 200 into a binary image. At this time, the binary image conversion step

The following equation

는 5x5 화소 영역에 대한 평균

와 표준 편차

를 이용해서 계산하고, 이 식을 이용하여 Sauvola 적응적 이진화 규칙을 적용하여 이진 영상을 생성하는데 R은 편차의 최대인 128로 하고 k는 0.2에서 0.5 사이 값으로 설정할 수 있다.here, (

≪ / RTI >< RTI ID = 0.0 &

And standard deviation

And the Sauvola adaptive binarization rule is applied to generate a binary image using this equation. R can be set to a value of 128, which is the maximum deviation, and k can be set to a value between 0.2 and 0.5.

By introducing the concept of integral image, the mean and standard deviation can be calculated at high speed. Since the positions of the numbers found in the license plate detection process can be changed according to the geometric distortion correction of the license plate image and the image improvement result, individual character blobs can be extracted again according to the character string arrangement rule shown in FIG.

The character-based segmentation step 320 extracts individual images of the vehicle license plate by character segmentation based on the character string layout information according to the license plate form in the binary image conversion step 310, can do.

The individual character blob extracting step 330 may extract the individual character blobs to be recognized by the divided characters in the character unit division extracting step 320. [

The license plate character recognition step 400 may include a character segmentation normalization step 410 and an individual character recognition step 420 wherein the individual character blobs subdivided in the individual character segmentation step 300 are input to a Korean alphabet Can be recognized by using

The character segmentation normalization step 410 may normalize the characters segmented in the individual character segmentation step 300.

The individual character recognition step 420 can recognize individual numbers and Korean characters using a neural network using vehicle image training, character training, and neural network training.

In the method of generating and matching the standard template of the characters to recognize the characters of the license plate, character strokes may be extracted accurately. Since the character strokes may not be extracted clearly in the license plate image inputted by using the CCTV, a neural network character recognizer which is known to have excellent generalization ability for character recognition can be implemented. It extracts learning characters and learns neural network by selecting images that can reflect various types of vehicle, type of license plate, font, daytime and nighttime input time difference, and distortion type from images acquired from CCTV. Since the individual characters may still appear in an inclined form despite the geometric distortion correction of the license plate, the characters rotated in the left and right direction of the standard training character are also included in the neural network learning. The local name recognizer, the Hangul recognizer, and the numeral recognizer can be separately implemented according to the type of characters to be recognized.

That is, the present invention is a vehicle license plate recognition method and apparatus for estimating distortion degree and correcting geometric distortion using position information of individual characters other than border information for physical deformation or damaged license plates.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: License plate area extraction step
110: License plate area division and labeling step
111: brightness image extraction step 112: second derivative image extraction step
113: Horizontal direction zero crossing extraction step 114: Character coloring image generation step
115: Character information extraction step 116: License plate area determination step
120: License plate area scanning step 130: License plate area detection step
200: License plate image correction step
210: filtering processing step 211: perspective correction step
212: noise removal step 220: filtering image enhancement step
221: Distortion correction step 222: Brightness image enhancement step
300: Individual character segmentation step
310: Binary image conversion step 320: Character unit division extraction step
330: Individual character blob extraction step
400: license plate character recognition step
410: Character segmentation normalization step 420: Individual character recognition step

Claims (17)

A license plate area extraction step of extracting license plate area from the inputted vehicle image by recognizing the license plate number;
A license plate image correcting step of removing noise of the license plate image obtained in the license plate area extracting step to perform a filtering and correcting operation;
An individual character segmentation step of segmenting recognition object individual character blobs in the image corrected in the image correction step; And
A license plate character recognizing step of recognizing the individual character blobs subdivided in the individual character division step using a Hangul and a number recognizer;
And,
The number plate region extracting step
A license plate area division and labeling step for performing border-based division and labeling on the input vehicle image;
A license plate area scanning step of selectively scanning a portion determined as a license plate area in the license plate area dividing and labeling step; And
A license plate area detecting step of detecting a license plate position and a color component using the image scanned in the license plate area scanning step;
The method comprising the steps of:
delete The method of claim 1, wherein the plate region segmentation and labeling step
A luminance image extracting step of extracting luminance image information by applying a DoG filter to the input vehicle image;
A second differential image extracting step of extracting image information by performing second derivative filtering on the entire vehicle image of the image obtained in the brightness image extracting step;
A horizontal direction zero crossing extracting step of extracting a zero crossing point in a horizontal direction with respect to an image filtered in the second differential image extracting step;
A character coloring image generation step of generating a character segment coloring image in such a manner that the same generation is allocated to the generated character stroke division by accumulating line frame partitions connecting the zero crossings extracted in the horizontal direction zero crossing point extraction step in the vertical direction ;
A character information extraction step of extracting character information from the frame-based segment coloring image generated in the character coloring image generation step; And
Determining a number plate blob corresponding to a predetermined pattern of the image information extracted at the character information extracting step using at least one predetermined number plate drawing information and determining a number plate area based on the number blob;
The method comprising the steps of:
4. The method of claim 3, wherein the numeric blob corresponding to the predetermined pattern comprises:
Wherein the banner is four consecutive blobs of the same size that are relatively large within the license plate area.
5. The method of claim 4, wherein the number plate region determination step
A left search step of searching leftward with respect to the four consecutive blobs and determining the presence or absence of remaining numbers and Hangeul regions, and the number of characters and the number of numbers;
An upper search step of searching upwards on the basis of the four consecutive blobs and judging the presence or absence of remaining numbers and Hangeul regions, the number of characters and the number of numbers;
A geometric distortion information extraction step of extracting geometric distortion information in an array (D6-D1) of six numbers in case of TYPE1c arranged in one line; And
A normalizing step of correcting the distortion and correcting the distortion by geometric transformation according to the character array information in the license plate extracted in the step of extracting the geometric distortion information, and normalizing the distortion;
The method comprising the steps of:
5. The method of claim 4, wherein the number plate region determination step
A left search step of searching leftward with respect to the four consecutive blobs and determining the presence or absence of remaining numbers and Hangeul regions, and the number of characters and the number of numbers;
A right search step of searching upwardly based on the four consecutive blobs and judging the presence or absence of remaining numbers and Hangeul regions, the number of characters and the number of numbers;
A geometric distortion information extraction step of extracting geometric distortion information from four numerical arrays (D4-D1) in case of TYPE2c arranged in two lines; And
A normalizing step of correcting the distortion and correcting the distortion by geometric transformation according to the character array information in the license plate extracted in the step of extracting the geometric distortion information, and normalizing the distortion;
The method comprising the steps of:
The method of claim 1, wherein the license plate image correction step
A filtering process step of performing a filtering process for noise removal of the license plate image obtained in the license plate area extracting step; And
A filtering image improving step of performing an improving operation on an image filtered in the filtering processing step;
The method comprising the steps of:
8. The method of claim 7, wherein the filtering processing step
A perspective perspective correction step of performing perspective perspective correction using the license plate image obtained in the license plate area extraction step; And
A noise removal step of removing noise by sequentially performing Gaussian smoothing and median filtering on the license plate image corrected in the perspective of perspective;
The method comprising the steps of:
[8] The method of claim 7,
A distortion correction step of performing a de-skew operation on the geometric distortion;
A bright image enhancement step applying an image enhancement technique in a brightness image to emphasize a stroke of characters corresponding to a license plate background;
The method comprising the steps of:
10. The method of claim 9, wherein the distortion correction step
The following equation

(Four corner coordinates T1, T2, T3 and T4 in the license plate image in the license plate image to be normalized, respectively

(X, y), S2 (x, y), and S4 (x, y) coordinates of the corresponding original image to x, y)
Wherein the license plate source images extracted in various shapes and sizes are normalized, and the geometric distortion is corrected by introducing a matching and an inverse matching method.
2. The method of claim 1,
A binary image conversion step of converting the image corrected in the license plate image correction step into a binary image;
A character-based segmentation step of segmenting the individual characters of the license plate by character segmentation based on the character string arrangement information according to license plate type in the binary image conversion step; And
An individual character blob extracting step of extracting individual character blobs to be recognized as the respective divided characters in the character unit division extracting step;
The method comprising the steps of:
The method of claim 1, wherein the license plate character recognition step
A character segmentation normalization step of normalizing the characters segmented in the individual character segmentation step; And
Recognition of individual characters and Korean characters using neural networks using vehicle image training, character training, and neural network training;
The method comprising the steps of:
An image collecting unit for acquiring a vehicle image;
A license plate region extracting unit for extracting a license plate area of the image acquired by the image collecting unit;
A license plate image correcting unit for correcting the image acquired by the license plate area extracting unit;
A character division unit for dividing individual characters of the image corrected by the image correction unit; And
A character recognition unit recognizing individual characters divided by the character division unit;
And,
The number plate region extracting unit
A license plate area dividing unit for dividing and labeling the license plate area of the image acquired by the image collecting unit;
A license plate area scanning unit for selectively scanning a portion determined as a license plate area by using information obtained from the license plate area division unit using at least one license plate drawing information determined in advance; And
A license plate area detector for detecting a license plate position and a color component using the image scanned by the license plate area scanner;
Wherein the vehicle license plate recognizing device comprises:
delete 14. The apparatus of claim 13, wherein the license plate area dividing unit
A brightness image extracting unit for extracting vehicle luminance image information by applying a DoG filter to the input vehicle image;
A second-order differential image extracting unit for performing second-order differential filtering on the entire vehicle image to extract image information;
A horizontal direction zero crossing point extracting unit for extracting a zero crossing point in a horizontal direction with respect to the image filtered by the second differential image extracting unit;
A character coloring image generating unit for generating a character segment coloring image in such a manner that the same segmentation is allocated to the generated character stroke segmentation by accumulating line segment segments connecting the zero crossings extracted by the horizontal direction zero crossing point extracting unit in the vertical direction, ;
A character information extracting unit for extracting character information from the frame-based segment coloring image generated by the character coloring image generating unit; And
A license plate area determination unit for finding a numeric blob corresponding to a predetermined pattern from the image information extracted by the character information extraction unit and determining a license plate area based on the number blob;
The vehicle license plate recognizing device comprising:
14. The apparatus of claim 13, wherein the license plate image correction unit
A filtering processor for performing a filtering process for noise removal of the license plate image obtained by the license plate area extracting unit; And
A filtering image enhancement unit performing an improvement operation of the filtered image in the filtering processing unit;
The vehicle license plate recognizing device comprising:
17. The apparatus of claim 16, wherein the filtering processor
A perspective sensing unit which performs perspective sensing using the license plate image obtained by the license plate region extracting unit; And
A noise eliminator for removing noise by sequentially performing Gaussian smoothing and median filtering on the license plate image calibrated by the perspective sensing unit;
The vehicle license plate recognizing device comprising:

Images