KR20110045971A - Recognition system of license plates and method thereof - Google Patents

Abstract

PURPOSE: A recognition system of license plates and a method thereof are provided to recognize a license plate by grouping an area which is distinguished by an edge. CONSTITUTION: In a recognition system of license plates and a method thereof, an edge discriminating unit(110) distinguish edge from the image of a license plate. A verification unit searches a charter area according to verification conditions of grouped area. A character-background determination unit(140) determines a character region and a background region by analyzing the intensity histogram of a character. A binary unit(150) binarizes the character region into a character area and a background area. A kind distinguish unit(180) distinguishes the kind of a license plate according the arrangement of the character in the character region.

Description

Recognition system of license plates and method

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a license plate recognition system and method, and more particularly, to a system and method capable of recognizing various kinds of license plates and increasing the detection rate or recognition rate of license plates.

The present invention is derived from a study performed as part of the IT source technology development of the Ministry of Knowledge Economy [Task Management Number: 2008-F-037-01, Task name: u-robot HRI solution and core device technology development].

Techniques for extracting and recognizing characters from license plates attached to vehicles have been developed for the purpose of preventing crimes using vehicles or managing vehicles.

Techniques for recognizing images of license plates released in Korea include edge, contrast information, and color. These related technologies have a problem that it does not secure a sufficient detection rate or recognition rate in a variety of indoor and outdoor lighting environment.

In addition, since there are many kinds of license plates, there is a need for a technology of accurately extracting and recognizing characters from these license plates by distinguishing various license plates.

The present invention has been made from the above problems, and an object of the present invention is to provide a system and method capable of recognizing various kinds of license plates and further increasing the detection rate or recognition rate of license plates.

An object of the present invention as described above, the method comprising the steps of grouping the region determined by the edge in the license plate image of the vehicle input, finding a candidate region of the character region by applying global verification conditions to the grouped region, and A license plate recognition method can be achieved including applying a local verification condition to a region to find a character region and binarizing the background portion of the character region and the character portion.

And when the grouped region does not satisfy the global verification condition or the candidate region does not satisfy some of the local verification conditions, resetting the threshold of the adaptive Sobel operation to determine the edges. Is also achievable.

In addition, by detecting four consecutive character strings in the binarized character area, verifying that binarization is performed well, and in case of incorrectly performing the result, resetting the variable value used for binarization to binarize the feedback. Is achievable.

Then, verify that the four consecutive strings are recognized as numbers to distinguish the type of license plate, and if any one of the four consecutive strings is not recognized as a number, repeat the other four consecutive strings. Is also achievable.

An object of the present invention as described above is another category, the edge determination unit for determining the edge in the license plate image of the vehicle, grouping the area determined as the edge, and applying the verification conditions to the grouped area to find the character area A verification unit, a character-background determination unit that analyzes the contrast histogram of the character area to determine the character part and the background part, a binarization part that binarizes the character area and the background part, and a license plate according to the character arrangement state of the character part. It is also achievable by a license plate recognition system that includes a class discriminating unit that distinguishes the types of.

According to one embodiment of the present invention as described above, it is possible to more accurately recognize various types of license plates, such as when the character portion present in the license plate is dark and the background part is bright or the character part is bright and the background part is dark.

In addition, it is possible to more accurately recognize a variety of license plates, such as when the left and right length of the license plate is different and accordingly the arrangement of the characters.

In addition, by performing a number of feedback processes, the license plate can be strongly recognized in lighting even in a harsh environment such as nighttime, daytime, and light reflection.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms, and only the present embodiments are intended to complete the disclosure of the present invention, and the general knowledge in the art to which the present invention pertains. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, “comprises” and / or “comprising” refers to the presence of one or more other components, steps, operations and / or elements. Or does not exclude additions.

Hereinafter, a configuration of a license plate recognition system according to the present invention will be described with reference to the accompanying drawings. 1 is a schematic structural block diagram of an embodiment of a license plate recognition system. The license plate recognition system includes an edge determination unit 110, a first labeling unit 120, a verification unit 130, a character-background determination unit 140, a binarization unit 160, a second labeling unit 160, and binarization verification. The unit 170, the type discriminating unit 180, and the like are included.

The edge determination unit 110 determines an edge from the license plate image of the vehicle photographed by the camera, and groups an area determined as an edge. The edge discrimination unit 110 determines the edge from the license plate image according to the adaptive Sobel operation. The mask of the adaptive Sobel operation is the same as the following Equation 1, and it is determined whether the edge using Equation 2.

의 값이다.In Equation 2, T _edge is an experimentally obtained threshold value according to the state of the input plate image, Mo of Equation 2

Is the value of.

Edges are determined according to Equation 2, and the edge discrimination unit 110 groups the regions determined as edges by connecting the edges. For example, run-length linking algorism may be used to link the edges.

For example, as shown in Equation 2, when the pixel determined as the edge is 1 and the pixel not as 0 is 0, the input image may be represented by a combination of 0 and 1 as shown in FIG. The edge determining unit 110 checks the run-length between the edges, and then connects the edge intervals having an interval less than or equal to a predetermined interval, and groups the regions determined as edges. For example, the maximum spacing between characters is 10 (Twidth = 10), and the edge spacing below this spacing is concatenated. This operation is performed in the vertical direction and the horizontal direction to group the areas identified as edges in the entire image. The grouped area becomes a candidate area with a high probability of being recognized as the character portion of the license plate. In Fig. 2, the predetermined interval Twidth between characters can be arbitrarily set by the user.

The first labeling unit 120 extracts information of the grouped areas. In extracting the information, for example, a labeling algorithm can be used. The information can be expressed as follows. start (x, y) is the position of the pixel where the grouped region starts, end (i, j) is the end position, and real_area is the actual labeled area existing in the minimum boundary rectangle (MBR).

label _i = {start (x, y), end (i, j), real_area}

The verification unit 130 finds a text area in the grouping area by applying a verification condition to the grouped area. Verification conditions include global verification conditions and local verification conditions. The verifier 130 first finds a candidate region of the character region by applying a global verification condition to the grouped regions, and then finds a character region by applying a local verification condition.

<Global Verification Conditions>

Global validation conditions can be divided into, for example, size constraints and ratio constraints. The size limit is the horizontal size limit and the vertical size limit of the grouped area, and the ratio limit is the limit on the ratio of the horizontal size and the vertical size and the ratio of the black pixels to the grouped area. Can be applied. Alternatively, some of the four global verification conditions may apply. In the above four global verification conditions, min and max of T1 to T4 are experimentally obtained values according to the resolution of the input image.

When the candidate area of the character area is found through the global verification, the verification unit 130 applies the local verification condition.

<Local verification conditions>

Under local verification conditions, the standard size, standard area, and standard rate are experimentally obtained for each of the long and short plates, and the standard location is the smallest candidate area of the character area located at the bottom of the input image. Set to have a value. When the value is determined according to the four local verification conditions as described above, the verification unit 130 calculates a final value by applying the following equation (3).

The largest value of Equation 3 is the value that satisfies the above four local verification conditions most. Among the above local verification conditions, the candidate area of the character area that satisfies the minimum local verification condition (eg, at least two) may be determined as the character area.

The text-background determination unit 140 analyzes the contrast histogram of the text area determined by the verification unit 130 to determine the text part and the background part in the text area. For example, the character-background determination unit 140 analyzes the contrast histogram to find two distinct peaks. Then, the brightness of the case where there are many pixels included in the periphery of the two peaks as the background is determined, and the brightness of the case where the pixel is relatively small is determined as a character. Among the plates are plates with dark letters on a light background (eg black letters on a white background) and plates with light letters on a dark background (eg white letters on a green background). The character-background determination unit 140 is for recognizing characters by distinguishing such license plates.

The binarization unit 160 binarizes the text area into a background part and a text part. As a method of binarization, for example, the Otsu algorithm may be used.

The second labeling unit 160 extracts size and position information of the character from the binary character portion.

The binarization verification unit 170 detects successive four character strings based on the size and position information of the character and verifies whether the binarization is well performed. The license plates exist in various ways as shown in FIG. 3, but all of the various license plates have a common number of four numbers. Therefore, if four strings that exist in this manner are detected, it is determined that binarization is well performed. If not, it is determined that binarization is incorrectly performed. If the binarization is incorrectly performed, the binarization verification unit 170 generates a predetermined signal and transmits the predetermined signal to the binarization unit 160, and the binarization unit 160 may re-binarize the binarization variable value. A detailed description of this is provided in the relevant section.

If the type discriminator 180 detects four consecutive strings by the binarization verification unit 170, the type discriminator 180 verifies whether the corresponding four strings are recognized as numbers. In this case, a previously learned number recognizer can be used. When the type discriminating unit 180 recognizes four strings as numbers, the type discriminating unit 180 may select any one of various types of license plates as shown in FIG. 3 according to an arrangement state of characters existing around the four strings. It can be recognized that

Hereinafter, a license plate recognition method according to the present invention will be described with reference to the accompanying drawings. 4 is a flowchart according to an embodiment of a license plate recognition method.

When the license plate image of the vehicle photographed by the camera is input, the edge determination unit 110 determines an edge according to the adaptive Sobel operation (S410). The mask used for the adaptive Sobel operation is as shown in Equation 1, and the discrimination criterion is as shown in Equation 2.

When the edge is determined, the edge determination unit 110 connects the edges distributed at predetermined intervals to group the areas determined as the edges (S420). As an example of how to join the edges, a continuous length concatenation algorithm may be used. An embodiment grouped using the continuous length connection algorithm is as described above with reference to FIG. 2.

The first labeling unit 120 extracts information of the grouped area (S430). In extracting the information, for example, a labeling algorithm can be used. The information may be extracted in the form of label _i = {start (x, y), end (i, j), real_area} as mentioned above.

The verification unit 130 finds a candidate area of the text area by applying global verification conditions to the grouped areas (S440). The global verification condition is the same as mentioned above, so it is replaced by the above description.

As a result of applying the global verification condition, when none of the regions satisfying the condition exist, the verification unit 130 outputs a predetermined control signal to the edge determination unit 110, and the edge determination unit 110 outputs the control signal. It is fed back to the step (S410) of determining the edge from the received image and performs a series of processes (S410 to S430) again. When the step of determining the _edge is performed again, the threshold value T _edge of the Sobel operation is reset in Equation 2 (S445). For example, decreasing the value of T _edge produces more edges, while increasing it produces fewer edges. Therefore, in consideration of the distribution of edges in the entire area of the input image before feedback, the T _edge value is increased (about 10%) if the edges are largely distributed, and the T _edge value is decreased if the edges are less distributed. (About 10%). The number of times to apply the global verification condition may be limited, for example, up to three times.

When global verification is performed, the verification unit 130 finds a text area by applying local verification conditions (S450). For example, the verification unit 130 applies the above-described local verification condition and calculates the result as shown in Equation 3 to find the character area. As a result of applying the local verification condition, if there is only one candidate area of the character area satisfying the condition, the area is determined as the character area.

On the other hand, when there are a plurality of candidate regions of the character region, there are also a plurality of Plate_scores, which are the result of Equation 3. The candidate area which can be determined as the character area among the candidate areas of the plurality of character areas is the candidate area having the largest Plate_score, and is the case where the four local verification conditions are most satisfied. In addition, the verification unit 130 may be set to determine the character area only when at least a part (eg, two or more) of the local verification conditions are satisfied.

As a result of applying the local verification condition, for example, when there is no candidate region satisfying at least two or more of the local verification conditions (S452), the verification unit 130 outputs a predetermined control signal to the edge discrimination unit 110. The edge determination unit 110 receives the control signal and feeds back to the step S410 of determining the edge from the input image and re-performs a series of processes S410 to S440. In the case of re-determining the edge, the threshold value T _edge of the Sobel operation in Equation 2 may be reset as described above (S455). The number of times of applying the local verification condition may be limited to, for example, up to three times.

The threshold value T _edge of the Sobel operation reset when fed back by applying the local verification condition may be set to be equal to the threshold value T _edge of the Sobel operation reset when fed back according to the global verification condition.

When each verification condition is applied to determine the text area, the text-background determination unit 140 analyzes the contrast histogram of the text area to determine the text part and the background part (S460). The contrast histogram of the character area shows two peaks. The brightness of the case where there are many pixels included around two peaks as a center is set as the background, and the brightness of the case where there are relatively few pixels is determined by characters.

The binarization unit 160 binarizes the text area into the background part and the text part (S470). As an example of binarization, an Otsu algorithm may be used.

The second labeling unit 160 extracts the size and position information of the character from the binary character portion (S480). Like the first labeling unit 120, a labeling algorithm may be used.

The binarization verification unit 170 verifies whether binarization has been well performed (S490). As an example of the verification method, the binarization verification unit 170 detects four consecutive strings in the binarized character part based on the size and position information of the character. If four consecutive strings of the same or similar size are detected, binarization is well performed, and the next step can be performed. If four consecutive character strings are not detected, the binarization is performed incorrectly, and the binarization verification unit 170 generates a predetermined signal and transmits the predetermined signal to the binarization unit 160. The binarization unit 160 resets the binarization variable value (for example, the variable value of the Otsu algorithm) (S495) and re-performs the binarization step. The binarization variable value may be increased or decreased by a predetermined size (eg, about 10%) to redistribute the binarization. Whether to increase or decrease is determined by considering the ratio of the area that the binarized result occupies in the area of the entire license plate.

The type discrimination unit 180 distinguishes the type of the license plate according to the character arrangement existing in the character portion (S500). Looking at one embodiment is as follows. The successive four strings detected by the binarization verification unit 170 are verified using a previously learned number recognizer (S502). If the verification result is recognized as a number, among the types of license plates as shown in FIG. 3 in consideration of the state of the array of characters located around four consecutive strings (for example, where there are characters other than four consecutive strings) Determining which one of the plates (S504). An optical character reader (OCR) can be used to account for the arrangement of characters.

If any one of the four consecutive strings is not recognized as a number (S506), an incorrect string is found. Therefore, in addition to the four consecutive strings already detected, it is checked whether there are other four consecutive strings (S508), and fed back to step S502, and the step S502 is repeated for the other consecutive strings.

As a result of repeating other successive character strings, if it is not recognized as a number, it is determined that there is no character part in the license plate (S510).

 As another embodiment of the present invention, an image of a license plate input to a license plate recognition system may be reduced. For example, if the license plate image photographed by the camera is 640 × 480, the image of the license plate input to the system may be reduced to a size of 320 × 240, which is 1/4. When the license plate recognition is performed by reducing the image of the license plate, the amount of computation can be reduced, thereby ensuring real-time performance of license plate recognition.

On the other hand, the present invention is not only used to detect the license plate, it is equally applicable to the detection of various recognition areas such as signs and symbols in the general natural image.

Although the present invention has been described in connection with the above-mentioned preferred embodiments, it is possible to make various modifications or variations without departing from the spirit and scope of the invention. Accordingly, the appended claims will include such modifications and variations as long as they fall within the spirit of the invention.

1 is a schematic block diagram of a license plate recognition system according to an embodiment of the present invention;

2 illustrates an embodiment of performing a continuous length concatenation algorithm,

Figure 3 shows the form by type of license plate,

4 is a flowchart according to an embodiment of a license plate recognition method according to the present invention.

<Explanation of symbols for the main parts of the drawings>

110: edge determination unit

120: first labeling unit

130: verification unit

140: character-background determination unit

150: binarization

160: second labeling unit

170: binarization verification unit

180: class distinction

Claims (10)

Grouping the areas identified as edges in the license plate image of the vehicle; Applying a global verification condition to the grouped area to find a candidate area of a text area; Finding a character area by applying a local verification condition to the candidate area; And Binarizing the text area into a background part and the text part; License plate recognition method comprising a. The method of claim 1, wherein the global verification condition, At least one of a constraint regarding a horizontal size of the grouped region, a constraint on a vertical size, a constraint on the ratio of the horizontal size and the vertical size, and a constraint on the ratio of black pixels to the grouped region License plate recognition method comprising one. The method of claim 1, wherein the local verification condition, A license plate recognition method comprising at least one of constraints on the size, area, ratio, and position of the candidate area. The method of claim 1, Determining an edge in the license plate image of the vehicle according to an adaptive Sobel operation; Determining the edge by resetting a threshold of the adaptive Sobel operation when the grouped region does not satisfy the global verification condition or the candidate region does not satisfy some of the local verification condition and the grouping License plate recognition method to repeat the steps. The method of claim 1, And analyzing a contrast histogram of the character area to determine a character part and a background part. The method of claim 1, Detecting four consecutive strings in the binarized character portion, And if the four consecutive strings are not detected, re-binding the variable by resetting the variable value used in the binarizing step. The method of claim 1, Verifying whether four consecutive strings existing in the character portion are recognized as numbers; And And recognizing the type of license plate according to an arrangement of other characters based on the four consecutive character strings recognized as the numbers. An edge determination unit for determining an edge in the license plate image of the vehicle and grouping an area determined as an edge; A verification unit which finds a text area by applying verification conditions to the grouped areas; A text-background determiner for analyzing a contrast histogram of the text area to determine a text part and a background part; A binarization unit for binarizing the character area into the character part and the background part; And A kind discrimination unit that distinguishes the kind of the license plate according to the character arrangement state of the character portion License plate recognition system comprising a. The method of claim 8, wherein the verification unit, And generating a candidate region of the character region by applying a global verification condition to the grouped region, and finding a character region by applying a local verification condition to the candidate region. The method of claim 8, And a binarization verification unit for verifying that binarization is performed by detecting four consecutive strings in the binarized character portion.

Images