JP2005250786A - Image recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition method capable of accurately and automatically recognizing the license plate of a car. <P>SOLUTION: A vehicle number 12 is captured by a color image pickup device 16, image pickup signals are digitized, and three-dimensional data based on the values of RGB color data are attached for the respective pixels of obtained image data. By the color clustering processing for a prescribed number of clusters to the three-dimensional data for the respective pixels, the respective pixels are classified for the respective clusters for the image data, and the obtained images of the respective clusters are used as character image data to be the object of character recognition. The character image data obtained by color clustering and a template image are compared and the character image data are discriminated. Then, the matching degree of the character image data and the template image is obtained and accumulated for the respective pixels of the character image data and identification of defining that the character image data for which an error value to respective templates becomes the minimum value for a calculated cumulative error value as the character of the template image is executed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image recognition method in which a license plate of a vehicle is imaged, compared with predetermined template data, and characters of the license plate are recognized.

  Conventionally, when character information is recognized from image data obtained by an image sensor, it is common to binarize an image and perform matching processing. In this method, first, a binarized image of a recognition target image is created so that it can be confirmed whether or not the template image matches. Next, template matching for confirming an appropriate match is performed. However, in this two-stage process, the reliability of recognition is reduced due to dirt or noise on the image. For example, when the target image state is not uniform, such as automatic reading of a license plate of a car, when the image example as shown in FIG. 8A is binarized, an image as shown in FIG. 9 is obtained as a result. Will get. Thus, image recognition cannot be performed with a simple binarized image for the entire image, or the recognition accuracy is low. Therefore, even if pattern matching is performed with such image recognition accuracy, for example, the character recognition performance of the license plate is extremely low.

On the other hand, in image recognition, as shown in Patent Documents 1 and 2, OCR technology has recently advanced. These image recognition methods perform recognition processing for each color as a plurality of image data obtained by separating color image data for each color. Accordingly, when the color of the character in the color original is different from the background color thereof, it is possible to prevent the characters (character information) from being lost because they are both converted to black (or white).
JP 2001-297303 A Japanese Patent Laid-Open No. 2004-21765

  However, in the case of character recognition of a license plate of a car where clean data such as a character manuscript cannot be obtained, it is difficult to identify the image itself due to shadows and dirt. Furthermore, not only hiragana, but also various types of characters such as kanji and 2-digit and 3-digit numbers, as well as small letters and numbers, are on the same plate, making it difficult to check the difference between them and dirt on the plate. It was difficult to improve the robustness of character recognition.

  The present invention has been made in view of the above-described problems of the prior art, and an object thereof is to provide an image recognition method capable of accurately and automatically recognizing a license plate of a car.

  The present invention applies a combination of generalized Hough transform and color clustering in pattern recognition, particularly a vehicle number recognition method, performs color clustering processing to divide into n clusters, and applies to pixels belonging to each cluster. Then, the inclusion relationship is examined to classify the pixels, and the obtained image is used as image data to be character recognition, and the image data to be character recognition is compared with the template image by generalized Hough transform. In this case, this is an image recognition method for recognizing a character based on a cumulative error value calculated and a minimum error value for each template based on the degree of matching between target image data determined as a character and a template image.

  That is, the present invention captures a vehicle number with a color imaging device, digitizes an imaging signal, and obtains each pixel of the obtained image data as three-dimensional data based on the value of RGB color data. A color clustering process of a predetermined number of clusters is performed on the three-dimensional data to classify each pixel in the image data for each cluster, and the obtained image of each cluster is used as character image data for character recognition as the color data. This is an image recognition method in which character image data obtained by clustering is compared with a template image to determine the character image data.

  Further, the degree of matching between the character image data and the template image is obtained and accumulated for each pixel of the character image data, and the calculated error value for each template is the minimum value for the accumulated error value. This is an image recognition method for identifying that data is a character of the template image. The number of clusters is preferably 3 to 5 from the relationship between processing speed and identification accuracy.

  According to the image recognition method of the present invention, the character recognition of the license plate of the vehicle can be performed with high accuracy. In particular, even a license plate containing dirt, shadows, etc. can be accurately recognized.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIGS. 1-8 shows one Embodiment of this invention, The vehicle image recognition method of this embodiment recognizes the number plate 12 of the vehicle 10, as shown in FIG. In this embodiment, an imaging device 16 having an imaging element 14 such as a semiconductor that reads the license plate 12 is provided, and the output of the imaging device 16 is input to the computer 20. The computer 20 includes a capture board 21 that receives image signals and performs predetermined processing, a CPU 22 that performs image processing and the like according to a predetermined program, a storage device 24 that stores information, and a monitor 26 that displays images. A keyboard 28 for performing predetermined input, a printer 30 as an output device, and the like are connected.

  In the image recognition method of this embodiment, the license plate 12 is recognized by dividing the pixels on the image into several clusters instead of the conventional binarization process before performing the template matching process of the captured image. Perform clustering. This clustering is performed using the values of the three primary colors of light possessed by each pixel, and is performed on each segmented image of the captured image based on RGB color information.

  Here, before the color clustering shown in step s2 of FIG. 2, in step s1 of FIG. 2, for example, as shown in FIG. 6, a process of segmenting a place that seems to correspond to a character on the license plate 12 is performed. , Judgment to determine the recognition target In this determination, the classification of templates to be compared is narrowed down in advance by classifying kanji, hiragana, and numbers (large and small) according to the position at the time of segmentation.

  Therefore, clustering will be described. Clustering refers to grouping together similar data from a large number of data sets as the same cluster and classifying them into a predetermined number of clusters. For example, the algorithm for N clusters is as shown in FIG.

  The number of clusters is N, and the points allocated to the clusters are x1, x2,... Xm. First, an arbitrary center is set for each cluster. For example, as shown in FIG. 4, c1, c2,... CN (random designation by random numbers) are set. Then, it is determined which cluster is located closest to the point (xi) of each pixel of the image sensor 14 or each pixel of the image in the processing of the computer 20. This operation clusters each point, and targets each point that is all pixels from the upper left to the lower right of the image data of the image sensor 14 or each point that is all pixels of the image in the processing of the computer 20. . Thereby, in all clusters, an average new cluster center is calculated from the positional relationship of each point in each class at the present time. When the center (average value) of the point group which is pixel data in each cluster changes, the process returns to the above-described new center calculation process and is repeated. Thereafter, the clustering process is terminated when there is no change in the center (average value) of the point group of the pixel data.

  Next, color clustering of this embodiment will be described. In the color clustering, the above-described clustering is expanded to the processing for each RGB of the image data. As shown in FIG. 5, each point xi of the image has 3 elements (R, G, B). Represented by a vector of dimensions. The clustering algorithm is the same as described above. Further, the distance from the center point vector value is expressed as follows by the Euclidean distance (color distance d: actually color difference).

The color distance d is obtained by the following formula. For example, if two points of a three-dimensional vector by RGB are x1 and x2,
x1 = (x1r, x1g, x1b), x2 = (x2r, x2g, x2b)
The distance d between the two points is d (x1, x2) = ‖ (x1r, x1g, x1b) − (x2r, x2g, x2b) ‖
= ‖ (X1r−x2r, x1g−x2g, x1b−x2b) ‖
= ((x1r−x2r) ² + (x1g−x2g) ² + (x1b−x2b) ² ) ^1/2
It is expressed.

  The color clustering process is performed as follows. Here, assuming that the license plate 12 has the contents shown in FIG. 8A, when the picked-up image has shadows and dirt as shown in FIG. 8A, the pick-up data in FIG. By performing color clustering of 4 clusters and binarizing, data as shown in FIG. 8B is obtained.

Here, the image is denoted by I (x, y), and is divided into n pixel groups with the number of clusters, and the names are C ₁ , C ₂ ,... C _n .
In each set,

Means that the pixels are in the same cluster.

  As a result, the center value (average color of R, G, B) of all pixels composed of the point group in each cluster is calculated. In this embodiment, the number of clusters is 4, and the temporary center point having the smallest value among the distances d between the pixel to be investigated and the four temporary center points is set to the center of the cluster to which the target point belongs (pixels of the same color). ). Then, this process is repeated until the center value (average color) of all the pixels in each cluster does not change. In this embodiment, the number of clusters in the color clustering is “4”, which is an optimum value obtained experimentally and empirically. The color level of each cluster image that appears varies dynamically depending on the state of the image being processed.

  Next, based on each segment image obtained by the color clustering process, a comparison template for image recognition is prepared in step s3 of FIG. 2, and the following pattern matching process is performed. For example, as shown in FIG. 7, for the character segment “Toyama”, the inclusion state of each pixel in each cluster is examined for the entire one-segment image. Since the clusters are classified into four by the above color clustering process, four binarized images are acquired from the investigation result of the inclusion relation. Here, for the sake of simplicity, the image is once replaced with a black and white image. In practice, a binarized image is acquired from a color clustering image. Pixels belonging to the cluster are represented as “black: value = 1”, and “white: value = 0” when they do not belong.

  Then, in the routine r1 of FIG. 2, the position of the target image is adjusted for each template image for each classification, and the matching state with the clustered target images is checked. A common area between each acquired image and the template image and a different area (error) are obtained, and the matching status is examined by sequentially switching all the templates being classified. Then, the sum of the common area and the different areas is calculated. When the sum of the common areas is larger in area than the sum of the different areas, the target image is processed as a portion representing “character”. When the different area is large, it is processed as a “background” portion. Here, the reason why the “character” and “background” determination processing is necessary is to determine which of the obtained four binarized cluster images represents what appears to be “character”. This process is performed sequentially for all clusters as shown in the routine r2 of FIG.

  Regarding the target image processed as “character”, the image with the least error (difference) during the above processing is selected from the template as the target character (recognition result).

This pattern matching process is performed as follows.
For all classes, i = 1 ·· n,

And The best matching of Ii is searched for the template image J. Let T be a transformation that changes Ii to J.

Means the minimum value.
Next, for all clusters, j = 1... N, j ≠ i

The class belongs to the foreground.

If Err (i) <Err, the degree of matching is higher than the previous matching
Err indicates a minimum matching error that is reached by the clustering for the pattern J.

In the case of the above sample, four clusters are used. I ₁ , I ₂ , I ₃ , and I ₄ are shown in order from the left as shown in FIG. For simplicity, it is displayed in black and white. Each image corresponds to the state shown when any one of the clusters is “ON”.

  A normal pattern matching algorithm (generalized Hough transform) is used to position each template image J at the closest match, as shown in the flowchart of FIG. Here, the transformation is represented as T. T is

It is the conversion shown by. In other words, searching for the best match

Means searching for (tx, ty) that best matches J (x, y).

  After the best matching is searched for one cluster image Ii in the routine r1 of FIG. 2, the routine r2 further determines which of the other clusters belongs to the foreground (character portion) and which is in the background. Check if it belongs. In the case of this example, it can be seen that cluster 4 belongs to the foreground indicating the Chinese character part “Toyama”, and clusters 1 and 2 belong to the foreground of the numeric part.

  Then, it is determined which template related to the character that the image given by the clustering of the character image and determined as the character best matches in advance. In step s4 of FIG. 2, it is assumed that the template having the smallest error that gives the best matching template among all templates is the target character. At this stage, the algorithm indicates that the character is recognized. In step s5, the character of the template is selected, and the image recognition is completed.

  Here, in the above algorithm, the parameter n of the number of clusters has decisive importance. If n is large, each cluster is small, and good results (small error values) are obtained for all characters. It is therefore not possible to identify which is the best. However, the problem that occurs when n is large is that the processing speed of the algorithm decreases. As a result, it is important that all the clusters include portions that well represent the character features. From experience, a value of 3 to 5 gives the best results. For this number, each class contains useful information about the foreground / background and the algorithm is faster.

  According to the image recognition method of this embodiment, the recognition accuracy of the entire image recognition algorithm is improved. Although the previous algorithm had a recognition rate of about 90%, this algorithm showed a recognition rate of 98%, which can be said to be a level with system reliability. Based on this algorithm, when used for character recognition of car license plates, it became possible to distinguish between foreground and background effectively, and a significant improvement in recognition results was recognized compared to license plate recognition. .

It is the schematic which shows the system of the image recognition method of one Embodiment of this invention. It is a flowchart which shows the process of the image recognition method of this embodiment. It is a flowchart which shows the color clustering process of the image recognition method of this embodiment. It is a conceptual diagram which shows the clustering of the image recognition method of this embodiment. It is a conceptual diagram which shows the color clustering of the image recognition method of this embodiment. It is a figure which shows the character of the recognition target of the image recognition method of this embodiment. It is a conceptual diagram which shows the state which carried out the color clustering of the character of the recognition target of the image recognition method of this embodiment. It is the figure which shows the (a) which shows the imaging state of the license plate of the recognition object of the image recognition method of this execution form, and the figure (b) which shows the number plate of the state where color clustering is done. It is a figure which shows the license plate binarized in the conventional image recognition method.

Explanation of symbols

10 car 12 license plate 14 image sensor 16 image pickup device 20 computer 22 CPU
24 storage device 26 monitor 28 keyboard 30 printer

Claims (3)

  A color imaging device captures the vehicle number and digitizes the imaging signal. For each pixel of the obtained image data, it is set as three-dimensional data based on the value of RGB color data. The color clustering process with a predetermined number of clusters is used to classify the pixels of the image data for each cluster, and the obtained image of each cluster is obtained as a character image data for character recognition by the color clustering. An image recognition method comprising: comparing character image data with a template image and determining the character image data.   The degree of coincidence between the character image data and the template image is obtained and accumulated for each pixel of the character image data, and the character image data for which the error value for each template is the minimum value is calculated for the accumulated error value calculated. 2. The image recognition method according to claim 1, wherein the template image is identified as a character. The image recognition method according to claim 1, wherein the number of clusters is 3 to 5.