JP4901676B2 - License plate information processing apparatus and license plate information processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for highly precisely extracting a license plate region. <P>SOLUTION: This license plate information processing method includes: detecting a plurality of character candidate regions from an image; voting the position information on each of a plurality of character candidate regions to a voting space configured of the whole or a part of geometrical parameters; selecting voting space coordinates whose vote becomes the largest or the maximum or a prescribed threshold or more; predicting the arrangement of a plurality of character regions in a license plate candidate region from the plurality of character candidate regions according to the value of the parameter corresponding to the voting space coordinates; geometrically converting the template corresponding to the license plate on the basis of the arrangement prediction; approximating the license plate candidate region by the geometrically converted template; and extracting the license plate region on the basis of the approximation result. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to a technology for extracting a license plate region from a vehicle photographed image.

  Conventionally, as a technique for extracting a license plate region, by utilizing the fact that a large number of horizontal edges are detected in the character portion of the license plate, binarization is performed by applying a horizontal edge detection filter and a similar filter to obtain edge pixels. A method for extracting a region having a large amount of noise has been proposed (see Patent Document 1).

  In addition, a method has been proposed in which a character (pattern) position is first detected and a pattern string is extracted by collating with a predetermined pattern arrangement (see Patent Document 2).

  Furthermore, after narrowing the range using something similar to an edge detection filter, a method is proposed in which character recognition is performed to check the presence of a character, and a portion where characters are adjacent to each other is extracted as a license plate region (patent) Reference 3).

Also, in order to detect linear features such as license plate frames and road white lines, Hough transforms that vote the edge pixels to a linear parameter space and extract the straight line represented by the parameter that maximizes the voting result However, it is known as a general technique in the field of image processing.
Japanese Patent No. 3679958 Japanese Patent No. 3762189 Japanese Patent No. 3455454

  However, the method of cutting out the license plate area using the image that has been subjected to the edge detection filter may erroneously detect an area having a lot of edge components in the horizontal direction other than the license plate portion, such as a vehicle headlight or a car body pattern. is there. If there are many edge components in the area adjacent to the license plate, the end of the license plate area cannot be cut out correctly.

  The method of extracting the license plate area by extracting the characters such as the vehicle number and comparing it with the arrangement of the characters on the license plate is that the arrangement of the characters is known in advance, that is, the direction and size of the license plate in the image Is assumed to be unchanged. If the vehicle was shot in a different direction or distance than expected, or if the shooting position / angle is unknown, the size of the characters in the license plate in the image and the angle of the character string are prepared in advance. It cannot be verified because it is different.

  In addition, the method of collating the character arrangement sequentially one character at a time or the method of detecting the area where the character is adjacent will extract the license plate area when the character detection fails and the character string is missing. Can not.

  An object of the present invention is to provide a technique for extracting a license plate region with high accuracy.

  The license plate information processing apparatus according to one embodiment of the present invention provides a plurality of the plurality of character means for detecting a plurality of character candidate regions from an image and a voting space constituted by all or part of geometric figure parameters. Vote each position information of the character candidate area, select a voting space coordinate having the maximum or maximum number of votes or a predetermined threshold value or more, and select a voting space coordinate among the plurality of character candidate areas according to a parameter value corresponding to the voting space coordinate. Predicts the arrangement of multiple character areas in the license plate candidate area, geometrically transforms the template corresponding to the license plate based on this arrangement prediction, approximates the license plate candidate area using this geometrically transformed template, and results of approximation And a license plate area extracting means for extracting a license plate area based on the above.

  The license plate information processing apparatus according to an embodiment of the present invention is configured to detect the plurality of character candidate regions from an image and a plurality of voting spaces configured by all or part of geometric transformation parameters. Vote the position information of each character candidate area, select the voting space coordinates with the maximum or maximum number of votes, or a predetermined threshold value or more, and geometrically shape the template corresponding to the license plate according to the parameter value corresponding to the voting space coordinates. License plate area extraction means for converting, approximating a license plate candidate area with the geometrically converted template, and extracting the license plate area based on the approximation result.

  The license plate information processing method according to an embodiment of the present invention detects a plurality of character candidate regions from an image, and the plurality of character candidate regions with respect to a voting space constituted by all or part of geometric figure parameters. The voting space coordinates with the maximum or maximum number of votes or a predetermined threshold value or more are selected, and the number plate is selected from among the plurality of character candidate areas according to the parameter value corresponding to the voting space coordinates. Predict the placement of multiple character areas in the candidate area, geometrically transform the template corresponding to the license plate based on this placement prediction, approximate the license plate candidate area using this geometrically transformed template, and number based on the approximation result Extract plate area.

  The license plate information processing method according to an embodiment of the present invention detects a plurality of character candidate regions from an image, and the plurality of character candidate regions with respect to a voting space configured by all or part of geometric transformation parameters. Voting the position information of each, selecting the voting space coordinates where the number of votes is the maximum or maximum or a predetermined threshold or more, geometrically transform the template corresponding to the license plate by the value of the parameter corresponding to the voting space coordinates, The license plate candidate area is approximated by the geometrically converted template, and the license plate area is extracted based on the approximation result.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which extracts a license plate area | region with high precision can be provided.

  The license plate image processing apparatus (number plate information processing apparatus) according to an embodiment of the present invention extracts the license plate area as follows, for example. First, vote the character candidate area that has been recognized from the image cut out from the image to a voting space consisting of parameters representing the license plate area or a part of it, select the coordinates with the most votes, and select the corresponding number. The plate area is taken as the extraction result. The parameters constituting the voting space are, for example, parameters for geometrical transformation (affine transformation, projective transformation, etc.) for overlaying a template shaped like a license plate seen from the front as shown in FIG. Part of it.

  Another form of voting space is a geometric figure that determines the arrangement of license plate components in the image (a straight line that passes through a 4-digit vehicle number, a square frame that borders the license plate, a license plate seal, etc.) There are parameters or combinations thereof. That is, a rectangular frame bordering the license plate is detected from the image, and a license plate area is extracted as described later using the detected frame. Alternatively, a license plate seal is detected from the image, and a license plate region is extracted as described later using the detected seal.

  For voting, not only the position information of the character candidate area, but also geometric information (height, width) and character recognition information (character recognition result, character recognition score, character inclination, etc.) And so on.

  When the voting space is a geometric figure parameter for determining the arrangement of the license plate components in the image, the license plate area is determined based on the geometric figure.

  Thereafter, post-processing is performed as necessary to extract a license plate area in the input image. The post-processing varies depending on the design of the voting space, and includes, for example, correction of errors generated due to parameter quantization by voting, and extraction / recognition of character regions and character string regions that have failed to be extracted.

  Details of the above-described license plate image processing apparatus will be described below.

  First, the outline of a Japanese license plate will be explained with reference to FIG. The license plate includes a vehicle number 101, a land transportation branch office name 102, a vehicle type code 103, and a hiragana 104. The vehicle number is a number from 1 to 4 digits. If the number is 3 digits or less, "." Is entered instead of a number, and "-" between the 2nd and 3rd digits is omitted.

  Next, a schematic configuration of the above-described license plate image processing apparatus will be described with reference to FIG. As shown in FIG. 2, the license plate image processing apparatus includes an image input device 201, a camera interface unit 202, an image processing unit 203, and a recognition result presentation device 208. The image processing unit 203 includes an input / output interface unit 204, a character detection / recognition unit 205, a license plate area extraction unit 206, and a post-processing unit 207.

  The image input device 201 is, for example, a digital camera, images a vehicle including a license plate at a toll gate or the like, and inputs the captured image to the image processing unit 203 through the camera interface unit 202. The character detection / recognition unit 205 detects a character candidate from the input image and recognizes the detected character candidate. The license plate area extraction unit 206 extracts the license plate area based on the arrangement of the detected / recognized characters. The post-processing unit 207 performs post-processing described later as necessary. The recognition result presentation device 208 presents the recognition result processed by the image processing unit 203 to the user.

  Hereinafter, the license plate recognition process for an automobile will be described with reference to the flowchart of FIG. First, an image is acquired by the image input device 201 (ST301), and preprocessing is performed on the acquired image (ST302). The preprocessing is processing for realizing character region segmentation and character recognition, such as extraction of components below a specific frequency, binarization, and the like by expansion / reduction processing. FIG. 4 is a diagram illustrating an example of the pre-processed image 400.

  Next, a character candidate area is cut out from the preprocessed image 400 using a method such as labeling (ST303). An example of the cutout result image 500 by labeling is shown in FIG. As illustrated in FIG. 5, the cutout result image 500 by labeling includes a circumscribed rectangle 501 of the cutout area. Actually, since many candidates other than characters are also cut out, an area whose size and aspect ratio are far from the target character area is excluded. Further, the extraction of the target character may fail, but this is dealt with by post-processing (ST308) described later.

  Thereafter, as shown in FIG. 6, an image 600 including a plurality of narrowed character candidate regions is obtained, character recognition is performed on the character candidate region 601 in the image 600 (ST304), and the character recognition result is displayed in the character candidate region 601. Character recognition information 602 such as a character recognition score is assigned.

  Next, a voting space serving as a voting destination for the character candidate area 601 is created (ST305), and the character candidate area is voted for the voting space (ST306). Here, three examples of voting space and voting method will be described.

  FIG. 8A shows an example of a license plate template 701 and an example of processing for approximating a license plate 802 in an image 801 by affine transformation of the template 701. FIG. 8B is a conceptual diagram of the voting space 803. As shown in FIG. 7, in the coordinate system of the template 701, the upper left is the origin, the right direction is the positive direction of the xp axis, and the lower direction is the positive direction of the yp axis. Template 701 is license plate frame 702, vehicle number 1st digit 703, 2nd digit 704, 3rd digit 705, 4th digit 706, land transportation branch and vehicle type code area 707, hiragana 708, and each rectangular vertex corresponding to each The coordinate value in the license plate coordinate system (xp, yp). The number plate 802 in the image 801 is approximated by transforming this with affine transformation parameters. In the coordinate system of the image 801, the upper left is the origin, the right direction is the positive direction of the x axis, and the lower direction is the positive direction of the y axis.

The coordinate values (xp, yp) of the license plate template 701 are converted into coordinates (x, y) in the image 801 by affine transformation. The conversion formula is as shown in Formula 1 below.

  The affine transformation parameters are: x-axis translation component x_translate, y-axis translation component y_translate, x-axis scaling component x_scale, y-axis scaling component y_scale, x-axis shear component x_shear, There are six y-shear components in the y-axis direction. The voting space 803 is a 6-dimensional space composed of these 6 parameters, and an arbitrary point in the voting space is a set of values taken by the parameters. When the character candidate area 601 is made to correspond to each element of the template 701, voting is performed for all possible parameter combinations in the voting space.

  In the voting method, not only the position information of the character candidate area 601 but also the geometric information (width and height) and the character recognition information 602 can be used to limit the coordinates of the voting destination. For example, the values of x_scale and y_scale to be voted are determined according to the height and width of the character candidate area. When information such as the inclination of the character is obtained from the character recognition result, the value of x_shear is determined accordingly. This is a restriction on the vote destination based on the character recognition information.

  Also, the influence of noise can be reduced by weighting the vote by the character recognition information, instead of voting equally for all the character candidate areas. For example, if the character recognition result is a number and the character recognition score is relatively high, voting is limited to the vehicle number or vehicle type code, and the number of votes is also higher than other character candidate areas. . Conversely, a candidate character area having a low character recognition score as a number is voted for the name of the land transportation branch office so that only a small number is voted.

  Depending on the image quality and the clipping method of the original image, there may be a deviation in the clipping result of the character candidate area. Therefore, it is desirable to vote not only for one vote but also for the surrounding area.

  At the stage where voting is completed for all the character candidate areas 601, the coordinates in the voting space 803 where the number of votes is the maximum, the threshold value is exceeded, or the maximum is selected (ST307), and the template 701 is selected according to the parameters corresponding to the coordinates. The license plate area approximated by the geometric transformation is set as the selected license plate candidate area.

  Here, the voting space does not necessarily need to use all six dimensions. For example, when the direction in which the vehicle passes is determined in advance, the y_shear component can be omitted due to restrictions on the direction of the license plate relative to the vehicle. In this case, x_scale and y_scale can be set as one parameter scale by utilizing the fact that the ratio of x_scale and y_scale is constant.

  FIG. 9A is a diagram illustrating an example of a license plate template 701 and an example of a process for projectively transforming the template 701 to approximate the license plate 902 in the image 901. FIG. 9B is a conceptual diagram of the voting space 903. As in the case of affine transformation, the voting space is composed of transformation parameters. In the case of projective transformation, since there are six independent transformation parameters, the voting space has a maximum of six dimensions.

  FIG. 10B is a diagram showing an example of the voting space 1003 when a straight line passing through the vehicle number is detected. FIG. 10A is a diagram showing an example of a straight line 1004 represented by parameters of the voting space 1003. Basically, it is composed of a parameter θ for determining the angle of the straight line and a parameter ρ for determining the position of the straight line, in the same manner as in the straight line detection method by the normal Hough transform. In addition, the character height S is also included in the elements of the voting space by utilizing the fact that each character of the vehicle number has the same height. By doing so, it is possible to prevent detection of a column of character candidate regions having completely different heights. That is, this corresponds to making the geometric information of the character candidate area a part of the axis of the voting space. Similarly, by utilizing that the vehicle numbers in the same license plate are all inclined to the same extent, the inclination of the character obtained by character recognition may be included in the elements of the voting space. That is, the character recognition information in the character candidate area may be part of the voting space axis.

  After all the votings are finished, a coordinate in which the number of votes is the maximum, equal to or greater than the threshold, or the maximum is selected in the voting space 1003, and the corresponding straight line is set as a straight line 1003 passing through the vehicle number.

  Post-processing after detecting a straight line 1003 passing through the vehicle number will be described with reference to the flowchart shown in FIG. 11 and FIGS. 12A to 12F. First, paying attention to the character candidate area 1202 voted for the detected straight line 1201, the number of vehicle number candidates on the straight line is determined by the value obtained by normalizing the distance 1203 between the character candidate areas by the width of the character candidate area 1202. The primary vehicle number arrangement candidate 1204 is derived (ST1101).

  If the character candidate area on the straight line is indistinguishable as in the case of the 1st to 2nd digits and the 3rd to 4th digits, both candidates are left. Next, duplicates of the primary vehicle number arrangement candidates 1204 are removed (ST1102), secondary vehicle number arrangement candidates 1205 are generated, and the license plate template 701 is overlapped with the corresponding character candidate area by the vehicle number portion. The license plate candidate area 1206 is obtained by superimposing the image into a number plate.

  After selecting the license plate candidate area 1206, post-processing as shown in FIGS. 13 and 14 is performed (ST308).

  FIG. 13 is a diagram for explaining position correction for a license plate candidate area selected from a voting space and post-processed as necessary. By voting in the voting space, the value of each parameter is quantized. Therefore, there may be a deviation between the voting source character candidate area and the license plate candidate area obtained by voting. Here, for example, the license plate candidate area is translated so that the distance between the center 1302 of the character candidate area corresponding to the vehicle number and the center 1303 of the vehicle number position of the template is minimized.

  FIG. 14 is a diagram showing character re-cutout and character recognition for the estimated license plate candidate area 1401. Due to various factors such as the image quality of the original image, the preprocessing method, the contamination of the license plate, and the hiding, the character candidate area may sometimes fail to be cut out. If there is no corresponding character candidate area for each character position in the template, the image is viewed again, the corresponding area is cut out, and character recognition is performed.

  At this time, since a plurality of license plate candidate regions are extracted for one image, they are narrowed down to one by a method such as scoring (ST309) and output to the recognition result presentation device (ST310).

  As described above, the license plate image processing apparatus votes each character candidate area constituting the license plate in a voting space formed by parameters representing the license plate area or a part thereof, and the coordinate value having a large number of votes. Therefore, even if the direction of the license plate to be extracted is unknown, the character candidate string corresponding to the license plate can be extracted.

  In addition, even if the extraction of some character candidate areas fails, if the number of votes from other character candidate areas is sufficient, the license plate area is extracted, and the recognition result of the part that has failed to be extracted by post-processing Can be obtained.

  In the above description, the voting space is created in ST305. However, the present embodiment is not limited to this. The voting space is created at a predetermined timing regardless of the process up to ST304. For example, a voting space may be created first. Speaking of specific processing, space is not designed but mere memory area is secured.

  In addition, what figure or conversion is adopted is determined at the design stage of the program (= the parameter is determined and the voting space is determined). For example, if a straight line is adopted, θ and ρ are selected, and if affine transformation is adopted, six parameters (or some of the parameters) used for affine transformation are selected. Then, a set of possible parameter values is calculated from the character candidate area. For example, vote for (θ, ρ) = (30,320), (31,320), (32,319), (33,318), (34,318), .... When voting is completed for all character candidate areas, coordinates (= parameter values) that take local maximum values and the like are selected from the voting space. This is the same flow as the straight line detection using the Hough space used in image processing. By using a character candidate area for voting, using character recognition information, and using part of the voting space axis as such information, the license plate area can be extracted with high accuracy.

  Note that the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention in the implementation stage. In addition, the embodiments may be appropriately combined as much as possible, and in that case, the combined effect can be obtained. Further, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be obtained as an invention.

It is a figure for demonstrating the outline | summary of a Japanese license plate. It is a figure for demonstrating schematic structure of a license plate image processing apparatus. It is a flowchart which shows an example of the recognition process of the number plate of a motor vehicle. It is a figure which shows an example of the image after a pre-processing. It is a figure which shows an example of the cutting-out result by labeling. It is a figure which shows an example of the character candidate area | region which gave restrictions with the magnitude | size with respect to the cut-out area | region and recognized the character. It is a figure which shows an example of the template for license plates. It is a figure which shows an example of the template for number plates, and shows an example of the process which approximates the number plate in an image by affine-transforming a template. It is a conceptual diagram of the voting space comprised with the parameter of an affine transformation. It is a figure which shows an example of the template for license plates, and also shows an example of the process which carries out projective transformation of a template and approximates the license plate in an image. It is a conceptual diagram of the voting space comprised by the parameter of projective transformation. It is a figure which shows an example of the straight line represented by the parameter of voting space. It is a figure which shows an example of the voting space in the case of detecting the straight line which passes a vehicle number. It is a flowchart which shows an example of post-processing. It is a figure which shows an example of the process which derives a primary vehicle number arrangement | positioning candidate from a character candidate area | region. It is a figure which shows an example of the process which derives a primary vehicle number arrangement | positioning candidate from a character candidate area | region. It is a figure which shows an example of the process which derives a primary vehicle number arrangement | positioning candidate from a character candidate area | region. It is a figure which shows an example of the process which derives | leads-out a license plate candidate area | region from a secondary vehicle number arrangement | positioning candidate. It is a figure which shows an example of the process which derives | leads-out a license plate candidate area | region from a secondary vehicle number arrangement | positioning candidate. It is a figure which shows an example of the process which derives | leads-out a license plate candidate area | region from a secondary vehicle number arrangement | positioning candidate. It is a figure which shows an example of the post-process which correct | amends the shift | offset | difference between the character candidate area | region of a voting origin, and the license plate candidate area | region obtained by voting. It is a figure which shows an example of the post-process which cuts out a corresponding area | region and performs character recognition, when the corresponding character candidate area | region does not exist with respect to each character position of a template.

Explanation of symbols

201 ... Image input device 202 ... Camera interface unit 203 ... Image processing unit 204 ... Input / output interface unit 205 ... Character detection / recognition unit 206 ... Number plate area extraction unit 207 ... Post-processing unit 208 ... Recognition Result presentation device

Claims (6)

Detecting means for detecting a plurality of character candidate regions from the image;
Voting position information of each of the plurality of character candidate areas is voted for a voting space constituted by all or part of geometric figure parameters, and voting space coordinates having a maximum or maximum number of votes or a predetermined threshold value or more are selected. The placement of a plurality of character regions in the license plate candidate region is predicted from the plurality of character candidate regions according to the parameter value corresponding to the voting space coordinates, and a template corresponding to the license plate is determined based on the placement prediction. License plate area extraction means for performing geometric transformation, approximating a license plate candidate area with the geometrically transformed template, and extracting a license plate area based on the approximation result;
License plate information processing device with Detecting means for detecting a plurality of character candidate regions from the image;
Voting a geometric conversion parameter for converting a character position of a template including character arrangement information of a license plate into a position in the image of the plurality of character candidate areas for a voting space constituted by all or part of the geometric conversion parameter Then, the voting space coordinates with the maximum or maximum number of votes or a predetermined threshold value or more are selected, the template corresponding to the license plate is geometrically transformed according to the value of the parameter corresponding to the voting space coordinates, and the geometrically transformed template is used. A license plate area extracting means for approximating a license plate candidate area and extracting the license plate area based on an approximation result;
License plate information processing device with   The license plate area extracting unit recognizes each character of the plurality of character candidate areas, and calculates a weighting coefficient for each of the plurality of character candidate areas based on each character recognition result of the plurality of character candidate areas. 3. The license plate information processing apparatus according to claim 1, wherein the voting of the position information of the plurality of character candidate areas is weighted by the weighting coefficient. The number plate area extracting means configures the voting space based on parameters obtained from position information of the plurality of character candidate areas and parameters obtained from geometric information of the plurality of character candidate areas. The license plate information processing apparatus according to 1 or 2. Detect multiple character candidate areas from the image,
Voting position information of each of the plurality of character candidate areas for a voting space constituted by all or part of geometric figure parameters,
Select the voting space coordinates where the number of votes is maximum, maximum or above a predetermined threshold,
Predicting the arrangement of a plurality of character regions in the license plate candidate region from among the plurality of character candidate regions by the value of the parameter corresponding to the voting space coordinates,
Based on this placement prediction, the template corresponding to the license plate is geometrically transformed,
Approximate the license plate candidate area with this geometrically transformed template,
Extract license plate area based on approximate result,
A license plate information processing method. Detect multiple character candidate areas from the image,
Voting a geometric conversion parameter for converting a character position of a template including character arrangement information of a license plate into a position in the image of the plurality of character candidate areas for a voting space constituted by all or part of the geometric conversion parameter And
Select the voting space coordinates where the number of votes is maximum, maximum or above a predetermined threshold,
The template corresponding to the license plate is geometrically transformed by the value of the parameter corresponding to the voting space coordinates,
Approximate the license plate candidate area with this geometrically transformed template,
Extracting the license plate region based on the approximation result;
A license plate information processing method.

Images