JP2005182236A - Vehicle number reader - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more certainly prevent recognition leakage to recognize a series of numbers with high accuracy. <P>SOLUTION: An image signal of an image including an image of a license plate of a vehicle picked up by a camera 1 is A/D converted and is stored in an image memory 3. A processing part 4 extracts a license plate candidate area that is a candidate of a license plate area from the image, and recognizes each character of the series of the numbers on the basis of the image of the area. When the number of the recognized characters are two or three, the processing part 4 acquires a recognition leakage character candidate area, and newly recognizes the characters of the series of the numbers on the basis of the image of the area. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a vehicle number reading device that processes a captured image including an image of a license plate of a vehicle and reads a vehicle number.

  Patent Document 1 below discloses a number plate serial designation number (also referred to as “serial number”) cutting device used as part of a vehicle number reading device. This clipping device includes clipping means for cutting out a serial number from an image in the license plate area, and clipping interpolation means for estimating a character position that could not be cut out by the cutting means and cutting out the character position (Japanese Patent Application Laid-Open No. 2003-122867). Claim 4, paragraph numbers [0023] to [0032]). In the vehicle number reading device using this clipping device, the serial number is recognized by performing processing for recognizing the characters of the serial number with respect to the character region of the serial number clipped by the clipping device.

  According to the cutout device, the cutout interpolation unit estimates the character position that could not be cut out by the cutout unit, and cuts out the character position, so that the reliability of cutting out the character position of the serial number is increased. Therefore, according to the vehicle number reading device using the cutout device, it is possible to reduce omission of recognition of serial numbers as compared with the case where the cutout interpolation unit is not provided.

Further, in the vehicle number reading device, the number plate is often imaged from an oblique direction when an image including the vehicle number is captured. In this case, the image of the license plate is subject to rotational distortion and other distortions with respect to the image taken from the front. For this reason, if you try to recognize the vehicle number using the image of the license plate with distortion as it is, the character recognition accuracy will decrease, or many character patterns that take distortion into consideration will be prepared as character patterns used for character recognition. Have to do. Therefore, conventionally, after extracting the image of the license plate area from the captured image, distortion correction is performed on the extracted image of the license plate area, and character recognition processing is performed on the image of the license plate area after distortion correction. Various vehicle number reading devices are provided. An example of a vehicle number reading device that performs such distortion correction is disclosed in Patent Document 2 below.
JP-A-9-326209 JP 2002-7961 A

  However, the clipping device disclosed in Patent Document 1 simply cuts out the character position of the serial number as a pre-processing for character recognition of the serial number, and the clipping means and the clipping interpolation means themselves do not recognize any character. Is not intended to be used. Therefore, in the clipping device disclosed in Patent Document 1, the region of the character position of the serial number clipped by the clipping means is not a character region but actually cut out incorrectly when trying to recognize a character. There is a possibility that the region of the character position interpolated by the cut-out interpolation means may be a region cut out by mistake. For this reason, the vehicle number reading device using the clipping device disclosed in Patent Document 1 cannot recognize the serial number with sufficient accuracy.

  Further, in the conventional vehicle number reading device that performs distortion correction as disclosed in Patent Document 2, after extracting the image of the license plate area from the captured image, the extracted image of the license plate area is extracted. Only perform distortion correction. In this case, there is no problem if the extracted license plate area is always correct. However, in the conventional vehicle number reading apparatus that performs distortion correction, distortion correction is performed only on the image of the license plate area extracted first, so it was found that the license plate area extracted first is incorrect later. Sometimes, when character recognition processing is performed for a newly identified correct license plate area, distortion correction must be performed again for the newly identified correct license plate area, and double distortion correction is required. Therefore, the processing cannot be performed efficiently.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a vehicle number reading device that can more reliably prevent a recognition failure and recognize a serial number with high accuracy.

  In addition, the present invention can more reliably prevent a recognition omission and recognize a serial number with high accuracy, and can also recognize a serial number with higher accuracy by using distortion correction. It is an object of the present invention to provide a vehicle number reading device that can perform processing well.

  In order to solve the above problems, a vehicle number reading device according to a first aspect of the present invention is a vehicle number reading device that reads a vehicle number by processing a captured image including an image of a vehicle license plate, License plate candidate area acquisition means for obtaining a license plate candidate area that is a candidate for a license plate area based on the captured image, and a serial number based on the image of the license plate candidate area of the captured image Character recognition processing means for performing recognition processing for recognizing each character, and when the number of characters recognized by the character recognition processing means is two or three, the character recognition processing of the serial numbers Acquired recognition missing character candidate area that obtains a recognition missing character candidate area that is a candidate for a character area that may have been recognized by the means. And when the number of characters recognized by the character recognition processing means is two or three, the serial number is based on the image of the recognition missing character candidate area in the captured image. And a character re-recognition processing means for performing re-recognition processing for re-recognizing the character.

  According to the first aspect, after the character recognition processing means recognizes each character of the serial number based on the image of the license plate candidate area, the recognition is performed when the number of recognized characters is two or three. The recognized leaked character candidate area is acquired by the leaked character candidate area acquiring means, and the character having the serial number is re-recognized based on the image of the leaked character candidate area by the character re-recognition processing means. As described above, according to the first aspect, with respect to the serial number, the character recognition itself is skillfully used for acquiring the recognition missing character candidate area, and the recognition recognition character thus acquired is re-recognized. Therefore, according to the first aspect, the vehicle number is read using the clipping device disclosed in Patent Document 1 that cuts out a character area that is completely separated from character recognition as character recognition preprocessing for serial numbers. Compared to the device, it is possible to more reliably prevent the recognition omission and recognize the serial number with higher accuracy.

  A vehicle number reading device according to a second aspect of the present invention includes, in the first aspect, a distortion correction unit that corrects distortion with respect to a region including the license plate candidate region in the captured image, The character recognition processing means performs recognition processing for recognizing each character of the serial number based on an image of an area corresponding to the license plate candidate area in the image corrected by the distortion correction means. is there.

  According to the second aspect, since the character recognition processing means recognizes each character of the serial number based on the image corrected by the distortion correction means, the recognition omission can be more reliably performed as in the first aspect. Thus, the serial number can be recognized with high accuracy, and the serial number can be recognized with higher accuracy by using distortion correction.

  The vehicle number reading device according to a third aspect of the present invention is the vehicle number reading device according to the second aspect, wherein the region in the captured image that is to be corrected by the distortion correction unit is detected by the leaked character candidate region acquisition unit. Including a region corresponding to a region that may be a character candidate region, and the character re-recognition processing unit has two or three characters recognized by the character recognition processing unit. The re-recognition processing for re-recognizing the character having the serial number is performed based on the image of the recognition-missing character candidate area among the images corrected by the distortion correcting unit.

  In the second aspect, the region in the captured image that is to be corrected by the distortion correcting unit may be only the license plate candidate region. However, in the third aspect, an area in the captured image that is to be corrected by the distortion correcting unit may be the recognized missing character candidate region by the leaked character candidate region acquiring unit. The re-recognition processing means performs re-recognition processing based on the image of the recognition missing character candidate area in the image corrected by the distortion correction means. A recognition missing character candidate area that is not included can be re-recognized with an image after distortion correction without performing distortion correction again. Therefore, according to the third aspect, while obtaining the same advantage as the second aspect, there is no need to perform double distortion correction, and it is possible to obtain an advantage that processing can be performed efficiently. .

  In the vehicle number reading device according to the fourth aspect of the present invention, in the second or third aspect, the distortion correction means corrects rotational distortion. The fourth aspect is an example of distortion correction. However, in the second and third aspects, other various distortion corrections may be performed. For example, the fourth aspect is disclosed in Patent Document 2. It is also possible to adopt the current distortion correction.

  The vehicle number reading device according to the fifth aspect of the present invention is the vehicle number reading device according to any one of the first to fourth aspects, wherein the recognition-missing character candidate area acquisition means is (i) a character recognized by the character recognition processing means. When the number of is two, the center interval on the image of the two characters, the character width on the image of at least one of the two characters, and the at least one character are recognized as “1”. Whether the two characters are the first digit and the second digit character of the serial number or the first pattern that is the third digit and the fourth digit character based on whether or not , Whether the two characters correspond to the second pattern of the second and third digits of the serial number, whether the two characters are the first and third digits of the serial number Or in the 2nd and 4th digit characters First to determine whether the second pattern corresponds to the third pattern and whether the two characters correspond to the fourth pattern which is the first digit and the fourth digit of the serial number. And (ii) whether the two characters are the first and second digits of the serial number when the first determination unit determines that the first pattern is satisfied. Or (iii) a case where it is determined by the first determination means that the character corresponds to the third pattern, if it is a character of the third digit and the fourth digit of the serial number; A third determination means for determining whether the two characters are the first and third digits of the serial number or the second and fourth digits of the serial number; iv) Based on the determination results by the first to third determination means, Means for obtaining a character candidate region Re 識漏, is intended to include.

  The fifth aspect is a specific example of a method for acquiring a recognition-missing character candidate region when the number of characters recognized by the character recognition processing unit is two.

  The first to fourth aspects are not limited to such a method. For example, instead of the first determination means, based on the ratio between the center distance and the vertical dimension of the two characters. A determination unit that determines which of the first to fourth patterns is applicable may be employed. However, in this case, since the ratio varies depending on the elevation angle when the license plate is imaged, the accuracy of determining which of the first to fourth patterns falls is correspondingly reduced. End up. On the other hand, in the fifth aspect, the first determination means includes the center interval on the image of the two characters, the character width on the image of at least one of the two characters, and the at least the character. Based on whether or not one character is recognized as “1”, it is determined which one of the first to fourth patterns is applicable, so basically the ratio between the center interval and the character width is determined. By using this, it is possible to remove the influence of fluctuations in the elevation angle when the license plate is imaged, and the accuracy of determination as to which of the first to fourth patterns is met. Therefore, it is preferable to employ the determination method as in the fifth aspect.

  The vehicle number reading device according to a sixth aspect of the present invention is the vehicle number reading device according to the fifth aspect, wherein the second determination means has a classification number on the upper side of the position of the lower digit of the two characters as a reference. The determination is performed according to whether or not there is a region corresponding to a part of (vehicle type code). The sixth aspect is a specific example of the determination method by the second determination means.

  The vehicle number reading device according to a seventh aspect of the present invention is the vehicle number reading device according to the fifth aspect, wherein the second determination means is a hiragana character estimated based on the position of one of the two characters. The determination is made according to whether or not hiragana characters can be recognized from the region. The seventh aspect is another specific example of the determination method by the second determination means.

  In the vehicle number reading device according to the eighth aspect of the present invention, in any one of the fifth to seventh aspects, the third determination means uses the position of the lower-order character of the two characters as a reference. The determination is performed depending on whether or not an area corresponding to a part of the classification number exists in the upper part. The eighth aspect is a specific example of the determination method by the third determination means.

  In the vehicle number reading device according to the ninth aspect of the present invention, in any one of the fifth to seventh aspects, the third determination means uses the position of one of the two characters as a reference. The determination is performed depending on whether or not the hiragana character can be recognized from the estimated hiragana character region. The ninth aspect is another specific example of the determination method by the third determination means.

  According to a tenth aspect of the present invention, in the vehicle number reading device according to any one of the first to eighth aspects, the recognition missing character candidate area obtaining unit has a number of characters recognized by the character recognition processing unit. In the case of three, based on the center interval on the image of the two lower digits of the three characters and the center interval on the image of the two upper digits of the three characters, Whether the three characters correspond to the fifth pattern which is the first digit, second digit and third digit character of the serial number, whether the three characters are the first digit, second digit of the serial number Whether or not the sixth pattern is a digit and fourth digit character, and whether the three characters are the seventh pattern that is the first digit, third digit and fourth digit character of the serial number And whether the three characters are the second digit of the serial number, the second Whether corresponds to eighth pattern of a digit and the fourth digit of the character, a fourth determination means for determining is intended to include.

  The tenth aspect is a specific example of a method for acquiring a recognition-missing character candidate area when the number of characters recognized by the character recognition processing unit is three. In the aspect of 8, it is not limited to such a method. However, in the tenth aspect, the fourth determination means includes the center interval on the image of the two lower digits of the three characters and the two upper digits of the three characters. Since it is determined which of the fifth to eighth patterns corresponds to the center interval on the image, the ratio of both the center intervals is used to image the license plate. The influence of fluctuations in the elevation angle can be removed, and the accuracy of determining which of the fifth to eighth patterns is applicable. Therefore, it is preferable to employ the determination method as in the tenth aspect.

  According to an eleventh aspect of the present invention, in the vehicle number reading device according to any one of the first to tenth aspects, the license plate candidate area obtaining means preliminarily calculates the license plate area based on the captured image. And a means for preliminarily recognizing each character of the serial number based on an image of the license plate preliminary candidate area of the captured image. Character preliminary recognition processing means for performing processing, and when the serial number is recognized by the character preliminary recognition processing means, the license plate preliminary candidate area is made the license plate candidate area, and the character preliminary recognition processing When the serial number is not recognized by the means, the license plate preliminary candidate area is Means not members plate candidate region, is intended to include.

  According to the eleventh aspect, the character recognition process is performed on the license plate preliminary candidate area, and only the license plate preliminary candidate area recognized by one or more characters is set as the license plate candidate area. Therefore, since the reliability of the license plate candidate area is increased, the serial number can be recognized with higher accuracy.

  A vehicle number reading device according to a twelfth aspect of the present invention is the vehicle number reading device according to any one of the first to eleventh aspects, on the image of the serial number character recognized by the character recognition processing means and the character re-recognition processing means. And a means for recognizing a character other than the serial number from the area obtained by the area acquisition means.

  According to the twelfth aspect, the character area other than the serial number is obtained with reference to the position of the character of the serial number recognized by the character recognition processing means and the character re-recognition processing means on the image, and from this area Recognizes characters other than serial numbers. Therefore, since the character area other than the serial number is obtained based on the position of the character with the serial number recognized with high accuracy, the character area other than the serial number can be obtained with high accuracy, and consequently the serial number. Characters other than can be recognized with high accuracy.

  A vehicle number reading device according to a thirteenth aspect of the present invention is the vehicle number reading device according to any one of the first to eleventh aspects, wherein the arrangement of the characters of the serial number recognized by the character recognition processing means and the character re-recognition processing means. Consistency determining means for determining whether or not a pattern matches an appropriate arrangement pattern is provided.

  According to the thirteenth aspect, since the consistency determining unit is provided, the serial number recognized by the character recognition processing unit and the character re-recognition processing unit is determined to be matched by the consistency determining unit. By using only as the recognition result of the serial number, the serial number can be recognized with higher accuracy.

  The vehicle number reading device according to the fourteenth aspect of the present invention is the serial number which is recognized by the character recognition processing means and the character re-recognition processing means in the thirteenth aspect and is determined to be matched by the consistency determination means. An area acquisition means for obtaining a character area other than the serial number based on the position of the character on the image, and a means for recognizing a character other than the serial number from the area obtained by the area acquisition means. Is.

  According to the fourteenth aspect, with reference to the position on the image of the character of the serial number recognized by the character recognition processing unit and the character re-recognition processing unit and determined to be matched by the consistency determination unit, The character area is acquired, and characters other than the serial number are recognized from this area. Therefore, since the character area other than the serial number is acquired based on the position of the character of the serial number recognized with higher accuracy, the character area other than the serial number can be obtained with higher accuracy. As a result, characters other than the serial number can be recognized with higher accuracy.

  According to the present invention, it is possible to provide a vehicle number reading device that can prevent a recognition failure more reliably and recognize a serial number with high accuracy.

  In addition, according to the present invention, it is possible to more reliably prevent a recognition omission and recognize a serial number with high accuracy, and to recognize a serial number with higher accuracy using distortion correction. Thus, it is possible to provide a vehicle number reading device that can perform processing efficiently.

  Hereinafter, a vehicle number reading device according to the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic block diagram showing a vehicle number reading device according to an embodiment of the present invention.

  As shown in FIG. 1, the vehicle number reading apparatus according to the present embodiment includes a camera 1 as an imaging unit, an A / D converter 2 for A / D converting an image signal obtained from the camera 1, and an A / D An image memory 3 for storing the D-converted image signal, and a processing unit 4 including a microcomputer or the like for processing the image stored in the image memory 3 to recognize the vehicle number and output the vehicle number. I have.

  Although not shown in the drawings, the camera 1 is installed so as to capture an image of a vehicle, particularly in the vicinity of a place where the license plate 5 passes, and captures an image including the image of the license plate 5.

  Here, a schematic plan view of the license plate 5 of the automobile is shown in FIG. As shown in FIG. 2, the license plate 5 has a classification number (vehicle type code) 5a and a land transportation branch office name 5b arranged on a horizontally long rectangular flat plate, a hiragana character 5c arranged on the left side, and represented in large letters. The serial number 5d is arranged in the center, and these constitute the vehicle number (registration number) as a whole. The serial number 5d is composed of 1 to 4 digits. When the serial number 5d consists of four digits (characters), as shown in FIG. 2, from the first digit character m1, the second digit character m2, the third digit character m3, and the fourth digit character m4. Become. If the serial number 5d is 3 digits or less, it is placed right-justified in order from the least significant digit, and the most significant digit side is “·” indicating that the digit of the digit does not exist, but the character position of each digit Is fixed regardless of the number of digits.

  FIG. 3 is a diagram showing the defined positional relationship between the letters m1 to m4 of each digit of the serial number 5d on the license plate 5. As shown in FIG. In the case of a license plate for an ordinary car, the center distance between the first digit character m1 and the second digit character m2 and the center interval between the third digit character m3 and the fourth digit character m4 are both 55 mm. The center distance between the second digit character m2 and the third digit character m3 is 85 mm. In the case of license plates for large vehicles, the center distance between the first digit character m1 and the second digit character m2 and the center interval between the third digit character m3 and the fourth digit character m4 are both 80 mm. The center distance between the second digit character m2 and the third digit character m3 is 112.5 mm. Note that the character widths of the characters m1 to m4 are all 40 mm if the character is other than “1”, and narrower than 40 mm if the character is “1”.

  Next, the operation of the processing unit 4 will be described with reference to FIGS. 4 to 7 are schematic flowcharts showing an example of the operation of the processing unit 4.

  When the image including the image of the license plate 5 is picked up by the camera and the image signal is stored in the image memory 3, the processing unit 4 starts an operation described below.

  First, the processing unit 4 smoothes a multi-valued image captured by the camera (hereinafter referred to as “original image”) by a known method (step S1).

  Next, the processing unit 4 performs a process of extracting an edge component by a well-known method for the image smoothed in step S1 (step S2), and binarizes this image (step S3). An example of the binarized image is shown in FIG.

  Next, the processing unit 4 performs a noise removal process for removing a minute region on the binarized image (step S4). Thereafter, with respect to the image after the noise removal processing, the expansion processing for expanding the pixels corresponding to the edge components (white pixels in the example shown in FIG. 8) and the connected state of the pixels connected by the expansion processing are maintained. The contraction process of contracting the connection area expanded by the expansion process is performed (step S5). An example of the image after the expansion / contraction process is shown in FIG.

  Next, the processing unit 4 determines, for example, the size of the circumscribed rectangle of each connected region, etc., for the image after the expansion / contraction process, so that a rectangular license plate preliminary candidate that becomes a preliminary candidate for the license plate region is used. The area is extracted as coordinates indicating the area (step S6). There may be no license plate candidate region, or there may be only one license plate region or a plurality of license plate candidate regions.

  Next, the processing unit 4 determines whether there are any license plate preliminary candidate regions extracted in step S6 that have not been processed in step S8 and subsequent steps (step S7). If no license plate preliminary candidate area has been extracted in step S6, or if one or more license plate preliminary candidate areas have been extracted in step S6, but the process in step S8 and subsequent steps has already been completed for that area The unit 4 ends the reading process of the license plate 5. On the other hand, if it is determined that there is an unprocessed license plate preliminary candidate area, the process proceeds to step S8.

  In step S8, the processing unit 4 selects one unprocessed license plate preliminary candidate region from the license plate preliminary candidate regions extracted in step S6.

  Next, the processing unit 4 binarizes the image of the extracted area selected in step S8 in the original image (step S9). Next, the processing unit 4 performs a labeling process on the image binarized in step S9 to obtain a connected region (step S10).

  After that, the processing unit 4 narrows down only those that satisfy a predetermined narrowing-down condition from the circumscribed rectangular areas of the connected areas obtained in step S10 as the areas of the respective characters with serial numbers (step S11). This narrowing-down process includes a series of characters such as a character area of classification number 5a, a character area of land transportation branch name 5b, a character area of hiragana character 5c, and a circumscribed rectangular area of a connected area that may be generated by other noises. This is a process of excluding those other than each character area of the number. As the narrowing-down conditions, for example, the horizontal length of the rectangular area is in a predetermined range, the vertical length of the rectangular area is in a predetermined range, and the area of the rectangular area is in a predetermined range, One or a combination of two or more of them can be mentioned.

  Next, the processing unit 4 performs a well-known character recognition (numeral recognition) process as a serial number preliminary recognition process for each region narrowed down in step S11 (step S12).

  Next, the processing unit 4 determines the number of characters recognized in step S12 (step S13). If the number of recognized characters is 0, the process returns to step S7. If the number of recognized characters is 1, the process proceeds to step S23. If the number of characters is 2 to 4, the process proceeds to step S14.

  In step S23, the processing unit 4 regards the character recognized in step S13 as the first digit character, and whether or not the recognized character is 1, the character width on the image, and the character position on the image. Based on the above, the license plate area on the image binarized in step S9 is determined. Next, based on the license plate area determined in step S23, the processing unit 4 performs other areas on the image binarized in step S9 (area of classification number 5a, area of land transportation branch office name 5b, And the area of the hiragana character 5c) (step S24). Thereafter, the processing unit 4 performs processing on each region determined in step S9 on the image binarized in step S9, and recognizes the classification number 5a, the land transportation branch office name 5b, and the hiragana character 5c (steps). S25), the license plate reading process is terminated. If the number of characters recognized in step S12 is 1, the processing unit 4 considers that a serial number consisting of only one digit with the number recognized in step S12 as the first digit has been read.

  In step S14, the processing unit 4 calculates the inclination θ of the character string (numeric string) recognized in step S12. A specific example of the calculation method of the inclination θ is schematically shown in FIG. For example, as shown in FIG. 10, the inclination θ is determined by the center of the rectangular area of the most significant character (leftmost character) and the least significant character (rightmost character) among the characters recognized in step S12. ) Can be calculated as the slope of a straight line connecting the center of the rectangular area. After step S14, the process proceeds to step S15.

  In step S15, the processing unit 4 determines the license plate preliminary candidate area selected in step S8 as a license plate candidate area.

  Next, the processing unit 4 performs well-known rotational distortion correction on the original image based on the inclination calculated in step S14 so that the inclination becomes 0 ° (step S16). At this time, in the present embodiment, not only the license plate candidate area determined in step S15 in the original image but also a recognition missing character candidate area in steps S37 to S42 and S53 to S56 described later. An area in the original image corresponding to a characteristic area is also subject to rotational distortion correction. Specifically, for example, an area obtained by extending the license plate candidate area determined in step S15 by two characters in the width direction to the upper digit side (left side) and the lower digit side (right side) (however, rotation correction is performed). The rotation correction may be performed for a region that is further enlarged in consideration. It should be noted that the license plate candidate area determined in step S15 is corrected to an area considering the rotation correction simultaneously with the rotation correction in step S16.

  Next, the processing unit 4 binarizes the image after the rotational distortion correction in step S16 (step S17). The license plate 5 includes a license plate whose ground color is white and letters are green, and a license plate whose ground color is green and letters are white. Then, the binary relationship is made so that the pixel corresponding to the character is a white pixel (of course, it may be a black pixel) for any license plate by determining the magnitude relationship between the gradation value of the character and the ground color. Process. FIG. 11 shows an example of a corrected license plate candidate area image (hereinafter simply referred to as “corrected license plate candidate area image”) of the images binarized in step S17.

  Thereafter, the processing unit 4 cuts out the lower region, which is the region of the hiragana characters 5c and the serial number 5d, from the image of the license plate candidate region after the correction (step S18). The lower region can be cut out as shown in FIGS. 12 and 13, for example. That is, a projection value obtained by projecting the image of the corrected license plate candidate area onto the Y axis (vertical axis in FIGS. 11 to 13) is obtained, and the position of the valley of the projection value is determined as a division position in the vicinity that is predicted in advance. Ask. FIG. 12 shows the Y-axis projection value and division position when the corrected image of the license plate candidate area is the one shown in FIG. Then, an area below this division position (area below the white line in FIG. 13) is cut out as the lower area.

  Next, the processing unit 4 performs a labeling process on the lower region cut out in step S18 to obtain a connected region (step S19).

  Next, the processing unit 4 narrows down only those that satisfy a predetermined narrowing-down condition from the circumscribed rectangular area of each connected area obtained in step S19 as the area of each serial number character (step S20). This narrowing-down process is a process of excluding the character areas of the hiragana characters 5c and the circumscribed rectangular areas of the connected areas that may be caused by other noises, etc., except for the character areas with serial numbers. As the narrowing-down conditions, for example, the horizontal length of the rectangular area is in a predetermined range, the vertical length of the rectangular area is in a predetermined range, and the area of the rectangular area is in a predetermined range, One or a combination of two or more of them can be mentioned.

  Note that, as in the present embodiment, the lower region is cut out in step S18, a connected region is obtained for the lower region in step S19, and the connected region is narrowed down in step S20. This is preferable for accurate extraction. However, in the present invention, instead of performing step S18, a labeling process is performed on the image of the corrected license plate candidate area in step S19 to obtain a connected area, and this connected area is narrowed down in step S20. Also good.

  After step S20, the processing unit 4 performs a recognition process on the circumscribed rectangular area image of each connected area that has been narrowed down in step S20 among the binarized images in step S17. Each character is recognized (step S21).

  Next, the processing unit 4 determines the number of characters recognized in step S21 (step S22). If the number of recognized characters is 2, the processing unit 4 proceeds to step S31 in FIG. 5, and if the number of recognized characters is 3, in FIG. If the number of recognized characters is 1 or 4, the process proceeds to step S63 in FIG.

  In step S31 in FIG. 5, the processing unit 4 determines whether or not a character other than “1” is recognized in step S21. If a character other than “1” is recognized, the processing unit 4 proceeds to step S32. If only “1” is recognized, the process proceeds to step S33.

  In step S32, the processing unit 4 determines the center interval L between the two characters recognized in step S21 and the character width of the two characters other than “1” (both the two characters are “1”). If it is a character other than, the average value of the character width on the image may be used.) The ratio L / W with W is calculated, and the process proceeds to step S34.

  In step S33, the processing unit 4 displays the center distance L between the two characters recognized in step S21 on the image and the character width of one of the two characters (the character width of the two characters). Average value may be used.) Ratio L / (α · W) of product α · W with W and coefficient (coefficient for converting width of character “1” into character width other than “1”) α And the process proceeds to step S34.

  Based on the ratio L / W calculated in step S32 or the ratio L / (α · W) calculated in step S33, the two characters recognized in step S21 are characters in any digit of the serial number. From this point of view, it can be classified into first to fourth patterns. This point will be described with reference to FIGS.

  In the first pattern, as shown in FIG. 14A, the two recognized characters are the first and second digit characters m1 and m2, or the third and fourth digit characters m3 and m4. This is the case. In this case, the prescribed L / W and L / (α · W) are 1.375 (= 55/40) for a license plate of a normal car and 2.000 (= 80/40) for a license plate of a large car. )

  In the second pattern, as shown in FIG. 14B, the two recognized characters are the second and third digit characters m2 and m3. In this case, the prescribed L / W and L / (α · W) are 2.125 (= 85/40) for a license plate of a normal car and 2.813 (= 112.5) for a license plate of a large car. / 40).

  In the third pattern, as shown in FIG. 14C, the two recognized characters are the first and third digit characters m1 and m3, or the second and fourth digit characters m2 and m4. This is the case. In this case, the prescribed L / W and L / (α · W) are 3.500 (= (55 + 85) / 40) for a license plate of a normal car and 4.813 (= ( 80 + 112.5) / 40).

  In the fourth pattern, as shown in FIG. 14D, the two recognized characters are the first and fourth digit characters m1 and m4. In this case, the prescribed L / W and L / (α · W) are 5.625 (= (85 + 55 + 85) / 40) for a license plate of a normal car and 7.625 (= ( 112.5 + 80 + 112.5) / 40).

  The specified L / W and L / (α · W) theoretically agree with the L / W and L / (α · W) on the image with high accuracy even if the shooting angle changes.

  Therefore, a predetermined intermediate value between 2.000 and 2.125 is a threshold value between the first pattern and the second pattern, and a predetermined intermediate value between 2.813 and 3.500. Is a threshold value between the second pattern and the third pattern, and a predetermined intermediate value between 4.813 and 5.625 is a threshold value between the third pattern and the fourth pattern. By comparing L / W calculated in step S32 or L / (α · W) calculated in step S33 with these threshold values, any one of the first to fourth patterns can be obtained. Whether it is applicable can be determined. In the present embodiment, the processing unit 4 makes this determination in step S34. In the case where only an integer ratio is handled due to restrictions of a program or the like, the values of L / W and L / (α · W) may be obtained by bit shift or integer multiplication.

  The determination in step S34 may always correspond to one of the first to fourth patterns as described above, but depending on the value of L / W or L / (α · W), It may be determined so as to obtain a determination result that does not correspond to any of the first to fourth patterns. In this case, for example, numerical ranges are defined in the vicinity of the above-described values 1.375, 2.000, 2.125, 2.813, 3.500, 4.813, 5.625, 7.625, respectively. When L / W or L / (α · W) falls within one of these numerical ranges, it is determined that the corresponding pattern is satisfied, and L / W or L / (α · W) falls within the numerical range. If it does not fall into any of the above, it may be determined that none of the first to fourth patterns is applicable. At this time, if it is determined that none of the first to fourth patterns is applicable, the process may return to step S7.

  If it is determined in step S34 that it corresponds to the first pattern, the process proceeds to step S35. If it is determined that it corresponds to the second pattern, the process proceeds to step S39. If it is determined to correspond to the third pattern, step S36 is performed. If it is determined that the pattern corresponds to the fourth pattern, the process proceeds to step S42.

  In step S35, the processing unit 4 determines whether the two characters recognized in step S21 are the first and second digit characters m1 and m2, or the third and fourth digit characters m3 and m4. Is determined. If it is determined that the characters are m1 and m2 of the first digit and the second digit, the process proceeds to step S37, while the process proceeds to step S38 if the characters are m3 and m4 of the third digit.

  A first specific example of the determination in step S35 will be described. In the first specific example, first, the processing unit 4 assumes that the two characters recognized in step S21 are the first digit and the second digit characters m1 and m2, and the recognized character is 1. The area of the hiragana character 5c is obtained based on whether or not it exists, the character width on the image, and the character position on the image. Next, the processing unit 4 performs a hiragana character recognition process on the obtained region of the hiragana character 5c in the image binarized in step S17. Then, when the hiragana character 5c is recognized from the area, the processing unit 4 determines that the two characters recognized in step S21 are the first digit and the second digit characters m1 and m2. If the hiragana character 5c is not recognized from the area, it is determined that the two characters recognized in step S21 are the third and fourth digit characters m3 and m4. In the example described here, the region of the hiragana character 5c is estimated on the assumption that the two characters recognized in step S21 are the first and second digit characters m1 and m2. Assuming that the two characters recognized in step 3 are the third and fourth digit characters m3 and m4, the same determination can be made by estimating the area of the hiragana character 5c.

  A second specific example of the determination in step S35 will be described. In the second specific example, the processing unit 4 determines whether or not a part of the region of the classification number 5a exists above the lower-order character (right character) of the two characters recognized in step S21. By determining whether or not, the determination in step S35 is performed. As shown in FIG. 2, the classification number 5a does not exist above the first digit character, but a part of the land transportation branch office 5b exists above the third digit character. More specifically, for example, assuming that the processing unit 4 is an area above the lower digit character of two recognized characters (for example, the lower digit character is “5” in FIG. 15, A rectangular area surrounded by a white broken line in FIG. 15) is obtained with reference to the position “5”, and the ratio of the number of pixels corresponding to characters in the area of the upper area is calculated. Then, when the ratio is less than the predetermined value, the processing unit 4 determines that the area of the classification number 5a does not exist above the character of the lower digit, and thus the two characters recognized in step S21 are It is determined that the characters are m1 and m2 of the first digit and the second digit. On the other hand, when the ratio is equal to or greater than the predetermined value, it is determined that a part of the area of the land transportation branch office 5b exists above the lower digit character, and thus the two characters recognized in step S21 are the third digit. And the fourth digit characters m3 and m4.

  In step S36, the processing unit 4 determines whether the two characters recognized in step S21 are the first and third digit characters m1 and m3 or the second and fourth digit characters m2 and m4. Is determined. If it is determined that the characters are m1 and m3 of the first digit and the third digit, the process proceeds to step S40, whereas the process proceeds to step S41 if the characters are m2 and m4 of the second digit.

  A first specific example of the determination in step S36 will be described. In this first specific example, first, the processing unit 4 assumes that the two characters recognized in step S21 are the first digit and the third digit characters m1 and m3. The area of the hiragana character 5c is obtained based on whether or not it exists, the character width on the image, and the character position on the image. Next, the processing unit 4 performs a hiragana character recognition process on the obtained region of the hiragana character 5c in the image binarized in step S17. Then, when the hiragana character 5c is recognized from the area, the processing unit 4 determines that the two characters recognized in step S21 are the first digit and the third digit characters m1 and m3, If the hiragana character 5c is not recognized from the area, it is determined that the two characters recognized in step S21 are the second and fourth digit characters m2 and m4. In the example described here, the region of the hiragana character 5c is estimated on the assumption that the two characters recognized in step S21 are the first and third digit characters m1 and m3. Assuming that the two characters recognized in step 2 are the characters m2 and m4 of the second digit and the fourth digit, the same determination can be made even if the region of the hiragana character 5c is estimated.

  A second specific example of the determination in step S36 will be described. In the second specific example, the processing unit 4 determines whether or not a part of the region of the classification number 5a exists above the lower-order character (right character) of the two characters recognized in step S21. By determining whether or not, the determination of step S36 is performed. As shown in FIG. 2, the classification number 5a does not exist above the first digit character, but a part of the classification number 5a exists above the second digit character. More specifically, for example, assuming that the processing unit 4 is an area above the lower digit character of two recognized characters (for example, the lower digit character is “5” in FIG. 15, A rectangular area surrounded by a white broken line in FIG. 15) is obtained with reference to the position “5”, and the ratio of the number of pixels corresponding to characters in the area of the upper area is calculated. Then, when the ratio is less than the predetermined value, the processing unit 4 determines that the area of the classification number 5a does not exist above the character of the lower digit, and thus the two characters recognized in step S21 are It is determined that the characters are m1 and m3 of the first digit and the third digit. On the other hand, if the ratio is equal to or greater than the predetermined value, it is determined that a part of the region of the classification number 5a exists above the lower-digit character, and thus the two characters recognized in step S21 are the second digit. And it determines with it being the characters m2 and m4 of the 4th digit.

  In step S37, the processing unit 4 assumes that the two characters recognized in step S21 are the first and second digit characters m1 and m2, and determines the character position on the image and the character width on the image. Based on whether or not the character is recognized as “1”, the rectangular regions of the third and fourth digit characters m3 and m4 are estimated, respectively, and these on the image binarized in step S17. Are acquired as recognition missing character candidate regions. Thereafter, the process proceeds to step S61 in FIG.

  In step S38, the processing unit 4 assumes that the two characters recognized in step S21 are the first and second digit characters m3 and m4, the character position on the image, the character width on the image, Based on whether or not the character is recognized as “1”, the rectangular areas of the first-digit and second-digit characters m1 and m2 are estimated, and these on the image binarized in step S17. Are acquired as recognition missing character candidate regions. Thereafter, the process proceeds to step S61 in FIG.

  In step S39, the processing unit 4 assumes that the two characters recognized in step S21 are the second and third digit characters m2 and m3, the character position on the image, the character width on the image, Based on whether or not the character is recognized as “1”, the rectangular areas of the first-digit and fourth-digit characters m1 and m4 are estimated, and these on the image binarized in step S17. Are acquired as recognition missing character candidate regions. Thereafter, the process proceeds to step S61 in FIG.

  In step S40, the processing unit 4 assumes that the two characters recognized in step S21 are the first and third digit characters m1 and m3, the character position on the image, the character width on the image, Based on whether the character is recognized as “1” or not, the rectangular areas of the characters m2 and m4 of the second digit and the fourth digit are respectively estimated, and these on the image binarized in step S17. Are acquired as recognition missing character candidate regions. Thereafter, the process proceeds to step S61 in FIG.

  In step S41, the processing unit 4 assumes that the two characters recognized in step S21 are the second and fourth digit characters m2 and m4, the character position on the image, the character width on the image, Based on whether or not the character is recognized as “1”, the rectangular areas of the first-digit and third-digit characters m1 and m3 are estimated, and these on the image binarized in step S17. Are acquired as recognition missing character candidate regions. Thereafter, the process proceeds to step S61 in FIG.

  In step S42, the processing unit 4 assumes that the two characters recognized in step S21 are the first and fourth digit characters m1 and m4, the character position on the image, the character width on the image, Based on whether or not the character is recognized as “1”, the rectangular areas of the second and third digit characters m2 and m3 are estimated, respectively, and these on the image binarized in step S17. Are acquired as recognition missing character candidate regions. Thereafter, the process proceeds to step S61 in FIG.

  In step S51 in FIG. 6, the processing unit 4 determines the center interval L1 on the image of the two lower-order characters of the three characters recognized in step S21 and the upper-order 2 of the three characters. A ratio L2 / L1 with the center interval L2 on the image of one character is calculated.

  Next, based on L2 / L1 calculated in step S51, the processing unit 4 determines whether the three characters recognized in step S21 are the characters of which digit of the serial number. It can be classified into an eighth pattern. This point will be described with reference to FIGS.

  In the fifth pattern, as shown in FIG. 16A, the three recognized characters are the first-digit, second-digit, and third-digit characters m1, m2, and m3. In this case, the specified L2 / L1 is 1.545 (= 85/55) for a license plate of a normal vehicle and 1.406 (= 112.5 / 80) for a license plate of a large vehicle.

  In the sixth pattern, as shown in FIG. 16B, the three recognized characters are the first-digit, second-digit, and fourth-digit characters m1, m2, and m4. In this case, L2 / L1 in the regulation is 2.545 (= (55 + 85) / 55) for a license plate of a normal car, and 2.406 (= (80 + 112.5) // for a license plate of a large car. 80).

  In the seventh pattern, as shown in FIG. 16C, the three recognized characters are the first-digit, third-digit, and fourth-digit characters m1, m3, and m4. In this case, the specified L2 / L1 is 0.393 (= 55 / (55 + 85) for a license plate of a normal car and 0.416 (= 80 / (80 + 112.5) for a license plate of a large car.

  In the eighth pattern, as shown in FIG. 16D, the three recognized characters are the second, third, and fourth digit characters m2, m3, and m4. In this case, the prescribed L2 / L1 is 0.647 (= 55/85) for a license plate of a normal car and 0.711 (= 80 / 112.5) for a license plate of a large car.

  Theoretically defined L2 / L1 is theoretically consistent with L2 / L1 on the image even if the shooting angle changes.

  Therefore, a predetermined intermediate value between 0.416 and 0.647 is a threshold value between the seventh pattern and the eighth pattern, and a predetermined intermediate value between 0.711 and 1.406. Is a threshold value between the eighth pattern and the fifth pattern, and a predetermined intermediate value between 1.545 and 2.406 is a threshold value between the fifth pattern and the sixth pattern. By comparing L2 / L1 calculated in step S51 with these threshold values, it can be determined which of the fifth to eighth patterns corresponds. In the present embodiment, the processing unit 4 makes this determination in step S52. In the case where only an integer ratio is handled due to restrictions such as a program, the value of L2 / L1 may be obtained by bit shift or integer multiplication. Needless to say, L1 / L2 may be used instead of L2 / L1.

  The determination in step S52 may always correspond to any of the fifth to eighth patterns as described above, but depending on the value of L2 / L1, any of the fifth to eighth patterns may be determined. It may be determined so as to obtain a determination result that does not correspond to the above. In this case, for example, numerical ranges are defined in the vicinity of the above-described values 0.393, 0.416, 0.647, 0.711, 1.406, 1.545, 2.406, 2.545, respectively. When L2 / L1 falls into any of those numerical ranges, it is determined that the corresponding pattern is satisfied, and when L2 / L1 does not fall within any of these numerical ranges, the fifth to eighth patterns It may be determined that none of the above applies. At this time, if it is determined that none of the fifth to eighth patterns is applicable, the process may return to step S7.

  If it is determined in step S52 that it corresponds to the fifth pattern, the process proceeds to step S53. If it is determined that it corresponds to the sixth pattern, the process proceeds to step S54. If it is determined to correspond to the seventh pattern, step S55. If it is determined that the pattern corresponds to the eighth pattern, the process proceeds to step S56.

  In step S53, the processing unit 4 determines that the three characters recognized in step S21 are the first digit, the second digit, and the third digit characters m1, m2, and m3, Based on the character width on the image and whether or not the character is recognized as “1”, the rectangular region of the fourth-digit character m4 is estimated, and this region on the image binarized in step S17. Is acquired as a recognition missing character candidate region. Thereafter, the process proceeds to step S61 in FIG.

  In step S54, the processing unit 4 assumes that the three characters recognized in step S21 are the first digit, second digit, and fourth digit characters m1, m2, and m4, and character positions on the image and Based on the character width on the image and whether or not the character is recognized as “1”, a rectangular region of the third-digit character m3 is estimated, and this region on the image binarized in step S17. Is acquired as a recognition missing character candidate region. Thereafter, the process proceeds to step S61 in FIG.

  In step S55, the processing unit 4 assumes that the three characters recognized in step S21 are the first digit, third digit, and fourth digit characters m1, m3, and m4, and character positions on the image and Based on the character width on the image and whether or not the character is recognized as “1”, the rectangular region of the second-digit character m2 is estimated, and this region on the image binarized in step S17. Is acquired as a recognition missing character candidate region. Thereafter, the process proceeds to step S61 in FIG.

  In step S56, the processing unit 4 assumes that the three characters recognized in step S21 are the second digit, third digit, and fourth digit characters m2, m3, and m4, and character positions on the image and Based on the character width on the image and whether or not the character is recognized as “1”, the rectangular region of the first digit character m1 is estimated, and this region on the image binarized in step S17. Is acquired as a recognition missing character candidate region. Thereafter, the process proceeds to step S61 in FIG.

  In step S61 in FIG. 7, the processing unit 4 performs serial number character recognition processing on the recognition missing character candidate areas acquired in steps S37 to S42 and S53 to S56 on the image binarized in step S17. (Re-recognition processing) is performed, and the process proceeds to step S63. In step S61, a recognition missing character candidate region that protrudes from the license plate candidate region after being rotationally corrected determined in step S15 (for example, the third digit and the fourth digit acquired as the recognition missing character candidate region in step S37). The areas of the digit characters m3 and m4 protrude from the corrected license plate candidate area to the left, and the areas of the first and second digit characters m1 and m2 acquired as the recognition missing character candidate areas in step S38 are: Also, the character recognition process can be performed as it is without performing rotation correction again. In the present embodiment, this is not limited to the license plate candidate area in the rotation correction target area in step S16, but may be regarded as a recognition missing character candidate area in steps S37 to S42 and S53 to S56. This is because a region in the original image corresponding to a certain region is also included. As described above, according to the present embodiment, it is not necessary to perform double rotational distortion correction, so that the processing can be performed efficiently.

  Note that the re-recognition process in step S61 does not always re-recognize the characters with serial numbers for all of the recognition-missing character candidate areas.

  After step S61, the processing unit 4 determines whether or not the arrangement pattern of the character recognized in step S21 and the character re-recognized in step S62 matches an appropriate arrangement pattern (step S62). Since serial numbers are arranged in order from the first digit, the most significant digit (the most significant digit can be any of the first digit, the second digit, the third digit, and the fourth digit). The lower digit of the number cannot be blank. Therefore, specifically, for example, if the lower-order digit character is not recognized with respect to the highest-order character among the characters recognized in step S21 and the character re-recognized in step S62, the On the other hand, if it is determined that the character does not match the correct arrangement pattern, if all the characters in the lower digits are recognized, it may be determined that the character matches the appropriate arrangement pattern. If NO is determined in step S62, the process returns to step S7. If YES is determined in step S62, the process proceeds to step S63.

  If it is determined in step S22 in FIG. 4 that the number of recognized characters is 1 and the process proceeds to step S63, in step S63, the processing unit 4 regards the character recognized in step S21 as the first digit character. Then, based on whether or not the recognized character is 1, the character width on the image and the character position on the image, the license plate area on the image binarized in step S17 is determined.

  When it is determined in step S22 in FIG. 4 that the number of recognized characters is 4, and the process proceeds to step S63, in step S63, the processing unit 4 determines whether one of the characters recognized in step S21 is 1 and its image. Based on the upper character width and the character position on the image, the license plate area on the image binarized in step S17 is determined.

  If YES is determined in step S62 and the process proceeds to step S63, in step S63, the processing unit 4 determines whether one of the character recognized in step S21 and the character re-recognized in step S62 is 1. Based on the character width on the image and the character position on the image, the license plate area on the image binarized in step S17 is determined. In this case, the determined license plate candidate region always includes the recognition missing character candidate region acquired in steps S37 to S42 and S53 to S56. Therefore, the determined license plate candidate area may be an extension of the corrected license plate candidate area described above. However, even in that case, it is not necessary to perform the rotation correction again for the expanded area, and the coordinates need only be corrected. In the present embodiment, this is not limited to the license plate candidate area in the rotation correction target area in step S16, but may be regarded as a recognition missing character candidate area in steps S37 to S42 and S53 to S56. This is because a region in the original image corresponding to a certain region is also included. As described above, according to the present embodiment, it is not necessary to perform double rotational distortion correction, so that the processing can be performed efficiently.

  After step S63, based on the license plate area determined in step S62, the processing unit 4 performs other areas on the image binarized in step S17 (area of classification number 5a, land transportation branch name 5b). A region and a region of the hiragana character 5c) are determined (step S64). Thereafter, the processing unit 4 performs processing on each region determined in step 63 on the image binarized in step S17, and recognizes the classification number 5a, the land transportation branch office name 5b, and the hiragana character 5c (steps). S65), the license plate reading process is terminated.

  According to the present embodiment, after the recognition processing of each character of the serial number 5d is performed based on the image of the license plate candidate area in step S21, when the number of recognized characters is two or three, steps S37 to S37 are performed. A recognition missing character candidate area is acquired in S42 and S53 to S56, and a character having a serial number is re-recognized based on the image of the leakage character candidate area in step S61. As described above, according to the present embodiment, character recognition itself is skillfully used to acquire a recognition missing character candidate region for a serial number, and re-recognition is performed on the acquired recognition missing character. Therefore, according to the present embodiment, a vehicle number reading device using the clipping device disclosed in Patent Document 1 that cuts out a character area completely separated from character recognition as a pre-processing of character recognition with respect to serial numbers. Compared to the above, the serial number can be recognized with higher accuracy by preventing the recognition omission more reliably.

  Further, according to the present embodiment, since each character of the serial number is recognized in step S21 based on the image corrected in step S16, the serial number is recognized with higher accuracy using distortion correction. Can do. However, in the present invention, it is not always necessary to perform distortion correction. In this embodiment, rotational distortion correction is performed as distortion correction. However, other various distortion corrections may be performed. For example, the distortion correction disclosed in Patent Document 2 is employed. You can also.

  Furthermore, according to the present embodiment, the region to be corrected for distortion correction in step S16 corresponds to a region that may be considered a recognition-missing character candidate region in steps S37 to S42 and S53 to S56. Since the region is included, as described above, there is also an advantage that the processing can be efficiently performed without the need for double distortion correction.

  In this embodiment, Steps S1 to S15 correspond to license plate candidate area obtaining means for obtaining a license plate candidate area based on the original image. However, in Step S6, a license plate preliminary candidate area is obtained. In step S12, the serial number is preliminarily recognized based on the image of the license plate preliminary candidate region, and the number of recognized characters is determined in step S13. If the number of recognized characters is 0, the license plate preliminary candidate region is determined as the license plate candidate. Not an area. Therefore, since the reliability of the license plate candidate area is increased, the serial number can be recognized with higher accuracy. However, in the present invention, the license plate candidate area may be acquired by using various conventionally known methods, and it is not always necessary to obtain the license plate candidate area using character recognition.

  Furthermore, according to the present embodiment, when the number of characters recognized in step S21 is two or three, step S63 is based on the position of the serial number character recognized in steps S21 and S61 on the image. After step S64, an area of characters other than the serial number is acquired, and in step S65, characters other than the serial number are recognized. Therefore, since the character area other than the serial number is obtained based on the position of the character with the serial number recognized with high accuracy, the character area other than the serial number can be obtained with high accuracy, and consequently the serial number. Characters other than can be recognized with high accuracy. In the present embodiment, after the license plate area is once determined in step S63, each other area is determined in step S64 using this license plate candidate area. However, step S63 is removed, It is also possible to obtain each other area directly from the information used for determining the license plate area in S63 without obtaining the license plate area.

  Furthermore, according to the present embodiment, since the consistency of the arrangement pattern of the character with the serial number recognized in step S62 is verified, the serial number can be recognized with higher accuracy. However, in the present invention, it is not always necessary to verify the consistency in step S62.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this embodiment.

It is a schematic block diagram which shows the vehicle number reading apparatus by one embodiment of this invention. It is a schematic plan view which shows the number plate of a motor vehicle. It is a figure which shows the positional relationship in which the character of each digit of the serial number in the license plate 5 is prescribed | regulated. It is a schematic flowchart which shows a part of operation | movement of the process part in FIG. It is a schematic flowchart which shows another part of operation | movement of the process part in FIG. 6 is a schematic flowchart showing still another part of the operation of the processing unit in FIG. 1. 6 is a schematic flowchart showing still another part of the operation of the processing unit in FIG. 1. It is a figure which shows the example of the image after step S3. It is a figure which shows the example of the image after step S5. It is a figure which shows typically the specific example of the calculation method of inclination. It is a figure which shows the example of the image of the number plate candidate area | region after correction | amendment. It is explanatory drawing which shows the method of cutting out a lower area | region from the number plate candidate area | region after correction | amendment. It is another explanatory drawing which shows the method of cutting out a lower area | region from the number plate candidate area | region after correction | amendment. It is a figure which shows the pattern in case the number of recognition characters of a serial number is two. It is a figure which shows the example of the area | region of the upper part with respect to a character. It is a figure which shows the pattern in case the number of recognition characters of a serial number is three.

Explanation of symbols

1 Camera 2 A / D Converter 3 Image Memory 4 Processing Unit 5 License Plate 5a Serial Number

Claims (14)

A vehicle number reading device that reads a vehicle number by processing a captured image including an image of a vehicle license plate,
License plate candidate area acquisition means for obtaining a license plate candidate area that is a candidate for a license plate area based on the captured image;
Character recognition processing means for performing recognition processing for recognizing each character of a serial number based on the image of the license plate candidate area in the captured image;
When the number of characters recognized by the character recognition processing means is two or three, it becomes a candidate for a character area that may have been recognized by the character recognition processing means in the serial number. A recognition missing character candidate area obtaining means for obtaining a recognition missing character candidate area;
When the number of characters recognized by the character recognition processing means is two or three, the characters of the serial number are determined based on the image of the recognition-recognized character candidate region in the captured image. Character re-recognition processing means for performing re-recognition processing for re-recognition;
A vehicle number reading device comprising: Distortion correction means for correcting distortion for an area including the license plate candidate area in the captured image,
The character recognition processing means performs a recognition process for recognizing each character of the serial number based on an image of an area corresponding to the license plate candidate area in the image corrected by the distortion correction means.
The vehicle number reading device according to claim 1. The region in the captured image to be corrected by the distortion correction unit includes a region corresponding to a region that is likely to be the recognition leaked character candidate region by the leaked character candidate region acquisition unit,
The character re-recognition processing means, when the number of characters recognized by the character recognition processing means is two or three, the recognition-rejected character candidate region in the image corrected by the distortion correction means Re-recognition processing for re-recognizing the characters of the serial number based on the image of
The vehicle number reading device according to claim 2.   The vehicle number reading device according to claim 2, wherein the distortion correction unit corrects rotational distortion.   When the number of characters recognized by the character recognition processing unit is two, the recognition-missing character candidate region acquisition unit (i), when the number of characters recognized by the character recognition processing unit is two, Based on the character width on the image of at least one character and whether or not the at least one character is recognized as “1”, the two characters are the first and second digit characters of the serial number. Or whether the two characters correspond to the first pattern which is the third digit and the fourth digit, or whether the two characters correspond to the second pattern which is the second digit and the third digit of the serial number Whether or not the two characters are the first and third digit characters of the serial number or the third pattern that is the second and fourth digit characters, and The two characters are the first and fourth digits of the serial number. First determination means for determining whether or not a certain fourth pattern is applicable; and (ii) the two characters when the first determination means determines that the first pattern is satisfied. Second determining means for determining whether the character is a first digit and second digit character of the serial number or a third digit and fourth digit character of the serial number; and (iii) the first The two characters are the first and third digits of the serial number, or the second digit and the fourth digit of the serial number. And (iv) means for obtaining the recognition-missing character candidate area based on the determination results by the first to third determination means. The vehicle according to any one of claims 1 to 4, wherein Bar reader.   The second determination means performs the determination based on whether or not there is a region corresponding to a part of the classification number above the character position of the lower digit of the two characters. The vehicle number reading device according to claim 5.   The second determination means performs the determination according to whether or not a hiragana character can be recognized from a hiragana character region estimated on the basis of the position of one of the two characters. 6. The vehicle number reading device according to claim 5, wherein:   The third determination means performs the determination based on whether or not there is an area corresponding to a part of the classification number above the position of the lower digit of the two characters as a reference. The vehicle number reading device according to any one of claims 5 to 7, wherein   The third determination means performs the determination according to whether or not a hiragana character can be recognized from a region of hiragana characters estimated with reference to the position of one of the two characters. The vehicle number reading device according to claim 5, wherein the vehicle number reading device is a vehicle number reading device.   When the number of characters recognized by the character recognition processing unit is three, the recognition-missing character candidate region acquisition unit and the center interval on the image of the two lower-order characters of the three characters and the Based on the center interval on the image of the two upper-order characters of the three characters, the three characters are the first digit, second digit, and third digit characters of the serial number. Whether the three characters correspond to a sixth pattern that is the first digit, second digit, and fourth digit of the serial number, and whether the three characters correspond to the sequence. Whether the first digit, the third digit and the fourth digit of the number correspond to the seventh pattern, and the three characters are the second digit, the third digit and the fourth digit of the serial number Including fourth determination means for determining whether or not the eighth pattern is a character of Vehicle number reading device according to any one of claims 1 to 8, symptom.   The license plate candidate area acquisition means acquires a license plate preliminary candidate area that is a preliminary candidate for a license plate area based on the captured image, and the license plate preliminary area of the captured image. Character preliminary recognition processing means for performing a recognition process for preliminary recognition of each character of the serial number based on the image of the candidate area, and one or more characters of the serial number are recognized by the character preliminary recognition processing means In this case, the license plate preliminary candidate area is used as the license plate candidate area, and the license plate preliminary candidate area is not regarded as the license plate candidate area when the serial number is not recognized by the character preliminary recognition processing means. 11. The vehicle according to claim 1, further comprising: means. Number reading device. An area acquisition means for obtaining an area of a character other than a serial number, based on a position on the image of a serial number character recognized by the character recognition processing means and the character re-recognition processing means;
The vehicle number reading device according to any one of claims 1 to 11, further comprising means for recognizing characters other than the serial number from the area obtained by the area acquisition means.   It is provided with consistency determining means for determining whether or not the arrangement pattern of the character with the serial number recognized by the character recognition processing means and the character re-recognition processing means matches an appropriate arrangement pattern. The vehicle number reading device according to any one of claims 1 to 11. A region for obtaining a character region other than a serial number based on a position on the image of a character with a serial number recognized by the character recognition processing unit and the character re-recognition processing unit and determined to be matched by the consistency determination unit. Acquisition means;
14. The vehicle number reading device according to claim 13, further comprising means for recognizing characters other than the serial number from the area obtained by the area acquisition means.

Images