JPH06215293A - Device for recognizing vehicle number - Google Patents

Abstract

PURPOSE:To provide a vehicle number recognizing device capable of cter excellently recognizing 'KANJI' (Chinese character) and cursive 'KANA' chara-(Japanese syllabary) in addition to numerals from the image of a number plate photographed by a usual television (TV) camera. CONSTITUTION:A plate extracting part 1 segments a plate area from an input image, a serial number extracting part 2 extracts a serial number from the plate area, a respective-code area coordinate calculating part 3 obtains the area coordinated of respective codes for a branch of the land transportation department, a vehicle sort, a purpose, etc., by using the coordinate string of the obtained serial number and respective codes are respectively recognized by a numeral recognizing part 5, a purpose code recognizing part 6 and a land transportation branch code recognizing part 7 in accordance with respective code areas. Each recognizing part recognizes a various density picture obtained by subtracting a background image from the input image as a featured value. Since the various density picture is recognized, 'KANJI' and cursive 'KANA' character can be effectively recognized by a usual TV camera even if brightness, dirt, a fold, etc., are generated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a vehicle number recognition device for automatically recognizing characters on a car license plate (hereinafter referred to as a plate), although various measurements are performed by image processing.

[0002]

2. Description of the Related Art Many types of plate character recognition have been developed so far, and recently, an all character recognition vehicle number recognition system capable of recognizing kanji and hiragana has been developed.

By the way, in kanji character recognition such as ordinary OCR, characters are often recognized with a resolution of about 32 × 32 pixels per character. When the license plate is photographed with such a resolution, the field of view becomes extremely narrow. Therefore, as in JP-A-62-121586, multiple television cameras are used to divide the image to recognize the Land Transport Office code, application code, etc. on the plate. I often do Alternatively, it is often the case that the image is processed by shooting with a special TV camera having a high resolution.

In a normal vehicle number recognition system, an input image is optimally binarized, and the binary image is processed to recognize the characters on the plate. The loss of information makes it impossible to recognize all the characters in the plate from the low resolution image. Therefore, high-resolution imaging is performed.

On the other hand, as a method of recognizing a face image using a grayscale image, for example, "identification of a man and a woman by a mosaic face having different resolutions using a neural network", Journal of Television Society, Vol. 46, No. 1 (1
992) is known. In this method, the resolution of a face image is reduced in a mosaic manner, and each pixel data of the dropped image is input as a feature amount to a neural network for recognition.

[0006]

In the method of photographing one vehicle with a plurality of television cameras described above, it is necessary to identify which television camera the plate is photographed, and image processing for recognition processing is performed. The device itself needs to have a configuration capable of inputting a plurality of video signals, which results in a very high cost. In addition, in a system using a high-resolution TV camera, not only is the camera expensive and the input system (A / D converter) for image processing has special specifications, but the number of target pixels for image processing becomes enormous. Therefore, there is a problem that an ultra-high-speed image processor is required.

Further, when the above-described mosaic image recognition method from a grayscale image is applied to character recognition of a plate, a change in the color of the plate, a change in the brightness of the plate itself, and a large change in the position of the recognized character are caused. However, there is a problem that the face image recognition processing method cannot be directly applied to the character recognition of the plate.

An object of the present invention is to provide a vehicle number recognizing device capable of satisfactorily recognizing all characters on a plate with one ordinary TV camera.

[0009]

In order to achieve the above object, according to the present invention, a license plate area extracting section for cutting out a license plate area from an input image taken in from a television camera and inputted in an image memory, and a cut out license plate. Using the serial number extraction unit that extracts the serial number from the area as a binary image, and the coordinates of each numeral that constitutes the extracted serial number, the area of the Land Transport Office code, vehicle type code, and usage code in the license plate Each code area coordinate calculation unit that determines the coordinates, recognizes the Land Transport Office code by using the feature amount of the grayscale image of each determined code area, and recognizes the vehicle type code by using the Land Transport Office code recognition unit and the feature amount Car number recognition from a character recognition unit that has a number recognition unit and a usage code recognition unit that recognizes a use and a code using the feature quantity One in which you configure the location.

[0010]

According to the present invention, the license plate is cut out from the input image taken in from the television camera, and the serial number of uppercase letters in the cut out license plate is extracted as a binary image. From the coordinate sequence of this extracted serial number, the Land Transport Office code in the license plate,
The areas of the vehicle type code and the usage code are roughly determined. More specifically, an image of only the characters in which the background density is removed is created in each of the determined rough regions, and the region occupied by the characters is determined in detail for each region using this character image. It From the image of the determined detailed character area, m × n grayscale feature amounts are extracted, the grayscale data is normalized to a real number of 0 to 1, and the normalized data is input to the neural network. By recognizing the Land Transport Office code, the vehicle type code, and the application code, it is possible to recognize the plate even from a low-resolution input image without being affected by the color, brightness change, or position change of the plate.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the construction of the plate recognition apparatus of the present invention. The process flow will be briefly described. A plate extraction unit 1 that extracts a plate from an input image, a serial number extraction unit 2 that extracts a serial number in the plate, and a Land Transport Office code from the coordinates of the extracted serial number. Each code area coordinate calculation unit 3 for calculating each coordinate position of the vehicle type code and the usage code, and the character recognition processing unit 4 for recognizing each code. Since the character recognition unit knows in advance what kind of character exists in each code area, the number recognition unit 5,
It is divided into a land transportation branch code (place name) recognition section 7 and a usage code (hiragana) recognition section 6. Before describing the plate recognition procedure, first, the character recognition procedure from the grayscale image will be described. Since the character position of the plate is standardized as shown in FIG. 2 (the unit of dimension is mm), for example, if the coordinates of the serial number are obtained, each character area can be relatively specified from the coordinates. However, in reality, the region cannot be accurately specified due to the bending or bending of the plate. Therefore, the character area is roughly set from the coordinates of the serial number, and an accurate character area is determined and recognized from the area. Therefore, the recognition processing for the case where the character area is roughly set as the area A will be described below with reference to FIG. The processing procedure is (1) background difference processing 8,
(2) Detailed coordinate extraction 9 of the character area, (3) grayscale feature amount extraction 10, and (4) recognition 11.

(1) Background subtraction processing Background subtraction processing for extracting only image information of character lines is executed so as to cope with changes in plate brightness and changes in plate color. In this processing, a background image is created directly from the recognition target image, and the processing target image 10 is generated as shown in FIG.
A background image 101 of the entire area A of 0 is created. For example,
When the characters of the processing target image 100 are darker than the background, the local maximum value filter is executed several times, and similarly the local minimum value filter processing is executed several times to create the background image. A background difference image 102 can be created by subtracting the processing target image 100 from the background image 101. The purpose of performing such processing is to make the character line always bright and to convert the background into a value of "0" to simplify the recognition processing. If the text is brighter than the background (for Japanese cars, green plates, etc.), the local minimum value filter is executed several times, and the local maximum value filter is executed several times to create the background image. Subtract the background image.

In the local maximum value filtering process, for example, the maximum density of a local area of 3 × 3 pixels is obtained, and the process of replacing the density with the density at the center position is performed by raster scanning, and is binary. This corresponds to image expansion processing. The local minimum value filtering process is a process of replacing the minimum density of the local region with the density of the central position, and corresponds to the contraction process of the binary image. Such background difference processing is disclosed in Japanese Patent Laid-Open No.
No. 3-153682.

(2) Extraction of Detailed Coordinates of Character Area The background difference processing described above roughly processes the area A. However, if this area is recognized as it is, it will be erroneously recognized because of character misalignment. Is likely to occur. Therefore, the detailed coordinates of the Kanji area for recognition are calculated using the background difference image 102. In this process, as shown in FIG. 3, cumulative projection distributions of densities are calculated in the vertical direction (x-axis projection distribution) and in the horizontal direction (y-axis projection distribution) with respect to the background difference processed image 102, and this frequency has a predetermined threshold value. The coordinates that have been exceeded are obtained for each direction, and the coordinates of the character area B are obtained. In this case, since the background image is drawn and the density is almost "0" except for the character lines, the threshold value may be set to about the minimum density of the character lines. In the case of an image with a lot of noise, the background density does not become "0" completely, so the cumulative value of the background density has an effect, but in this case, it is less than the threshold value beforehand (for example, the maximum of rough areas). It is only necessary to create an image from which density pixels of 1 / n of the density or half of the minimum density of the character line have been removed, and to calculate the coordinates using the density projection distribution of this image (or the threshold value). Only the above concentrations are accumulated).

Although it is possible to directly obtain the detailed coordinates of the character area from the image to be processed, it is very difficult to set the above threshold value when the brightness of the plate changes, and the detailed area cannot be determined. . On the other hand, in the area determination from the background difference image, since the density of characters other than the character can be set to almost “0”, the threshold value can be easily set, and the detailed area determination can be easily executed. As a result, it is possible to favorably determine the area even when the plate is bent or bent.

When the above processing is executed, the size of the detailed region obtained is extremely small when there is no density projection distribution above the threshold value or even when there is a density distribution above the threshold value. In this case, it can be determined that there is no target character.

(3) Feature Extraction Next, after the detailed coordinates of the Chinese character region are obtained as the region B, the feature amount for recognition is obtained. Here, the density value of the grayscale image is used as the feature amount, but some processing is conceivable. (A) Since the character area is small, all the density data of the detailed area is used as the feature amount.

(B) Extracting m × n feature quantities from the image of the detailed area (mosaicing). and so on.

In the method (a), the field of view of 1.5 m is 5
When the image is captured with 12 × 256 pixels, the number of pixels of one Kanji character is about 14 × 7 pixels, so the number of feature amounts is about 100, and especially when recognizing with a neural network, it takes time to learn and recognize. Takes. On the other hand, since the method (b) reduces the data, the position of the pixel data to be sampled may be misaligned, which may cause an erroneous recognition, but it is effective in terms of processing time. Therefore, the method (b) is adopted in the present invention. For the problem of sampling, the detail region is smoothed and m × n data is extracted from the smoothed image so that there is no problem even if the position of sampling as the feature amount is slightly shifted. There is. Specific processing is shown in FIG.

(A) The detail region B of the background difference image 102 is smoothed several times to blur the image (image 104).

(B) The area B of the blurred image 104 is m × n
It is reduced to pixels (image 106). It is also conceivable that the degree of smoothing described above is changed by this reduction rate.

(C) The density data f (i, j) of the m × n pixel reduced image 106 is read.

When recognizing the feature amount data thus obtained by the neural network, it is usually necessary to convert the data into a real number of 0 to 1. Here, the conversion is performed by the calculation as shown in Expression 1.

T (i, j) = {f (i, j) -ming} / (maxg-ming) Equation 1 Here, t (i, j) is the feature amount data at the (i, j) coordinate, f (i, j) is the density data at the (i, j) coordinate, and ming, maxg are the minimum density and the maximum density in the image of m × n pixels.

In addition to the above, the following method can be considered as a normalization method for extracting m × n feature quantities.

(A) With respect to the background difference image 102, the background difference image is subjected to the smoothing process 104 in consideration of the deviation of the sample points, and m × n pieces of data are directly extracted from the image.

(B) For the background difference image 102, m × n
The pixels are divided into meshes, and the average density of each mesh is calculated.

When the grayscale feature amount data is obtained by the above processing, learning and recognition processing by a neural network is performed. Since learning and recognition processing by the neural network may be general processing, description thereof will be omitted.

As described above, by performing the background subtraction process, it is possible to extract the required feature amount from the grayscale image and recognize the character without being affected by the brightness change or the position change of the recognition target. Become.

Although the character recognition method has been described above, the procedure when applied to plate recognition will be described below. The point that plate recognition differs from general character recognition is: (1) I do not know where the plate is in the input image.

(2) There are several types of plate colors.

(3) The size of the plate changes (the size of the characters changes) depending on the position where the entire surface of the vehicle is photographed.

(4) The size of the plate includes medium size and large size (the size ratio of medium size and large size character is 3: 2).

(5) The number of characters to be recognized is limited to some extent.

(6) Since the Land Transport Office code represents one place name with two or three Chinese characters, the entire code area may be processed to determine the place name pattern. And so on.

Therefore, it is important to execute a plate extraction process, a plate color discrimination process, a recognition process that does not depend on the size of characters, and the like. On the other hand, since the number of characters to be recognized is smaller than that in general character recognition as in (5), even a low-resolution character can be recognized by using a grayscale image. Further, since the pattern of the place name is discriminated as in (6), it is not necessary to discriminate similar characters such as "bird", "island" of "Tottori", "Shimane". The recognition procedure is shown below.

FIG. 5 shows a plate recognition procedure. The flow of processing is (1) image input 12. (2) Identification of plate position 13. (3) Plate color determination 14. (4) Extraction of serial number 15. (5) Obtain each coordinate of the serial number and determine 16 the coordinate of each code area. (6) Recognition of serial number 1
7. (7) Recognition of vehicle type code 18. (8) Recognition of Land Transport Bureau Code 19. (9) Recognition 20 of the usage code. Becomes
The details will be sequentially described below.

(1) Image input The image from the television camera is taken into the image memory. Since an electronic shutter camera is usually used, only a field image can be captured, so the vertical resolution is poor. Here, description will be given below assuming that an image is captured with 512 pixels in the horizontal direction and 256 pixels in the vertical direction. If the field of view is 1.5m, kanji,
Hiragana characters are 40 x 40 mm per character, so about 1
The size is about 4 pixels × 7 pixels, and it is necessary to recognize a considerably smaller character than the resolution of 32 to 40 pixels in general character recognition. However, the number of recognition targets is small.

(2) Plate Position Determination Where the plate is in the captured image varies depending on the traveling position of the vehicle and the mounting position of the plate. Therefore, it is necessary to recognize the position of the plate from the input image. There are various methods for this, but here, Japanese Patent Laid-Open No.
-1441087 The method described in the official gazette is adopted. That is, FIG.
As shown in FIG. 11, the input image 108 is subjected to horizontal differentiation processing 110, binarized by a predetermined binarization threshold value, and then dilation processing 112 is performed to shape the binary image. . Since most of the pattern on the entire surface of the vehicle is a horizontal striped pattern, it is a method that focuses on the fact that the background other than the plate can be removed by performing differentiation (emphasizing vertical edges) in the horizontal direction. Therefore, the size of each area of the obtained shaped image (area, vertical, horizontal size, etc.)
An area where is similar to the shape of the plate is the plate candidate position. It should be noted that the plate candidates are preferentially searched from the lower side of the screen.

(3) Judgment of plate color When the plate candidate position can be specified, the color of the area is judged. Japanese cars are roughly divided into green characters on a white background and white characters on a green background, so it is necessary to judge whether the characters are brighter or darker than the background. Here, the determination is performed using the density frequency distribution of the obtained area. That is, as shown in FIG. 7, when comparing the area occupied by the characters on the plate with the area occupied by the characters other than the characters on the plate, it is sure that characters other than the characters on the plate (background)
Is wider, the background is white (white background plate) if the density maxg showing the maximum frequency of the frequency distribution is brighter than the average density avg, and the background is green (green background plate) if dark.

This determination result is used in the following character extraction.
Although the color determination may be erroneous, if the serial number cannot be extracted, the color determination is set in reverse and the process is repeated.

In addition to the above, there is also a method of comparing the number of pixels brighter than the average density and the number of dark pixels in the plate color determination method. However, in terms of processing time, the density of the maximum frequency is compared with the average density. Is good.

(4) Extraction of serial number It is only necessary that the positioning of each code can be obtained directly from the grayscale image of the plate area, but it is quite difficult. Therefore, the serial number is extracted as a binary image and relatively determined from the coordinates of the numbers. Therefore, if the color can be determined, the serial number is extracted as a binary image (the character size of the serial number is larger than that of the Land Transport Office code or vehicle type code, so that the binary image is less likely to be crushed). The extraction method at this time uses the background difference processing method. As described above, this method replaces the density of the character image with the density of the background image to create an image of the background only,
By subtracting this background image from the binarization target grayscale image, it is possible to obtain density information of only characters (hereinafter referred to as background difference image). As a result, it is possible to easily set the binarization threshold value for extracting characters, so that a clear binary image can be created. At this time, as described above, the Japanese plate is roughly divided into a white background and a green background, so the method of creating the background is switched depending on the color of the characters.

When the character color is darker than the background (white background plate) (a) The local maximum value filter is executed several times for the binarization target grayscale image to fill the character area with the background density.

(B) The local minimum value filter is executed the same number of times as the local maximum value filter.

(C) In order to reduce noise, smoothing processing is performed to create a background image.

(D) The binary image to be binarized from the created background image is subjected to difference processing by calculation for each pixel.

(E) The difference image in the background area of the plate has a value close to "0", and the density of the contrast of the character can be extracted from the character area. Therefore, a clear image can be created by binarizing the threshold value with a value near "0" without extracting noise.

When the character color is brighter than the background (green plate) When the character color is brighter than the background, the processing of (a) and (b) of the above processing is performed in the order of the local minimum value filter and the local maximum value filter. A background is created by performing a process of removing the density of the background density with the surrounding background density, and the background image is subtracted from the binarization target grayscale image. Alternatively, the grayscale image to be binarized is inverted,
You may perform the process of the said white background plate. The inversion process executes f (x, y) = 255-g (x, y) if the captured image is an 8-bit image.

Here, f (x, y) is an inverted image, g
(X, y) is a binarization target grayscale image.

By the above processing, the binary image of the serial number can be clearly extracted.

If the background position difference processing is executed when the plate position of the captured image is in the screen frame, there may be a case where characters cannot be extracted. This cuts out the plate area from the input image 108 as shown in FIG.
3. If local maximum value filter processing is performed on this plate area, the density expansion processing cannot be performed on the area on the left side in the figure, such as 114. If the local minimum value filtering process is executed on this image, a background image with reduced density on the left side is created as in 115. Therefore, when the difference processing from the background image 115 to the plate image is performed, an image in which the character on the left side is missing like 116 is obtained. This is because the maximum value filter processing cannot be processed in the area beyond the screen frame on the image memory, so only the minimum value filter is processed, and as a result, the background difference image cannot extract character images near the screen frame. This is because it ends up. Therefore, in the present invention, if the plate area is obtained,
As shown in FIG. 9, the image in the area is moved to the center of the screen (it is enough to be separated from the screen frame by the number of times of the local maximum filter processing) 117, and the background difference processing is executed at that position. By this processing, even if the plate is photographed in the vicinity of the screen frame, the local maximum value filter can be accurately executed, so that characters can be extracted well.

(5) Determining coordinates of each code area After obtaining each coordinate area of the binary image of the extracted serial number and checking whether it is noise or a character by the size of the area of the binary image or the arrangement rule of the character string, etc. The position of the Land Transport Office code, vehicle type code, and usage code is specified from the center coordinates of each number image of the serial number, the average height (pixels), and the digit spacing. Since the position coordinates of each code are different for large and medium plates as shown in Fig. 2, use the average height of the characters of the serial number.
When the height of the character is equal to or larger than a predetermined value, it is determined that the plate is a large plate, and the plate positional relationship is calculated according to each plate type. Here, if the image resolution is calculated from the height of the character and the horizontal relative position is obtained with this resolution, the height of the character changes due to the downward tilt of the plate, and the horizontal position of each code area is accordingly changed. The coordinates cannot be determined accurately. Therefore, for the positioning of each code area, the resolution (mm / pixel) in the vertical direction (y direction) is calculated from the average height of the characters, and the character interval (interval of the center X coordinate of each character of the extracted character pattern) is calculated. The resolution (mm / pixel) in the horizontal direction (x direction) is calculated, and the coordinates of each code area are calculated at each resolution. In this way, by calculating the coordinates of each code area with the resolution in each of the x and y directions,
Although the size of the plate changes depending on the downward inclination of the plate and the position where the plate is taken in, it is possible to favorably cope with such a change.

However, the areas calculated here are, as shown in FIG. 10, the Land Transport Office code area 51, the vehicle type code area 52,
The usage code area 53 has a larger area setting. The detailed positioning is determined by the detailed area calculation process described above. In particular, since the number of characters of the Land Transport Bureau code is unknown, it is assumed that it is a two-character place name, and the plate mounting screw 54 is used as the starting point for the serial number 2
The end point X coordinate of the digit is obtained as the X coordinate of the area, and the positioning is performed in detail by the recognition process described later.

The plate has a serial number of 1 to 4 digits, but since the character spacing can be found in 2 or more digits, if it is 1 digit, it is estimated from the character height. Alternatively,
It is estimated from the intervals of the dots that are marked instead of the numbers.

(6) Recognizing Characters There is a method for recognizing characters by extracting a feature amount from a binary image and inputting the feature amount into a neural network for recognition. , Characters are likely to be crushed or missing. Therefore, the recognition with the grayscale image as described above is executed. Since the color of each recognition target area and plate has been obtained by the above processing, the following processing is executed for each area.

(A) A background difference image corresponding to the plate color is created for each area (a background difference image of the entire plate area, which is executed at the time of extracting the serial number, may be used.
Considering processing time, plate stains, etc., it is desirable to perform optimal background subtraction processing for each area).

(B) Density projection distribution (x,
A detailed character area is extracted from (y direction).

(C) The image of the detailed character area is normalized to m × n pixels.

(D) Learning with m × n feature amount data,
recognize.

The above processing is a method of the character recognition processing described above.

By the way, in the plate recognition, it is determined in advance what kind of character is in each character area, so three neural networks are prepared here.

That is, for uppercase and lowercase numeral recognition,
It is for recognition of Land Transport Office code (Kanji, Hiragana) and usage code (Hiragana). The outline of the network is as follows (the network scale is an example).

(A) Network of Uppercase Numbers and Lowercase Numbers The network scale is as shown in FIG.
6), intermediate 12 and output 10. As for the number recognition, the size of the characters is secured to some extent,
It can be recognized even if a general binary image recognition method is adopted. However, in the case of a binary image, it is difficult to deal with a chipping or the like, so it is desirable to perform the recognition process on the grayscale image within the processing time range. Further, although the uppercase and lowercase letters are recognized in the same network here, it may be possible to divide the network into each.

(B) Network for Land Transport Branch Code The network scale is as shown in FIG.
4), intermediate 32, and output 100. Here, since the Land Transport Office code recognizes the entire code area as one pattern, only the number of place names needs to be separated. Therefore, about 100 output neurons are sufficient. By doing this, the “island” of “Shimane” and “Tottori”
It is no longer necessary to identify similar characters of "bird" and distinguish between hiragana and kanji. Such a method of recognizing the whole character as a symbol is described by Kim et al .: Vehicle number recognition using a neural network; 1990 Spring National Convention of the Institute of Electronics, Information and Communication Engineers 7-228 (image data is input image. Since you are entering the data itself, it is necessary to learn by plate color).

(C) Use code network The network scale is as shown in FIG.
4), intermediate 16 and output 46. Regarding the hiragana usage code recognition, since there are similar characters such as "ha", "ho", "sa", and "ki", it is necessary to finally perform similar character discrimination. There are several possible similar character discrimination processes. One is a method of re-identification with a network in which the number of features is increased (the resolution is increased).
One is a method of checking the characteristic position again and making a determination. In the method of increasing the feature amount, the feature amount is first classified into 8 × 4 feature amounts. For example, when the recognition result is “ha” or “ho”, the mosaic is densely performed, for example, 12 × 6 pixels. Then, the feature amount of this image is input to the neural network, and "ha" and "ho" are identified. Therefore, as many detailed classification networks as the number of similar characters are provided.

FIG. 14 shows a method of checking a characteristic position. The difference between "ha" and "ho" is that there is one or two horizontal bars in the upper right of the character as shown in Fig. 14 (a).
“Sa” and “ki” are the differences in the horizontal bars at the top, as shown in FIG. Therefore, the number of horizontal bars may be checked from the projected distribution of density in the horizontal direction for this image. The checking method may be a method of checking the number of peaks or troughs of the density projection distribution, or a method of inputting the density projection distribution itself as feature data to a neural network for determination. Alternatively, a method may be considered in which the usage code area is divided as shown in (1) to (8) of FIG. 15 and the pixel data of each area is determined as a feature amount. For example, "ha" and "ho" are divided by (2), "nu" and "me" are divided by (4),
“Ma” and “yo” are classified by dividing (5).
The cases of "nu" and "me" are shown in FIGS. 15 (9) and 15 (10). In the case of such a classification method, as shown in FIG. 16, learning using only similar characters is performed in advance by using a feature amount suitable for the similar character determination, and the resulting weighting factors 22 of a plurality of neural networks are acquired. , 23, 24 (Fig. 1
6 shows only three, but in the case of FIG. 15, it is 8
The individual weighting factors are stored in the weighting factor storage unit 25. At this point in time, it is determined which character is used for which characteristic amount to determine. The flow of recognition is based on the recognition output result of the application code recognition unit 6 described above, the method determination unit 21 determines which weighting coefficient 22, 23, 24 is used for the similarity determination, and the determination processing unit 26 by the neural network.
Then, the similarity determination is performed by the neural network recall process using the weighting factor determined by the method determination unit 21. By such processing, it is possible to improve the recognition rate of the usage code.

Here, the order in which each code is recognized is as follows: the serial number, the vehicle type code, the Land Transport Office code, and the usage code are recognized in this order. The reason for this ordering is to accurately obtain the area of the Land Transport Office code. That is, the Land Transport Office code has 2 to 4 characters for the place name (for example,
"Adachi", "Kasukabe", "Owari Komaki"), but in the above-mentioned area coordinate calculation process, the coordinates are determined assuming that the number of characters of the place name is the most frequent two characters, so the final recognition process Then, it is necessary to fine-tune this coordinate. Therefore, after the process of cutting out the vehicle type code is executed, the end point coordinate of the x coordinate of the Land Transport Office code is set as the start point x coordinate coordinate of the first digit of the vehicle type code.

Further, since the characters used in the usage code are limited according to the vehicle type code, it is possible to check the validity of the recognition result of the usage code based on the recognition result of the vehicle type code. For this reason, the usage code is recognized after the vehicle type code is recognized.

As described above, by using the image data mosaiced to the background difference image as the feature quantity,
Since the feature amount that does not affect the brightness variation can be extracted, the character recognition rate can be improved. Furthermore,
Since the character image can always be extracted brightly regardless of the plate color, it is not necessary to learn the data for each plate color, and the recognition can be simplified. Further, since the detailed coordinate determination of each character area can be performed with the grayscale image as it is, the alignment accuracy is high. For this reason, there is a problem that it is difficult to specify the character position and it is not possible to cope with the brightness change in the method of simply mosaicing the input image like the face image recognition. However, by mosaicing the background difference image, Since these problems can be solved, the recognition rate can be improved.

The use of such a background difference image is effective not only for plate recognition, but also when the character color changes or when the brightness level changes significantly.

FIG. 17 shows a hardware configuration of the present invention. An A / D converter 31 for converting an analog video signal from the television camera 30 into a digital signal, and an image memory for storing the digital signal. Image processor 3 for performing image processing on 32 and the image memory
3 and a CPU 34 for controlling various arithmetic operations and various hardware. The image memory 32 is a memory for storing an image of a normal television camera, images in the middle of image processing, and the like, and a plurality of memories each having a capacity of about 512 × 512 pixels are provided. This image memory is provided with not only a grayscale image but also a plurality of binary image memories. The image processor 33 includes a local maximum value filter of the image, a local minimum value filter process, a pixel-by-pixel difference process, a smoothing process, a differential process, a binarization process, a binary image expansion process, a labeling process, a histogram process, and the like. Performs general image processing functions. Although the calculation time of the neural network becomes long, the calculation may be performed by the general CPU 34 or a dedicated DSP or neurochip may be used, but in any case, the required performance can be secured.
Next, a method for improving processing performance will be described.

(1) Improvement of recognition rate Since the appearance frequency of the Land Transport Office code varies greatly depending on the installation location of the recognition device, it is possible to improve the recognition rate by inputting the installation position in the system in advance. . In the frequency distribution of the types of Land Transport Bureau codes, for example, when shooting in the vicinity of “Mito”, the “Mito” number occupies about 80%, and the remaining 20% is “Tsuchiura” or the number outside the prefecture. In the Tokyo metropolitan area, there are many “Yokohama”, “Narashino”, and “Adachi”. Thus, especially in the frequency distribution of plates in rural areas, the frequency of only a specific Land Transport Office code is extremely high. Therefore, if you output an ambiguous recognition result,
It is possible to improve the recognition rate by forcibly recognizing the Land Transport Bureau code as the code with the highest frequency of appearance.

More specifically, assuming that the recognition rate of the Land Transport Office code for all vehicles is 90% and the ratio of "Mito" numbers is 80%, the number of recognized "Mito" numbers is 80% × 90% = 72% can be recognized. In addition, the code for the Land Transport Office other than "Mito" is 20%, so 20%
× 90% = 18% can be recognized. That is, 72% + 1
8% = 90% can be recognized. In this case, if the recognition result of unrecognizable vehicles (excluding vehicles whose land support code cannot be completely seen) is forced to be "Mito", the recognition rate of "Mito" is 72.
% Is up to 80%, resulting in a total recognition rate of 98% (up to 8% improvement). Therefore, by entering the installation location in the system in advance, all ambiguous vehicles can be replaced with the most frequent place names in the area, and the apparent recognition rate can be improved. Such an effect can be expected to be more effective when installed in rural areas, and this method cannot be adopted when the frequency of several types of Land Transport Bureau codes is high, such as in the Tokyo metropolitan area.

The operation of the system will be described with reference to FIG. 18. The installation place is input from the place name input section 43 such as a keyboard, and the conversion code name for this place name is converted into the conversion table 4.
Draw from one. This conversion table 41 is stored for each place name as shown in FIG. When the recognition process starts,
The vehicle number recognition processing unit 40 processes the video of the TV camera 30.
If the recognition result of the Land Transport Office code at that time is input to the recognition result conversion processing unit 42 and an ambiguous recognition result is output, the conversion code name extracted from the conversion table is changed only in this case. Output to. Whether or not it is an ambiguous recognition result is determined, for example, when the outputs of all output neurons of the neural network are below a certain threshold,
When the value of the maximum output neuron and the value of the second output neuron are not so different, it is determined that the recognition is impossible, and when the value is normally recognized, the value of the output neuron is increased by a certain neuron, so this determination is easy. . Note that the place name input unit may directly input the converted place name instead of inputting the installation place. When the place name is not converted, a code to that effect is stored in the conversion table.

As described above, according to the installation position of the camera,
The recognition rate can be easily improved by replacing the unrecognizable character with the Land Transport Office code corresponding to the camera installation position and outputting the recognition result.

(2) Learning data collection method Since the object of plate recognition is a vehicle, there are infinite combinations of how the plate becomes dirty, how it is bent, how it bends, and how the plate looks. The problem with recognition with a neural network is that the recognition rate does not improve unless optimal learning is performed. That is, the deformed plate cannot be recognized only by instructing only a clean plate. No matter where you shoot the numbers and usage codes (Hiragana), the appearance frequency for each category is the same, so you can collect data such as the above deformation patterns, but the Land Transport Bureau code is for each region. Since the appearance frequency for each category is different,
It is impossible to collect deformation patterns of all categories in advance. Also, if you start operating the car number recognition system all over the country at the same time, you can collect data in a short period of time, but in reality it will take a considerable time to install it throughout the country, It takes a very long time. Therefore, the data collection method for learning the Land Transport Bureau code will be described.

Since the standard font is prepared as the font of the Land Transport Office code, this font will be learned in advance.
In this learning, there is no bending of the letters or stains. Therefore, when the characters on the actual plate are recognized, only characters that are not deformed can be recognized, and therefore many characters cannot be recognized at first. By the way, the frequency of appearance of the Land Transport Bureau code is different in each area as described above.For example, when shooting in the vicinity of Mito, when a code other than "Mito" is output as a recognition result (when the Mito number is erroneously recognized, If there is no deformation with a number other than or Mito) or if an unrecognizable result is output, the characteristic amount of that image is stored, and if the maintenance staff checks the image and misrecognizes it, the correct result is obtained. (Land Transport Office name and corresponding code number, etc.) are input, and learning data in which the feature amount data and the correct result are paired are collected. If data can be collected to some extent, learning with the learning data will improve the recognition rate by an amount commensurate with the number of data. Further, if similar data collection is performed, unrecognizable characters are gradually reduced (recognition rate is improved), and thus the data collection work is gradually reduced. As a result, the working time for improving the recognition rate can be significantly shortened. At this time, if this image itself is automatically stored in a file (VTR, laser disk, etc.), maintenance personnel can later create learning data (feature amount data and category name pair) while looking at these images. You can also

FIG. 20 shows the configuration of the maintenance process.
The system is provided with a frequency table 44 for storing one or a plurality of place names having a high appearance frequency with respect to the installation place, as shown in FIG. 21, and the installation place of the system is input in advance from the place name input unit 43. While performing the recognition process,
When the recognition result is input from the vehicle number recognition processing unit 40 and the recognition result other than the code for the corresponding place name in the frequency table 44 is obtained or it is output as unrecognizable, the image at that time is displayed on the input image display device 48. And inquire about the correct result. Check the plate image displayed on the input image display device by inquiry, and enter the name of the Land Transport Office or the corresponding code number of the image in the category input section 4
7, the feature amount and the category of the image obtained by the vehicle number recognition processing unit are paired and stored in the learning data storage unit 46. In the learning data storage unit 46, the feature amount for the image of the Land Transport Office code of a place name (a pattern that cannot be recognized by the existing learning data) of a place that is rare at the place where the system is installed or a plate that is severely deformed in the end is stored. ,
Category pairs are accumulated. Therefore, by re-learning the accumulated data, it is possible to improve the recognition rate of deformation patterns and the like.

When there is a large-capacity image file, if a recognition result is obtained for a place name other than the above-mentioned frequently-used place name, or if it is output as unrecognizable, the input image at that time is stored in the file and the feature amount data and time are stored. To a data file. When the data is accumulated to some extent, the maintenance staff inputs the correct recognition place name if the data file time and the image file time are matched and if the data is mistakenly recognized (or for all data). By doing so, it is possible to store the image data of the Land Transport Office code of the plate that rarely appears. Further, if these data can be collected all over the country, the learning process is finally executed to create a network that can recognize all of them.

By storing the name of the land transportation branch office having a high frequency of appearance corresponding to the camera installation position in this manner, the learning data of the category (land transportation branch office name) having a low appearance frequency and the deformation pattern at the camera installation position can be sequentially collected, Eventually, data such as various deformation patterns can be efficiently collected.

In the above embodiment, the network configuration is described in detail as an example, but the size of the network may be any size as long as it is recognizable. In addition, although recognition with a neural network has been described as an example, a method of classifying grayscale feature amounts by statistical processing,
Recognition by fuzzy processing is also possible.

[0084]

According to the present invention, by using the image data mosaiced to the background difference image as the feature amount, the feature amount that does not affect the brightness variation can be extracted, so that the character with low resolution can be extracted. Even with this, it is possible to improve the character recognition rate. Further, since the character image can always be extracted brightly regardless of the color of the plate, it is not necessary to learn the data for each color of the plate, and the recognition can be simplified.

Further, the use of the hiragana-similar character determination unit makes it possible to improve the recognition rate of the usage code.

By inputting the name of the place where the recognition device is installed into the system, processing (judgment for learning, improvement of recognition rate) according to the frequency of appearance of the Land Transport Office code can be executed,
It is possible to improve the system performance.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of the present invention.

FIG. 2 is a dimensional diagram of a license plate.

FIG. 3 is a diagram showing a procedure of character recognition from a grayscale image according to the present invention.

FIG. 4 is a diagram illustrating a feature amount extraction method from a grayscale image.

FIG. 5 is a diagram showing a procedure for recognizing characters on a license plate.

FIG. 6 is a diagram showing a method of extracting a license plate area from an input image.

FIG. 7 is a diagram showing a method of determining the color of a license plate.

FIG. 8 is a diagram illustrating a problem in the case of extracting characters when the shooting position of the license plate is near the screen frame.

FIG. 9 is a diagram showing a method for solving the problem shown in FIG.

FIG. 10 is an explanatory diagram for roughly determining each code area coordinate.

FIG. 11 is a diagram showing a configuration of a neural network for digit recognition.

FIG. 12 is a diagram showing a configuration of a neural network for use code recognition.

FIG. 13 is a diagram showing a configuration of a neural network for recognizing Land Transport Office code.

FIG. 14 is a diagram illustrating a method in which a projected distribution of density is used as a feature amount for determining similar characters in a usage code (hiragana).

FIG. 15 is a diagram showing types of feature amounts when image data obtained by dividing a character region into small blocks is used as a feature amount for determining similar characters of a usage code (hiragana).

FIG. 16 is a diagram showing a configuration in the case of performing similarity determination using a feature amount divided into small blocks.

FIG. 17 is a diagram showing a hardware configuration of the present invention.

FIG. 18 is a diagram showing a device configuration capable of improving the recognition rate of the Land Transport Office code.

FIG. 19 is a diagram showing a conversion table for storing a code name of the most frequent Land Transport Office code for a place where a license plate is recognized.

FIG. 20 is a diagram showing a system configuration for accumulating learning data of a Land Transport Office code.

FIG. 21 is a diagram showing a frequency table for storing a place name with a high frequency of Land Transport Office code for a place where a license plate is recognized.

[Explanation of symbols]

1 ... Plate extraction unit, 2 ... Serial number extraction unit, 3 ... Code area coordinate calculation unit, 4 ... Character recognition unit, 5 ... Number recognition unit,
6 ... Usage code recognition unit, 7 ... Land Transport Office code recognition unit, 3
2 ... Image memory, 33 ... Image processor.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Norio Tanaka 52-1 Omika-cho, Hitachi-shi, Ibaraki Hitachi Ltd. Omika factory

Claims (9)

[Claims] 1. A television camera for photographing a license plate of a vehicle, an A / D converter for digitizing a video signal from the television camera, and an image memory for storing a digital image from the A / D converter. A license plate area extracting unit that cuts out a license plate area from the input image input to the image memory, a serial number extracting unit that extracts a serial number as a binary image from the cut out license plate area, and the extracted serial number. Each code area coordinate calculation unit that determines the coordinates of each area of the Land Transport Office code, vehicle type code and usage code in the license plate by using the coordinates of each number constituting the number, and a grayscale image of each determined code area The Land Transport Office code recognition unit that recognizes the Land Transport Office code using the feature quantity of the Recognition unit and vehicle number recognition apparatus characterized by having a character recognition unit and a recognizing application code recognition unit applications and code using the feature quantity. 2. The code area coordinate calculating unit according to claim 1, wherein the image resolution in the vertical direction is calculated from the average height of the extracted serial numbers, and the image in the horizontal direction is calculated from the character spacing of the serial numbers. The resolution is calculated, the Y-direction coordinate of each code area is relatively determined from the calculated vertical image resolution from the position of the serial number, and the X-direction of each code area is calculated from the calculated horizontal image resolution. A vehicle number recognition device characterized in that the coordinates of the vehicle number are relatively determined from the position of the serial number. 3. The character recognition section according to claim 1, wherein the character recognition section creates a background difference processed image in which the background image of each code area of the input image is removed, and the image data of the background difference processed image is set to a predetermined size. A vehicle number recognition device characterized by performing a smoothing process to obtain a desired size, and performing character recognition using the normalized image data normalized by the smoothing process. 4. The character recognition unit according to claim 1, wherein the character recognition unit recognizes the vehicle type code, and controls the coordinates of the area of the Land Transport Office code by using the X start point coordinate of the first character of the recognized vehicle type code. At the same time, the vehicle number recognition device is characterized in that the validity of the recognition result of the usage code is checked based on the recognition result of the vehicle type code. 5. The usage code recognition unit according to claim 1, wherein a background difference processed image is created by removing a background image of the usage code area of the input image, and image data of the background difference processed image of the usage code area is created. Is smoothed to a predetermined size, character recognition is performed using the normalized image data normalized by the smoothing processing, and similar character identification processing is executed according to the recognition result. Vehicle number recognition device. 6. The vehicle number according to claim 5, wherein the X-direction density projection distribution or the Y-direction density projection distribution of the background difference processed image of the usage code area is used as the feature quantity in the similar character identification processing. Recognition device. 7. The image data obtained when the usage code area is divided into four with respect to the background difference processed image of the usage code area, the left and right half-division, or the feature quantity in the similar character identification processing according to claim 5. A vehicle number recognizing device characterized by using image data obtained by vertically dividing into two. 8. A TV camera for photographing a license plate of a vehicle, a car number recognition processing unit for processing a video signal from the TV camera to recognize characters on the license plate, and a place name where the camera is installed are input. The place name input means, a conversion table that stores the place name indicating the maximum frequency of the Land Transport Office code in the installation place name and the installation place name, and the recognition result of the Land Transport Office code obtained from the vehicle number recognition unit is ambiguous The vehicle number recognition device, further comprising: a recognition result conversion processing unit that outputs a maximum frequency place name in the conversion table corresponding to the place name input from the place name input means as a recognition result. 9. A TV camera for photographing a license plate of a vehicle, a car number recognition processing unit for processing a video signal from the TV camera to recognize characters on the license plate, and a place name where the camera is installed are input. A place name input means, a frequency table for storing at least one place name with a high frequency of occurrence of land transportation office code in the installation place name, and the recognition result of the land transportation office code obtained from the vehicle number recognition unit If it does not match the frequency place name in the frequency table, or if it is determined that it cannot be recognized, the image processed by the vehicle number recognition processing unit is displayed on the input image display device, and the input image is captured from the category input unit. Enter the place name or place name code number of the Land Transport Office code that is present, and the feature amount of the Land Transport Office code obtained by the vehicle number recognition processing unit and the place name. Corresponding vehicle number recognition apparatus characterized in that a learning data determination unit to be stored in the learning data storage unit as the learning data data.