JP2002007961A - License plate recognizing method and license plate recognizing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To conduct a simple and accurate license plate recognizing process from the images of a vehicle photographed at various distances and angles for generalizing the number plate recognizing process. SOLUTION: Even when the image of the vehicle M including a license plate is photographed by an image pickup device 11 from the oblique direction and stored in an image storage device 13, an image cutting device 14 extracts and cuts the license plate image from the photographed image, then an image correcting device 15 corrects the distortion generated when the license plate image is photographed from the oblique direction. An image normalizing device 16 normalizes the corrected license plate image into a fixed size, then a character recognizing device 17 conducts a character recognizing process from the license plate image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a license plate recognition method and a license plate recognition apparatus for capturing an image of a vehicle and recognizing characters displayed on a license plate of the captured vehicle to be read.

[0002]

2. Description of the Related Art Hitherto, various technologies for recognizing a license plate of a vehicle from a captured image have been developed and put into practical use. A license plate portion is extracted from a captured image of a vehicle. It is common to perform character recognition.

In this type of license plate recognition processing, when a camera is installed so as to take an image from an oblique direction with respect to the vehicle, the license plate on the image is distorted due to the angle between the camera and the vehicle. Character recognition becomes difficult. For this reason, if the photographing angle is limited to the front of the vehicle or photographed from an oblique direction, a parameter representing the magnitude of the distortion is set in advance so that the image of the license plate having the distortion can be recognized. There is one that prepares a character pattern in consideration of distortion in advance. However, depending on the situation of the installation location, it is not always possible to shoot from the front, and when the installation conditions of the camera change and the shooting angle changes, the magnitude of the distortion also changes. You need to adjust the characters and prepare multiple character patterns.
Another problem is that the size of the characters in the image also changes when the shooting distance changes, and the same complicated work is required.

To solve this problem, for example, FIG.
As shown in the figure, the license plate of the vehicle to be photographed when the entering vehicle is detected by the sensor is assumed to be on the assumed imaging plane, and according to the parameters given by the installation conditions according to the positional relationship with respect to the camera 101. Japanese Patent Laying-Open No. 6-274788 proposes that the license plate recognition is simplified by performing coordinate transformation of a captured image and correcting image distortion.

Further, when the width of the passage through which the vehicle passes is wide, if the image pickup range is widened to surely image the vehicle with only a single image pickup device, the license plate in the image becomes relatively small. There is a problem to be solved in that the accuracy of character recognition is reduced.

To solve this problem, for example, FIG.
As shown in FIG. 7, the vehicle M illuminated by the strobe 210 based on the detection of the sensors 211 to 213 expands the imaging range by installing a plurality of cameras 201 to 205 so as to cover the width direction of the passage R. Is reliably imaged, and the recognizing device 220 detects 3
Japanese Patent Application Laid-Open No. 5-35912 proposes that a wide range of license plate recognition be achieved by performing image processing and character recognition by compressing one captured image into one image. , A second conventional example).

[0007]

However, the first conventional example is effective when the vehicle enters the imaging range under almost constant conditions assumed by the vehicle, but is effective at the entrance to the imaging range. In an environment where the approach angle differs from vehicle to vehicle due to reasons such as wideness, the license plate at the time of shooting may not always be on the assumed imaging plane. In this case, the camera 101 and the license plate The positional relationship is lost, and there is a problem that the distortion of the captured image cannot be properly corrected with the parameters corresponding to the installation conditions according to the positional relationship.

Further, in the second conventional example, even when the license plate exists near the boundary of the imaging range of each of the cameras 201 to 205, the license plate is completely cut without any one of the license plates being cut off. The cameras 201 to 205 need to be installed so that the adjacent overlapping portion becomes an area equal to or more than one license plate, so that the entire imaging range is narrowed. In addition, there is a problem that the processing is complicated because the three captured images are compressed into one image to extract and recognize the license plate.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems and to broaden the scope of application of license plate recognition processing to achieve generalization. The present invention realizes a simple and accurate license plate recognition process from images of a vehicle taken at various distances and angles (first object), and does not cause problems as in the second prior art. A license plate recognizing method that achieves the above-mentioned object (solution object) by effectively using an image captured by a plurality of units to realize an efficient recognition process while expanding the image pickup range as a whole (second object); A license plate recognition device is provided.

[0010]

In order to achieve the first object of the present invention, a license plate recognition method includes a method of recognizing information on a license plate to be read from a captured image of a vehicle. The method
A license plate area is extracted from the image of the vehicle, and then the characters on the license plate are cut out from the extracted license plate area. Then, the size of the license plate area and the position of the cut out characters on the license plate Correcting the distortion of the image of the license plate imaged based on the size and the size and normalizing the size of the image of the license plate, after that, having a configuration to recognize information on the license plate,

The license plate recognizing device is a license plate recognizing device that recognizes information on a license plate to be read from a captured image of a vehicle, and stores an image of the vehicle captured by the image capturing device. An image storage device, an image cutout device that obtains a captured image from the image storage device and cuts out a license plate image of the vehicle, the size of the license plate image cut out by the image cutout device, and the size of the license plate image. An image correction device that detects distortion of the license plate image based on the position and size of the character and corrects the license plate image, and normalizes the size of the license plate image corrected by the image correction device. And an image normalizing device.

According to this configuration, the number plate is extracted from the size of the license plate area extracted from the captured image of the vehicle and the position and size of the character cut out from the license plate (from the character that should be the same size). In addition to detecting the distortion of the image and making corrections such as extending the image, the size of the image on the license plate is adjusted to the original shape (the aspect ratio of the circumscribed rectangle of the image is adjusted to the original ratio). It is possible to further normalize the size of the license plate by expanding the image, etc., and to perform recognition processing such as character information on the license plate from the processed image. Can be. Therefore, for example, even when the license plate is photographed in an oblique direction, the distortion of the captured image is corrected in accordance with the actual photographing angle, and simple recognition is always performed from a license plate image of a fixed size regardless of the photographing distance. Through the processing, character information and the like on the license plate can be recognized with high accuracy.

On the other hand, in order to achieve the second object of the present invention, the license plate recognition method is a license plate recognition method for recognizing information on a license plate to be read from a captured image of a vehicle. Simultaneously capturing a plurality of images with the edges of the imaging range overlapped so that the vehicle is included in the imaging target, and then extracting a character candidate on the license plate as a feature point for each of the plurality of captured images. Then, a corresponding point between the adjacent captured images is detected based on the extracted character candidate, and then the plurality of captured images captured based on the corresponding point of the detected character candidate are set to 1 Compositing into a sheet, then extracting a license plate area from the composite image, having a configuration to recognize information on the license plate,

The license plate recognizing device is a license plate recognizing device for recognizing information on a license plate to be read from a captured image of a vehicle, wherein the license plate recognizing device is installed such that edges of adjacent imaging ranges overlap with each other. A plurality of imaging devices that simultaneously capture a plurality of images so as to include the vehicle to be imaged; an image storage device that stores images captured by the imaging device; and obtaining the captured images from the image storage device. A feature extraction device that extracts a character candidate to be a feature, and a corresponding point detection device that detects a corresponding point between the images captured by the adjacent imaging devices based on the character candidate extracted by the feature extraction device. An image synthesizing device for synthesizing the images captured by the plurality of image capturing devices into one based on the corresponding points detected by the corresponding point detecting device; An image cutout device that extracts a license plate image from a composite image by the image synthesis device, and a character recognition device that performs character recognition processing on the license plate image extracted by the image cutout device and reads information on the license plate. , Is provided.

With this configuration, it is possible to simultaneously capture a plurality of images in which the edges of the imaging range overlap each other so as to include the vehicle, and to superimpose adjacent captured images by character candidates extracted from the plurality of captured images. A plurality of captured images are combined into one by superimposing adjacent captured images with corresponding points of the character candidates by detecting points, and license plate extraction and recognition processing is performed from the combined image. it can. Therefore, even if the license plate straddles the adjacent captured image, the license plate can be extracted and recognized without excess and deficiency, and the overlapping of the imaging ranges is set to be smaller than the width of the license plate to efficiently perform wide imaging. It is possible to secure a range, and it is possible to easily and efficiently recognize only one synthesized image rather than processing a plurality of captured images, thereby recognizing character information of a license plate.

Here, as a license plate recognition method, the character candidates on the license plate are extracted from a binarized image binarized for each of the plurality of captured images, and the character candidates between the captured images are extracted. Corresponding points are configured to be detected using the extracted binary character candidates, and as a license plate recognition device, the feature extraction device includes:
A binarizing device for binarizing the captured image from the image storage device to extract the character candidate from the binarized image of the captured image; It is preferable that the corresponding point is detected between the adjacent captured images using the character candidates extracted from the image, because the processing amount of the corresponding point detection can be reduced and the speed can be increased.

Also, as a license plate recognition method,
Only when the character candidate to be processed is extracted in a portion where the imaging ranges between the adjacent captured images overlap, detection of the corresponding point between the captured images and synthesis of the captured images are performed. And a determining device for receiving a result of extracting the character candidate by the feature extracting device and determining a processing method of the captured image as a license plate recognizing device, and performing the correspondence based on the determining result by the determining device. The detection of the corresponding points between the captured images by the point detection device and the synthesis of the captured images by the image synthesizing device are performed, so that the synthesis / recognition processing of a plurality of captured images can be performed at high speed. It is possible and preferable.

Further, as a license plate recognizing method, the character candidates on the license plate are obtained by converting a plurality of the captured images into binary values under two conditions, one for a general vehicle license plate and the other for a business vehicle license plate. And then extracting from each of the two binarized images for each binarized captured image, the corresponding points between the captured images are the two binarized images for each of the binned images And detecting each of the binary character candidates extracted from the digitized image, and as a license plate recognizing device, the feature extracting device converts the captured image from the image storage device into a general vehicle number. A first binarizing device for binarizing under conditions for plates and binarizing the captured images from the image storage device under conditions for license plates for business vehicles A second binarizing device, the first and second binarizing devices for each of the captured images extract the character candidates from the two types of binarized images, and the corresponding point detecting device includes: The configuration for detecting the corresponding point between the adjacent captured images using each of the character candidates extracted from the two types of the binarized images for each captured image is a general vehicle / business Any license plate for cars can be used, and to detect corresponding points in two types of images, the registration at the time of image synthesis is performed with high accuracy, and a license plate using a highly reproducible synthesized image is used. Is preferable because the recognition processing can be performed.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings. 1 to 3 show a first embodiment of a license plate recognition method and a license plate recognition device according to the present invention.

In FIG. 1, a license plate recognition device 10 includes an imaging device 11 installed to capture an image of a vehicle M including a license plate, and the imaging device 1.
The image processing device 12 is configured by an image processing device 12 that processes a captured image, the image processing device 12 stores an image storage device 13 that stores the captured image, and an image cutout device that extracts a license plate image of the vehicle M from the captured image. 14 and
An image correction device 15 that detects and corrects the distortion of the license plate image based on the position and size of the character in the cut-out license plate image, and an image normalization device that normalizes the size of the corrected license plate image. Device 16
And character recognition processing on the corrected and normalized license plate image to recognize the character information on the license plate.
And a character recognition device 17 for reading.

The image pickup device 11 is installed so as to pick up an image including a license plate of a vehicle M passing through a passage (not shown) somewhere in the image. After that, the image clipping device 14 clips the image of the license plate portion of the vehicle M from the stored captured image by a known method, and transfers the image to the image correction device 15. Note that the imaging device 11 does not necessarily need to be installed so as to be able to image all vehicles M in a place where there is a possibility of passing.
The license plate at the position where most of the vehicles M pass may be set to be the center of the captured image so that a certain range can be covered.

When the image pickup device 11 is installed so as to pick up an image of the vehicle M from an oblique lateral direction, for example, the license plate image P11 is distorted as shown in FIG. Therefore, the distortion is removed and the characters are corrected so that the characters become evenly sized, and the image normalizing device 16 sets the corrected license plate image P14 to have a fixed size. The image is enlarged and normalized, and the character recognition device 17 recognizes character information and the like on the license plate by the character recognition processing after the correction and normalization.

More specifically, an example in which an image is corrected and normalized using a 4-digit serial number on a license plate will be described.

For example, the image correcting device 15 cuts out a serial number from the license plate image P11 by a known character cutting-out method, and then checks the size and position of each number, and determines the size according to the position of each number. The photographing direction of the license plate is determined from, and if the size of each of the serial numbers decreases toward the right of the image P11, it is determined that the photograph was taken from the front right of the vehicle M. If it is larger, it is determined that the image was taken from the left front. In addition, the cutout from the image of the character or the like is performed, for example, by using so-called labeling to binarize the license plate image P11 and determine that the size and the aspect ratio of the black pixel portion are within the set range as the serial number. You can also cut out.

When it is determined that the image is photographed from the right side, the image correction device 15 displays the leftmost numeral C1.
When it is determined that the image was taken from the left side, the Y axis is matched with a line passing vertically through the center of the number C1 or C4 with reference to the rightmost number C4, and X is drawn on a line passing horizontally through the lower end. By imagining the XY coordinates for matching the axes, a straight line L1 connecting the upper ends of the serial number numbers C1 to C4 and a straight line L2 connecting the lower ends are set, and the coefficients and constants of the following equations representing these straight lines L1 and L2 are obtained. Straight line Y _L1 = a1 · X (x) + b1 straight line Y _L2 = a2 · X (x) The elongation ratio and the conversion formula shown in the following formula are calculated by multiplying the y coordinate of each x coordinate of the license plate image P11 by: Elongation rate c = b1 / ((a1-a2) × x) Conversion equation y ′ = y × c (x) Conversion is performed to make the straight lines L1 and L2 parallel, and the license plate image 2 shown in FIG. To be corrected.

Similarly, since the serial number after this correction is distorted in the horizontal direction and the horizontal width is changed, the image correction device 15 also has the same ratio as the x-coordinate of the license plate image at the same ratio. Is corrected by the following conversion equation multiplying the expansion rate of the conversion equation, and the conversion equation x ′ = x × c (x) FIG. 3 (a) in which the numbers C1 to C4 of the serial number are equal in length and width. A correction process for making the license plate image P13 shown is performed.

Further, in the image correction device 15, the size of the serial number in the license plate image P13 is unified as shown in FIG. Since M is photographed obliquely from the horizontal direction, the distortion reduced in the horizontal direction from the number C1 based on the size remains, so the license plate image P
13 is also performed so that the aspect ratio becomes the same as the original aspect ratio. For example, the ratio of the width Wx1 to the height Wy1 in the circumscribed rectangle S1 of the corrected license plate portion is determined. , As shown in FIG.
The width W of the actual license plate in the circumscribed rectangle S2
The license plate image P14 is obtained by performing image correction processing so that x2 and the vertical width Wy2 are obtained, thereby correcting distortion caused by the image capturing apparatus 11 capturing an image in an oblique direction.

Further, the image normalizing device 16 outputs the license plate image P14 corrected by the image correcting device 15.
Since the size of the vehicle M in the captured image differs depending on the imaging distance and the imaging conditions such as the model of the imaging device 11, this is normalized to a certain size, and the license plate image P14 By making the size constant, the influence of the photographing conditions when the character recognition device 17 performs the character recognition processing is reduced.

The character recognition device 17 performs character recognition on the license plate image normalized by the image normalization device 16 to accurately recognize and read the serial number and number information displayed on the license plate. can do.

For this reason, the license plate recognition device 10
First, the image cutout device 14 extracts (cuts out) the license plate image P11 from the captured image of the vehicle M captured by the imaging device 11, and outputs characters such as serial numbers and numbers one by one from within the license plate area. Cut out and check its position and size, and then the image correction device 1
After the characters 5 having the same size on the license plate have the same size on the license plate, each portion in the license plate area is expanded and corrected to the license plate images P12 and P13, and the corrected number is obtained. Plate image P
The aspect ratio of the circumscribed rectangle of the 13 license plate areas is checked, the image is expanded so that the original ratio is obtained, and then the size of the license plate image recognized by the image normalizing device 16 becomes constant. After that, the distortion caused by the character recognition device 17 obliquely photographing the license plate is corrected, and the character recognition processing of the license plate image of a fixed size is always performed regardless of the photographing conditions. The information on the license plate of the vehicle M can be easily and accurately recognized and acquired.

It should be noted that the license plate image correction
The normalization processing is not limited to the above. For example, the width of each number C1 to C4 and the width of the space between characters are set based on the width of one number (C1 in FIG. 2A) in the license plate image P11. In addition to calculating the ratio, the original license plate is determined by the rules for character width and character spacing, and the ratio is also known, so compare both to determine the expansion rate at each point and correct the license plate image Normalization may be performed, and even more accurate correction can be performed by using not only the four-digit serial number numeral portion but also characters on other license plates in a similar manner.

As described above, in the present embodiment, the license plate distortion is detected and corrected from the captured image irrespective of the parameters such as the camera installation conditions, and the size is normalized to obtain the license plate number. The plate can be recognized accurately.

Therefore, the versatility of the license plate recognition process can be improved by reducing the restrictions imposed by the installation position of the imaging device 11 in the license plate recognition process.

Next, FIGS. 4 and 5 show a second embodiment of a license plate recognition method and a license plate recognition apparatus according to the present invention.

In FIG. 4, a license plate recognition device 20 is constructed by a plurality of imaging devices 21a to 21n for capturing images of the front side of a vehicle M including a license plate N, and an image processing device 22 for processing the captured images. The image processing device 22 includes the imaging devices 21 a to 21 a.
1n, and a plurality of feature extraction devices 24a for extracting character candidates displayed on the vehicle M including the license plate N from each of the captured images. ~ 24n
And corresponding point detection devices 25a to 25 (n-1) for detecting corresponding points (coordinates) between the captured images based on character candidates extracted from the captured images of the adjacent imaging devices 21a to 21n.
And the image storage device 23 based on the detection result of the corresponding point.
a to 23n, an image synthesizing device 26 for synthesizing adjacent captured images, an image clipping device 27 for cutting out an image of a license plate N from the synthesized image, and performing character recognition processing on the cut out license plate image. A character recognition device 28 that recognizes character information on the license plate N.

The image pickup devices 21a to 21n overlap the image pickup ranges with each other in an area smaller than the license plate N so as to effectively use the image pickup ranges of the adjacent objects, and pass through a wide passage (not shown). The image storage device 23 is installed such that any one of them captures an image of the license plate N attached to any height of the vehicle M to be driven.
a to 23n are connected to the imaging devices 21a to 21n, respectively.
21n captured images (including the vehicle M) are stored.

The feature extraction devices 24a to 24n extract character candidates from the captured images stored in the image storage devices 23a to 23n. For example, FIG.
As shown in, when the captured images P21 and P22 are stored in the image storage devices 23a and 23b, the feature extraction device 24a outputs “horizontal”, “beach”,
While "O", "2", "3", and "-" are extracted, the feature extraction device 24b extracts "Hama", "3",
“0”, “0”, “3”, “−”, “4”, and “5” are extracted. For example, as a method of extracting a character candidate, a method of labeling an image and extracting a character having the same size, area, and aspect ratio as characters on the license plate N may be used.

Corresponding point detecting devices 25a to 25 (n-1)
Focuses on an area on the side of the image so that adjacent captured images stored in the image storage devices 23a to 23n are overlapped with each other, and overlaps images in a state where the images are matched by character candidates extracted in the area. The corresponding points that can be detected are detected. For example, the corresponding point detection device 25a is configured as shown in FIG.
From the captured images P21 and P22 shown in FIG.
As shown in the figure, the captured image P2
“Hama”, “3”, extracted from the area p23 where 1 overlaps,
“−”, “Hama”, “3”, “Hama” extracted from the area p24 where the captured image P22 overlaps by the feature extraction device 24b.
A corresponding point overlapping “−” is detected.

The image synthesizing device 26 includes an image storage device 23a.
To 23n are overlapped with the corresponding points detected by the corresponding point detection devices 25a to 25 (n-1), for example, the captured images P21 and P2.
2 is a character candidate "Hama", as shown in FIG.
The regions p23 and p24 are overlapped by being shifted in the vertical direction so that “3” and “−” are overlapped, and the captured images P2i and P2j for which corresponding points could not be detected are overlapped with the top and bottom to form one sheet. The image is synthesized with the captured image P25. The image cutout device 27 outputs the composite image P25
, An image of the license plate N is cut out by a known method, and the character recognition device 28 performs character recognition processing on the image of the license plate N to recognize and read the character information on the license plate N.

More specifically, as shown in FIG. 5A, the imaging devices 21a and 21b capture an image of the vehicle M including the license plate N and store the captured images P21 and P22 in the image storage devices 23a and 23b. The case where the information is stored will be described as an example.

The feature extraction devices 24a to 24n are, for example,
In the captured images stored in the image storage devices 23a to 23n, the character candidate area is determined from the shading change pattern of the character displayed on the license plate N, or by using an edge present in the outline of the character. Extract.
For example, the feature extracting device 24a determines “horizontal”, “beach”, “oh”, “2”, “3”, “3” from the captured image P21 stored in the image storage device 23a shown in FIG. −), The feature extraction device 24b extracts the image storage device 23
From the captured image P22 stored in b, “Hama”,
"3", "0", "0", "3", "-", "4",
"5" is extracted. Note that extraction of character candidates is not limited to the method described here.

Corresponding point detectors 25a to 25 (n-1)
Is calculated in advance in all the overlapping regions between the adjacent captured images in the image storage devices 23a to 23n by calculating the correlation of the degree of coincidence of the images while shifting the overlapping regions little by little in the vertical direction. The corresponding point that is the same or more than the threshold value is set as the corresponding point. For example, as shown in FIG.
The character candidates "hama" extracted by the feature extraction devices 24a and 24b by superimposing the regions p23 and p24 of P21 and P22,
A corresponding point where "3" and "-" overlap is detected. As a method of calculating the correlation, a typical one has a normalized correlation. However, when the image of the character candidate is binarized, the pixel values of the two images are more simply matched. The ratio of pixels may be counted.

The image synthesizing device 26 overlaps the side regions of the plurality of captured images in the image storage devices 23a to 23n at the corresponding points detected by the corresponding point detecting devices 25a to 25 (n-1). At this time, as a pixel value to be used as an image of the overlapping area, either one of the pixel values is preferentially employed, or a pixel value obtained by calculating an average of both or a pixel obtained by weighting and adding both values The captured images P21 and P22 in which the corresponding points are detected by adopting values, for example.
In the space, as shown in FIG. 5B, the side area is overlapped by the corresponding point. The upper and lower edges of the captured images for which a corresponding point could not be detected (there was no need to adjust) are made to match, and all the captured images are combined into one composite image P25. Thereafter, the image cutout device 27 cuts out a license plate image from the composite image P25, and the character recognition device 28 recognizes and reads a serial number and the like on the license plate N.

Note that the image synthesis is performed without using the images in the image storage devices 23a to 23n described above.
1, P22 may be used as it is. In this case, the composite image is as shown in FIG. If the image cutout device 27 cuts out a license plate image based on such character candidates, the image storage devices 23a to 23-2
When a composite image is created using the 3n image, character candidates must be newly extracted. However, when combining using a character candidate image, the character candidate extraction process is omitted and a license plate image is cut out. Can be simplified.

For this reason, the license plate recognition device 20
First, an image of the vehicle M passing by the imaging devices 21a to 21n arranged so as to overlap with each other on the side regions set to have a smaller width than the license plate N is reliably captured. Next, a character candidate such as a serial number on the license plate N is extracted by extracting a character candidate from each of the plurality of captured images. Thereafter, using the extracted character candidates, a corresponding point at which a character image in a portion where the side region of an adjacent captured image overlaps is matched and overlapped is detected, and a plurality of captured images are detected based on the corresponding point. Can be combined into one sheet. Subsequently, a license plate recognition method of extracting (extracting) the image of the license plate N from the composite image P25 using the extracted character candidates and recognizing and reading the serial number and the like on the license plate N is executed. The information on the license plate N of the vehicle M can be easily used by effectively using the imaging range of the plurality of imaging devices 21a to 21n.
And it can be recognized and acquired with high accuracy.

As described above, in the present embodiment, in order to effectively use the images picked up by the plurality of image pickup devices 21a to 21n, the side of the image pickup range is set to the license plate N.
By setting to overlap in a smaller area,
Even when the image of the license plate N to be recognized straddles the adjacent captured images P21 and P22, a plurality of captured images are combined into one, and the license plate N is reliably cut out to extract a serial number and the like.・ We can recognize.

Therefore, a plurality of image pickup devices 21a to 21a-2
1n, a wide range can be efficiently captured, and the captured images to be recognized can be combined into one to recognize the character information of the license plate by a simple and efficient recognition process. The versatility of the recognition processing can be improved.

Further, since the synthesis is performed using the character candidates of the license plate N as the feature points, a synthesized image in which the license plate portion best matches after the synthesis can be obtained.
In general, it is possible to reduce the influence of discontinuity at a synthesis boundary, which is likely to occur in image synthesis, to obtain a synthesized image suitable for license plate recognition.

As another aspect of the present embodiment, processing is performed only on a region where the side of the captured image overlaps, and the entire image is combined into one using the corresponding points detected from the region. You may do so. In this case, the processing amount for creating one composite image can be reduced. However, a method in which the character candidates described above are also used for cutting out the license plate N cannot be adopted.

Next, FIG. 6 is a view showing a third embodiment of a license plate recognition method and a license plate recognition apparatus according to the present invention. Note that the present embodiment is configured substantially in the same manner as the above-described second embodiment, and therefore, the same components are denoted by the same reference numerals and the drawings are diverted to describe the characteristic portions (other components described below). The same applies to the embodiment).

In FIG. 6, the license plate recognition device 30 includes image storage devices 23a to 23n, feature extraction devices 34a to 34n, and corresponding point detection devices 25a to 25 (n−
1), image synthesizing device 26, and image clipping device 27
Processing device 32 including a character recognition device 28
Are connected to the imaging devices 21a to 21n, and the feature extraction devices 24a to 24n in the second embodiment described above.
The feature extraction devices 34a to 34n in place of 4n extract binarization devices 35a to 35n that convert captured images stored in the image storage devices 23a to 23n into binary images, and extract character candidates from the images. Character candidate extraction devices 36a-3
It is interposed in front of 6n.

Therefore, the feature extraction devices 34a to 34n
Then, since it is sufficient to perform a process of extracting a character candidate from a binarized captured image, the process can be performed faster than a case of extracting a character candidate from an original image. (N-1), the image storage device 23a
To 23n can be processed at high speed while slightly shifting the overlapping area between adjacent captured images in the vertical direction.

For this reason, the license plate recognition device 30
Performs binarization processing for each of the plurality of captured images after the imaging of the passing vehicle M by the imaging devices 21a to 21n.
Character candidates can be extracted at high speed with a small amount of processing, and subsequently, corresponding points that can be matched and overlapped in the side regions of the binarized captured image can be similarly quickly processed with a small amount of processing. Can be detected.

As described above, in the present embodiment, in addition to the operation and effect of the second embodiment, the captured image to be processed is binarized to extract character candidates from the captured image and synthesize the captured image. The number of processes for detecting corresponding points to be performed can be reduced, and the license plate recognition process can be completed more quickly.

Therefore, the versatility of the license plate recognition processing can be further improved by eliminating the application limit due to the recognition processing speed.

FIGS. 7 and 8 show a license plate recognition method and a license plate recognition apparatus according to a fourth embodiment of the present invention.

In FIG. 7, the license plate recognition device 40 includes image storage devices 23a to 23n, feature extraction devices 24a to 24n, and determination devices 43a to 43 (n-1).
And an image processing device 42 including a corresponding point detecting device 25a to 25 (n-1), an image synthesizing device 26, an image segmenting device 27, and a character recognizing device 28.
21n, and the image processing device 42
Are provided with determination devices 43a to 43 (n-1) in front of the corresponding point detection devices 25a to 25 (n-1).

The judging devices 43a to 43 (n-1) receive the extraction results of the character candidates by the feature extracting devices 24a to 24n, and according to the state of the region of the character candidates, the corresponding point detecting devices 25a to 25 (n-1). When it is determined whether or not to perform a process of detecting a corresponding point in a side region in order to perform synthesis between adjacent captured images according to n-1), and it is determined that the process of detecting the corresponding point is necessary. In the same manner as in the second embodiment, the captured images in the image storage devices 23a to 23n are transferred to the corresponding point detection devices 25a to 25 (n-1). If it is determined that the corresponding point detection processing is unnecessary, the captured image is transferred to the image synthesizing device 26 as it is so as to be synthesized vertically, and the corresponding point detecting devices 25a to 25 (n-
1) is bypassed, and the necessity of the captured image itself is also determined. If it is determined that the captured image is unnecessary, the captured image itself is disposed of.
(N-1) is set so as to determine the necessity of corresponding point detection processing as follows.

When there is a character candidate area in a portion overlapping with an image of an adjacent imaging range, for example, as shown in FIG.
When a character candidate exists in 43 and p44, it is determined that detection processing of a corresponding point from within the areas p43 and p44 is necessary to overlap the areas p43 and p44 and synthesize the captured images P41 and P42. .

When there is not a sufficient character candidate area in a portion overlapping an image in an adjacent imaging range, for example, a preset threshold value such as the number of detected character candidates, the area, and the positional relationship between the character candidates is set to If a character candidate that exceeds the number of characters is not extracted, as shown in FIG.
5. It is determined that there is no character candidate in the area p47 or p48 where the side of P46 overlaps, and that there is a character candidate in another area, and it is not necessary to perform processing for detecting a corresponding point from the area p47 or p48. Is determined.

If there is not a sufficient character candidate area in all the areas in the captured image, that is, if there is a captured image from which a character candidate exceeding the threshold cannot be extracted, for example, FIG. ), The side region p52
In the picked-up images P49 to P51 that overlap with p55 to p55, since the picked-up image P50 has no character candidate itself, it is determined that the picked-up image P50 is unnecessary for this license plate recognition process and is excluded from the processing target. It is determined that the process of superimposing and combining the side regions p52 and p55 is effective.

Therefore, the corresponding point detecting devices 25a to 25a
In (n-1), in order to combine adjacent captured images, it is possible to prevent unnecessary processing for detecting a corresponding point from a side area to be overlapped even though no character candidate is included. Also, in the image synthesizing device 26 and the image clipping device 27, it is possible to avoid unnecessary processing of a captured image that does not include a character candidate, that is, does not include an image of the license plate N.

For this reason, the license plate recognition device 40
Is a process for detecting a corresponding point for overlapping side regions of adjacent captured images in accordance with a result of extracting character candidates from the captured images, which is performed subsequent to the imaging of the passing vehicle M by the imaging devices 21a to 21n. By determining the necessity of the captured image itself together with the necessity, only the necessary corresponding points are detected, and only the required captured image can be synthesized at high speed.
Thereafter, as in the second embodiment, the license plate recognition method of cutting out (extracting) the image of the license plate N and recognizing and reading the license plate N can be completed quickly.

As described above, in the present embodiment, in addition to the operation and effect of the second embodiment described above, only the necessary captured image and its side region are processed, and the corresponding points from the side region of the captured image are processed. To reduce the amount of processing when detecting the license plate and cutting out the license plate N from the captured image,
The license plate recognition process can be completed at a higher speed.

Next, FIG. 9 is a diagram showing a fifth embodiment of a license plate recognition method and a license plate recognition apparatus according to the present invention.

In FIG. 9, the license plate recognition device 50 includes image storage devices 23a to 23n shown in FIG.
Corresponding point detection devices 25a to 25 (n-1), image synthesis device 26, image cutout device 27, and character recognition device 28
In the image processing device connected to the imaging devices 21a to 21n, the feature extraction device 24 according to the second embodiment is provided.
a to 24n instead of the feature extraction devices 54a to 54n (5
4a) (shown only in FIG. 4a), and the feature extraction devices 54a to 54n include the image storage devices 23a to 23n.
3n, the images captured by the imaging devices 21a to 21n are
After the binarization is performed by the first binarization device 55 under the condition for a general vehicle license plate, character candidates are extracted by the character candidate extraction device 57 from the binarized captured image, and similarly, image storage is performed. After the captured images in the devices 23a to 23n are binarized by the second binarization device 56 under the conditions for a license plate for business vehicles, the character candidate extraction device 58 uses the character candidate extraction device 58 from the binarized captured image. Are provided.

Here, the license plate for general vehicles has a color display in which characters are displayed in green on a white background or characters are displayed in black on a yellow background, whereas the license plates for business vehicles are displayed in white or on a green background. The colors are displayed in yellow on a black background, and the brightness is reversed for general vehicles and business vehicles. At the time of imaging by the imaging devices 21a to 21n, since it is not possible to determine whether the license plate N of the vehicle M is for a general vehicle or a business vehicle, the first and second binarization devices 55, 5
6, the binarization or inversion binarization processing is separately performed under both conditions, and character candidates are extracted from the binarized separate captured images (binary images) by the character candidate extraction devices 57 and 58, respectively. I do.

Since the character candidate extracting devices 57 and 58 need only perform the process of extracting the character candidates from the binarized image of the captured image, it is possible to process the two captured images at high speed. , And the corresponding point detection devices 25a to 25
In the case of (n-1) as well, a corresponding point for overlapping captured images can be detected from the side region of the same type of binarized image at high speed and with high accuracy.

For this reason, the license plate recognition device 50
After the passing vehicle M is imaged by the imaging devices 21a to 21n, two types of binarization processing are performed for each of the plurality of captured images, so that character candidates can be extracted with high accuracy and high speed with a small processing amount. The corresponding points that can be matched and overlapped from the side regions of the binarized captured image can be similarly detected at high speed with a small amount of processing, and thereafter, As in the second embodiment,
The license plate recognition method of creating a composite image of the captured image based on the corresponding points, cutting out (extracting) the image of the license plate N, and recognizing and reading the license plate N can be quickly completed.

As described above, in the present embodiment, in addition to the operation and effect of the second embodiment, the captured image is binarized by two types of threshold values for a general vehicle license plate and a business vehicle license plate. By performing the conversion process, it is possible to extract character candidates from the captured image and to detect corresponding points for synthesizing the captured image, so that the license plate recognition process can be completed more accurately and quickly.

Therefore, the versatility of the license plate recognition processing can be further improved by eliminating the application limit due to the recognition processing speed.

[0072]

As described above, the present invention detects and corrects the image distortion based on the size of the license plate in the captured image of the vehicle and the position and size of the character on the license plate. At the same time, the size of the image of the license plate can be normalized to a certain value,
Without setting parameters for each installation condition in advance,
Regardless of the imaging direction or imaging distance, character information on the license plate can be recognized with high accuracy by a simple recognition process from a license plate image of a fixed size without distortion,

Even when the imaging range is overlapped to the extent that the license plate straddles an adjacent captured image, the license plate extraction / recognition processing is performed after combining them into one by using corresponding points of overlapping character candidates in the captured image. The character information on the license plate can be efficiently recognized by a simple recognition process while ensuring a wide imaging range by effectively using a plurality of captured images.

Therefore, in the license plate recognition processing, restrictions due to the installation position of the imaging device, the imaging range, and the like can be reduced, and the license plate recognition processing can be generalized.

[Brief description of the drawings]

FIG. 1 is a diagram showing a first embodiment of a license plate recognizing apparatus for performing a license plate recognizing method according to the present invention, and is a block diagram showing an overall configuration thereof.

FIG. 2 is a diagram showing a captured image for explaining the recognition processing.

FIG. 3 is a diagram showing a captured image for explaining the recognition processing.

FIG. 4 is a diagram illustrating a second embodiment of a license plate recognition apparatus that performs a license plate recognition method according to the present invention, and is a block diagram illustrating an overall configuration thereof.

FIG. 5 is a diagram showing a captured image for explaining the recognition processing.

FIG. 6 is a diagram showing a third embodiment of a license plate recognizing device that performs a license plate recognizing method according to the present invention, and is a block diagram showing an entire configuration thereof.

FIG. 7 is a diagram showing a fourth embodiment of a license plate recognizing device for performing a license plate recognizing method according to the present invention, and is a block diagram showing an entire configuration thereof.

FIG. 8 is a diagram showing a captured image for explaining the recognition processing.

FIG. 9 is a diagram showing a fifth embodiment of a license plate recognition apparatus that performs a license plate recognition method according to the present invention, and is a block diagram illustrating a main configuration thereof.

FIG. 10 is a diagram illustrating the related art (first conventional example).

FIG. 11 is a diagram illustrating the related art (second conventional example).

[Explanation of symbols]

10, 20, 30, 40, 50 License plate recognition device 11, 21a to 21n Imaging device 12, 22, 32, 42 Image processing device 13, 23a to 23n Image storage device 14, 27 Image cutout device 15 Image correction device 16 Image Normalization device 17, 28 Character recognition device 24a to 24n, 34a to 34n, 54a Feature extraction device 25a to 25 (n-1) Corresponding point detection device 26 Image synthesis device 35a to 35n Binarization device 36a to 36n, 57, 58 Character candidate extraction device 43a to 43 (n-1) Judgment device 55 First binarization device 56 Second binarization device

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G08G 1/017 G08G 1/017 1/04 1/04 CF term (Reference) 5B029 BB02 CC30 EE04 EE05 EE06 EE15 5B057 AA16 BA02 CA12 CA16 CB12 CB16 CD12 CE10 5H180 CC04 CC30 DD10

Claims (10)

[Claims] 1. A license plate recognition method for recognizing information on a license plate to be read from a captured image of a vehicle, comprising: extracting a license plate region from the image of the vehicle; Cut out the characters on the license plate, correct the distortion of the image of the license plate taken based on the size of the license plate area and the position and size of the cut out characters on the license plate and correct the distortion of the license plate. A license plate recognition method comprising: normalizing the size of an image and recognizing information on the license plate. 2. A license plate recognition device for recognizing information on a license plate to be read from a captured image of a vehicle, comprising: an image storage device configured to store an image of the vehicle captured by an imaging device; An image cutout device that obtains a captured image from an image storage device and cuts out a license plate image of the vehicle, a size of the license plate image cut out by the image cutout device, and a position and a size of a character in the license plate image An image correction device that detects distortion of the license plate image based on the image correction device and corrects the license plate image, and an image normalization device that normalizes the size of the license plate image corrected by the image correction device. A license plate recognition device, comprising: 3. A license plate recognizing method for recognizing information on a license plate to be read from a captured image of a vehicle, comprising: a plurality of license plates having edges of an imaging range overlapped with each other so as to include the vehicle as an imaging target. Images are captured simultaneously, character candidates on the license plate are extracted for each of the plurality of captured images, and corresponding points between the adjacent captured images are detected based on the extracted character candidates. Combining the plurality of captured images captured based on the corresponding points of the character candidates into one, extracting a license plate area from the composite image, and recognizing information on the license plate. License plate recognition method. 4. A license plate recognizing device for recognizing information on a license plate to be read from a captured image of a vehicle, wherein the license plate recognition device is installed such that edges of adjacent imaging ranges overlap with each other. A plurality of imaging devices that simultaneously capture a plurality of images so as to include, an image storage device that stores images captured by the imaging device, and character candidates that are obtained by obtaining the captured images from the image storage devices. A feature extraction device to be extracted, a corresponding point detection device that detects a corresponding point between the images captured by the adjacent imaging devices based on the character candidates extracted by the feature extraction device, and a corresponding point detection device An image synthesizing device for synthesizing the images captured by the plurality of image capturing devices into one based on the detected corresponding points; An image cutout device that extracts a bar plate image, and a character recognition device that performs character recognition processing on the license plate image extracted by the image cutout device and reads information on the license plate, License plate recognition device. 5. The character candidate on the license plate is extracted from a binarized image binarized for each of the plurality of captured images, and the corresponding point between the captured images is extracted from the extracted binary image. 4. The license plate recognition method according to claim 3, wherein the detection is performed using a value character candidate. 6. The feature extracting device includes a binarizing device that binarizes the captured image from the image storage device, and extracts the character candidate from a binarized image of the captured image. 5. The apparatus according to claim 4, wherein the corresponding point detecting device detects the corresponding point between adjacent captured images using the character candidates extracted from the binarized image.
License plate recognition device according to 1. 7. The detection of the corresponding point between the captured images and the imaging only when the character candidate to be processed is extracted at a portion where the imaging range between the adjacent captured images overlaps. The license plate recognition method according to claim 3, wherein images are combined. 8. A determining device for receiving a result of extracting the character candidate by the feature extracting device and determining a processing direction of the captured image, wherein the image capturing by the corresponding point detecting device is performed based on the determination result by the determining device. The license plate recognition device according to claim 4, wherein the corresponding point is detected between images and the captured image is synthesized by the image synthesis device. 9. The character candidate on the license plate is obtained by binarizing a plurality of the captured images under two conditions, one for a general vehicle license plate and the other for a business vehicle license plate. And extracted from each of the two types of binarized images for each of the captured images, and the corresponding point between the captured images is 2 for each of the captured images.
8. The license plate recognition method according to claim 3, wherein the detection is performed using each of the binary character candidates extracted from the binarized image of the seed. 10. A first binarizing device for binarizing the captured image from the image storage device under conditions for a general vehicle license plate, and the captured image from the image storage device. A second binarizing device for binarizing the character under conditions for a business vehicle license plate, and the character from the two types of binarized images by the first and second binarizing devices for each of the captured images Along with extracting a candidate, the corresponding point detection device uses each of the character candidates extracted from the two types of the binarized images for each of the captured images to determine the corresponding point between the adjacent captured images. 9. The license plate recognition device according to claim 4, wherein detection is performed.