JP4224328B2 - Car number recognition device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a monitoring system using a television camera applied to a road or the like, and more particularly to a vehicle number recognition device that captures characters by photographing a license plate of an automobile.
[0002]
[Prior art]
In recent years, a system that recognizes the car number of a car using a television camera (television camera) for grasping road traffic conditions and statistical management has become widespread. Is installed, the license plate of the passing vehicle is photographed, and the number is automatically recognized by character recognition.
[0003]
Therefore, in this system, a certain degree of high resolution is required for reading an image by the television camera. For this reason, if a normal NTSC system television camera is used, a field of view width of only about 1000 mm can be secured.
[0004]
Therefore, in order to monitor the entire road width, it is common to use a dedicated high-definition TV camera. In this case, however, the TV camera becomes expensive, and the signal input system for image processing has a special specification. In addition, since the number of pixels to be processed becomes enormous, an extremely high-speed image processor is required.
[0005]
Here, also in the prior art, an apparatus for recognizing a car number using a single television camera has been proposed (see, for example, Patent Document 1).
[0006]
Therefore, this conventional technique will be described below. Here, first, FIG. 7 is shown. In this example, a TV camera 40, an A / D converter 41 that converts an analog video signal output therefrom to a digital signal, and a digital video signal are recorded. An image memory 4732, an image processor 43, and a CPU 44 that executes various operations and controls various hardware.
[0007]
Here, the TV camera 40 is of a general NTSC system, and the image memory 42 can store a normal TV camera image, an image in the process, and the like in this case. A plurality of memories having a storage capacity of about 512 × 512 are provided.
[0008]
The image processor 43 also performs local maximum value filter processing and local minimum value filter processing of the image, difference processing for each pixel, smoothing processing, differentiation processing, binarization processing, binarized image expansion processing, and labeling processing. It performs functions necessary for general image processing such as histogram processing.
[0009]
In this prior art, the car number recognition process is performed by first cutting out a plate from the image, then cutting out a serial number of up to 4 digits, and following this, various code areas (land transportation branch code of place name, vehicle type code, use code) (Area etc.) is cut out, and each character is recognized according to a series of flows.The calculation of various code areas at this time is obtained from the size of the serial numbers and their positions. ing.
[0010]
Therefore, the process according to a series of flows at this time will be briefly described with reference to FIG. 8. This process includes a plate extraction unit 1 that extracts a plate region from an input image and a serial number that extracts a serial number in the plate. Executed by the extraction unit 2, the code area coordinate calculation unit 3 for calculating the coordinate positions of the land transportation branch office code, the vehicle type code, and the use code from the extracted serial number coordinates, and the character recognition processing unit 4 for recognizing each code. Is done.
[0011]
As a result, since the character recognition processing unit 4 knows in advance what kind of character exists for each code area, the character recognition unit 5, the land transportation branch code (location name) recognition unit 6, and the application code ( Hiragana) It is divided into the recognition part 7.
[0012]
Next, the plate recognition processing according to this prior art will be described. Here, the license plate of the automobile will be described first. FIG. 9 shows the shape of the license plate in Japan (Japan). This is composed of a serial number 11 of up to four digits, a land transportation branch office code 12 of a place name, a vehicle type code 13, and a use code 14.
[0013]
At this time, as shown in FIGS. 9 (a) and 9 (b), there are two types of Japanese plates, medium size and large size. The size of the medium size serial number 11 is 40 × 80 mm, The size is determined to be 60 × 120 mm, but the land transportation branch code 12, vehicle type code 13, and application code 14 are 40 mm in height for both large and medium types. Therefore, the height of the serial number is 2 of the other codes. It is more than double the size.
[0014]
As shown in FIG. 8, the license plate recognition procedure is a procedure of plate cutting, serial number extraction, code area coordinate calculation, and character recognition of each code area. Therefore, hereinafter, the license plate recognition procedure will be sequentially described with reference to the processing flow of FIG.
[0015]
At this time, the imaging field size (or resolution) is given as a parameter in advance, and threshold values (hereinafter referred to as threshold values) for the plate size, character size, etc. are stored in advance.
[0016]
(1) Image input (not shown)
An image captured by the television camera 40 is taken into the image memory 32. Here, it is assumed that an NTSC system camera is used as the TV camera 40, and an image is captured with a width of 512 pixels and a height of 240 pixels.
[0017]
(2) Plate cutting (process 15 in FIG. 10)
Where the plate is located in the captured image varies depending on the traveling position of the automobile and the mounting position of the plate. Therefore, it is necessary to recognize the position of the plate from the input image. There are various methods for this purpose, but here, a method is adopted in which a region having a strong edge is extracted and shaped to detect the plate.
[0018]
For this reason, the threshold values used for extraction include a minimum and maximum plate width, a minimum and maximum height, and the detailed description is omitted.
[0019]
(3) Extraction of serial number (process 16 in FIG. 10)
A serial number is extracted as a binary image for the plate area. There are various binarization methods at this time, but detailed description thereof is omitted here. Then, an object having a size within a specified range (a range of a field number set in advance or a sequence number determined by resolution) is extracted from the binary image, and only the objects arranged in a line are serial number candidates. Extract as
[0020]
When extracting the serial number, the area coordinates (X start point, X end point, Y start point, Y end point) of each character are obtained, and the center X, Y coordinate, character height, width, etc. of each character are obtained from the values. .
[0021]
(4) Determination of plate size (process 17 in FIG. 10)
As shown in FIG. 9, the position coordinates of each code differ between the large plate and the medium plate. Therefore, since the relative distance from the coordinate position of the extracted serial number is also different, it is necessary to determine the plate size.
[0022]
The specific processing at this time is processing that uses the average height of the characters with serial numbers, and determines that the plate is a large plate if the height of the characters is equal to or greater than a predetermined value (a threshold determined by a preset field size). It becomes.
[0023]
(5) Calculation of resolution in X direction and Y direction (process 18 in FIG. 10)
Since the size (number of pixels) of the serial number varies depending on the position at the time of image capture and the downward angle of the plate, it is necessary to calculate the vertical and horizontal resolution from the extracted number of pixels of the serial number.
[0024]
At this time, since the character width is 40 mm in the case of the medium plate, the resolution in the X direction is 40 mm / character width (number of pixels). On the other hand, the vertical resolution is 80 mm / character height (number of pixels) since the character height is 80 mm in the case of a medium-sized plate.
[0025]
In the case of a large size, similarly, the resolutions in the X and Y directions are calculated using values of width 60 mm and height 120 mm. Note that the horizontal resolution is not the character width as described above, but the calculation of the resolution from the first and fourth character intervals for four-digit characters and the three-digit interval for three-digit characters. There is little error.
[0026]
In case of 1 digit, it is calculated by the character width. At this time, the number “1” has a different width. Therefore, it is necessary to devise such as calculating from the distance between the dots or estimating from the character height. There are many, but it is unavoidable.
[0027]
(6) Coordinate calculation of each code area (process 19 in FIG. 10)
Since the position coordinates of each code are different between the large plate and the medium plate as described with reference to FIG. 9, the coordinates of each code area are calculated using the resolution in the X direction and Y direction calculated in the processing 18. For example, since the vehicle type code area is in the range of 40 mm height from 15 mm above the serial number, the pixel position is calculated by dividing 15 mm and 55 mm by the Y direction resolution (mm / pixel).
[0028]
Also, the distance from the first digit center coordinate from the left of the serial number to the center coordinate of the application code is 55 mm away and 40 mm wide in the case of a medium plate, so the horizontal range of the application code is a pixel in the range of 35 mm to 75 mm. Find from position.
[0029]
In the case of a large plate, since it is 40 mm wide at a distance of 87.5 mm, it is obtained from a pixel position corresponding to a range of 67.5 to 107.5 mm.
[0030]
As shown in FIG. 11, such processing is executed for all the code areas, and the coordinates of the land transportation branch office code 22, the vehicle type code 23, and the use code 24 are calculated. Note that the coordinates of each code area are set to an area wider than the actual dimension in consideration of error, positional deviation, plate bending, and the like.
[0031]
(7) Character area extraction (process 20 in FIG. 10)
The code area at this time is a slightly wider area in consideration of errors and the like as described above. For this reason, the actual region to be recognized is determined using the density projection distribution.
[0032]
In this process, as shown in FIG. 4, the code area coordinates 25 are set for the input image, and the density is accumulated in the vertical direction (x-axis projection distribution) and the horizontal direction (y-axis projection distribution), respectively. The projection distribution is obtained, the coordinates whose frequency exceeds a predetermined threshold value are obtained for each direction, and the coordinates of the character area 26 in the recognized range are obtained.
[0033]
(8) Character recognition (process 21 in FIG. 10)
Here, the character area of each code obtained as described above is recognized using, for example, a neural network. At this time, the license plate characters have fewer categories than general character recognition. Therefore, it can be recognized even at a relatively low resolution. For example, since the application code 14 is hiragana and has a size of 40 mm × 40 mm, it can be recognized even with about 10 × 10 pixels.
[0034]
In this way, if the area can be determined, even about 4 mm / pixel can be recognized, and the vehicle number can be recognized by one television camera 40. There are various other character recognition methods at this time, but they are omitted here.
[0035]
[Patent Document 1]
JP-A-6-215293
[0036]
[Problems to be solved by the invention]
In general, character recognition in a car number recognition device requires a resolution of 2 mm / pixel. Therefore, in the case of hiragana, since it is 40 mm × 40 mm, at least about 20 × 20 pixels are required.
[0037]
However, the character recognition itself after cutting out the character area is about 3 to 4 mm / pixel, that is, in the case of hiragana, it is 40 mm × 40 mm, so it is empirically known that there is no problem even with about 10 × 10-13 × 13 pixels. It has been.
[0038]
However, with such a low resolution of about 3 to 4 mm / pixel, it is difficult to cut out a character area, and as a result, character recognition cannot be performed.
[0039]
For this reason, when shooting at a resolution of 2 mm / pixel, for example, even when a high-definition camera of about 1000 × 500 pixels is used, the horizontal field of view of 2 m is full, and with an NTSC camera, 640 × 240 pixels Therefore, the field of view can be secured only about 1m horizontally.
[0040]
Moreover, since the actual road width is at least about 3 m, it is often overlooked in the monitoring range of 1 m. Therefore, in the prior art, it is necessary to install a plurality of NTSC cameras or use expensive high-definition cameras. .
[0041]
The character cut-out described above is determined by calculating a relative position to each code area calculated from the coordinates and size of the serial number. Therefore, for example, even when it is calculated that the number is 14.8 pixels to 20.4 pixels in the upward direction of the serial number, since the image data handles only integer values, the result is a range of 14 to 20 pixels. The higher the resolution, the greater the possibility that the area will shift.
[0042]
Even when the detailed area of each code is obtained with a density projection distribution, it is desirable to set the thresholds for determining the start and end coordinates of the projection distribution to real numbers, but errors occur because only integer values can be handled. Therefore, in the prior art, as a result, the region coordinates are misaligned, resulting in erroneous recognition.
[0043]
The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a vehicle number recognition device capable of recognizing a wide range with a single television camera.
[0044]
[Means for Solving the Problems]
In order to achieve the above object, the present invention has been made by paying attention to the fact that the size of the serial number in the license plate is different from the size of other codes such as the place name (land transportation branch office code), vehicle type code, and application code. For this reason, a series number of a relatively large character size is cut out from a video photographed with a resolution of about 3 to 4 mm / pixel, and the resolution (or field size) at that time is calculated, from the predetermined resolution (or field size) Is larger (when the plate is smaller than the predetermined size), the entire area of the license plate is enlarged to a predetermined size (for example, resolution of 2 mm / pixel), and the vehicle type code, land transportation branch code, and application code are An area is cut out and recognized.
[0045]
As described above, with a resolution of 2 mm / pixel, a normal plate can be recognized almost 100%. With such a method, a wide field of view can be monitored by an inexpensive TV camera, and the vehicle number is recognized. Will be able to.
[0046]
For example, in the case of an NTSC camera, conventionally, only a field of view of about 1 m can be monitored. However, according to the present invention, a field of view of about 1.5 m, for example, can be monitored. Degree) can be monitored, and oversight of a vehicle traveling with an inexpensive configuration can be greatly reduced.
[0047]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a vehicle number recognition apparatus according to the present invention will be described in detail with reference to an embodiment shown in the drawings. Here, FIG. 1 is a license plate recognition procedure according to an embodiment of the present invention. The hardware configuration is the same as that of the prior art described in FIG.
[0048]
Also in this embodiment of the present invention, the vehicle number recognition process is the same as the prior art described in FIG. 8, and the plate extraction unit 1 for extracting a region from the input image and the serial number in the plate A serial number extracting unit 2 for extracting a code, a code area coordinate calculating unit 3 for calculating each coordinate position of a land transportation branch code, a vehicle type code, and a use code from the coordinates of the extracted serial number, and a character recognition processing unit for recognizing each code 4 is executed by each unit.
[0049]
Accordingly, in this embodiment, as is apparent from FIG. 1, in the license plate recognition procedure in the prior art described with reference to FIG. 10, the following processing is added when the field of view is wide.
[0050]
As described above, in the license plate, the character size of the serial number is larger than that of the land transportation branch office code and the vehicle type code, and therefore, even if the field of view is slightly expanded, the characters are not easily crushed.
[0051]
Therefore, in the present invention, after the serial number is cut out, the current resolution is calculated, the entire area of the plate is enlarged so that the resolution becomes a predetermined value (for example, 2 mm / pixel), and the image is again processed. The serial number was cut out and the area settings for various code areas were executed.
[0052]
In this way, it is possible to determine the same area as a video shot with a resolution equivalent to that of a video shot with a resolution of 2 mm / pixel, and it can be recognized with a low resolution. It will come to an end.
[0053]
Hereinafter, the embodiment of the present invention will be described in detail with reference to the flowchart of FIG. In FIG. 1, the processes having the same numbers as those in FIG. 10 are the same as those in the prior art, so the description thereof will be omitted, and only the processes 30 to 34 added between the processes 17 and 18 will be described. .
[0054]
(1) Calculation of visual field size (process 30 in FIG. 1)
Similar to the prior art described in FIG. 10, processing 15 to processing 17 is completed, and the center X coordinate of each character of the serial number extracted in a wide visual field is obtained, and the visual field size in the horizontal direction (x direction) is obtained from this interval. Is calculated. The field size at this time is slightly different from the preset field size parameter. This is because the farther the plate loading position is, the larger the calculated visual field is.
[0055]
Therefore, the field of view size is calculated using the coordinates of the cut out serial number. Here, the character spacing differs depending on the plate size, so if the plate type is medium, the first digit and the fourth digit of the serial number. Since the distance between the eyes is 195 mm as shown in FIG. 9A, the number of pixels in the image memory is 512 pixels, and the number of pixels in the first and fourth digits of the extracted serial number is P. _1-4 Then, the visual field size S at this time _M Is calculated by the following equation 1.
[0056]
Field size S _M = 512 × (195 / P _1-4 ) ………… (Formula 1)
[0057]
When it is determined that the plate is a large plate, the distance between the first and fourth digits is 272.5 mm as shown in FIG. _L Is calculated by the following equation 2.
[0058]
Field size S _L = 512 × (272.5 / P _1-4 ) ………… (Formula 2)
[0059]
Here, as an example, the width reference is obtained at the positions of the first digit and the fourth digit. However, other methods may be used for the reference width, and any example is not included in the gist of the present invention. There is no.
[0060]
Here, the visual field size is calculated, but since the value obtained by multiplying the resolution by the number of horizontal pixels of the image memory is the visual field size, the same processing can be performed even if the resolution is calculated.
[0061]
Therefore, the resolution A in the case of a medium license plate _M Is expressed in Equation 3, and the resolution A for a large size _L Is shown in Equation 4.
[0062]
Medium resolution A _M = 195 / P _1-4 ............ (Formula 3)
Large resolution A _L = 272.5 / P _1-4 ............ (Formula 4)
[0063]
(B) Judgment of visual field size (process 31 in FIG. 1)
In this process, it is determined whether the visual field size is equal to or larger than a predetermined value. At this time, if the TV camera 40 is of the NTSC system, it is determined whether it is 1.2 m or more, for example. When the visual field size is equal to or smaller than the predetermined value, the process proceeds from here to process 18, and at this time, the same process input as in the prior art is obtained.
[0064]
Note that if the resolution is determined instead of the field of view size, it may be determined whether or not the resolution is greater than 2 mm / pixel.
[0065]
(C) Plate area enlargement process (process 32 in FIG. 1)
When the visual field size obtained in the process 31 is equal to or larger than the predetermined value, the plate is small, and the enlargement process is executed for the plate area. At this time, in order to speed up the processing, as shown in FIG. 2, only the input image of the plate area 35 is cut out and enlarged to a predetermined resolution size.
[0066]
For example, when standardizing to 2 mm / pixel, the standard visual field size is 512 pixels × 2 mm / pixel = 1024 mm, so that the visual field size calculated from the serial number is enlarged by 1024.
[0067]
Strictly speaking, it is necessary to change the enlargement magnification in the X direction and the Y direction, but here, the Y direction is enlarged as much as the magnification in the X direction.
[0068]
Here, there are methods such as nearest neighbor interpolation, 4-point linear interpolation (bilinear method), and 16-point interpolation (bicubic method) for image enlargement processing, but considering the high processing speed and image quality, Satisfactory results are obtained with 4-point linear interpolation.
[0069]
At this time, since the serial number is extracted, the enlargement area can be increased in speed by calculating the plate area size determined from the serial number and enlarging only this area.
[0070]
(D) Re-extraction of serial number from enlarged image (process 33 in FIG. 1)
Next, the serial number is extracted again from the enlarged image. Then, the coordinates of the serial number are calculated from the extracted image in the same manner as in the prior art. At this time, the coordinates of each code area determine the relative position from the coordinates of the serial number. Therefore, the coordinates of the serial number need to be calculated as accurately as possible.
[0071]
Here, the reason why the serial number is cut out again is that the character size and position can be obtained more correctly in the binary image when the character is larger than the binary image when the character is smaller.
[0072]
(E) Determination of plate size (process 34 in FIG. 1)
Here, the average height of the characters with the serial number is used, and if the height of the characters is equal to or greater than a predetermined value, it is determined that the plate is a large plate. Unlike the initial plate size determination (FIG. 1, that is, the process 17 in FIG. 10), the threshold used for the determination at this time is a character size calculated in accordance with the resolution at the time of enlargement. Will be used.
[0073]
Here, since the processing 33 in FIG. 1 is unified to 2 mm / pixel, the field width becomes 1024 mm, and the character size when the image is taken with this field width, that is, the initial field size is 1500 mm. Since the large serial number has an average height of 100 mm, a threshold corresponding to the size is set.
[0074]
If the field size after enlargement is 1000 mm, the plate size is determined using a value that is increased by about 3/2 as a threshold value.
[0075]
It should be noted that this plate size determination has a low possibility of erroneous determination even if the determination result (process 17) when the field of view is wide is used as it is, but it is determined again here just in case.
[0076]
After the above processing, the vehicle number recognition is performed by executing the resolution calculation processing 18, the code area extraction processing 19, the character extraction processing 20, and the character recognition processing 21 as in the prior art.
[0077]
As described above, in the embodiment of the present invention, the serial number is cut out from the image of the wide photographing field of view, the field size (or resolution) is calculated from the coordinates of the serial number, and the plate region corresponds to the predetermined resolution by this field size. The serial number is extracted again from the image, the resolution is calculated again from the coordinates of the serial number, the position of each code area is obtained from the relative distance from the serial number, and character recognition is performed.
[0078]
As a result, according to the above embodiment, the character area can be correctly positioned even if the character of each code is small. As a result, character extraction and character recognition can be correctly executed, and erroneous recognition can be prevented.
[0079]
With such a process, in the above embodiment, it is possible to increase the field of view with a general NTSC inexpensive television camera without using an expensive high-definition television camera. The probability of missing a number can be greatly reduced.
[0080]
If the field of view size is too large, the serial number will be crushed and cannot be extracted, resulting in difficulty in recognizing all characters. Therefore, it is necessary to set the visual field size within a range in which the resolution (about 3 mm / pixel) that can correctly extract the serial number can be maintained.
[0081]
By the way, in the processing according to the above embodiment, the visual field size and the resolution are calculated from the character pitch of the serial number, and this is enlarged so as to become a predetermined value. However, in the case of a fixed camera, the visual field size is often fixed.
[0082]
Therefore, even in the present invention, when a fixed TV camera is used, instead of extracting the serial number and calculating the visual field size, the plate area is enlarged at a fixed magnification so as to have a predetermined resolution in advance. You may implement by performing the process after a serial number extraction process from an image.
[0083]
FIG. 3 is a flowchart showing an embodiment of the present invention in such a case, but here again, the same numbers as those in the flowcharts of FIGS.
[0084]
When the plate is cut out in the process 15, the plate area is obtained as indicated by 35 in FIG. Therefore, if the field of view size is, for example, 3000 mm and the number of pixels is 1000 pixels, the plate area 35 is enlarged to 2 mm / pixel in this case (3 in FIG. 3).
[0085]
Thus, even if the plate region is always enlarged at a fixed magnification, the same effect as that of the embodiment described with reference to FIG. 1 can be obtained.
[0086]
However, if there are many plate candidates, enlargement processing occurs for each region, and processing time increases. When it is necessary to deal with various field sizes, it is desirable to calculate the field size (or resolution) and enlarge based on the value as in the embodiment of FIG.
[0087]
By the way, when the field of view that can be recognized is widened by the processing in the above embodiment, the plate size and the character size are greatly different between the lower right and upper left of the screen. In particular, the difference becomes even larger when the lane is photographed obliquely from the side of the road.
[0088]
In the plate detection process in the above embodiment, the minimum maximum height and the minimum maximum width of the plate are set as thresholds, and the minimum character height and the maximum character height are set for the serial number extraction process. , I try to cut out objects that fall within that range.
[0089]
At this time, if the field of view is not so wide, it is sufficient to define one kind of the minimum and maximum thresholds. However, if the field of view becomes wider, one kind of threshold cannot be used.
[0090]
In addition, the plate size determination (medium size, large size determination) processing is also performed with the character height threshold set to one value, but this can also be handled with a single threshold when the field of view of the TV camera is increased. For example, it is necessary to take measures such that the lower right corner of the screen is a large plate with a series number of 20 pixels or more, a middle plate is less, and the upper left is 15 pixels or more.
[0091]
In this license plate recognition process, if plate detection, serial number extraction, and plate size determination are mistaken, all subsequent processes will malfunction, so setting a threshold value for these is an extremely important matter.
[0092]
Here, FIG. 4 shows a coordinate system when the TV camera 40 is installed beside the road and the road 41 is photographed obliquely from the side. In this case, the TV camera 40 is the origin, the left direction is the X coordinate, When the depth direction is the Y coordinate, the distance L from the TV camera 40 to the point P (X, Y) is determined by the following equation (5).
[0093]
Distance L = √ (X ² + Y ² ) ………… (Formula 5)
[0094]
On the other hand, “Computer Vision (Technical Review and Future Prospects)”: Takashi Matsuyama and others, as described in “Chapter 3 Camera Calibration” published by New Technology Communications, world coordinates (world coordinates) and image coordinates (of image memory) The relationship of (coordinates) can be uniquely determined.
[0095]
That is, as shown in FIG. 5, the coordinate system x, y of the image memory and the world coordinates X, Y can be defined as a function of X (x, y), Y (x, y) (previously the coordinates of the image memory). And world coordinates in multiple pairs).
[0096]
Therefore, calculation methods for the plate detection threshold, the serial number extraction threshold, and the plate size determination threshold will be described below.
[0097]
(1) Calculation of plate detection threshold
To make it possible to accurately extract serial numbers even when the field of view is widened, calculate the minimum maximum height, minimum maximum width, etc. of the license plate for the position coordinates on each image. Just remember.
[0098]
First, the world coordinates P (X, Y) are obtained from the coordinates (x, y) on the image. In this case, since the distance L from the TV camera can be calculated by the above equation 5, as shown in FIG. 6, the mounting height of the camera 40 is H, the vertical viewing angle of the camera 40 is φ, and the height of the large plate is h, where the mounting height of the plate is h0, the camera angle θ1 up to the coordinates at the distance L1 can be calculated by the following equation 6, and the angle θ2 to the upper end of the plate at the plate height h is It can be calculated by the following formula 7.
[0099]
Angle θ1 = tan ^-1 ((H−h0) / L1) ………… (Formula 6)
Angle θ2 = tan ^-1 ((H−h0) / L2) ………… (Formula 7)
[0100]
If the angles θ1 and θ2 can be calculated in this way, the number of pixels PN used for the maximum height threshold of the plate can be calculated by the following equation (8). Here, Py is the number of pixels in the vertical direction of the image memory.
[0101]
Number of pixels PN = (θ1, θ2) / (φ / Py) (Equation 8)
[0102]
Further, the threshold value of the minimum height can be obtained from the same equation 8 by substituting the medium plate height h into the equation 6. However, the threshold value of the minimum height is set to be wider by an estimated attachment angle because the attachment angle of the plate actually changes and the size slightly varies.
[0103]
By the way, the above is a method in which the threshold value is calculated for each pixel, but in this case, it may be difficult to store the threshold value for the size of the image memory due to the limitation of the memory capacity. is there.
[0104]
Therefore, in this case, after calculating the threshold value of each pixel position, the image memory may be divided into n × m, an average value of the range may be obtained for each block, and this value may be stored as the threshold value.
[0105]
As described above, since at least the threshold value for detecting the plate is unknown on the image, it is necessary to calculate and store the threshold value for each position in advance.
[0106]
It is also possible to determine whether or not the candidate object is within a predetermined threshold range by using the center coordinates of the image memory coordinate system x and y obtained when the plate candidates are extracted and calculating the threshold value each time. It is.
[0107]
(2) Calculation of threshold for sequence number extraction
Since the serial number extraction process already knows the position of the plate that has been imaged by the plate cutout process, the extraction threshold of the serial number may be calculated each time using the coordinate position of the extracted plate.
[0108]
Therefore, when the world coordinate P (X, Y) is obtained from the plate center x coordinate and the bottom coordinate (x, y) on the image obtained when the license plate is cut out, the distance L from the camera is expressed by the above formula. 5 can be calculated.
[0109]
If the distance L can be calculated, the mounting height H of the TV camera 40, the vertical viewing angle φ, the height h of the serial number of the large plate, the mounting height h0 of the plate (which varies depending on the vehicle) as described above. Therefore, if the empirical numerical value is substituted), the camera angle θ1 up to the plate lower end coordinates at the position of the distance L1 can be calculated by the above equation 6.
[0110]
On the other hand, the angle θ2 to the upper end of the serial number in the case of a character with a height h can be calculated by the following equation (9).
[0111]
Angle θ2 = tan ^-1 ((H−h0−h) / L1) (Equation 9)
[0112]
When the angles θ1 and θ2 are calculated in this way, the number of pixels used for the maximum height threshold of the serial number can be calculated by the above equation 8. Similarly, the minimum threshold value is calculated using the character height of the medium plate. At this time, since the mounting angle of the plate actually changes and the size slightly fluctuates, the threshold value is also set wider by an estimated mounting angle as in the case of the threshold value for plate detection.
[0113]
Note that the serial number extraction by the process 16 in FIG. 1 (or FIG. 3, FIG. 10) is determined by the above method, and the serial number extraction from the enlarged image by the process 33 in FIG. It is only necessary to correct the scale.
[0114]
(3) Calculation of plate size judgment threshold
Here, the plate size is large, and the average value of the heights of the serial numbers of the medium plates is the threshold. Therefore, when the world coordinates P (X, Y) are obtained from the lower end coordinates (x, y) on the image obtained when the serial number is cut out, the distance L from the camera can be calculated by Equation 5.
[0115]
If this distance L can be calculated, the height H of the TV camera 40, the vertical viewing angle φ of the camera 40, and the average height of the serial numbers of the large plate and the medium plate are set to h (the medium size and the medium size) as described above. (Large average height is 100mm), plate mounting height h0, the camera angle θ1 to the bottom coordinate of the serial number at the position of distance L1 can be calculated by the above formula 6, and the character of height h In this case, the angle θ2 to the upper end of the serial number can be calculated by the above formula 7 or the above formula 9.
[0116]
Here, in the case of Equation 7, the world coordinates X2, Y2 are calculated from the image memory coordinates (x2, y2) at the upper end of the serial number, and the distance L2 is obtained therefrom.
[0117]
When the angles θ1 and θ2 can be calculated in this way, the number of pixels used for the plate determination threshold can be calculated by the above equation 8.
[0118]
By the way, it is desirable to execute these processes for each serial number. However, in order to shorten the processing time, the determination threshold may be calculated using the average x and y coordinates of the entire serial number. This is because the threshold values at the right end and the left end of the serial number do not change much.
[0119]
There are several other possible expressions for the correspondence between the image memory and the world coordinates at this time, but in any case, all methods for obtaining the determination threshold values for the medium and large plates from the serial number coordinates are included.
[0120]
Here again, the plate size determination by the process 17 in FIG. 1 is determined by the above-described method, and the original threshold is set to the image enlargement for the plate determination after extracting the serial number from the enlarged image by the process 34 in FIG. It is sufficient to correct the scale.
[0121]
As described above, according to the embodiment of the present invention, the field of view of plate detection, serial number extraction, and plate size determination is widened by calculating various threshold values on the screen from the relationship between the image coordinates and the world coordinates. Can be executed accurately, and as a result, the character recognition accuracy of the plate can be improved.
[0122]
Therefore, according to the above-described embodiment, the vehicle number is recognized by photographing with a resolution of about 2 mm / pixel, and the recognition rate is ensured. However, the plate portion is cut out and the image is enlarged to a predetermined size. Thus, the field of view can be widened to a resolution of about 3 mm / pixel, and as a result, the road can be widely monitored with a low-cost TV camera.
[0123]
In addition, the size of the plate size and the serial number varies depending on which part of the screen was taken as the field of view widened. In the above embodiment, the plate detection and the serial number are determined from the image memory coordinates of the plate and the serial number. By automatically calculating the threshold values used for extraction and plate size determination, various types of clipping can be accurately executed, and the character recognition accuracy of the plate can be improved even when the field of view is widened.
[0124]
【The invention's effect】
ADVANTAGE OF THE INVENTION According to this invention, the vehicle number recognition apparatus which can recognize a vehicle number reliably over a wide range only by using one television camera can be provided easily.
[Brief description of the drawings]
FIG. 1 is a flowchart for explaining a vehicle number recognition process executed by an embodiment of a vehicle number recognition device according to the present invention.
FIG. 2 is an explanatory diagram of a plate region cut out from an input image.
FIG. 3 is a flowchart for explaining vehicle number recognition processing executed by another embodiment of the vehicle number recognition device according to the present invention.
FIG. 4 is an explanatory diagram showing a positional relationship between a television camera and a road.
FIG. 5 is an explanatory diagram showing a relationship between a coordinate system of an image memory and a world coordinate system.
FIG. 6 is an explanatory diagram when the positional relationship between the television camera and the vehicle is viewed from the side.
FIG. 7 is a block diagram illustrating an example of a hardware configuration of a vehicle number recognition device.
FIG. 8 is an explanatory diagram of a vehicle number recognition process according to a series of flows.
FIG. 9 is an explanatory diagram of a license plate in a Japanese automobile.
FIG. 10 is a flowchart for explaining vehicle number recognition processing executed by a conventional vehicle number recognition device.
FIG. 11 is an explanatory diagram of processing for setting various code areas in a license plate.
FIG. 12 is an explanatory diagram of processing for determining a character area from the density distribution of various code areas;
[Explanation of symbols]
30 Field size calculation processing
32 Plate area expansion processing
33 Sequence number extraction process
34 Plate size judgment processing

Claims (5)

In the car number recognition system that processes the image captured by the TV camera and recognizes the characters on the license plate,
Plate cutting means for cutting out the license plate region from the image;
Serial number extraction means for extracting the serial number of the license plate from the cut out license plate region;
Using the average height of the character with the extracted serial number, it is determined that the character plate is a large plate when the character height is equal to or greater than a predetermined threshold value, and is determined to be a medium plate when the threshold value is not reached. Plate size determination means to perform,
The center coordinates of each character of the extracted serial number is obtained, the number of pixels in the horizontal direction of two characters selected arbitrarily, and the horizontal interval between the two characters of the medium or large plate determined by the plate size determination means Visual field size calculating means for calculating the visual field size from the number of pixels of the image memory ;
When the calculated visual field size is a predetermined value or more, an image enlarging unit that makes an image near the plate an enlarged image of a predetermined size;
An area calculating means for calculating character area coordinates other than the serial number from the serial number extracted from the enlarged image by the serial number extracting means;
A vehicle number recognition apparatus configured to recognize each character including a serial number of a license plate based on the position coordinates of these serial numbers and the character region coordinates other than the serial numbers. In the car number recognition system that processes the image captured by the TV camera and recognizes the characters on the license plate,
Plate cutting means for cutting out the license plate region from the image;
Serial number extraction means for extracting the serial number of the license plate from the cut out license plate region;
Using the average height of the character with the extracted serial number, it is determined that the character plate is a large plate when the character height is equal to or greater than a predetermined threshold value, and is determined to be a medium plate when the threshold value is not reached. Plate size determination means to perform,
The horizontal spacing between the two characters of the medium or large plate determined by the plate size determining means of the first and fourth digits of the extracted serial number, and the horizontal pixels of the first and fourth digits Resolution calculating means for calculating horizontal pixel resolution (dimension / pixel) from the number;
When the calculated resolution is equal to or higher than a predetermined value, an image enlarging unit that makes an image near the plate an enlarged image with a predetermined resolution;
An area calculating means for calculating character area coordinates other than the serial number from the serial number extracted from the enlarged image by the serial number extracting means;
A vehicle number recognition apparatus configured to recognize each character including a serial number of a license plate based on the position coordinates of these serial numbers and the character region coordinates other than the serial numbers. In the invention according to claim 1 or claim 2,
A vehicle number recognizing device, characterized in that means for automatically calculating a threshold value used in the plate cutout means based on a distance from the television camera corresponding to each image position in advance is provided. In the invention according to claim 1 or claim 2,
The threshold used by the serial number extracting means and the threshold used by the plate size determining means are automatically calculated based on the distance from the television camera corresponding to the image coordinates of the plate extracted by the plate extracting means, respectively. Means for providing a vehicle number is provided. In the invention according to claim 3 or claim 4,
Means are provided for automatically calculating a threshold used by the plate size determining means based on a distance from the television camera corresponding to the image coordinates of the serial number extracted by the serial number extracting means. Car number recognition device.

Images