JP4849262B2 - License plate extraction system, license plate extraction device, license plate extraction method and program - Google Patents

Description

  The present invention relates to a license plate recognition technique, and in particular, a license plate extraction system and a license plate that can reduce the search time for a license plate when the position and size of the license plate in a photographed image are unknown. The present invention relates to an extraction device, a license plate extraction method, and a program.

  In general license plate recognition technology, a license plate position is extracted from an image taken by a camera, and an OCR (Optical Character Recognition) technology is applied to the license plate to read characters on the license plate. Cameras used to shoot license plates vary from those installed on elevated roads to those carried by hand, but in general, the relative position between the license plate and the camera is constant. Is the most stable. This means that the size and position of the license plate in the photographed image are constant. In such a case, license plate recognition can be easily performed, but conditions such as the installation of a camera and the like, and the shooting scene are limited, and the degree of freedom tends to be impaired. On the other hand, in a case where the degree of freedom is high, such as carrying around by hand, the size and position of the license plate in the image become indefinite instead of being able to deal with various shooting situations, and the difficulty of license plate recognition increases.

  Under such circumstances, a license plate recognition technique with a higher degree of freedom has been developed due to recent improvements in computer performance. At this time, a particular problem is a method for obtaining the position and size of the license plate, which is referred to as “number plate extraction” in this specification. In this license plate extraction, since the size and position of the license plate are often indefinite, the processing time tends to increase, and countermeasures are necessary.

The following are disclosed as technical techniques related to the present invention.
Japanese Patent Application Laid-Open No. 2004-228688 discloses a method of obtaining vertical and horizontal edges and setting a combination that matches a predetermined threshold from a combination of arbitrary vertical and horizontal edges as a license plate position. However, with this method, when the background is complicated, many edges are extracted, and as a result, the number of combinations of vertical and horizontal edges increases, and the processing time is considered to increase.
In Patent Document 2, after differentiation, an image is divided into 16 × 32 dot sub-windows to form a search window composed of 3 × 7 sub-windows, the entire image is searched in this search window, and the differentiation in each search window A method is disclosed in which whether a license plate is determined by comparing the sum of values with a predetermined threshold value is disclosed. However, with this method, there are cases where areas where trees or rough roads are in the image are also recognized as license plates.
Japanese Patent Laid-Open No. 10-172091 Japanese Patent Application Laid-Open No. 08-190691 JP 11-73514 A

From the above, license plate extraction has the following problems.
The first problem is that it is necessary to recognize license plate extraction at high speed. This is because the size and position of the license plate in the image are indefinite, and it is necessary to search with various patterns.
The second problem is to reduce misrecognition of license plate extraction. The reason is that there are cases in which objects with many edges such as trees and rough road surfaces are erroneously recognized.

  The present invention has been made in view of the above circumstances. A license plate extraction system, a license plate extraction device, a license plate extraction method, and a program that can reduce erroneous recognition and perform license plate extraction at high speed. The purpose is to provide.

  In Patent Document 3, the brightness value of the license plate candidate area is projected in each of the vertical and horizontal directions, and whether or not the license plate area is a license plate area by comparing the projected shape (histogram) with a predetermined shape. Is similar to the present invention in that the license plate is extracted using a histogram, but in the present invention, all the edges in the vertical and horizontal directions of the license plate candidate area are obtained as feature amounts, This is different in that the feature amount is converted into a histogram, a threshold value is set for the histogram by a discrimination classification method, and the separated upper portion (portion having a large amount of feature) is used as a license plate candidate region.

In order to achieve such an object, a first aspect of the present invention includes a photographing means for photographing a license plate image, an image input means for dropping the image into an image, and a license plate for extracting a license plate area from the image. An extraction processing means; a character extraction processing means for extracting a character from the extracted license plate region; and a character recognition processing means for recognizing the extracted character . The license plate extraction processing means is an image input means. A feature processing unit that performs feature processing on an image input from the image processing unit, and a feature amount in each rectangular region while moving a rectangular region having a predetermined size with respect to the entire region of the image subjected to the feature processing. A histogram showing the feature value is created, a first threshold value for equally dividing the feature value shown in the histogram is found, and a feature larger than the first threshold value is obtained. A candidate region extraction unit that extracts a candidate region of the rectangular area can be a license plate candidate is, the area determination processing unit for determining whether a license plate on the extracted candidate regions, in that it comprises Features.

A second aspect of the present invention is a license plate extraction device that extracts a license plate area from an image in which a license plate is photographed , a feature processing unit that performs feature processing on the image, and a feature processing The feature quantity in each rectangular area is obtained while moving a rectangular area of a predetermined size with respect to the entire area of the applied image , a histogram showing the feature quantity is created, and the feature quantity shown in the histogram is A candidate area extraction processing unit that obtains a first threshold value that is equally divided and extracts a rectangular area that is a feature amount greater than the first threshold value as a candidate area that can be a license plate candidate; and for the extracted candidate area and having an area determination unit that determines whether the license plate, the.

According to a third aspect of the present invention, there is provided a license plate extraction method for extracting a license plate area from an image in which a license plate is photographed. The feature quantity in each rectangular area is obtained while moving a rectangular area of a predetermined size with respect to the entire area of the applied image, a histogram showing the feature quantity is created, and the feature quantity shown in the histogram is A candidate area extraction processing step for obtaining a first threshold value that is equally divided and extracting a rectangular area having a feature amount larger than the first threshold value as a candidate area that can be a license plate candidate; and for the extracted candidate area And an area determination processing step for determining whether or not it is a license plate .

According to a fourth aspect of the present invention , there is provided a program for extracting a license plate area from an image in which a license plate is photographed. The feature quantity in each rectangular area is obtained while moving a rectangular area of a predetermined size with respect to the entire area of the applied image, a histogram showing the feature quantity is created, and the feature quantity shown in the histogram is A candidate area extraction process for obtaining a first threshold value that is equally divided and extracting a rectangular area having a feature amount larger than the first threshold value as a candidate area that can be a license plate candidate, and a number for the extracted candidate area And a region determination process for determining whether or not the plate is a plate .

  According to the present invention, the processing efficiency is improved by performing license plate extraction in multiple stages, and it is possible to perform license plate extraction at high speed and with few erroneous recognitions.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram showing the configuration of the license plate extraction system of the present invention. As shown in FIG. 1, a license plate extraction system of the present invention includes a video camera (photographing means) 101, an image input means 102, a license plate extraction means (apparatus) 103, a character extraction processing means 104, and character recognition. And processing means 105. The license plate extraction processing means (device) 103 according to the present invention includes a feature processing unit 106, a candidate region extraction processing unit 107, and a region determination processing unit 108.
Each of these means operates as follows.

  The video camera 101 is a general video camera, and in some cases, there is near-infrared light used for night vision or the like. This video camera takes a picture of a vehicle with a license plate.

  The image input unit 102 is a part that captures video captured by the video camera 101 into a computer, captures one frame from consecutive frames, and develops it as an image on a memory or a hard disk.

  The license plate extraction processing unit 103 performs processing for specifying an area that appears to be a license plate for the image developed by the image input unit 102. The license plate extraction processing unit 103 includes a feature processing unit 106, a candidate region extraction processing unit 107, and a region determination processing unit 108.

  The character extraction processing unit 104 specifies a character region with respect to the license plate region extracted by the license plate extraction processing unit 103, and generally performs binarization processing and labeling processing.

  The character recognition processing unit 105 specifies characters using the OCR technique or the like for the extracted character area.

  The feature processing unit 106 performs processing for obtaining image features by performing image processing such as edge detection on the image developed by the image input unit 102.

  The candidate area extraction processing unit 107 searches the image subjected to the feature processing by the feature processing unit 106 with an appropriate rectangle for all the regions in the image. The likelihood is quantified, and a candidate rectangle for the license plate is selected based on an appropriate threshold. Here, the narrowing is performed by about 50% to 70% with respect to all the rectangles. Specifically, since the vehicle body and window on which the license plate is installed have no features and are uniform, those portions are rejected from the search area from the feature amount. Conversely, there are fine roughness on the road and trees on the side of the road, which tend to increase the amount of edges. In order to cope with this, a distinction is made between the license plate area, road roughness, and trees based on the difference between the feature quantities in the horizontal and vertical directions. This is because the license plate characters are composed of approximately the same amount of features in both the horizontal and vertical directions, but the camera's viewpoint direction is often oblique to the road surface on the road. The feature amount in the horizontal direction is attenuated and the feature amount in the horizontal direction is maintained, so that the feature amount in the horizontal direction becomes larger than the vertical direction. Further, since the leaves grow in the horizontal direction because the leaves receive sunlight, the feature amount in the horizontal direction is larger than that in the vertical direction.

  The region determination processing unit 108 determines several license plate regions by performing processing such as pattern recognition on the candidate region selected by the candidate region extraction processing unit 107.

  Next, the license plate extraction operation of the present invention will be described in detail with reference to the flowchart of FIG. 2, FIG. 3, and FIG. 2 describes the operation of the license plate extraction processing means 103 in FIG. The license plate extraction processing means 103 executes the following license plate extraction operation according to the read control program.

  First, for the entire area of the image developed by the image input means 102, the vertical and horizontal feature quantities are respectively obtained (steps S201 and S202).

  Thereafter, an initial value of the rectangular size to be searched is set (step S203). A plurality of rectangular sizes are prepared so that the license plates can be extracted even if the size of the license plate in the image is large or small.

  Next, the position of the search rectangle (start point of the search area) is initialized (step S204). Here, the upper left coordinate (302 in FIG. 3) of the search rectangle is aligned with the upper left of the developed image (301 in FIG. 3).

  Next, the sum of the feature amounts in the search rectangle is obtained in each of the vertical direction and the horizontal direction (steps S205 and S206).

  Next, the sum of the feature amount in the vertical direction and the sum of the feature amount in the horizontal direction obtained in steps S205 and S206 is further calculated (step S207).

  The feature amount obtained in step S207 is added to the histogram (403 in FIG. 4) (step S208). Specifically, the horizontal axis of the histogram is a feature amount (402 in FIG. 4), and the vertical axis is the number of search rectangles (401 in FIG. 4) with respect to the horizontal axis.

  Next, the search rectangle 302 in FIG. 3 is moved vertically or horizontally (step S209). This movement is repeated, a search is performed for the entire area of the developed image (301 in FIG. 3) (step S210), the respective feature amounts are obtained, and the histogram 403 in FIG. 4 is completed.

  A threshold value (404 in FIG. 4) is set so that the feature amount is equally divided from the histogram 403 (step S211). The threshold 404 or less is excluded from the search rectangle candidate targets (step S212).

  Next, in each target search rectangle, the absolute value of the difference between the vertical feature quantity and the horizontal feature quantity is taken (step S213). If this difference is larger than the predetermined threshold, the search rectangle candidate is excluded (step S214).

  Next, the search rectangle size is changed (step S215). If this size does not exceed the predetermined value (step S216 / NO), the process returns to steps S205 and S206, and if it exceeds (step S216 / YES). Then, pattern recognition is performed on each remaining search rectangle to determine whether it is a license plate or not (step S217).

Next, the license plate extraction operation of the present invention described with reference to FIG. 2 will be described using a specific embodiment.
An example of an image input from the image input unit 102 is an image shown in FIG. As shown in FIG. 5, this image is obtained from a vehicle 501, a license plate 502 mounted on the vehicle 501, trees 503 beside the road 504, a road 504 on which the vehicle 501 is traveling, and a road surface roughness 505. It is configured. For convenience, the texture is omitted.

  For such an image, when the vertical feature quantity is performed using a vertical Sobel filter, the result is as shown in FIG. 6, and when the horizontal feature quantity is performed using a horizontal Sobel filter, FIG. As shown in FIG.

  When FIG. 6 and FIG. 7 are added or ORed, the result is as shown in FIG. 8, resulting in an image with enhanced boundary lines. Here, the search rectangle 801 is executed so that the entire screen is scanned pixel by pixel from the upper left, and the sum of the feature amounts is obtained at each position to create a histogram as shown in FIG. For this histogram, a threshold value is obtained by discriminant analysis so as to separate a region with a large feature amount and a region with a small feature amount. Since the license plate has many features in both the vertical and horizontal directions, search areas that are lower than the threshold are rejected. At this time, the trees 503 and the road surface roughness 505 remain unrejected.

  In order to cope with these, when the absolute value of the difference between FIG. 6 and FIG. 7 is obtained, it becomes as shown in FIG. 9, and trees 503 and road surface roughness 505 remain. Similarly for FIG. 9, the entire screen is scanned pixel by pixel by the search rectangle 901, the sum of feature amounts is obtained at each position, and if this is larger than a predetermined threshold, the region is rejected. A pattern recognition process is performed on the search rectangular area remaining in the two-stage process, and it is finally determined whether or not it is a license plate.

As described above, the present invention has the following effects.
The first effect is to perform license plate extraction accurately. The reason is that region extraction processing is performed in multiple stages, and accurate extraction is possible by performing region extraction from different viewpoints in each stage.
The second effect is that license plate extraction is performed at high speed. The reason is that the region extraction process is performed in multiple stages. Initially, the search region is roughly limited, and then the region determination is performed in detail, thereby enabling high-speed extraction.
The third effect is that it is possible to reduce misrecognition of a vehicle front grille, boundary line, road roughness, trees and the like by license plate extraction.

  Although the present invention has been described above, the present invention is not limited to the above description, and various modifications can be made without departing from the scope of the invention.

  For example, the above-described operation in the present invention can be executed by hardware, software, or a combined configuration of both.

  In addition, when executing processing by software, the program recording the processing sequence of FIG. 2 is installed in a memory in a computer incorporated in dedicated hardware and executed, or various processing is executed. It is possible to install and execute the program on a possible general-purpose computer.

For example, the program can be recorded in advance on a hard disk or a ROM (Read Only Memory) as a recording medium. Alternatively, the program is temporarily stored on a removable recording medium such as a floppy (registered trademark) disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. Or can be stored (recorded) permanently.
Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. On the other hand, the computer can receive the transferred program and install it on a recording medium such as a built-in hard disk.

  Further, not only is it executed in time series according to the processing operation described with reference to FIG. 2, but also the processing capability of the device that executes the processing, or it is constructed so that it is executed in parallel or individually as required. It is also possible.

  In addition, the license plate extraction system of the present invention can be constructed to have a logical set configuration of a plurality of devices or to mix the functions of each device.

  The present invention can be applied to uses such as identification and monitoring of vehicles in facilities such as airports and harbors where security is required in addition to license plate recognition. Also, it can be applied to security related uses such as identification of stolen vehicles and wanted vehicles.

It is a block diagram which shows the structure of the license plate extraction system of this invention. It is a flowchart which shows operation | movement of the license plate extraction means (apparatus) of this invention. It is a conceptual diagram which shows operation | movement of a search area | region. It is a conceptual diagram of the histogram which shows distribution of a feature-value. It is a conceptual diagram of the example of an input image from a video camera. It is the conceptual diagram which calculated | required the feature-value of the perpendicular direction with respect to FIG. It is the conceptual diagram which calculated | required the feature-value of the horizontal direction with respect to FIG. It is the conceptual diagram which calculated | required the sum or OR of each feature-value of each vertical direction of FIG. 6, FIG. It is the conceptual diagram which calculated | required the absolute value of the difference of FIG. 6 and FIG.

Explanation of symbols

101 Apparatus for photographing video (photographing means)
102 Means (image input means) for taking captured images into a computer
103 Means for Identifying License Plate-Like Region (Number Plate Extraction Means / Number Plate Extraction Device)
104 Means for specifying character area (character extraction processing means)
105 Means (character recognition processing means) for specifying characters using the OCR technique or the like for the extracted character area
106 Means for obtaining license plate features (feature processing unit)
107 Means for Rejecting Area Not Like License Plate (Candidate Area Extraction Processing Unit)
108 Means for determining whether or not a license plate (area determination processing unit)
201 Means for Obtaining Vertical Features 202 Means for Finding Horizontal Features 203 Means for Initializing Search Rectangle Size 204 Means for Initializing Search Rectangle Position 205 Means for Obtaining Sum of Vertical Feature Quantity in Search Rectangle 206 Means for Finding Sum of Horizontal Feature Amount in Search Rectangle 207 Means for Finding Sum of Vertical Feature Amount in Search Rectangle and Sum of Horizontal Feature Amount in Search Rectangle 208 Feature in Vertical and Horizontal Direction Means for adding quantity to histogram 209 Means for moving search rectangle vertically or horizontally 210 Means for completing histogram with vertical and horizontal feature quantities 211 Threshold value is obtained so that feature quantities are equally divided from histogram Means 212 Means to exclude search rectangle candidate below threshold value 213 Absolute difference between vertical feature quantity and horizontal feature quantity Means for taking the value 214 Means for excluding the search rectangle candidate when the difference between the vertical feature quantity and the horizontal feature quantity is greater than a predetermined threshold value 215 Means for changing the search rectangle size 216 Search rectangle size has a predetermined value Means for determining whether or not the number exceeds 217 Means for performing pattern recognition on the search rectangle and determining whether or not it is a license plate 301 Image expanded in memory or file 302 Search rectangle 401 Feature quantity 402 Number of search rectangles for feature quantity 403 Histogram of feature amount 404 Threshold for equally dividing feature amount 501 Vehicle 502 License plate 503 Trees 504 Road 505 Roughness of road surface 801 Search rectangle 901 Search rectangle

Claims (8)

Photographing means for photographing a license plate image;
Image input means for dropping the video into the image;
License plate extraction processing means for extracting a license plate region from the image;
Character extraction processing means for extracting characters from the extracted license plate region;
Character recognition processing means for recognizing the extracted characters ,
The license plate extraction processing means includes:
A feature processing unit for performing feature processing on the image input from the image input unit;
The feature amount in each rectangular region is obtained while moving a rectangular region having a predetermined size with respect to the entire region of the image subjected to the feature processing, a histogram indicating the feature amount is created, and the histogram A candidate area extraction processing unit that obtains a first threshold value that equally divides the feature amount shown in (2) and extracts a rectangular area that is a feature quantity that is larger than the first threshold value as a candidate area that can be a license plate candidate;
An area determination processing unit for determining whether the extracted candidate area is a license plate;
License plate extraction system, characterized in that it comprises a. The candidate area extraction processing unit
For a rectangular area having a feature quantity greater than the first threshold, a difference between the vertical feature quantity and the horizontal feature quantity is obtained, and a rectangular area with the difference greater than a predetermined second threshold is determined as the candidate. 2. The license plate extraction system according to claim 1, wherein the license plate extraction system is excluded from the area. A license plate extraction device that extracts a license plate area from an image of a license plate photographed,
A feature processing unit for performing feature processing on the image;
The feature amount in each rectangular region is obtained while moving a rectangular region having a predetermined size with respect to the entire region of the image subjected to the feature processing , a histogram indicating the feature amount is created, and the histogram A candidate area extraction processing unit that obtains a first threshold value that equally divides the feature amount shown in (2) and extracts a rectangular area that is a feature quantity that is larger than the first threshold value as a candidate area that can be a license plate candidate ;
An area determination processing unit for determining whether the extracted candidate area is a license plate;
A license plate extraction apparatus comprising: The candidate area extraction processing unit
For a rectangular area having a feature quantity greater than the first threshold, a difference between the vertical feature quantity and the horizontal feature quantity is obtained, and a rectangular area with the difference greater than a predetermined second threshold is determined as the candidate. 4. The license plate extraction device according to claim 3, wherein the license plate extraction device is excluded from the area. A license plate extraction method for extracting a license plate area from an image in which a license plate is photographed,
A feature processing step for performing feature processing on the image;
The feature amount in each rectangular region is obtained while moving a rectangular region having a predetermined size with respect to the entire region of the image subjected to the feature processing , a histogram indicating the feature amount is created, and the histogram A candidate area extraction processing step for obtaining a first threshold value for equally dividing the feature quantity shown in (2) and extracting a rectangular area having a feature quantity larger than the first threshold value as a candidate area that can be a license plate candidate ;
A region determination processing step for determining whether or not the extracted candidate region is a license plate;
A license plate extraction method comprising: In the candidate area extraction processing step,
For a rectangular area having a feature quantity greater than the first threshold, a difference between the vertical feature quantity and the horizontal feature quantity is obtained, and a rectangular area with the difference greater than a predetermined second threshold is determined as the candidate. 6. The license plate extraction method according to claim 5, wherein the license plate is excluded from the area. A program for executing extraction of a license plate area from an image in which a license plate is photographed,
Feature processing for performing feature processing on the image;
The feature amount in each rectangular region is obtained while moving a rectangular region having a predetermined size with respect to the entire region of the image subjected to the feature processing , a histogram indicating the feature amount is created, and the histogram A candidate area extraction process for obtaining a first threshold value that equally divides the feature quantity shown in (2) and extracting a rectangular area that is a feature quantity larger than the first threshold value as a candidate area that can be a license plate candidate ;
An area determination process for determining whether the extracted candidate area is a license plate;
A program that causes a computer to execute. As the candidate area extraction process,
For a rectangular area having a feature quantity greater than the first threshold, a difference between the vertical feature quantity and the horizontal feature quantity is obtained, and a rectangular area with the difference greater than a predetermined second threshold is determined as the candidate. The program according to claim 7, wherein the program is excluded from the area.

Images