JPH0554196A - Number plate recognition device - Google Patents

Abstract

PURPOSE:To accurately perform a character recognition by easily coping with the change of the installation environment of a number plate recognition device even if the environment changes to some degrees. CONSTITUTION:The picture data of vehicles for which an image pickup is performed by cameras C1 to C5 and a prescribed picture processing is performed in a picture processing part 4 is inputted in a character recognition part 5 using a nuero computer having a learning function by a nueral network and characters are made to be recognized. As for unrecognized characters, a learning is performed by inputting right characters using a keyboard 13 by comparing a picture monitor 6, an original picture displayed on a console monitor 12 and the recognized character data, and the errors in the recognition are corrected.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is used in the technical field of, for example, a parking lot management device for managing entry / exit of a parking lot, and more specifically, a license plate for a vehicle entering or leaving. The present invention relates to a license plate recognition device capable of reading a license.

[0002]

2. Description of the Related Art As a device for recognizing characters (land support code, classification number, classification character, serial designation number) displayed on a vehicle license plate, conventionally, a camera is used to capture an image of the vehicle license plate. Camera-type recognition devices that can confirm characters (numbers) by applying predetermined image processing to image information are widely used.
As this recognition device, for example, Japanese Patent Publication No. 2-389.
As disclosed in Japanese Patent Publication No. 97, the character information on the license plate is cut out from the image of the license plate sent from the plate cutting portion, and the character information is collated with a recognition dictionary to perform character recognition. Character recognition device is used.

However, in the case of a character recognition device equipped with the above-mentioned recognition dictionary, accurate character recognition is possible by collating with the recognition dictionary when the license plate is imaged from the front by a camera. When the license plate is imaged from an oblique direction, each character on the license plate is tilted, and when the surroundings are too bright, the characters on the license plate become thin, so the characters are recognized correctly depending on the imaging conditions. There was a problem that I could not do it.

Therefore, conventionally, for example, Japanese Patent Laid-Open No. 2-8379.
As can be seen in Japanese Patent No. 8 publication, by adding an algorithm for vertically tilting the imaged characters, or by modifying the recognition dictionary or adjusting the light intensity of the strobe at the time of imaging, The problems described above have been solved.

[0005]

However, in order to carry out the above-mentioned solution, it is necessary to perform various and tedious adjustment operations such as image pickup angle adjustment and light intensity adjustment when the system (recognition device) is installed. Moreover, since it requires a high degree of specialized knowledge to change programs and dictionary data, it takes time to start up the system, and if the system installation environment is slightly different, troublesome adjustment work is required each time. Therefore, there is a problem that the handling is complicated and complicated.

Therefore, a technical object of the present invention is to make it possible to accurately recognize a license plate even if the installation environment of the license plate recognizing device is slightly different, in response to the change in the environment. ..

[0007]

[Means for Solving the Problems] Means taken in the present invention for solving the above technical problems are as follows. In a license plate recognition device configured to recognize a license plate by applying predetermined image processing to image information of a vehicle imaged by a camera,

(1) Providing a memory means for accumulating the original image of the vehicle imaged by the camera and a learning mode selecting means for selecting a learning mode as needed. (2) Use a neuro computer with a learning function by a neural network for the character recognition unit that recognizes the license plate. (3) The learning mode is correct when the original image data captured by the camera and the character data recognized by the character recognition unit are displayed at the same time, and the character recognized by the character recognition unit is incorrect. Providing a character input means that can learn by inputting a character pattern into the character recognition unit.

Here, however, the neuro computer is a system for performing judgment and pattern recognition having a learning function by a neural network using a parallel information mechanism by neurons, using a human brain as a model. For example, it corresponds to a device for classifying and identifying patterns disclosed in Japanese Patent Laid-Open No. 61-193279.

[0010]

The above means operates as follows. The element of (1) above can store the original image of the vehicle taken by the camera for learning, and select the learning mode according to the system installation or the change of the installation environment. Allows you to adjust the conditions.

The requirement of the above (2) is that the neural network of the neuro computer acquires the characteristics that a large number of processing units (neurons) perform information processing in parallel (parallel distributed processing) and the processing ability by learning. Features (self-organization) and features (generalization of learning) that can respond to unlearned patterns based on past learning,
Since the memory has a feature (distributed expression) expressed as a connection between units, a neural network can be used without collating the captured image with the recognition dictionary or considering an algorithm suitable for the installation environment of the system. Allows you to accurately recognize the letters on the license plate.

In the element (3), since the original image data and the character data recognized by the system can be simultaneously displayed and compared for learning, a recognition error can be easily found, and Since this error can be accurately corrected using the character input means, it is possible to easily input knowledge data for character recognition that matches the installation environment even after the system is installed, and repeat learning by this pattern input. , It is possible to classify the pattern space by generating neurons and modifying threshold values, and to reliably recognize the characters on the license plate. As described above, the technical problems described above can be solved by the above means, and the problems of the conventional technology can be solved.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the license plate recognition apparatus according to the present invention described above will be described in detail below with reference to the accompanying drawings. FIG. 1 is a block diagram showing an embodiment of a license plate recognition device according to the present invention, in which R is a passage for a vehicle CR such as an entrance of a parking lot,
RL indicates a photographing line (photographing position) at which the license plate NP of the vehicle CR is photographed by a plurality of cameras C1, C2, C3, C4, C5. In the illustrated embodiment, the entire width of the passage R is A. Divide into 5 areas of ~ E,
The structure is such that these five areas A to E are covered by a total of five cameras C1 to C5. The number of divisions of these passage widths and the number of cameras are an example of implementation, and the change in passage width It is needless to say that the number of divisions and the number of cameras increase or decrease in accordance with the above, and in some cases, one camera may be used for shooting.

Reference numeral 1 generally indicates a license plate recognition control device (computer) which constitutes a main part of the present invention. The control device 1 is synchronized with the shutter timing of the camera. The flash 7 that is turned on (emits light), the first infrared sensor 8 for vehicle detection provided in front of the above-described shooting line PL, and the shooting line PL
Second infrared sensor 9 provided behind the vehicle to check vehicle passage
And an ultrasonic sensor 1 for detecting the lateral width and the vehicle center position of the vehicle CR provided in front of the photographing line PL.
0 is connected.

In order to calculate the center position of the vehicle with the ultrasonic sensor 10, that is, the position of the license plate NP, the left and right distance data to the vehicle CR measured by the left and right ultrasonic sensors 10, 10 are used. Based on the above, it can be determined by the following formula. Vehicle width = passage width- (right side distance data + left side distance data-right side margin Sa-left side margin Sb) Plate position = vehicle width / 2 + right side distance data-right side margin Sa

FIG. 2 is a block diagram showing the electrical construction of the license plate recognition control apparatus described above.
Reference numeral 2 is an image memory to which the above-mentioned cameras C1 to C5 are connected, and reference numeral 3 is an imaging control unit in which the above-described strobe 7, the first and second infrared sensors 8 and 9 and the ultrasonic sensor 10 are connected to each other. C1 to C5 are configured so that the shutter is always opened in synchronization with the vertical retrace line. When an image capture instruction is input to the image memory 2 from the image pickup control unit 3, the flash timing is synchronized with the shutter timing signal. 7 is turned on (ON), and the image memory 2 is used by each of the cameras C1 to C.
It is designed to capture images in synchronization with the vertical retrace line of 5.

In addition, the image is selected in accordance with the position of the license plate NP detected by the ultrasonic sensor 10, a camera which covers the center position and each region on the left and right sides, that is, five cameras in the embodiment shown in FIG. Camera C1
Of the five images taken by C5, the cameras C2, C1, which cover the areas A and C on the left and right of the area B as the center
When a total of three images taken by C3 are selected, and if the position of the license plate NP is located at one end like the area E in FIG. 1, two areas adjacent to this area E are centered. Three cameras C that cover a total of three areas D and C
The three images taken by C3, C4, and C5 are selected from the five images, and the image pickup control section 3 sends the image selection instruction to the image memory 2.

Further, in FIG. 2, 4 is the image memory 2 described above.
The image processing unit 4 connected to the image processing unit 4 reduces the image data sent from the image memory 2, that is, the selected three images to 1/4, and further reduces the image data. Image compression processing that collects these three images into one multi-valued image into one memory and removes dirt and noise from the license plate NP shown in this multi-valued image Preprocessing for emphasizing the image of the "sequential designation number", binarization processing for converting the multivalued image subjected to this preprocessing to a binary image of black and white, and the connection of white or black images into one block (label ) As a labeling process for extracting position and size information, and a candidate extraction process for extracting license plate candidates (plural) from the size and positional relationship of the “series designated number” based on the label data, and the extracted plural Number Image cut-out processing for "series designated number" is performed by cutting out the image of the plate candidate from the original image and collecting it in one frame memory (not shown). For the value image, the same preprocessing as described above is performed to extract the “category character”, “classification number”, and “land support code” of the license plate NP, and the preprocessed multivalued image is converted into a black and white binary image. Binarization processing to convert to the image of the number, labeling processing to extract the above "classification character", "type number", "land support code", and based on the label data, the license plate candidate seems to be the most license plate License plate determination processing that selects one and determines this as the license plate, and cuts out character images from the area determined for the license plate of each image, Was equal cut "series designation number", "classification character", "type number", the image cutout processing to aggregate one frame memory a character image of "Riku支 code", which like cut license plate N
Various means (programs) for performing a transmission process of sending the image information of P to the character recognition unit 5 of the host computer shown in FIG. 2 are also provided.

The character recognition section 5 has a program section 11 in which software for a neuro computer is stored as shown in FIG. 2, and learning for displaying the recognition result of the character recognized by the character recognition section 5. Console monitor 12 for
A keyboard 13 used to select a learning mode or input a correct character to the character recognition unit 5 during learning to correct an error, and an NLS memory storing various information necessary for character recognition and learning. 14 and the image processing unit 4
The storage unit 1 having an area for accumulating the image information of the license plate NP sent from the computer and an area for storing the recognition data of the license plate NP recognized by the character recognition unit 5.
5 is connected to the image processing unit 4 described above.
An image monitor 6 that displays the image information of the license plate NP called from the storage unit 15 in the learning mode is connected. During learning, the original image displayed on the image monitor 6 and the console monitor 12 are connected. It is designed to find mistakes by comparing the displayed recognition data.

Further, the above-mentioned character recognition section 5 recognizes the characteristic of each character extracted from the character information sent from the above-mentioned image processing section 4 and the characteristic of the extracted character. A recognition process for determining the number of the license plate NP, and a memory for sending the original image of the license plate NP processed by the image processing unit 4 and the number of the license plate NP recognized by the character recognition unit 5 to the storage unit 6 for storage. Each means (program) for performing processing is provided.

A character recognition system using a neural network developed by Nestor Incorporated, USA (see Japanese Patent Laid-Open No. 61-193279) is used as the character recognition unit 5 described above. Detailed description is omitted.

The structure of the license plate recognition device according to the present invention is as described above. Next, the license plate N of the vehicle CR actually traveling in the passage R using the present invention is used.
The procedure of each of the image capturing process, the image process, and the recognizing process for recognizing P, and the process procedure of the learning mode will be described with reference to the illustrated flowcharts.

(Explanation of Imaging Process Shown in FIG. 3) First, in step S1, the first vehicle detection provided in the passage R is provided.
It is determined whether or not the infrared sensor 8 has detected the traveling vehicle CR, and when it is detected, the process proceeds to the next step S2, and the imaging control unit 3 captures an image in the image memory 2 as shown in FIG. Send a step-in instruction and proceed to the next step S3.

In step S3, the strobe 7 is turned on (emits light) in synchronization with the shutter timing of the cameras C1 to C5, but in the present invention, a total of five cameras C1 to C5 connected to the image memory 2 are operated. , The shutter is always released in synchronization with the vertical blanking line, and the image memory 2 is configured to capture images in synchronization with the vertical blanking lines of the cameras C1 to C5. Upon receiving the capture instruction from the control unit 3, the image memory 2 can capture a clear image captured by the five cameras C1 to C5 that are brightly illuminated by the strobe 7. (Step S4)

In the next step S5, the ultrasonic sensor 1
The data from 0 is read, then in step S6, the width and center position of the vehicle CR, that is, the position of the license plate NP is calculated from the data of the ultrasonic sensor 10 by the method described above, and the process proceeds to step S7 to proceed to the vehicle CR. The image memory 2 is instructed to select three pieces of image data from the five pieces of image data in the same manner as described above with reference to the center position of the image memory 2.
To step S8. In step S8, the image data of three sheets selected by the selection instruction sent from the imaging control unit 3 is transferred to the image processing unit 4.

(Explanation of the image processing shown in FIG. 4) The image processing unit 4 compresses the three sent images to 1/4 size and collects them in the frame memory as one image. The process (step S9), the pre-process for emphasizing the image of the image including the serially designated number of the license plate NP (step S10), the binarization process of this image (step S11), and the labeling process of this image ( Step S12) and an extraction process for extracting license plate candidates with a series of designated numbers from this image (Step S13)
And a process of cutting out images of license plate candidates from the original image and collecting them in one memory (step S14)
And preprocessing for emphasizing the image of the image including the division characters, the classification characters, and the land support code of the license plate NP from the multivalued images collected in this one sheet (step S15)
And a binarization process for converting this image into a black and white binary image (step S16), and a labeling process for extracting information on the position and size of characters in the binarized image (step S17). ), A process for selecting the most likely license plate from the label data and determining this as the license plate (step S18), and a cutting process for cutting out a character image from the area determined for the license plate of each image (step S19). Then, the process of collecting the cut-out character images in one area (step S20) and the process of transmitting the collected image data collectively to the character recognition unit 5 of the host computer 1H (step S21) are sequentially executed. After that, the system is designed to proceed to step S22 shown in FIG.

In the above-described character image cutout process (step S19), the series of designated numbers of the license plate NP is extracted in step S13, and the compressed multi-valued image is extracted from the held label information. It is cut out from the binarized image, the classification character and the classification character are cut out from the label information in step S17, the binarized image is cut out from the image binarized in step S16, and the land support code is extracted from the label information in step S17. It is configured to cut out from the images collected into one sheet in S14.

(Explanation of the mode selection process shown in FIG. 5) In step S22, it is determined whether or not the learning mode is selected by operating the keyboard 13. If the learning mode is selected, the process proceeds to the next step S23. The learning process shown in FIG. 7 is executed, and when the learning mode is not selected, the process proceeds to step S24, and the license plate NP recognition process shown in FIG. 6 is executed.

(Explanation of the recognition process shown in FIG. 6) In the first step S30, the characteristics of the character image sent from the image processing unit 4 to the character recognition unit 5 are extracted, and then the process proceeds to step S31 for extraction. The specified feature amount is passed to the neuro system, and the process proceeds to the next step S32.

In step S32, it is judged whether or not there is a neuron that excites (reacts) to the feature amount of the input character image. If yes, the process proceeds to the next step S33 and the excited neuron is the only neuron. It is determined whether or not it is unique, and if it is unique, it proceeds to step S37 to evaluate as one pattern (identified). If it is evaluated as uncertain among a plurality of classes, and it is evaluated as definite, the process proceeds to step S38, and the character is recognized and output. If it is evaluated as uncertain, the process proceeds to step S35. Multiple candidate characters and their confirmation rate (%) are output,
Further, when it is determined that there is no excited neuron in step S32 described above, the process proceeds to step S36, is evaluated as unidentified, and the process proceeds to the next step S39, respectively.

In step S39, it is determined whether or not the above-described evaluation process has been performed on all the extracted feature amounts, and if there is a feature amount that has not been evaluated, the process returns to step S30 and the above-described process is performed. Is repeated, and when the evaluations for all the feature amounts are completed, the next step S4
The position of the recognized character and the recognition result are proceeded to 0 and the storage unit 15
, And the process proceeds to the next step S41.

In step S41, the above-mentioned step S3 is performed.
It is determined whether or not the processing from 0 to S40 has been performed on all the characters of the license plate NP. If there is a character that has not been processed, the process returns to step S30 and the processing is repeated to process all the characters. If finished, step S
In step 42, the number of the license plate NP recognized (determined) is stored in the storage unit 15, and the process is ended.

(Explanation of the learning process shown in FIG. 7) Further, in the present invention, due to the learning function of the neural network, the recognition process of FIG. 6 is uncertain (step S3).
4) or if there is a character evaluated as unknown (step S36), the learning mode is selected in the processing shown in FIG. 5 and the learning processing shown in FIG. Allows you to correctly recognize unknown characters.

That is, in order to perform learning, first, the keyboard 13
, The original image of the license plate NP that could not be recognized is called from the storage unit 15 and displayed on the image monitor 6 of the image processing unit 4 (step S50), and then step S50.
In step 51, the result recognized by the recognition process shown in FIG. 6 is called from the storage unit 15 and displayed on the console monitor 12, and the process proceeds to step S52.

In step S52, it is determined again whether or not the indeterminate or unknown character cannot be recognized. If it is determined that the character cannot be recognized, the process proceeds to the next step S53 and the image is displayed on the image monitor 6. It is determined whether or not the original image cannot be determined.If it is not determined, that is, if it is determined that the character cannot be read from the original image, it is determined that learning is impossible and the process ends, and the determination is possible, that is, the original image. When it is determined that the image can be read, the process proceeds to step S54, and the keyboard 13 is operated while watching the image monitor 6 to input correct data.

On the other hand, when it is determined in step S52 that the system can recognize it, the process proceeds to the next step S55, in which it is determined whether the system recognition is correct, and it is determined that the recognition is correct. If it is done, learning is not necessary and the learning process ends.
If it is determined that the recognition is wrong, the process proceeds to step S54, the correct data is input through the keyboard 13, and the process proceeds to the next step S56.

In the next step S56, a plurality of feature quantities of the characters of the original image (for example, the area obtained by subdividing the characters) are extracted, and then the process proceeds to step S57.
When the correct data and the feature amount input in 54 are passed to the neuro system for learning, the process proceeds to the next step S58 to generate a neuron or to correct the threshold value (radius) of the neuron so that the pattern space Are sequentially classified into classes to correct the recognition error with respect to the original image data, and then the process proceeds to step S59 to send the learning data to the NLS memory 14 to update the contents, thereby ending the processing.

When the present invention is used as a storage management device for a parking lot, the determined number is stored in the storage unit 15 and then the entrance gate (not shown) provided at the position of the photographing line RL of the passage R described above. When the second infrared sensor 9 for passage confirmation detects the passage of the vehicle CR, the entrance gate is closed and the storage process is finished. The storage time is stored together with the plate number stored in the storage unit 15,
It is used for calculating charges when leaving the warehouse.

[0039]

As described above, according to the license plate recognition device of the present invention, it is possible to accurately recognize the characters of the license plate traveling in the passage. In the character recognition part that recognizes the characters on the license plate imaged, model the human brain,
Since a neuro computer with a learning function by a neural network that uses a parallel information mechanism by neurons is used, if there is an error in character recognition or there is an uncertain place such as recognition uncertainty, learn by inputting correct data By doing so, it is possible to classify the pattern space by correcting the generation of neurons or the threshold value and correct the recognition error. Therefore, even if the installation environment of the license plate recognition device is slightly different. By learning, it is possible to easily respond to environmental changes and reliably perform character recognition, and thus it is possible to easily bring up the recognition device in any environment in a short time. It is suitable for use in a parking lot entry / exit management device and the like.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a license plate recognition device according to the present invention.

FIG. 2 is a block diagram illustrating an electrical configuration of a license plate recognition control device.

FIG. 3 is a flowchart illustrating a procedure of image pickup processing according to the present invention.

FIG. 4 is a flowchart similarly illustrating the procedure of image processing.

FIG. 5 is a flowchart illustrating the procedure of learning mode selection processing.

FIG. 6 is a flowchart illustrating a procedure of character recognition processing in the same manner.

FIG. 7 is a flowchart similarly illustrating a procedure of learning processing.

[Explanation of symbols]

 R passage CR vehicle NP license plate C1 to C5 camera 1 license plate recognition control device 2 image memory 3 imaging control device 4 image processing unit 5 character recognition unit (host computer) 6 image monitor 11 neuro computer software 12 console monitor 13 keyboard 14 NLS memory 15 memory

Front page continued (72) Inventor Takuma Sawa 275 Soyado-cho, Kohoku-ku, Yokohama-shi, Kanagawa Amano Co., Ltd.

Claims (1)

[Claims] 1. A license plate recognition device configured to recognize a license plate by applying predetermined image processing to image information of the vehicle imaged by the camera, wherein an original image of the vehicle imaged by the camera is obtained. A neuro computer having a memory means for accumulating images and a learning mode selecting means capable of selecting a learning mode as needed, and a character recognition section for recognizing a license plate having a learning function by a neural network. In addition to using the above, in the learning mode, when the original image data captured by the camera and the character data recognized by the character recognition unit are simultaneously displayed, and the character recognized by the character recognition unit is incorrect, The character input means that can input the correct character pattern into the character recognition unit and make it learn Number plate recognition system, characterized in that the.