JP7061977B2 - Pattern learning system, pattern learning method and pattern learning program - Google Patents

Description

The present invention relates to pattern matching for recognizing license plates.

With the spread of ETC, the impact of the suspension of the ETC system is increasing. Therefore, the importance of the license plate recognition camera for complementing the ETC system is increasing. And it is becoming more important to improve the performance of license plate recognition.

Currently, in order to recognize characters cut out from an image, pattern matching is performed using a dictionary (pattern) learned in advance.
The dictionary used in pattern matching is generated using multiple still images with different license plate accuracy in a static environment.

In a real environment, a moving vehicle is photographed by a license plate recognition camera. The license plate shown in the image obtained by shooting is matched with each pattern registered as a dictionary. Then, based on the matching result, the license plate of the running vehicle is recognized.

Patent Document 1 discloses a license plate recognition device.
This license plate recognition device displays the original image of the misrecognized character on the monitor, and learns based on the correct character input using the keyboard.

Japanese Unexamined Patent Publication No. 5-54196

An object of the present invention is to improve the recognition performance of a license plate.

The pattern learning system of the present invention is
The information reception unit that receives the license plate information transmitted from the on-board unit of the vehicle to the roadside communication device,
An image reception unit that receives vehicle images obtained by the roadside camera that captured the vehicle, and an image reception unit.
An image recognition unit that generates a license plate recognition result that is compared with the license plate information that is accepted by pattern matching to the accepted vehicle image.
It is provided with a pattern learning unit that updates the pattern data used for pattern matching based on the received vehicle image when the generated license plate recognition result does not match the received license plate information.

According to the present invention, the pattern data used for pattern matching for recognizing the license plate can be updated at any time. As a result, the recognition performance of the license plate is improved.

The block diagram of the FFETC system 100 in Embodiment 1. FIG. The block diagram of the charge collecting apparatus 200 in Embodiment 1. FIG. The flowchart of the charge collection method and the pattern learning method in Embodiment 1. The flowchart of the image recognition process (S110) in Embodiment 1. The figure which shows the specific example of the license plate 103 in Embodiment 1. FIG. The schematic diagram of the pattern matching in Embodiment 1. The flowchart of the pattern learning process (S130) in Embodiment 1.

In embodiments and drawings, the same or corresponding elements are designated by the same reference numerals. Descriptions of elements with the same reference numerals as the described elements will be omitted or simplified as appropriate. The arrows in the figure mainly indicate the flow of data or the flow of processing.

Embodiment 1.
A mode for updating the pattern data used for pattern matching for recognizing the license plate at any time will be described with reference to FIGS. 1 to 7.

*** Explanation of configuration ***
The configuration of the FFETC system 100 will be described with reference to FIG.
The FFETC system 100 is a system for collecting tolls for a vehicle 101 traveling on a toll road.
FFETC is an abbreviation for free flow ETC.
ETC is an abbreviation for Electronic Toll Collection.

The vehicle-mounted device 102 is mounted on the vehicle 101.
An ETC card is inserted into the vehicle-mounted device 102.
The ETC card is an IC card for collecting tolls. IC is an abbreviation for Integrated Circuit.
Various information for collecting charges is recorded on the ETC card. Various information recorded on the ETC card is called card information.

Further, the license plate information is registered in the vehicle-mounted device 102.
The license plate information indicates a character string written on the license plate 103 of the vehicle 101.

The FFETC system 100 includes a roadside machine 110 and a toll collection device 200.
The roadside machine 110 is installed around the toll road (for example, above the road).

The roadside machine 110 includes a roadside communication device 111 and a roadside camera 112.
The roadside communication device 111 wirelessly communicates with the vehicle-mounted device 102 of the vehicle 101. For example, the roadside communication device 111 receives vehicle data from the vehicle-mounted device 102. The vehicle data is data including various information about the vehicle 101. Vehicle data includes card information and license plate information.
The roadside camera 112 obtains a vehicle image by photographing the vehicle 101. The vehicle image is an image showing the vehicle 101. The roadside camera 112 is also referred to as a license plate recognition camera.

The toll collection device 200 is a device that collects tolls for a vehicle 101 passing through a toll road.
The toll collection device 200 collects tolls from the vehicle 101 based on the vehicle data communicated from the vehicle-mounted device 102 to the roadside communication device 111 and the vehicle image obtained by the roadside camera 112 that captures the vehicle 101. conduct.

The FFETC system 100 has the function of a pattern learning system. The toll collection device 200 has a function of a pattern learning device.
Pattern learning is a function for machine learning pattern data used for pattern matching on vehicle images. Pattern matching for the vehicle image is performed to recognize the license plate 103 using the vehicle image.

The configuration of the toll collection device 200 will be described with reference to FIG. 2.
The toll collection device 200 is a computer including hardware such as a processor 201, a memory 202, an auxiliary storage device 203, a communication device 204, and an input / output interface 205. These hardware are connected to each other via a signal line.

The processor 201 is an IC (Integrated Circuit) that performs arithmetic processing, and controls other hardware. For example, the processor 201 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit).
The memory 202 is a volatile storage device. The memory 202 is also referred to as a main storage device or a main memory. For example, the memory 202 is a RAM (Random Access Memory). The data stored in the memory 202 is stored in the auxiliary storage device 203 as needed.
The auxiliary storage device 203 is a non-volatile storage device. For example, the auxiliary storage device 203 is a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. The data stored in the auxiliary storage device 203 is loaded into the memory 202 as needed.
The communication device 204 is a receiver and a transmitter. For example, the communication device 204 is a communication chip or a NIC (Network Interface Card).
The input / output interface 205 is a port to which an input device and an output device are connected. For example, the input / output interface 205 is a USB terminal, the input device is a keyboard and a mouse, and the output device is a display. USB is an abbreviation for Universal Serial Bus.

The toll collection device 200 includes elements such as an information reception unit 211, an image reception unit 212, an image recognition unit 213, a pattern learning unit 214, and a charge collection unit 215. These elements are realized by software.

The auxiliary storage device 203 stores a charge collection device program for operating a computer as an information reception unit 211, an image reception unit 212, an image recognition unit 213, a pattern learning unit 214, and a charge collection unit 215. The toll collection program is loaded into memory 202 and executed by processor 201.
Further, an OS (Operating System) is stored in the auxiliary storage device 203. At least a portion of the OS is loaded into memory 202 and executed by processor 201.
That is, the processor 201 executes the toll collection program while executing the OS.
The data obtained by executing the toll collection program is stored in a storage device such as a memory 202, an auxiliary storage device 203, a register in the processor 201, or a cache memory in the processor 201.

The memory 202 functions as a storage unit 220. However, another storage device may function as the storage unit 220 instead of the memory 202 or together with the memory 202.
The storage unit 220 stores pattern data 230 and the like used for pattern matching with respect to the vehicle image.

The toll collection device 200 may include a plurality of processors that replace the processor 201. The plurality of processors share the role of the processor 201.

The toll collection program can be computer-readablely recorded (stored) on a non-volatile recording medium such as an optical disc or flash memory.
The toll collection program includes a pattern learning program.

*** Explanation of operation ***
The operation of the toll collection device 200 corresponds to a toll collection method and a pattern learning method. Further, the procedure of the charge collection method corresponds to the procedure of the charge collection program, and the procedure of the pattern learning method corresponds to the procedure of the pattern learning program.

A fee collection method and a pattern learning method will be described with reference to FIG.
In step S101, the information receiving unit 211 receives the vehicle data communicated from the vehicle-mounted device 102 of the vehicle 101 to the roadside communication device 111.
Vehicle data includes card information and license plate information.

In step S102, the image receiving unit 212 receives the vehicle image obtained by the roadside camera 112 that captured the vehicle 101.

In step S110, the image recognition unit 213 recognizes the license plate by pattern matching with respect to the vehicle image.

The procedure of the image recognition process (S110) will be described with reference to FIG.
In step S111, the image recognition unit 213 detects one or more characters in the license plate 103 reflected in the vehicle image.

A specific example of step S111 will be described with reference to FIG.
The vehicle 101 is shown in the vehicle image 120.
On the license plate 103 of the vehicle 101, the character string "Shonan 55 a 42-49" is written.
"Shonan" identifies the Transport Branch Office.
"55" identifies the vehicle type.
"A" identifies the use.
"42-49" is a serial number.

In this case, the image recognition unit 213 displays nine characters of "Sho", "South", "5", "5", "A", "4", "2", "4" and "9" as vehicle images. Detect from 120.

The Transport Branch Office, vehicle type, use, and serial number are attributes of the vehicle 101, respectively. In the license plate 103, the place where each attribute is written is determined.
The image recognition unit 213 detects two characters, "Sho" and "South", from the description for the Transport Branch Office.
The image recognition unit 213 detects two characters "5" and "5" from the description location for the vehicle model.
The image recognition unit 213 detects one character "A" from the description location for the purpose.
The image recognition unit 213 detects the four characters "4", "2", "4", and "9" from the description for the serial number.

Returning to FIG. 4, the description will be continued from step S112.
In step S112, the image recognition unit 213 cuts out one or more image portions corresponding to the detected one or more characters from the vehicle image.
The cut out image portion is referred to as a "character image".

In step S113, the image recognition unit 213 uses the pattern data 230 to perform pattern matching on one or more character images.
As a result, the image recognition unit 213 selects one or more character patterns for the detected one or more characters from the pattern data 230.
The pattern matching method is arbitrary. For example, the image recognition unit 213 performs pattern matching in the conventional method.

A specific example of step S113 will be described with reference to FIG.
The pattern data 230 includes a plurality of character patterns 231 corresponding to a plurality of characters. Each character pattern 231 is a standard size image and represents one character. An identifier of the corresponding character is added to each character pattern 231. For example, a character code is added to each character pattern 231.

First, the image recognition unit 213 changes the size of the character image 121 to the standard size by enlarging or reducing the character image 121.
Then, the image recognition unit 213 compares the standard size character image 121 with each character pattern 231 and selects the character pattern 231 most similar to the standard size character image 121.
The image recognition unit 213 selects one character pattern 231 for each character image 121.

Returning to FIG. 4, step S114 will be described.
In step S114, the image recognition unit 213 generates a license plate recognition result based on one or more selected character patterns.
The license plate recognition result indicates one or more characters identified by one or more selected character patterns. The characters shown in the license plate recognition result are called "recognition characters".

Specifically, the image recognition unit 213 writes the recognition character string for each vehicle attribute in the license plate recognition result. That is, the license plate recognition result indicates a recognition character string for the Transport Branch Office, a recognition character string for the vehicle type, a recognition character string for the purpose, and a recognition character string for the serial number.

Returning to FIG. 3, the description is continued from step S120.
In step S120, the toll collection unit 215 collects the toll from the vehicle 101 based on the card information or the license plate recognition result.
For example, when vehicle data is obtained as a result of normal wireless communication (road-vehicle communication) between the roadside communication device 111 and the on-board unit 102, the toll collection unit 215 collects card information from the vehicle data. The fee is collected for the vehicle 101 based on the extracted card information.
For example, when vehicle data cannot be obtained as a result of normal road-to-vehicle communication, the toll collection unit 215 identifies the vehicle 101 based on the license plate recognition result, and the specified vehicle 101 is used. Collect the fee.
The method of collecting fees is optional. For example, the charge collection unit 215 collects charges by a conventional method.

In step S130, the pattern learning unit 214 performs machine learning of the pattern data 230 based on the license plate information and the license plate recognition result.
At this time, the pattern learning unit 214 may perform machine learning of the pattern data 230 based on, for example, a plurality of past data of the license plate information and the license plate recognition result. Further, this machine learning may be performed using artificial intelligence such as a dual network, deep learning, and various types of filtering.
The license plate information is extracted from the received vehicle data.

Specifically, the pattern learning unit 214 compares the license plate recognition result with the license plate information. Then, when the license plate recognition result does not match the license plate information, the pattern learning unit 214 updates the pattern data 230 based on the received vehicle image.

The pattern learning unit 214 updates the pattern data 230 as follows.
First, the pattern learning unit 214 compares one or more recognition characters shown in the license plate recognition result with the character string shown in the license plate information. Then, the pattern learning unit 214 detects the recognition character that does not match the corresponding character in the character string shown in the license plate information among one or more recognition characters. The detected recognition character is referred to as a "misrecognition character".
Next, the pattern learning unit 214 selects a character pattern corresponding to the corresponding character compared with the erroneously recognized character from the pattern data 230.
Then, the pattern learning unit 214 updates the character pattern selected from the pattern data 230 based on the character image corresponding to the erroneously recognized character.

The procedure of the pattern learning process (S130) will be described with reference to FIG. 7.
In step S131, the pattern learning unit 214 selects one unselected recognition character from the license plate recognition result.

In step S132, the pattern learning unit 214 selects a character corresponding to the selected recognition character from the license plate information. The selected character is referred to as a "corresponding character".

The pattern learning unit 214 selects the corresponding character as follows.
Each of the license plate recognition result and the license plate information indicates a character string for each vehicle attribute. That is, each of the license plate recognition result and the license plate information indicates a character string for the Transport Branch Office, a character string for the vehicle type, a character string for the purpose, and a character string for the serial number.
The vehicle attribute of the character string including the recognition character is referred to as "target attribute". A number indicating the order of recognized characters in a character string for a target attribute is referred to as a "target number".
First, the pattern learning unit 214 refers to the license plate recognition result and specifies the target attribute and the target number.
Then, the pattern learning unit 214 selects a character string for the target attribute from the license plate information, and selects a character of the target number from the selected character string. The selected character is the corresponding character.

In step S133, the pattern learning unit 214 compares the selected recognition character with the corresponding character.
If the selected recognition character matches the corresponding character, the process proceeds to step S137.
If the selected recognition character does not match the corresponding character, the process proceeds to step S134. The selected recognition character is referred to as a "misrecognition character".

In step S134, the pattern learning unit 214 selects a character pattern corresponding to the corresponding character from the pattern data 230.
For example, the pattern learning unit 214 selects a character pattern to which the same character code as the character code of the corresponding character is added from the pattern data 230.

In step S135, the pattern learning unit 214 selects a character image corresponding to the erroneously recognized character from one or more character images (see step S112 in FIG. 4) cut out from the vehicle image.

The pattern learning unit 214 selects a character image corresponding to the misrecognized character as follows.
First, the pattern learning unit 214 selects one or more character images cut out from the description location for the target attribute. This target attribute is a vehicle attribute for a character string including a misrecognized character.
Then, the pattern learning unit 214 selects the character image of the target number from one or more selected character images. This target number is a number indicating the order of misrecognized characters in the character string for the target attribute.

In step S136, the pattern learning unit 214 updates the character pattern selected in step S134 based on the character image selected in step S135.
The method of updating the character pattern is arbitrary.

In step S137, the pattern learning unit 214 determines whether or not there is a recognition character (unselected recognition character) not selected in step S131.
If there is an unselected recognition character, the process proceeds to step S131.
If there are no unselected recognition characters, the process ends.

*** Summary of Embodiment 1 ***
In the FFETC system, the vehicle is specified by wireless communication with the on-board unit. Then, the fee is collected. However, wireless communication may not function in the event of a failure. In preparation for such a case, a license plate recognition camera has been introduced as a complementary system for identifying the vehicle.
The first embodiment aims to improve the vehicle recognition performance by the license plate recognition camera.

In the first embodiment, the performance is improved by adding the pattern information acquired in the actual environment to the pattern registered in advance as a dictionary.
In the FFETC system, it is possible to acquire the license plate information of the vehicle in ETC communication. When the on-board unit is set up, the license plate information of the vehicle is stored in the on-board unit, and the license plate information is acquired from the on-board unit by ETC communication.
Therefore, it is possible to update the dictionary at any time by linking the image acquired in the actual environment with the dictionary, assuming that the license plate information acquired by ETC communication is positive.

*** Effect of Embodiment 1 ***
According to the first embodiment, the pattern data used for pattern matching for recognizing the license plate can be updated at any time. As a result, the recognition performance of the license plate is improved.

*** Other configurations ***
The pattern learning device may be provided as a device separate from the toll collection device 200. In that case, the pattern learning device does not need the toll collection unit 215.

*** Supplement to the embodiment ***
The embodiments are examples of preferred embodiments and are not intended to limit the technical scope of the invention. The embodiment may be partially implemented or may be implemented in combination with other embodiments. The procedure described using the flowchart or the like may be appropriately changed.

Each device described in the embodiment may be realized by a plurality of devices. That is, each device described in the embodiment may be realized as a system.
Each element of the device described in the embodiments may be implemented in any of software, hardware, firmware or a combination thereof.
"Part" may be read as "processing" or "process".

100 FFETC system, 101 vehicle, 102 in-vehicle device, 103 license plate, 110 roadside machine, 111 roadside communication device, 112 roadside camera, 120 vehicle image, 121 character image, 200 charge collection device, 201 processor, 202 memory, 203 auxiliary storage Device, 204 communication device, 205 input / output interface, 211 information reception unit, 212 image reception unit, 213 image recognition unit, 214 pattern learning unit, 215 charge collection unit, 220 storage unit, 230 pattern data, 231 character patterns.

Claims (6)

The information reception unit that receives the license plate information transmitted from the on-board unit of the vehicle to the roadside communication device,
An image reception unit that receives vehicle images obtained by the roadside camera that captured the vehicle, and an image reception unit.
An image recognition unit that generates a license plate recognition result that is compared with the license plate information that is accepted by pattern matching to the accepted vehicle image.
A pattern learning system including a pattern learning unit that updates the pattern data used for pattern matching based on the received vehicle image when the generated license plate recognition result does not match the received license plate information. .. The image recognition unit is
One or more characters in the number plate reflected in the vehicle image are detected, and one or more image portions corresponding to the detected one or more characters are used as one or more character images from the vehicle image. One or more character patterns corresponding to the one or more characters detected by cutting out and pattern matching to the one or more character images are selected from the pattern data and identified by the selected one or more character patterns. The pattern learning system according to claim 1, wherein data indicating one or more recognition characters to be recognized is generated as the number plate recognition result. The license plate information indicates a character string written on the license plate of the vehicle.
The pattern learning unit
By comparing one or more recognition characters shown in the number plate recognition result with the character string shown in the number plate information, the one or more recognition characters do not match the corresponding characters in the character string. The recognized character is detected as a misrecognized character, a character pattern corresponding to the corresponding character compared with the misrecognized character is selected from the pattern data, and the selected character pattern is based on the character image corresponding to the misrecognized character. The pattern learning system according to claim 2, which is updated. On the license plate of the vehicle, a character string is written for each vehicle attribute.
Each of the license plate recognition result and the license plate information indicates a character string for each vehicle attribute.
The pattern learning unit
The recognition characters are selected one by one from the number plate recognition result, the vehicle attribute of the character string including the selected recognition character is specified as the target attribute, and the selected recognition is performed in the character string including the selected recognition character. A number indicating the order of characters is specified as a target number, a character string of the target attribute is selected from the number plate information, a character of the target number is selected as the corresponding character from the selected character string, and the selected character is selected. The recognized characters are compared with the corresponding characters, and when the selected recognition characters do not match the corresponding characters, the selected recognition characters are detected as the erroneous recognition characters, and the character pattern corresponding to the corresponding characters is detected. Is selected from the pattern data, and one or more character images cut out from the description location of the target attribute are selected from the one or more character images cut out from the vehicle image, and one or more selected characters are selected. The character image of the target number is selected as the character image corresponding to the misrecognized character from the character image of, and the selected character pattern is updated based on the character image corresponding to the misrecognized character.
The pattern learning system according to claim 3. The information reception unit receives the license plate information transmitted from the on-board unit of the vehicle to the roadside communication device,
The image reception unit receives the vehicle image obtained by the roadside camera that photographed the vehicle, and receives the image.
The image recognition unit generates a license plate recognition result to be compared with the license plate information received by pattern matching for the received vehicle image.
A pattern learning method in which the pattern learning unit updates the pattern data used for pattern matching based on the received vehicle image when the generated license plate recognition result does not match the received license plate information. The information reception unit that receives the license plate information transmitted from the on-board unit of the vehicle to the roadside communication device,
An image reception unit that receives vehicle images obtained by the roadside camera that captured the vehicle, and an image reception unit.
An image recognition unit that generates a license plate recognition result that is compared with the license plate information that is accepted by pattern matching to the accepted vehicle image.
As a pattern learning unit that updates the pattern data used for pattern matching based on the accepted vehicle image when the generated license plate recognition result does not match the accepted license plate information.
A pattern learning program for making a computer work.

Images