JP2020144595A - Pattern learning system, pattern learning method and pattern learning program - Google Patents

Abstract

To improve the performance of recognizing a license plate.SOLUTION: A toll collection device 200 receives license plate information communicated from an in-vehicle device 102 of a vehicle 101 to a roadside communication device 111. The toll collection device 200 receives a vehicle image obtained by a roadside camera 112 that captures the vehicle 101. The toll collection device 200 generates a license plate recognition result to be compared with the license plate information received by pattern matching with respect to the vehicle image. When the generated license plate recognition result does not match the received license plate information, the toll collection device 200 updates pattern data used for pattern matching based on the received vehicle image.SELECTED DRAWING: Figure 1

Description

The present invention relates to pattern matching for recognizing license plates.

With the spread of ETC, the impact of the suspension of the ETC system is increasing. Therefore, the license plate recognition camera for complementing the ETC system is becoming more important. And it is becoming more important to improve the license plate recognition performance.

At present, pattern matching is performed using a dictionary (pattern) learned in advance in order to recognize characters cut out from an image.
The dictionary used in pattern matching is generated in a static environment using a plurality of still images with different license plate accuracy.

In a real environment, a moving vehicle is photographed by a license plate recognition camera. The license plate shown in the image obtained by shooting is matched with each pattern registered as a dictionary. Then, based on the matching result, the license plate of the running vehicle is recognized.

Patent Document 1 discloses a license plate recognition device.
This license plate recognition device displays the original image of the erroneously recognized characters on the monitor, and performs learning based on the correct characters input using the keyboard.

Japanese Unexamined Patent Publication No. 5-54196

An object of the present invention is to improve the recognition performance of a license plate.

The pattern learning system of the present invention
An information reception unit that receives license plate information communicated from the on-board unit of the vehicle to the roadside communication device,
An image reception unit that receives vehicle images obtained by the roadside camera that photographed the vehicle, and
An image recognition unit that generates a license plate recognition result that is compared with the received license plate information by pattern matching for the received vehicle image.
It is provided with a pattern learning unit that updates the pattern data used for pattern matching based on the received vehicle image when the generated license plate recognition result does not match the received license plate information.

According to the present invention, the pattern data used for pattern matching for recognizing the license plate can be updated at any time. As a result, the license plate recognition performance is improved.

The block diagram of the FFETC system 100 in Embodiment 1. FIG. The block diagram of the charge collecting apparatus 200 according to Embodiment 1. FIG. The flowchart of the charge collection method and the pattern learning method in Embodiment 1. The flowchart of the image recognition process (S110) in Embodiment 1. The figure which shows the specific example of the license plate 103 in Embodiment 1. FIG. The schematic diagram of the pattern matching in Embodiment 1. The flowchart of the pattern learning process (S130) in Embodiment 1.

In embodiments and drawings, the same or corresponding elements are designated by the same reference numerals. Descriptions of elements with the same reference numerals as the described elements will be omitted or simplified as appropriate. The arrows in the figure mainly indicate the flow of data or the flow of processing.

Embodiment 1.
A mode for updating the pattern data used for pattern matching for recognizing the license plate at any time will be described with reference to FIGS. 1 to 7.

*** Explanation of configuration ***
The configuration of the FFETC system 100 will be described with reference to FIG.
The FFETC system 100 is a system for collecting tolls for a vehicle 101 passing through a toll road.
FFETC is an abbreviation for free flow ETC.
ETC is an abbreviation for Electronic Toll Collection.

The vehicle 101 is equipped with an on-board unit 102.
An ETC card is inserted into the vehicle-mounted device 102.
The ETC card is an IC card for collecting tolls. IC is an abbreviation for Integrated Circuit.
Various information for collecting charges is recorded on the ETC card. Various information recorded on the ETC card is called card information.

Further, license plate information is registered in the vehicle-mounted device 102.
The license plate information indicates a character string written on the license plate 103 of the vehicle 101.

The FFETC system 100 includes a roadside machine 110 and a toll collection device 200.
The roadside machine 110 is installed around the toll road (for example, above the road).

The roadside machine 110 includes a roadside communication device 111 and a roadside camera 112.
The roadside communication device 111 wirelessly communicates with the vehicle-mounted device 102 of the vehicle 101. For example, the roadside communication device 111 receives vehicle data from the vehicle-mounted device 102. The vehicle data is data including various information about the vehicle 101. Vehicle data includes card information and license plate information.
The roadside camera 112 obtains a vehicle image by photographing the vehicle 101. The vehicle image is an image of the vehicle 101. The roadside camera 112 is also referred to as a license plate recognition camera.

The toll collection device 200 is a device that collects tolls for a vehicle 101 passing through a toll road.
The toll collection device 200 collects tolls from the vehicle 101 based on the vehicle data communicated from the vehicle-mounted device 102 to the roadside communication device 111 and the vehicle image obtained by the roadside camera 112 that captures the vehicle 101. Do.

The FFETC system 100 has the function of a pattern learning system. The toll collection device 200 has a function of a pattern learning device.
Pattern learning is a function for machine learning pattern data used for pattern matching on a vehicle image. Pattern matching for the vehicle image is performed to recognize the license plate 103 using the vehicle image.

The configuration of the toll collection device 200 will be described with reference to FIG.
The toll collection device 200 is a computer including hardware such as a processor 201, a memory 202, an auxiliary storage device 203, a communication device 204, and an input / output interface 205. These hardware are connected to each other via signal lines.

The processor 201 is an IC (Integrated Circuit) that performs arithmetic processing, and controls other hardware. For example, the processor 201 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit).
The memory 202 is a volatile storage device. The memory 202 is also referred to as a main storage device or a main memory. For example, the memory 202 is a RAM (Random Access Memory). The data stored in the memory 202 is stored in the auxiliary storage device 203 as needed.
The auxiliary storage device 203 is a non-volatile storage device. For example, the auxiliary storage device 203 is a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. The data stored in the auxiliary storage device 203 is loaded into the memory 202 as needed.
The communication device 204 is a receiver and a transmitter. For example, the communication device 204 is a communication chip or a NIC (Network Interface Card).
The input / output interface 205 is a port to which an input device and an output device are connected. For example, the input / output interface 205 is a USB terminal, the input device is a keyboard and a mouse, and the output device is a display. USB is an abbreviation for Universal Serial Bus.

The toll collection device 200 includes elements such as an information reception unit 211, an image reception unit 212, an image recognition unit 213, a pattern learning unit 214, and a toll collection unit 215. These elements are realized in software.

The auxiliary storage device 203 stores a charge collection device program for operating a computer as an information reception unit 211, an image reception unit 212, an image recognition unit 213, a pattern learning unit 214, and a charge collection unit 215. The toll collection program is loaded into memory 202 and executed by processor 201.
Further, the auxiliary storage device 203 stores an OS (Operating System). At least a portion of the OS is loaded into memory 202 and executed by processor 201.
That is, the processor 201 executes the toll collection program while executing the OS.
The data obtained by executing the toll collection program is stored in a storage device such as a memory 202, an auxiliary storage device 203, a register in the processor 201, or a cache memory in the processor 201.

The memory 202 functions as a storage unit 220. However, another storage device may function as the storage unit 220 instead of the memory 202 or together with the memory 202.
The storage unit 220 stores pattern data 230 and the like used for pattern matching with respect to the vehicle image.

The toll collection device 200 may include a plurality of processors that replace the processor 201. The plurality of processors share the role of the processor 201.

The toll collection program can be computer-readablely recorded (stored) on a non-volatile recording medium such as an optical disc or flash memory.
The toll collection program includes a pattern learning program.

*** Explanation of operation ***
The operation of the toll collection device 200 corresponds to the toll collection method and the pattern learning method. Further, the procedure of the toll collection method corresponds to the procedure of the toll collection program, and the procedure of the pattern learning method corresponds to the procedure of the pattern learning program.

A fee collection method and a pattern learning method will be described with reference to FIG.
In step S101, the information receiving unit 211 receives the vehicle data communicated from the vehicle-mounted device 102 of the vehicle 101 to the roadside communication device 111.
Vehicle data includes card information and license plate information.

In step S102, the image receiving unit 212 receives the vehicle image obtained by the roadside camera 112 that photographed the vehicle 101.

In step S110, the image recognition unit 213 recognizes the license plate by pattern matching with respect to the vehicle image.

The procedure of the image recognition process (S110) will be described with reference to FIG.
In step S111, the image recognition unit 213 detects one or more characters in the license plate 103 appearing in the vehicle image.

A specific example of step S111 will be described with reference to FIG.
The vehicle 101 is shown in the vehicle image 120.
On the license plate 103 of the vehicle 101, the character string "Shonan 55 a 42-49" is written.
"Shonan" identifies the transportation branch office.
"55" identifies the vehicle type.
"A" identifies the use.
"42-49" is a serial number.

In this case, the image recognition unit 213 displays the nine characters "Sho", "South", "5", "5", "A", "4", "2", "4" and "9" as the vehicle image. Detect from 120.

The transportation branch office, vehicle type, use, and serial number are attributes of the vehicle 101, respectively. In the license plate 103, the place where each attribute is written is determined.
The image recognition unit 213 detects two characters, "sho" and "south", from the description for the transportation branch office.
The image recognition unit 213 detects two characters "5" and "5" from the description location for the vehicle type.
The image recognition unit 213 detects the single character "a" from the description for the purpose.
The image recognition unit 213 detects the four characters "4", "2", "4", and "9" from the description for the serial number.

Returning to FIG. 4, the description continues from step S112.
In step S112, the image recognition unit 213 cuts out one or more image portions corresponding to the detected one or more characters from the vehicle image.
The cut out image portion is referred to as a "character image".

In step S113, the image recognition unit 213 uses the pattern data 230 to perform pattern matching on one or more character images.
As a result, the image recognition unit 213 selects one or more character patterns for the one or more detected characters from the pattern data 230.
The pattern matching method is arbitrary. For example, the image recognition unit 213 performs pattern matching in the conventional method.

A specific example of step S113 will be described with reference to FIG.
The pattern data 230 includes a plurality of character patterns 231 corresponding to a plurality of characters. Each character pattern 231 is a standard size image and represents one character. An identifier of the corresponding character is added to each character pattern 231. For example, a character code is added to each character pattern 231.

First, the image recognition unit 213 changes the size of the character image 121 to the standard size by enlarging or reducing the character image 121.
Then, the image recognition unit 213 compares the standard size character image 121 with each character pattern 231 and selects the character pattern 231 most similar to the standard size character image 121.
The image recognition unit 213 selects one character pattern 231 for each character image 121.

Returning to FIG. 4, step S114 will be described.
In step S114, the image recognition unit 213 generates a license plate recognition result based on one or more selected character patterns.
The license plate recognition result indicates one or more characters identified by one or more selected character patterns. The characters shown in the license plate recognition result are called "recognition characters".

Specifically, the image recognition unit 213 writes the recognition character string for each vehicle attribute in the license plate recognition result. That is, the license plate recognition result indicates a recognition character string for the transportation branch office, a recognition character string for the vehicle type, a recognition character string for the purpose, and a recognition character string for the serial number.

Returning to FIG. 3, the description continues from step S120.
In step S120, the toll collection unit 215 collects the toll from the vehicle 101 based on the card information or the license plate recognition result.
For example, when vehicle data is obtained as a result of normal wireless communication (road-vehicle communication) between the roadside communication device 111 and the on-board unit 102, the toll collection unit 215 collects card information from the vehicle data. The fee is collected for the vehicle 101 based on the extracted card information.
For example, when vehicle data cannot be obtained as a result of normal road-to-vehicle communication, the toll collection unit 215 identifies the vehicle 101 based on the license plate recognition result, and refers to the specified vehicle 101. Collect the fee.
The method of collecting the fee is optional. For example, the toll collection unit 215 collects tolls by a conventional method.

In step S130, the pattern learning unit 214 performs machine learning of the pattern data 230 based on the license plate information and the license plate recognition result.
At this time, the pattern learning unit 214 may perform machine learning of the pattern data 230 based on, for example, a plurality of past data of the license plate information and the license plate recognition result. In addition, this machine learning may be performed using artificial intelligence such as a dual network, deep learning, and various types of filtering.
The license plate information is extracted from the received vehicle data.

Specifically, the pattern learning unit 214 compares the license plate recognition result with the license plate information. Then, when the license plate recognition result does not match the license plate information, the pattern learning unit 214 updates the pattern data 230 based on the received vehicle image.

The pattern learning unit 214 updates the pattern data 230 as follows.
First, the pattern learning unit 214 compares one or more recognition characters shown in the license plate recognition result with the character string shown in the license plate information. Then, the pattern learning unit 214 detects a recognition character that does not match the corresponding character in the character string shown in the license plate information among one or more recognition characters. The detected recognition character is referred to as a "misrecognition character".
Next, the pattern learning unit 214 selects a character pattern corresponding to the corresponding character compared with the erroneously recognized character from the pattern data 230.
Then, the pattern learning unit 214 updates the character pattern selected from the pattern data 230 based on the character image corresponding to the erroneously recognized character.

The procedure of the pattern learning process (S130) will be described with reference to FIG. 7.
In step S131, the pattern learning unit 214 selects one unselected recognition character from the license plate recognition result.

In step S132, the pattern learning unit 214 selects a character corresponding to the selected recognition character from the license plate information. The selected character is referred to as a "corresponding character".

The pattern learning unit 214 selects the corresponding character as follows.
Each of the license plate recognition result and the license plate information indicates a character string for each vehicle attribute. That is, each of the license plate recognition result and the license plate information indicates a character string for the transportation branch office, a character string for the vehicle type, a character string for the purpose, and a character string for the serial number.
The vehicle attribute of the character string including the recognition character is referred to as "target attribute". A number indicating the order of recognized characters in a character string for a target attribute is referred to as a "target number".
First, the pattern learning unit 214 refers to the license plate recognition result and identifies the target attribute and the target number.
Then, the pattern learning unit 214 selects a character string for the target attribute from the license plate information, and selects the character of the target number from the selected character string. The selected character is the corresponding character.

In step S133, the pattern learning unit 214 compares the selected recognition character with the corresponding character.
If the selected recognition character matches the corresponding character, the process proceeds to step S137.
If the selected recognition character does not match the corresponding character, the process proceeds to step S134. The selected recognition character is referred to as a "misrecognition character".

In step S134, the pattern learning unit 214 selects a character pattern corresponding to the corresponding character from the pattern data 230.
For example, the pattern learning unit 214 selects a character pattern to which the same character code as the character code of the corresponding character is added from the pattern data 230.

In step S135, the pattern learning unit 214 selects a character image corresponding to the erroneously recognized character from one or more character images (see step S112 in FIG. 4) cut out from the vehicle image.

The pattern learning unit 214 selects a character image corresponding to the erroneously recognized character as follows.
First, the pattern learning unit 214 selects one or more character images cut out from the description location for the target attribute. This target attribute is a vehicle attribute for a character string including a misrecognized character.
Then, the pattern learning unit 214 selects the character image of the target number from one or more selected character images. This target number is a number indicating the order of misrecognized characters in the character string for the target attribute.

In step S136, the pattern learning unit 214 updates the character pattern selected in step S134 based on the character image selected in step S135.
The method of updating the character pattern is arbitrary.

In step S137, the pattern learning unit 214 determines whether or not there is a recognition character (unselected recognition character) not selected in step S131.
If there is an unselected recognition character, the process proceeds to step S131.
If there are no unselected recognition characters, the process ends.

*** Summary of Embodiment 1 ***
In the FFETC system, the vehicle is identified by wireless communication with the on-board unit. Then, the fee is collected. However, wireless communication may not function in the event of a failure. In preparation for such a case, a license plate recognition camera has been introduced as a complementary system for identifying the vehicle.
The first embodiment is intended to improve the vehicle recognition performance by the license plate recognition camera.

In the first embodiment, the performance is improved by adding the pattern information acquired in the actual environment to the pattern registered in advance as a dictionary.
In the FFETC system, it is possible to acquire the license plate information of the vehicle in ETC communication. When the on-board unit is set up, the license plate information of the vehicle is stored in the on-board unit, and the license plate information is acquired from the on-board unit by ETC communication.
Therefore, it is possible to update the dictionary at any time by linking the image acquired in the actual environment with the dictionary, assuming that the license plate information acquired by ETC communication is positive.

*** Effect of Embodiment 1 ***
According to the first embodiment, the pattern data used for pattern matching for recognizing the license plate can be updated at any time. As a result, the license plate recognition performance is improved.

*** Other configurations ***
The pattern learning device may be provided as a device separate from the toll collection device 200. In that case, the pattern learning device does not need the toll collection unit 215.

*** Supplement to the embodiment ***
The embodiments are examples of preferred embodiments and are not intended to limit the technical scope of the invention. The embodiment may be partially implemented or may be implemented in combination with other embodiments. The procedure described using the flowchart or the like may be appropriately changed.

Each device described in the embodiment may be realized by a plurality of devices. That is, each device described in the embodiment may be realized as a system.
Each element of the device described in the embodiments may be implemented in any of software, hardware, firmware or a combination thereof.
"Part" may be read as "process" or "process".

100 FFETC system, 101 vehicle, 102 on-board unit, 103 number plate, 110 roadside unit, 111 roadside communication device, 112 roadside camera, 120 vehicle image, 121 character image, 200 toll collection device, 201 processor, 202 memory, 203 auxiliary storage Device, 204 communication device, 205 input / output interface, 211 information reception unit, 212 image reception unit, 213 image recognition unit, 214 pattern learning unit, 215 charge collection unit, 220 storage unit, 230 pattern data, 231 character patterns.

Claims (5)

An information reception unit that receives license plate information communicated from the on-board unit of the vehicle to the roadside communication device,
An image reception unit that receives vehicle images obtained by the roadside camera that photographed the vehicle, and
An image recognition unit that generates a license plate recognition result that is compared with the received license plate information by pattern matching for the received vehicle image.
A pattern learning system including a pattern learning unit that updates the pattern data used for pattern matching based on the received vehicle image when the generated license plate recognition result does not match the received license plate information. .. The image recognition unit
One or more characters in the license plate appearing in the vehicle image are detected, and one or more image portions corresponding to the detected one or more characters are used as one or more character images from the vehicle image. One or more character patterns corresponding to the one or more characters detected by cutting out and pattern matching the one or more character images are selected from the pattern data and identified by the selected one or more character patterns. The pattern learning system according to claim 1, wherein data indicating one or more recognition characters to be generated is generated as the number plate recognition result. The license plate information indicates a character string written on the license plate of the vehicle.
The pattern learning unit
By comparing one or more recognition characters shown in the number plate recognition result with the character string shown in the number plate information, the one or more recognition characters do not match the corresponding characters in the character string. The recognized character is detected as a misrecognized character, a character pattern corresponding to the corresponding character compared with the misrecognized character is selected from the pattern data, and the selected character pattern is based on the character image corresponding to the misrecognized character. The pattern learning system according to claim 2, which is updated. The information reception department receives the license plate information communicated from the on-board unit of the vehicle to the roadside communication device,
The image reception unit receives the vehicle image obtained by the roadside camera that photographed the vehicle,
The image recognition unit generates a license plate recognition result to be compared with the received license plate information by pattern matching with respect to the received vehicle image.
A pattern learning method in which the pattern learning unit updates the pattern data used for pattern matching based on the received vehicle image when the generated license plate recognition result does not match the received license plate information. An information reception unit that receives license plate information communicated from the on-board unit of the vehicle to the roadside communication device,
An image reception unit that receives vehicle images obtained by the roadside camera that photographed the vehicle, and
An image recognition unit that generates a license plate recognition result that is compared with the received license plate information by pattern matching for the received vehicle image.
As a pattern learning unit that updates the pattern data used for pattern matching based on the received vehicle image when the generated license plate recognition result does not match the received license plate information.
A pattern learning program for making a computer work.

Images