JP2022181678A - Information processing system and program - Google Patents

Abstract

To provide an information processing system and a program for suitably collecting information related to a vehicle.SOLUTION: A program according to an aspect of present disclosure causes a computer to execute a detection step, an extraction step, a character recognition step, and an output control step. In the detection step, a license plate of a surrounding vehicle located around a vehicle is detected from a photographed image generated by photographing a landscape around the vehicle using a detection model that outputs a position of an image area showing the license plate contained in an input image. In the extraction step, an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle is cut out from the photographed image to generate an extraction image. In the character recognition step, character information is acquired from the extracted image using a character recognition model, and license plate information of the surrounding vehicle is generated based on the character information. In the output control step, the license plate information of the surrounding vehicle is output.SELECTED DRAWING: Figure 4

Description

TECHNICAL FIELD The present disclosure relates to an information processing system and program, and more particularly to an information processing system and program for acquiring information related to a vehicle.

Techniques for collecting vehicle-related information have been proposed in order to monitor vehicles traveling on roads and vehicles entering and exiting parking lots. For example, in Patent Document 1, a vehicle number recognition system obtains the position and size of a license plate from an image of a running vehicle captured by an in-vehicle camera, extracts a character image, and obtains the vehicle number from the character image by character recognition. is disclosed. The above-mentioned Patent Literature 1 discloses that the vehicle number recognition system records the acquired vehicle number together with the speed data and position data of the traveling vehicle.

Further, in Patent Document 2, character recognition is performed by detecting a vehicle body from a photographed image, detecting a license plate from the vehicle body image, extracting a character partial image from the extracted license plate image, and executing character recognition using the character partial image. An apparatus is disclosed.

JP-A-11-296785 Patent No. 6779491

However, the above-mentioned Patent Document 1 does not disclose a detailed method of acquiring the speed data and position data of the traveling vehicle, and does not disclose a detailed method of detecting the license plate of the traveling vehicle from the image captured by the in-vehicle camera. is also not disclosed.

In addition, the method described in Patent Document 2 above has a problem that the processing load is high because there are many processing procedures until character recognition is executed. It should be noted that the aforementioned Patent Document 2 does not disclose acquisition of information related to travel of other vehicles.
Therefore, there is a demand for more suitable collection of vehicle-related information.

An object of the present disclosure is to provide an information processing system and a program that suitably collect vehicle-related information in view of the above-described problems.

A program according to an aspect of the present disclosure causes a computer to execute an image acquisition stage, an identification information estimation stage, a driving information generation stage, and an output control stage. The image acquisition step acquires a photographed image generated by photographing the scenery around the vehicle. The identification information estimation step estimates identification information of surrounding vehicles located around the vehicle based on the captured image. The driving information generating step acquires first driving information about driving of the vehicle, and generates second driving information about driving of the surrounding vehicle based on the first driving information. The output control step causes the identification information and the second travel information to be output in association with each other.

A program according to an aspect of the present disclosure causes a computer to execute a detection stage, an extraction stage, a character recognition stage, and an output control stage. The detection step uses a detection model that outputs the position of an image area showing a license plate contained in an input image from a captured image generated by capturing the scenery around the vehicle to locate the area around the vehicle. Detect license plates of surrounding vehicles. In the extracting step, an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle is cut out from the photographed image to generate an extraction image. In the character recognition step, character information is obtained from the extracted image using a character recognition model, and license plate information of the surrounding vehicle is generated based on the character information. The output control step outputs license plate information of the surrounding vehicle.

An information processing system according to one aspect of the present disclosure includes an image acquisition unit, an identification information estimation unit, a travel information generation unit, and an output control unit. The image acquisition unit acquires a photographed image of scenery around the vehicle. The identification information estimation unit estimates identification information of surrounding vehicles positioned around the vehicle based on the captured image. The travel information generation unit acquires first travel information about travel of the vehicle, and generates second travel information about travel of the surrounding vehicle based on the first travel information. The output control unit causes the identification information and the second travel information to be output in association with each other.

An information processing system according to one aspect of the present disclosure includes a detection unit, an extraction unit, a character recognition unit, and an output control unit. The detection unit uses a detection model that outputs the position of the image area showing the license plate included in the input image from the captured image of the scenery around the vehicle, and detects the license plates of the surrounding vehicles located around the vehicle. Detect the plate. The extraction unit cuts out an extraction area corresponding to an image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image. The character recognition unit acquires character information from the extracted image using a character recognition model, and generates license plate information of the surrounding vehicle based on the character information. The output control unit outputs the license plate information of the surrounding vehicle.

According to the present disclosure, it is possible to provide an information processing system and a program that suitably collect vehicle-related information.

1 is a block diagram showing the configuration of an information processing system according to a first embodiment; FIG. 4 is a flow chart showing the flow of an information processing method according to the first embodiment; 2 is a block diagram showing the configuration of an information processing system according to a second embodiment; FIG. 9 is a flow chart showing the flow of an information processing method according to the second embodiment; FIG. 11 is a schematic configuration diagram of a vehicle management system according to Embodiment 3; FIG. 11 is a block diagram showing the configuration of an in-vehicle system according to Embodiment 3; FIG. 12 is a diagram showing an example of a photographed image according to the third embodiment; FIG. FIG. 11 is a diagram showing an example of an extracted image according to the third embodiment; FIG. FIG. 11 is a diagram for explaining character recognition processing according to the third embodiment; FIG. 11 is a diagram showing an example of a recognition result according to the third embodiment; FIG. 11 is a flow chart showing the flow of an information processing method according to the third embodiment; It is a figure which shows an example of a picked-up image. FIG. 12 is a diagram showing an example of a photographed image according to the first modified example of the third embodiment; FIG. FIG. 14 is a diagram for explaining a process of synthesizing license plate information according to the second modification of the third embodiment; FIG. 11 is a block diagram showing the configuration of an in-vehicle system according to a third modified example of the third embodiment; FIG. 11 is a schematic configuration diagram of a vehicle management system according to a fourth embodiment; FIG. 11 is a block diagram showing the configuration of a recognition training data generation device according to a fourth embodiment; FIG. 13 is a diagram showing an example of the data structure of annotation information according to the fourth embodiment; FIG. FIG. 14 is a diagram for explaining format information and drawing point information according to the fourth embodiment; FIG. 14 is a diagram showing an example of the data structure of a drawing point information DB according to the fourth embodiment; FIG. FIG. 14 is a flow chart showing the flow of a process of generating training data for recognition according to the fourth embodiment; FIG. FIG. 11 is a diagram for explaining a drawing method by an image generator according to the fourth embodiment; FIG. FIG. 12 is a diagram for explaining correction processing by a correction unit according to the fourth embodiment; FIG. 14 is a flowchart showing the flow of character recognition model learning processing according to the fourth embodiment; FIG. FIG. 11 is a schematic configuration diagram of a vehicle management system according to a fifth embodiment; FIG. 12 is a diagram for explaining character recognition processing according to the fifth embodiment; It is a figure which shows the structural example of a computer.

Hereinafter, the present disclosure will be described through embodiments, but the disclosure according to the scope of claims is not limited to the following embodiments. Moreover, not all the configurations described in the embodiments are essential as means for solving the problems. In each drawing, the same elements are denoted by the same reference numerals, and redundant description is omitted as necessary.

<Problem of Embodiment>
Here, the problem of the embodiment will be explained again. Vehicles are photographed by cameras fixedly installed at the entrances of roads and parking lots, and surveillance systems are used to recognize license plates from the photographed images. However, in the surveillance system described above, the cameras must be fixedly installed, and there is a problem that the cameras cannot be easily installed due to the high installation cost and constructional restrictions. For this reason, the above monitoring system currently covers only major roads. In recent years, a portable monitoring system has been developed, but it takes time and effort to install it.

Here, the vehicle number recognition system described in the above-mentioned Patent Document 1 discloses that a running vehicle is photographed by an in-vehicle camera instead of a fixedly installed camera, and a vehicle number is acquired. The vehicle number recognition system described in the above-mentioned Patent Document 1 discloses that the acquired vehicle number is recorded together with the speed data and position data of the traveling vehicle. Here, when recognizing a license plate using an in-vehicle camera, it is required to more preferably acquire traveling information such as speed data and position data of the traveling vehicle in order to more preferably monitor the vehicle. However, Patent Literature 1 mentioned above does not disclose a detailed method for acquiring the speed data and position data of the traveling vehicle.

It should be noted that the recognition target is not limited to the license plate of the running vehicle, and the same problem exists even if it is the type or body color of the running vehicle.

Moreover, the above-mentioned Patent Document 2 discloses a technique of acquiring license plate information based on a photographed image. However, as described above, the method described in Patent Literature 2 has a large processing load due to the large number of processing steps up to execution of character recognition. Therefore, when the recognition target is a license plate, it is required to obtain the license plate information more preferably.

Thus, there is a demand for more suitable collection of vehicle-related information. The present embodiment is made to solve such problems.

<Embodiment 1>
Next, Embodiment 1 of the present disclosure will be described. FIG. 1 is a block diagram showing the configuration of an information processing system 900 according to the first embodiment. The information processing system 900 is a computer system that acquires vehicle identification information and travel information as vehicle-related information. The information processing system 900 includes an image acquisition section 902 , an identification information estimation section 903 , a travel information generation section 907 and an output control section 908 .

The image acquisition unit 902 acquires a captured image generated by capturing the scenery around the vehicle. A vehicle is a mobile object such as an automobile, a train, or a bus. The vehicle is equipped with a camera (not shown) that captures the surrounding scenery. The camera may be a camera built into the drive recorder, but may be any other vehicle-mounted camera. Hereinafter, a vehicle in which the surrounding scenery is captured by a camera is sometimes referred to as a captured vehicle (own vehicle). The image acquisition unit 902 supplies the captured image to the identification information estimation unit 903 .

The identification information estimation unit 903 estimates identification information of surrounding vehicles located around the photographed vehicle based on the photographed image. The peripheral vehicle may be a vehicle traveling in front or behind the photographing vehicle, a vehicle traveling opposite the photographing vehicle, or a vehicle parked or stopped in the vicinity of the photographing vehicle. good too. The identification information is information that identifies the vehicle, and may be identification information that can be estimated from the appearance of the vehicle. The identification information may be, for example, license plate, vehicle type, or body color information.
The identification information estimation unit 903 then supplies the identification information of the surrounding vehicle to the output control unit 908 .

The travel information generation unit 907 acquires first travel information regarding travel of the photographing vehicle. The first travel information includes at least one of location information, traveling direction, and speed of the photographing vehicle. Based on the first travel information, the travel information generation unit 907 generates second travel information regarding the travel of the surrounding vehicle. The second travel information includes at least one of location information, traveling direction, and speed of surrounding vehicles. For example, the travel information generator 907 generates the same information as the first travel information as the second travel information. Further, for example, the travel information generation unit 907 generates second travel information based on the captured image and the first travel information. As an example, the travel information generation unit 907 calculates the relative position of the surrounding vehicle with respect to the photographed vehicle based on at least one of the size and position of the image area of the surrounding vehicle included in the photographed image. Then, the travel information generation unit 907 may calculate the positions of surrounding vehicles based on the position information and the relative positions of the shooting vehicle. In addition, the position information of the shooting vehicle is obtained by receiving positioning information for positioning the position of the own vehicle from a satellite positioning system such as GNSS (global navigation satellite system), for example GPS (Global Positioning System). can be done. As an example, the travel information generation unit 907 tracks the surrounding vehicles included in the captured image, and calculates the relative speed of the surrounding vehicles with respect to the captured vehicle based on the difference between the frames in the position and size of the image area of the surrounding vehicles. Calculate Then, the travel information generation unit 907 may calculate the speed of surrounding vehicles based on the speed and relative speed of the photographing vehicle. Note that the speed of the photographing vehicle can be acquired from the vehicle speed signal. Further, the traveling information generation unit 907 may calculate the traveling direction of the surrounding vehicle based on the traveling direction of the photographing vehicle and the positive/negative of the relative speed.
The travel information generator 907 then supplies the second travel information to the output controller 908 .

The output control unit 908 outputs the identification information and the second travel information in association with each other. Here, the output may be outputting the associated information to an output unit (not shown) of the information processing system 900 in a predetermined data format, or may be output to an external device (not shown) via a network. shown). The output may be storing the associated information in a storage unit (not shown) of the information processing system 900 .

FIG. 2 is a flow chart showing the flow of the information processing method according to the first embodiment. First, the image acquisition unit 902 of the information processing system 900 acquires a captured image generated by capturing the scenery around the vehicle (step S10). Next, the identification information estimation unit 903 estimates identification information of surrounding vehicles based on the captured image acquired in step S10 (step S11). Next, the travel information generator 907 acquires first travel information of the photographing vehicle (step S12). Next, the travel information generation unit 907 generates second travel information of surrounding vehicles based on the first travel information of the photographing vehicle (step S13). The processing shown in steps S12 and S13 may be performed before step S11 or may be performed in parallel with step S11. Next, the output control unit 908 outputs the identification information of the surrounding vehicle estimated in step S11 and the second travel information of the surrounding vehicle generated in step S13 in association with each other (step S14).

As described above, according to the first embodiment, the information processing system 900 can photograph surrounding vehicles from the photographing vehicle, thereby avoiding the problems of installation costs and architectural installation restrictions. For example, just by driving the photographing vehicle on the road, it is possible to acquire information related to the vehicle ahead or behind without special equipment investment. Also, for example, just by traveling on a road wide enough for the photographing vehicle to pass the oncoming vehicle, it is possible to acquire information related to the oncoming vehicle without requiring any special equipment investment.

The information processing system 900 can easily generate travel information of surrounding vehicles without requiring a complicated sensor, and can output the generated travel information in association with the identification information of the surrounding vehicles. Therefore, the information processing system 900 can suitably collect information related to the vehicle.

<Embodiment 2>
Next, Embodiment 2 of the present disclosure will be described. FIG. 3 is a block diagram showing the configuration of an information processing system 910 according to the second embodiment. The information processing system 910 is a computer system that acquires vehicle license plate information as vehicle-related information. The information processing system 910 includes a detection unit 914 , an extraction unit 915 , a character recognition unit 916 and an output control unit 918 .

A detection unit 914 uses a detector (also called a detection model) to detect the license plates of surrounding vehicles located around the vehicle from a captured image generated by capturing the scenery around the vehicle. As in the first embodiment, the vehicle is equipped with an in-vehicle camera (not shown) that captures the surrounding scenery. The camera configuration may be the same as in the first embodiment. The detection model outputs the positional information of the image area indicating the license plate included in the input image. The detection model is a model including a trained convolutional neural network (CNN=Convolutional Neural Network), such as Yolo (You Look Only Onse), SSD (Single Shot MultiBox Detector) or WSSD (Weighted Single Shot MultiBox Detector). you can For example, the detection model is trained using, as training data, an image to which position information of an image area showing a license plate is added as annotation information. The position information may be, for example, the position coordinates of the vertices of a rectangular area representing the license plate. Therefore, in this case, the detection unit 914 can acquire the position coordinates of the vertices of the rectangular area detected as the license plate of the surrounding vehicle by inputting the photographed image into the detection model. Note that the detection model is not limited to CNN, and may be FCN (Fully Convolution Network) or U-net, which is one of FCN.

The detection unit 914 supplies the extraction unit 915 with the positional information of the image area detected as the license plate of the surrounding vehicle. Below, the image area detected as the license plate may be referred to as the license plate image area.

The extraction unit 915 cuts out an extraction area corresponding to the license plate image area of the surrounding vehicle from the captured image to generate an extraction image. The extraction area may match the license plate image area, may be a predetermined area including the license plate image area, or may be an area including at least a predetermined area in the license plate image area. good. For example, the extraction area may be an image area of a predetermined range around the license plate image area. The extraction unit 915 supplies the extracted image to the character recognition unit 916 .

A character recognition unit 916 obtains character information from the extracted image using a character recognizer (also called a character recognition model). The character recognition model is trained using an image to which the position information of the image area of a figure representing a character (also called a character figure) and the character information indicated by the figure are added as annotation information, as training data. . The position information may be, for example, the position coordinates of the vertices of a rectangular area surrounding the figure. Therefore, in this case, by inputting the extracted image into the character recognition model, the character recognition unit 916 can acquire the character information indicated by the figure included in the license plate of the surrounding vehicle together with the position coordinates of the area surrounding the figure. can.

Then, the character recognition unit 916 generates license plate information of surrounding vehicles based on the acquired character information. Specifically, the character recognition unit 916 determines a combination of character information based on a plurality of pieces of character information and the positional coordinates of an area surrounding a graphic corresponding to each piece of character information. Thereby, the character recognition unit 916 can generate license plate information.

The character recognition section 916 supplies the generated license plate information to the output control section 918 .

The output control unit 918 outputs license plate information of surrounding vehicles. Here, the output may be outputting the associated information to an output unit (not shown) of the information processing system 910 in a predetermined data format, or may be output to an external device (not shown) via a network. shown). Also, the output may be storing the associated information in a storage unit (not shown) of the information processing system 910 .

FIG. 4 is a flow chart showing the flow of the information processing method according to the second embodiment. First, the detection unit 914 of the information processing system 910 uses the detection model to detect the license plates of the surrounding vehicles located around the photographed vehicle from the photographed image generated by photographing the scenery around the photographed vehicle ( step S20). Next, the extraction unit 915 cuts out an extraction area corresponding to the license plate image area of the surrounding vehicle from the captured image to generate an extraction image (step S21). Next, the character recognition unit 916 acquires character information from the extracted image using the character recognition model (step S22). Next, the character recognition unit 916 generates license plate information of surrounding vehicles based on the character information (step S23). Next, the output control unit 918 outputs license plate information of surrounding vehicles (step S24).

As described above, according to the second embodiment, the information processing system 910 can easily detect the license plate from the captured image, and as a result, can easily acquire the license plate information.

The information processing systems 900 and 910 are provided with processors, memories, and storage devices (not shown). Further, the storage device stores a computer program in which the processing of each information processing method according to the first and second embodiments is implemented. Then, the processor loads the computer program from the storage device into the memory and executes the computer program. As a result, the processor functions as an image acquisition unit 902, an identification information estimation unit 903, a travel information generation unit 907 and an output control unit 908, or a detection unit 914, an extraction unit 915, a character recognition unit 916 and an output control unit 918. Realize

Alternatively, the image acquisition unit 902, the identification information estimation unit 903, the driving information generation unit 907, the output control unit 908, the detection unit 914, the extraction unit 915, the character recognition unit 916, and the output control unit 918 are respectively dedicated hardware. may be implemented. Also, part or all of each component of each device may be realized by general-purpose or dedicated circuitry, processors, etc., or combinations thereof. These may be composed of a single chip, or may be composed of multiple chips connected via a bus. A part or all of each component of each device may be implemented by a combination of the above-described circuits and the like and programs. As the processor, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array), or the like can be used.

Further, when a part or all of the components of the information processing systems 900 and 910 are realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged. and may be distributed. For example, the information processing device, circuits, and the like may be implemented as a form in which each is connected via a communication network, such as a client-server system, a cloud computing system, or the like. Also, the functions of the information processing systems 900 and 910 may be provided in a SaaS (Software as a Service) format.

<Embodiment 3>
Next, Embodiment 3 of the present disclosure will be described. The third embodiment is a specific embodiment of the first and second embodiments. FIG. 5 is a schematic configuration diagram of the vehicle management system 1 according to the third embodiment. The vehicle management system 1 is a computer system that manages vehicle license plate information and driving information as vehicle-related information. That is, in the third embodiment, the identification information of the vehicle is license plate information of the vehicle. Note that in the third embodiment, the travel information includes position information, traveling direction, and speed. The vehicle management system 1 includes in-vehicle systems 10-1, 10-2, 10-3 and a management server 20, which are connected via a network N so as to be communicable. The network N is a wired or wireless communication line.

Each of the in-vehicle systems 10-1, 10-2 and 10-3 is a computer system installed in each vehicle. In the following description, the in-vehicle systems 10-1, 10-2, and 10-3 may simply be referred to as the in-vehicle system 10 when they are referred to without distinction. The number of in-vehicle systems 10 in the vehicle management system 1 is not limited to three, and may be two or less, or four or more.

The in-vehicle system 10 acquires information related to surrounding vehicles using an in-vehicle camera (not shown), and transmits the information related to the surrounding vehicles to the management server 20 via the network N. FIG. Specifically, the in-vehicle system 10 uses an in-vehicle camera to acquire the license plate information of the surrounding vehicle and the driving information of the surrounding vehicle, and transmits the vehicle-related information in which the license plate information and the driving information are associated with each other via the network. N to the management server 20 . The vehicle-related information may further include an extracted image cut out according to the photographing time, the photographed image (full-view image), and the license plate image area.

The management server 20 collects information related to the surrounding vehicles from the in-vehicle system 10 and records the information related to the surrounding vehicles in a storage unit (not shown). Specifically, when the management server 20 receives the vehicle-related information from the in-vehicle system 10 via the network N, the management server 20 associates the license plate information and the driving information included in the vehicle-related information with each other and stores them in the storage unit. . When the vehicle-related information includes the photographing time, the photographed image, and the extracted image, these are stored in the storage unit in association with the license plate information.

The management server 20 may notify the operator of the vehicle-related information by displaying it on a display unit (not shown). At this time, the management server 20 may display the shooting location on the map based on the position information included in the travel information.

FIG. 6 is a block diagram showing the configuration of the in-vehicle system 10 according to the third embodiment. The in-vehicle system 10 includes an information processing system (hereinafter referred to as an information processing device) 100 , an in-vehicle camera 120 and one or more sensors 140 .

The in-vehicle camera 120 is mounted at an arbitrary position of the photographing vehicle, photographs the scenery around the photographing vehicle, and generates photographing data, which is the data of the photographed image. In-vehicle camera 120 is communicably connected to information processing apparatus 100 . The in-vehicle camera 120 generates, for example, 30 frames per second (30 fps) photographed data, and supplies the generated photographed data to the information processing apparatus 100 every 1/30th of a second. The shooting data is, for example, H.264. 264 or H.264. It may be generated using a scheme such as H.265. In the third embodiment, the vehicle-mounted camera 120 is a front camera that captures the scenery in front of the vehicle. However, the vehicle-mounted camera 120 may be a rear camera, left camera, or right camera that captures the scenery behind, left, or right of the vehicle. Alternatively, vehicle-mounted camera 120 may be configured to include a plurality of the cameras described above.

One or a plurality of sensors 140 are mounted at arbitrary positions of the imaging vehicle and are sensors that measure the position information of the imaging vehicle and the vehicle speed of the imaging vehicle. For example, sensor 140 may be a sensor that measures vehicle speed and produces a vehicle speed signal. The sensor 140 may directly transmit the vehicle speed signal to the information processing device 100, or may transmit the vehicle speed signal to an ECU (Electronic Control Unit), and the ECU may transmit the vehicle speed signal to the information processing device 100. . Further, the sensor 140 may function as a positioning information receiving unit that receives positioning information for positioning the position of the photographing vehicle from a satellite positioning system such as GPS as position information. The sensor 140 may transmit the received position information directly to the information processing device 100 , or may transmit the position information to the ECU, and the ECU may transmit the position information to the information processing device 100 .

The information processing device 100 is a computer that generates information related to surrounding vehicles. The information processing device 100 includes a communication unit 101 , an image acquisition unit 102 , an identification information estimation unit 103 , a travel information generation unit 107 , an output control unit 108 and a storage unit 109 .

A communication unit 101 is a communication interface with the network N. FIG.

The image acquisition unit 102 is an example of the image acquisition unit 902 described above. The image acquisition unit 102 acquires a photographed image of the scenery around the photographing vehicle from the vehicle-mounted camera 120 . The image acquisition unit 102 then supplies the captured image to the identification information estimation unit 103 .

The identification information estimation unit 103 is an example of the identification information estimation unit 903 described above. The identification information estimation unit 103 estimates the license plate information of the surrounding vehicle as the identification information of the surrounding vehicle based on the captured image. The identification information estimation unit 103 has a detection unit 104 , an extraction unit 105 and a character recognition unit 106 .

The detection unit 104 is an example of the detection unit 914 described above. The detection unit 104 detects license plates of surrounding vehicles from the captured image using the detection model described above. Then, the detection unit 104 acquires the position information of the license plate image area NA of the surrounding vehicle from the detection model, and supplies it to the extraction unit 105 . The position information of the license plate image area NA may be the position coordinates of the vertices of the rectangular area representing the license plate.

Extraction unit 105 is an example of extraction unit 915 described above. The extraction unit 105 cuts out an extraction area corresponding to the license plate image area NA of the surrounding vehicle from the captured image to generate an extraction image. Note that the extraction unit 105 may determine the range of the extraction area so that the size of the extraction area matches the size of the input image of the subsequent character recognition model. Alternatively, the extraction unit 105 cuts out a predetermined extraction region from the captured image, and resizes the cutout image to the size of the input image of the subsequent character recognition model to generate an extraction image. may If the license plate image area NA of the surrounding vehicle is tilted with respect to the clipped image, the extracted image may be generated after correcting the tilt.
The extraction unit 105 supplies the extracted image to the character recognition unit 106 .

The character recognition unit 106 is an example of the character recognition unit 916 described above. The character recognition unit 106 uses the above-described character recognition model to obtain a plurality of pieces of character information and the position coordinates of the area surrounding the figure corresponding to each character information from the extracted image. In the third embodiment, the position coordinates of the area surrounding the figure are the position coordinates of the representative point of the area surrounding the figure, and may be, for example, the position coordinates of the vertices or the position coordinates of the center of gravity of the rectangular area surrounding the figure.

Then, the character recognition unit 106 generates license plate information of surrounding vehicles based on the plurality of character information and the position coordinates of the area surrounding the graphic corresponding to each character information. The character recognition unit 106 supplies the generated license plate information to the output control unit 108 .

The travel information generator 107 is an example of the travel information generator 907 described above. The travel information generation unit 107 acquires, directly or indirectly, first travel information (position information, traveling direction, and vehicle speed) regarding travel of the photographing vehicle from the sensor 140 . Then, as described above, the travel information generation unit 107 generates second travel information (position information, traveling direction, and vehicle speed) regarding travel of surrounding vehicles based on the first travel information.
Then, the travel information generator 107 supplies the second travel information to the output controller 108 .

The output control unit 108 is an example of the output control unit 908 and the output control unit 918 described above. The output control unit 108 outputs the license plate information of the surrounding vehicle and the second travel information in association with each other. Specifically, the output control unit 108 transmits, via the communication unit 101, the vehicle-related information in which the license plate information of the surrounding vehicle, the second travel information, the photographing time, the photographed image, and the extracted image are associated with each other. is sent to the management server 20 via the network N. Note that the output control unit 108 may transmit the vehicle-related information to the management server 20 each time the license plate information and the second travel information of the surrounding vehicle are generated. However, instead of this, the output control unit 108 accumulates vehicle-related information in the storage unit 109, and manages the vehicle-related information periodically or when the amount of accumulated information exceeds a predetermined amount. You may transmit to the server 20. Alternatively, output control unit 108 may transmit vehicle-related information accumulated in storage unit 109 to management server 20 when transmission of vehicle-related information is requested by management server 20 .

The storage unit 109 is a storage medium that stores information necessary for information processing of the information processing apparatus 100 . The storage unit 109 may also store vehicle-related information.

FIG. 7 is a diagram showing an example of a captured image 300 according to the third embodiment. The captured image 300 includes a vehicle image area V and other image areas showing scenery. Here, the detection unit 104 inputs the captured image 300 to the detection model. For example, the detection model detects a rectangular license plate image area NA in the captured image 300 and outputs the position information of the vertex of the license plate image area NA in the captured image 300 . Thereby, the detection unit 104 can acquire the position information of the vertex of the license plate image area NA from the captured image 300 .

Then, the extraction unit 105 cuts out a rectangular area including the license plate image area NA from the captured image 300 based on the positional information of the vertices of the license plate image area NA, thereby generating an extracted image.

FIG. 8 is a diagram showing an example of an extracted image 310 according to the third embodiment. Extracted image 310 includes license plate image area NA. The license plate image area NA includes a plurality of figures corresponding to character information. The character information constitutes each identification information of the license plate information.

Here, the license plate information is composed of a plurality of pieces of identification information of different types (sometimes called classification items). Specifically, the plurality of pieces of identification information are land transportation branch office T, classification number C, hiragana U, and series designation number S. For example, the land transport branch office T is represented by kanji characters or kana characters, and is "Adachi" in this figure. The classification number C is a number or the like, and is "330" in this figure. Hiragana, etc. U is a kana character, an alphabet, or the like, and is "e" in the figure. The series designation number S is a number or the like, and is "12-34" in this figure.

FIG. 9 is a diagram for explaining character recognition processing according to the third embodiment. The character recognition unit 106 inputs the extracted image 310 to the character recognition model CRM. The character recognition model CRM detects a rectangular area surrounding a figure from the extracted image 310, and outputs character information estimated to be indicated by the figure for each rectangular area. For example, the character recognition model CRM has "Adachi", "3", "3", "0", "E", "1", "2", "3" and "4" as character information, and each character A recognition result 320 including position coordinates of representative points (for example, vertices) of the rectangular area corresponding to the information is output. Then, the character recognition unit 106 determines which identification information each piece of character information indicates, based on the positional coordinates of the representative point of the rectangular area. For example, the character recognition unit 106 indicates that "Adachi" indicates land transportation branch office T, "3", "3" and "0" indicate classification number C, "E" indicates hiragana U, etc., "1", It is determined that "2", "3" and "4" indicate the series designation number S. Regarding the classification number C and the series designation number S, the character information is concatenated in order from the character information located on the left side of the extracted image 310 based on the positional coordinates of the representative point of the rectangular area on the horizontal axis. As a result, the character recognition unit 106 generates license plate information "(Land Transport Office T) Adachi/(Classification number C) 330/(Hiragana U) E/(Serial designation number S) 12-34".

When the character recognition unit 106 cannot detect character information at a predetermined position, or when the distance between representative points of rectangular areas corresponding to adjacent character information in the same identification information is equal to or greater than a predetermined threshold value, Identification information may be generated by adding a predetermined symbol “?” instead of character information. For example, if the serial designation number S "2" cannot be detected, the character recognition unit 106 generates license plate information "Adachi/330/E/1?-34". As a result, the character recognition unit 106 can easily determine "missing" in subsequent processing.

Note that the character recognition model CRM may output, for each rectangular area, the reliability of the assumed character information in addition to the character information assumed to be indicated by the graphic.
FIG. 10 is a diagram showing an example of the recognition result 320 according to the third embodiment. For example, the recognition result 320 includes character information, the position coordinates of the representative point of the rectangular area corresponding to each character information, and the reliability of each character information. As an example, the character recognition result "E" indicating hiragana U has a reliability of 32%, and "0" indicating a part of the classification number C has a reliability of 90%.

Here, the character recognition model CRM may output a plurality of pieces of character information for the same figure. In this case, the character recognition unit 106 selects a predetermined number of pieces of character information from the plurality of pieces of character information based on the plurality of pieces of character information and the reliability of each piece of character information, and recognizes the number based on the selected predetermined number of pieces of character information. Plate information may be generated. For example, the character recognition unit 106 recognizes the character information having the highest degree of reliability as the identification information that constitutes the license plate information for the identification information composed of one piece of character information such as the land transport branch office T and hiragana U. select. On the other hand, for identification information composed of a plurality of pieces of character information, such as the classification number C and the series designation number S, the character recognition unit 106 selects character information of a predetermined number of digits with high reliability. good. For example, the character recognition unit 106 may select the character information with the highest four reliability levels for the serial designation number S. FIG.

Alternatively, the character recognition unit 106 selects a predetermined number of character information from the plurality of character information based on the plurality of character information and the position of the figure corresponding to each character information, and based on the selected predetermined number of character information. may generate license plate information. For example, for identification information composed of a plurality of pieces of character information, the character recognition unit 106 uses the position coordinates of the representative point of the rectangular area on the horizontal axis and the minimum width of the one-digit figure to identify characters of the same digit. It may be determined whether there is information. When determining that there is character information of the same digit, the character recognition unit 106 may select character information with the highest reliability among the character information of the same digit as the character information constituting the identification information. Also at this time, the character recognition unit 106 may adjust the selection of character information so that the number of digits is a predetermined number based on the reliability.

Then, the character recognition unit 106 connects the selected character information based on the positional coordinates of the representative points of the rectangular area on the horizontal axis to generate identification information constituting the license plate information.

FIG. 11 is a flow chart showing the flow of the information processing method according to the third embodiment. First, the image acquisition unit 102 acquires a photographed image of the surroundings of the vehicle from the vehicle-mounted camera 120 (step S100).

Next, the identification information estimation unit 103 executes detection processing shown in step S110. Step S110 may consist of steps S111 and S112. In step S111, the detection unit 104 detects license plates of surrounding vehicles from the captured image using the detection model. Then, in step S112, the extraction unit 105 cuts out an extraction area corresponding to the license plate image area NA from the captured image to generate an extraction image.

Next, the identification information estimation unit 103 executes character recognition processing shown in step S120. Step S120 may consist of steps S121 and S122. In step S1212, the character recognition unit 106 acquires character information from the extracted image using the character recognition model. Then, in step S122, the character recognition unit 106 generates license plate information based on the character information.

Next, the travel information generation unit 107 executes travel information generation processing shown in step S130. Step S130 is composed of steps S131 and S132. In step S<b>131 , the travel information generator 107 acquires first travel information from the sensor 140 . Then, in step S132, the travel information generator 107 generates second travel information based on the first travel information. Note that step S130 may be performed before step S110 or S120, or may be performed in parallel with step S110 or S120.

Next, the output control unit 108 transmits the vehicle management information in which the license plate information generated in step S122 and the second travel information generated in step S132 are associated with each other to the management server 20 via the communication unit 101. (step S140).

Thus, according to the third embodiment, the same effects as those of the first and second embodiments can be obtained. In addition, just by mounting the in-vehicle system 10 on the vehicle, it becomes possible to recognize the license plate even on a narrow road where a fixed camera cannot be installed. The in-vehicle system 10 can be realized by adding the information processing device 100 to an existing in-vehicle camera such as a drive recorder, and thus can be realized at a relatively low cost.

Further, in the third embodiment, the in-vehicle system 10 performs detection processing and character recognition processing. As a result, compared with the case where the management server 20 performs the detection process and the character recognition process, it is possible to avoid a situation in which the image quality deteriorates due to the communication environment and the accuracy of detection and character recognition decreases. However, if the communication environment is in place or if the accuracy of detection and character recognition is acceptable, instead of this, the in-vehicle system 10 transmits the captured image to the management server 20, and the management server 20 performs detection processing and character recognition processing. may be performed.

<First Modification of Embodiment 3>
In the third embodiment described above, the detection unit 104 detects license plates of surrounding vehicles from the captured image. However, the detection unit 104 may detect, as a license plate, a rectangular body having a size similar to that of a license plate, such as a signboard or a road sign, or a rectangular body containing graphic characters like a license plate.

FIG. 12 is a diagram showing an example of the captured image 330. As shown in FIG. This figure shows that the detection unit 104 has detected the license plate image area NA3 in addition to the license plate image areas NA1 and NA2. However, the license plate image area NA3 is a road sign (intersection name sign), not a license plate. In order to avoid such erroneous detection, in the first modification of the third embodiment, the detection unit 104 predetermines a detection target region (detection target region) in the captured image, and detects the license plate from the detection target region. to detect

FIG. 13 is a diagram showing an example of a captured image 330 according to the first modified example of the third embodiment. In this figure, the detection unit 104 defines the lower portion of the captured image 330 as the detection target area DA. More specifically, the detection unit 104 defines the detection target area DA such that the upper end of the detection target area DA is separated downward from the upper end of the captured image 330 by a predetermined length. For example, the detection unit 104 may determine the lower half or the lower ⅓ area of the captured image 330 as the detection target area DA. This is because there is a high possibility that the license plates of surrounding vehicles running in the vicinity of the photographed vehicle appear below the photographed image 330 . Therefore, signboards or road signs that are likely to appear on the upper side of the captured image 330 can be excluded from detection targets, and detection accuracy can be improved. Therefore, in this figure, the license plate image areas NA1 and NA2 included in the detection target area DA are detected, but the license plate image area NA3 is not detected.

Note that the detection unit 104 may detect a license plate image area NA of a small size, such as the license plate image area NA2 in this figure, when the vehicle to be photographed and the surrounding vehicle are separated by a predetermined distance or more. In this case, it becomes difficult for the character recognition unit 106 to perform highly reliable character recognition. Therefore, if the license plate image area NA detected by the detection unit 104 is smaller than a predetermined size, the subsequent extraction unit 105 may not generate an extracted image. For example, when the width of the license plate image area NA is less than the predetermined number of pixels, the extraction unit 105 does not need to generate the extraction image. Thereby, information processing by the information processing apparatus 100 can be speeded up.

<Second Modification of Embodiment 3>
In the third embodiment, the in-vehicle camera 120 continuously photographs the oncoming vehicle and the following vehicle, and the information processing device 100 generates the license plate information of the oncoming vehicle and the following vehicle for each photographed image. For this reason, the information processing device 100 transmits the license plate information of the same peripheral vehicle to the management server 20 multiple times. In particular, the information processing apparatus 100 transmits a plurality of pieces of information related to the same surrounding vehicle to the management server 20 when the photographing vehicle and surrounding vehicles are waiting for a signal. As a result, there is a problem that the capacity of the storage unit of the management server 20 and the capacity of the storage unit 109 of the information processing apparatus 100 are pressed.

Therefore, in the second modified example of the third embodiment, the character recognition unit 106 generates one piece of license plate information based on the license plate information generated for each of a plurality of shot images taken at different shooting timings. That is, the character recognition unit 106 synthesizes a plurality of pieces of license plate information to generate one piece of license plate information. Then, the output control unit 108 outputs the one piece of license plate information generated by the character recognition unit 106 in association with the second travel information.

FIG. 14 is a diagram for explaining processing for synthesizing license plate information according to the second modification of the third embodiment. This figure shows a plurality of extracted images 310a and 310b relating to the same surrounding vehicle. License plate information "Adachi/330/?/12-34" is obtained from the extracted image 310a, and license plate information "Adachi/3?0/eh/12-3?" is obtained from the extracted image 310b. have been obtained. In this case, the character recognition unit 106 integrates the two license plate information and generates one license plate information "Adachi/330/E/12-34" as the license plate information of the surrounding vehicle. For example, the character recognition unit 106 complements “?” indicating character information that could not be recognized in license plate information obtained from one extracted image with license plate information obtained from another extracted image, thereby obtaining 1 License plate information may be generated. Further, the character recognition unit 106 may generate one piece of license plate information based on the reliability of each character information output by the character recognition model CRM for each extracted image. For example, the character recognition unit 106 selects character information with the highest reliability among the extracted images for each piece of character information included in each piece of identification information, and combines the selected character information to generate one piece of license plate information. You can

As a result, the amount of data output by the output control unit 108 can be reduced, and the problem of pressure on the capacity of the storage unit of the management server 20 and the capacity of the storage unit 109 of the information processing apparatus 100 can be avoided. In addition, since the character recognition unit 106 can complement character information that cannot be recognized from one extracted image with another extracted image, the accuracy of estimating license plate information is improved.

Whether or not the extracted image 310a and the extracted image 310b are images corresponding to the same surrounding vehicle is determined by the character recognition unit 106 based on the redundancy of license plate information obtained from each extracted image. good. For example, the character recognition unit 106 determines license plate information with a degree of redundancy equal to or greater than a predetermined threshold as license plate information corresponding to the same surrounding vehicle. Also, in this figure, the character recognition unit 106 synthesizes license plate information obtained from two extracted images, but the number of extracted images is not limited to this.

<Third Modification of Embodiment 3>
The functions of the information processing device 100 according to the third embodiment may be implemented by a plurality of information processing devices.
FIG. 15 is a block diagram showing the configuration of an in-vehicle system 10 according to a third modified example of the third embodiment. The information processing device 100 included in the in-vehicle system 10 has an image analysis device 160 , a travel information generation device 107 b , an I/O 110 and a communication terminal 180 .

The image analysis device 160 is an information processing device that detects and recognizes the license plate based on the captured image captured by the vehicle-mounted camera 120 . The image analysis device 160 includes an image acquisition unit 102, an identification information estimation unit 103, a first output control unit 108a, and a first storage unit 109a. First output control section 108 a outputs the license plate information generated by identification information estimation section 103 to communication terminal 180 via I/O 110 . For example, the first output control unit 108a is connected to the I/O 110 via a LAN (Local Area Network). The first storage unit 109a is a storage medium that stores information necessary for processing of the image analysis device 160. FIG. The captured image may be stored in the first storage unit 109a.

The travel information generator 107b is connected to the sensor 140 and has the same function as the travel information generator 107. The travel information generation device 107b outputs the second travel information to the communication terminal 180 via the I/O 110. FIG. For example, the travel information generation device 107b is connected to the I/O 110 according to the USB 2.0 communication standard. Note that the communication standard is not limited to USB 2.0, and may be USB 2.0 or higher.

The I/O 110 is an interface that relays data input/output between the image analysis device 160 , the travel information generation device 107 b and the communication terminal 180 . For example, the I/O 110 is connected to the communication terminal 180 according to the USB 2.0 communication standard. Note that the communication standard is not limited to USB 2.0, and may be USB 2.0 or higher.

The communication terminal 180 is a communication terminal such as a tablet terminal or a smart phone. Communication terminal 180 may be a communication terminal that can use an LTE (Long Term Evolution) line. The communication terminal 180 associates the data analyzed by the image analysis device 160, the second travel information that can be obtained via the travel information generation device 107b, and the shooting time, and accumulates them as vehicle-related information. Communication terminal 180 also transmits vehicle-related information to management server 20 . Communication terminal 180 includes communication section 101, second output control section 108b, and second storage section 109b. The second output control unit 108b may have the same function as the output control unit 108 of the information processing device 100. FIG. The second storage unit 109b is a storage medium that stores information necessary for processing of the communication terminal 180. FIG. Vehicle-related information is accumulated in the second storage unit 109b.

<Embodiment 4>
Next, Embodiment 4 of the present disclosure will be described. FIG. 16 is a schematic configuration diagram of a vehicle management system 1a according to the fourth embodiment. The vehicle management system 1 a includes a detection teacher data generation device 30 , a detection learning device 40 , a recognition teacher data generation device 50 and a recognition learning device 60 in addition to the vehicle-mounted system 10 and the management server 20 . The detection teaching data generation device 30, the detection learning device 40, the recognition teaching data generation device 50, and the recognition learning device 60 are connected to a network N. FIG.

The detection teacher data generation device 30 is a computer that generates detection teacher data, which is teacher data for learning a detection model. The detection training data is data in which the position information of the image area showing the license plate is added as annotation information to the image. The detection training data generation device 30 generates detection training data by adding position information of an image area showing a license plate to the image as annotation information. Then, the detection teaching data generation device 30 outputs the generated detection teaching data to the detection learning device 40 via the network N. FIG.

The detection learning device 40 is a computer that learns a detection model using detection teacher data. The detection learning device 40 outputs the generated learned detection model to the in-vehicle system 10 via the network N. FIG.

The recognition training data generation device 50 is a computer that generates recognition training data, which is training data for learning a character recognition model. As the recognition training data generation device 50, for example, an image generation device described in Japanese Patent Application No. 2020-072831 by the inventors can be used. The recognition training data is data in which annotation information including character information and the position of a figure is added to a character image. The character image includes graphics corresponding to character information generated based on the character information. In the fourth embodiment, when a character image generated based on character information is used as a primary character image, the character image used as training data for recognition is obtained by correcting the pixel values of the primary character image. character image. First, the recognition training data generation device 50 generates an image including a figure based on character information, adds the character information and the position information of the image area of the figure as annotation information, and generates recognition training data. Generate. Then, the recognition teaching data generating device 50 outputs the generated recognition teaching data to the recognition learning device 60 via the network N. FIG.

The recognition learning device 60 is a computer that learns a character recognition model using recognition teacher data. The recognition learning device 60 outputs the generated learned character recognition model to the in-vehicle system 10 via the network N. FIG.

FIG. 17 is a block diagram showing the configuration of the recognition training data generation device 50 according to the fourth embodiment. The recognition training data generation device 50 has an acquisition unit 500 , an image generation unit 501 , a correction unit 502 , an annotation addition unit 503 and a storage unit 504 .

The storage unit 504 is a storage medium for storing various information and recognition teaching data necessary for generation of recognition teaching data. A storage unit 504 stores format information FM, a drawing point information database (DB) 505, and a learning DB 506. FIG.
The drawing point information DB 505 stores drawing point information DR for drawing a figure corresponding to character information. Details of the drawing point information DR and the format information FM will be described later.
The learning DB 506 stores the above-described recognition teacher data.

The acquisition unit 500 acquires various types of information such as character information related to recognition training data from the user. The acquisition unit 500 then supplies the acquired character information and the like to the image generation unit 501 . The acquisition unit 500 also acquires various types of information related to the drawing point information DR from the user. The acquisition unit 500 then stores various types of information related to the acquired drawing point information DR in the drawing point information DB 505 of the storage unit 504 .

The image generator 501 generates a character image including a figure corresponding to the character information from the obtained character information. Here, the image generation unit 501 generates a character image based on the drawing point information DR associated with the format information FM and the character information stored in the drawing point information DB 505 of the storage unit 504 . The image generation unit 501 then supplies the character image to the correction unit 502 and the character information to the annotation addition unit 503 .

The correction unit 502 converts the pixel values of the character image supplied from the image generation unit 501 and corrects the character image. The correction unit 502 supplies the corrected character image to the annotation adding unit 503 .

The annotation adding unit 503 generates annotation information using the character information supplied from the image generating unit 501 . Then, the annotation adding unit 503 adds annotation information to the character image supplied from the correcting unit 502 to generate teacher data for recognition. The annotation adding unit 503 stores the generated recognition teacher data in the learning DB 506 of the storage unit 504 .

FIG. 18 is a diagram illustrating an example of a data structure of annotation information according to the fourth embodiment; As shown in this figure, for example, the annotation information includes the type of identification information, the positional coordinates of the partitioned area P and the character area L, and the correct label, which are associated with each other. Here, the partitioned area P is an area (drawing area) that is assigned to each piece of identification information and in which a figure corresponding to the character information corresponding to the type of the identification information can be drawn in the character image. The character area L is a drawing area for each character assigned according to the number of characters of the identification information corresponding to each partition area P. FIG. The positional coordinates of the partitioned area P and the character area L may be the coordinates of the representative point of the area, such as the coordinates of the vertex that determines the area or the coordinates of the center of the area. Note that the partitioned area P and the character area L are determined by format information FM, which will be described later.

FIG. 19 is a diagram for explaining format information and drawing point information according to the fourth embodiment. This figure shows the format indicated by the format information FM and the drawing points indicated by the drawing point information DR on the character image. As shown in this figure, the format includes the positions, dimensions, and the like that define the partitioned area P and the character area L for each type of identification information. Therefore, the format information FM includes position information and dimension information that define the partition area P and the character area L for each type of identification information. Note that the format information FM may be determined based on the format of the license plate defined by laws, government ordinances, or ministerial ordinances.

A drawing point is a point for drawing a figure corresponding to character information included in a character image. Therefore, the drawing point information DR indicates the position coordinates of such a drawing point. The position coordinates here may be bitmap coordinates. In the fourth embodiment, the drawing point information DR may be the position coordinates of the drawing points forming the frame line including the outer frame and the inner frame defining the figure among all the drawing points included in the figure. Also, the drawing point information DR may be position coordinates of an intermediate point between the outer frame and the inner frame among all the drawing points included in the figure. By defining the drawing point information DR as the position coordinates of some of the drawing points in this way, the required memory capacity can be minimized compared to the case where the position coordinates of all the drawing points are stored in the drawing point information DB 505. . In addition, it becomes easy to fill in the color of a figure, which will be described later. Note that the drawing point information DR in the fourth embodiment may be relative positional coordinates of such a drawing point with respect to a representative point of the partition area P or the character area L. FIG.

FIG. 20 is a diagram showing an example of the data structure of the drawing point information DB 505 according to the fourth embodiment. For example, the drawing point information DB 505 contains the type of identification information, character information corresponding to the type of identification information, and the relative position coordinates of the drawing point with respect to the representative point of the partitioned area P or character area L corresponding to the type of the identification information. (drawing point information DR). Although the data structure of the drawing point information DB 505 is shown in a table format in FIG. The file may be a text file, such as an XML file. Each file may be associated with the type of identification information and text information corresponding to the type of identification information.

FIG. 21 is a flow chart showing the flow of recognition teacher data generation processing according to the fourth embodiment. First, the acquisition unit 500 of the recognition teaching data generation device 50 acquires character information that will be the correct label of the recognition teaching data for each type of identification information (step S210). At this time, the acquisition unit 500 acquires the recognition training data from a user terminal (not shown) via the network N or when an input unit (not shown) of the recognition training data generation device 50 receives an input from the user. Character information may be obtained. At this time, the acquisition unit 500 may acquire vehicle type information that determines the colors of the graphics and the background. The acquisition unit 500 outputs the type of identification information, character information, vehicle type information, and the like to the image generation unit 501 .

Next, the image generation unit 501 acquires the drawing point information DR associated with the character information corresponding to the identification information type from the drawing point information DB 505 of the storage unit 504 based on the identification information type and the character information. (Step S212). The image generation unit 501 also acquires the format information FM from the storage unit 504 .

Next, the image generator 501 draws a figure corresponding to the character information based on the obtained drawing point information DR and format information FM to generate a character image (step S214).
Here, FIG. 22 is a diagram for explaining the drawing method by the image generation unit 501 according to the fourth embodiment. In the fourth embodiment, as shown in FIG. 22, the image generation unit 501 plots the position coordinates included in the acquired drawing point information DR with reference to the representative point of the partition region P or the character region L, and An outer frame and an inner frame are formed. Then, the image generating unit 501 performs a process of painting out the area surrounded by the outer and inner frames of the figure and the background outside the outer frame using the color corresponding to the vehicle type information. The image generation unit 501 uses . NET Framework standard functions may be used to perform such fill processing. The image generation unit 501 then supplies the character image generated in this way to the correction unit 502 and supplies the character information for each type of identification information to the annotation addition unit 503 .

Returning to FIG. 21, the description is continued. Next, the correction unit 502 converts the pixel values of the character image generated by the image generation unit 501 to correct the character image (step S215). The correction unit 502 then supplies the corrected character image to the annotation addition unit 503 .

Here, FIG. 23 is a diagram for explaining correction processing by the correction unit 502 according to the fourth embodiment. As shown in this figure, the correction unit 502 may perform various pixel value conversion processes.
For example, as shown in FIG. 23(a), the correction unit 502 may rotate the character image by a predetermined angle. In particular, since the license plate of the oncoming vehicle is slightly tilted in the photographed image, it is effective to learn the character recognition model using the character image corrected as described above as the training data for recognition.
Further, as shown in FIG. 23(b), the correction unit 502 may detect the edges of the figure included in the character image and emphasize the contour of the figure.

Further, as shown in FIG. 23C, the correction unit 502 may reduce the resolution of the character image to make the outline unclear. For example, the correction unit 502 reduces the resolution of the character image by changing the interval between a predetermined pixel and surrounding pixels and interpolating the pixels therebetween. The resolution reduction rate (compression rate) may be determined according to the image quality and blurring of the extracted image generated based on the actual captured image. When the license plate image area NA in the extraction area is tilted, the extraction image referred to in determining the compression ratio may be an image obtained by clipping the extraction area and then correcting the tilt.

Further, as shown in FIG. 23D, the correction unit 502 may perform distortion correction on the character image.
Further, as shown in FIG. 23E, the correction unit 502 may convert the pixel values of the character image using filters such as a Gaussian filter, a median filter, and a bilateral filter.

Further, as shown in FIG. 23F, the correction unit 502 may add noise such as Gaussian noise or impulse noise to each pixel of the character image to smooth the character image. In recent years, there has been an increase in the number of license plates that include a pattern in the background. Using the character image corrected as described above as training data for recognition has the effect of suppressing the effect of the pattern.

Further, the correction unit 502 may combine the correction processes described above. For example, as shown in FIG. 23G, the correction unit 502 may rotate the character image by a predetermined angle and further compress the resolution of the character image to make the outline unclear. For example, the correction unit 502 may execute these processes using an application using OpenCV (registered trademark).

Returning to FIG. 21, the description is continued. Next, for the character image corrected in step S215, the annotation adding unit 503 creates annotation information in which the partitioned region P corresponding to the type of the identification information or the partitioned region P and the correct label that is the character information are associated with each other. is given (step S216). In this manner, the annotation adding unit 503 generates training data for recognition. Then, the annotation adding unit 503 stores the character image to which the annotation information is added in the learning DB 506 as training data for recognition (step S218).

It should be noted that acquisition of the vehicle type information by the acquisition unit 500 in step S210 may be omitted. In this case, the image generation unit 501 may estimate the colors of the figure and the background based on the character information corresponding to the classification number C and U such as hiragana in S214.

FIG. 24 is a flowchart showing the flow of character recognition model learning processing according to the fourth embodiment. First, the recognition learning device 60 acquires recognition teaching data from the recognition teaching data generating device 50 (step S220). Next, the recognition learning device 60 acquires a pre-learning character recognition model (step S221). Next, the recognition learning device 60 performs character recognition processing using the character recognition model before learning (step S222). At this time, the learning device for recognition 60 inputs the character image of the teacher data for recognition to the character recognition model, and acquires the output value output from the character recognition model. Next, the recognition learning device 60 calculates the error between the correct label indicated by the annotation information and the output value based on the annotation information attached to the input character image (step S223). Then, the recognition learning device 60 determines whether or not to end the learning (step S224). For example, the recognition learning device 60 may determine whether or not to end learning by determining whether or not the number of parameter updates has reached a predetermined number. Further, the recognition learning device 60 may determine whether or not to end learning by determining whether or not the calculated error is less than a threshold. If the learning device for recognition 60 ends the learning (Yes in step S224), the process proceeds to step S226; otherwise (No in step S224), the process proceeds to step S225.

In step S225, the recognition learning device 60 updates various parameters of the neural network of the character recognition model based on the error. The recognition learning device 60 then returns the process to S222.

In step S226, the recognition learning device 60 ends learning and determines various parameters and character recognition models. Then, the recognition learning device 60 outputs the character recognition model to the in-vehicle system 10 via the network N, and ends the process.

As described above, according to the fourth embodiment, the recognition training data generation device 50 generates a character image from the acquired character information, and automatically adds annotation information to the generated character image. Therefore, the recognition training data generation device 50 can obtain a large amount of character images. The recognition training data generation device 50 can efficiently add annotation information to character images while minimizing the workload of the administrator. As a result, it is possible to efficiently generate recognition training data to which annotation information is attached.

Further, in the fourth embodiment, a character recognition model learned using recognition teacher data generated based on character images subjected to correction processing such as rotation and blurring is used for character recognition processing. Here, until now, systems and solutions for recognizing license plate information have mainly used recognition methods based on perfect matching based on machine learning. In addition, there is a problem that an image captured by an in-vehicle camera from a running vehicle is not stable and clear because the size of the subject is not fixed and the relative speed is high. Therefore, a character recognition model trained using clear images has a problem of low character recognition accuracy and cannot be used at a practical level. However, according to the fourth embodiment, it is possible to improve the accuracy of character recognition of license plate information using an in-vehicle camera.

Note that the image generation unit 501 may supply the character image to the annotation addition unit 503 without going through the correction unit 502 . That is, the annotation adding unit 503 may add annotation information to an uncorrected character image to generate recognition training data. Some or all of the functions of the detection teacher data generation device 30, the detection learning device 40, the recognition teacher data generation device 50, and the recognition learning device 60 are provided in each in-vehicle system 10 or the management server 20. may For example, each in-vehicle system 10 or management server 20 may perform the above-described recognition teacher data generation processing and character recognition model learning processing.

<Embodiment 5>
Next, Embodiment 5 of the present disclosure will be described. Embodiment 5 is characterized in that there is a character recognition model for each type of identification information, and that the vehicle management system has a recognition teacher data generating device and a recognition learning device for each character recognition model.

FIG. 25 is a schematic configuration diagram of a vehicle management system 1b according to the fifth embodiment. The vehicle management system 1b replaces the recognition teaching data generation device 50 and the recognition learning device 60 with first to fourth recognition teaching data generation devices 50b-1 to 50b-4 and first to fourth recognition Learning devices 60b-1 to 60b-4 are provided. The first to fourth recognition training data generation devices 50b-1 to 50b-4 and the first to fourth recognition learning devices 60b-1 to 60b-4 are connected to a network N. FIG.

The first recognition teacher data generation device 50b-1 and the first recognition learning device 60b-1 are a recognition teacher data generation device and a recognition learning device corresponding to the land transportation branch office model CRM-1 that recognizes the land transportation branch office T. be. The second recognition teaching data generation device 50b-2 and the second recognition learning device 60b-2 are a recognition teaching data generation device and a recognition learning device corresponding to the classification number model CRM-2 that recognizes the classification number C. be. The third recognition teaching data generation device 50b-3 and the third recognition learning device 60b-3 are a recognition teaching data generation device and a recognition learning device corresponding to the hiragana etc. model CRM-3 that recognizes hiragana etc. U. be. The fourth recognition teacher data generation device 50b-4 and the fourth recognition learning device 60b-4 are recognition teacher data generation devices and recognition learning devices corresponding to the serial designation number model CRM-4 that recognizes the serial designation number S. It is a device.

In addition, arbitrary two identification information of land transportation branch office T, classification number C, hiragana etc. U and serial designation number S are called first identification information and second identification information, and each of the first identification information and second identification information A character recognition model for recognizing corresponding character information may be called a first character recognition model and a second character recognition model. Recognition teaching data corresponding to the first identification information and the second identification information may be called first recognition teaching data and second recognition teaching data, respectively.

FIG. 26 is a diagram for explaining character recognition processing according to the fifth embodiment.
First, the character recognition unit 106 of the in-vehicle system 10 inputs the extracted image 310 to the land transportation branch office model CRM-1, classification number model CRM-2, hiragana etc. model CRM-3, and series designation number model CRM-4.

In the training data for recognition of the land transportation branch office model CRM-1, annotation information including character information corresponding to the land transportation branch office T and the position of the figure indicating the character information is added to the character image representing the entire license plate. data. Land transportation branch office model CRM-1 to which extracted image 310 is input outputs a recognition result including at least "Adachi" as land transportation branch office T. FIG.

In the recognition training data of the classification number model CRM-2, annotation information including character information corresponding to the classification number C and the position of the figure indicating the character information is added to the character image representing the entire license plate. data. The classification number model CRM-2 to which the extracted image 310 is input has at least "3", "3", and "0" as the classification number C and the position coordinates of the representative points of the rectangular areas corresponding to each character information. Output recognition results including

The teacher data for recognition of the hiragana model CRM-3 is provided with annotation information including character information corresponding to the hiragana U and the position of the figure indicating the character information for the character image representing the entire license plate. data. The hiragana model CRM-3 to which the extracted image 310 is input outputs a recognition result including at least "e" as the hiragana U. FIG.

The training data for recognition of the sequence designation model CRM-4 is annotation information including character information corresponding to the sequence designation number S and the position of the figure indicating the character information for the character image representing the entire license plate. It is given data. The series designation number model CRM-4 to which the extracted image 310 is input has "1", "2", "3" and "4" as the series designation number S, and the representative points of the rectangular areas corresponding to each piece of character information. output a recognition result including at least position coordinates;

The character recognition unit 106 generates license plate information "Adachi/330/E/12-34" based on the recognition results obtained from each character recognition model.

In this way, by recognizing each piece of identification information using character recognition models learned independently for each type of identification information, compared to the case of recognizing four types of identification information from one character recognition model, Since there are fewer options for characters to be output, the accuracy of character recognition can be improved. In one example of recognizing four types of identification information from one character recognition model, the recognition success rates for land transportation branch office T, classification number C, hiragana, etc. U, and serial designation number S are 29.7% and 75.4%, respectively. , 69.2% and 93.2%. However, in one example according to the fifth embodiment, the recognition success rates for the land transportation branch T, the classification number C, the hiragana U, and the serial designation number S are 94.9%, 97.4%, and 99.1%, respectively. and 99.9%, and the recognition success rate could be improved.

Some or all of the functions of the first to fourth recognition training data generation devices 50b-1 to 50b-4 and the first to fourth recognition learning devices 60b-1 to 60b-4 are provided by each in-vehicle system 10. Alternatively, it may be provided in the management server 20 . For example, each in-vehicle system 10 or management server 20 may perform recognition teacher data generation processing and character recognition model learning processing for each type of identification information.

Next, the physical configurations of the information processing device, the management server, the detection teacher data generation device, the detection learning device, the recognition teacher data generation device, and the recognition learning device included in the vehicle management system will be described. FIG. 27 is a diagram showing a configuration example of a computer used as the above device. The computer 1000 has a processor 1010 , a storage unit 1020 , a ROM (Read Only Memory) 1030 , a RAM (Random Access Memory) 1040 , a communication interface (IF) 1050 and a user interface 1060 .

The communication interface 1050 is an interface for connecting the computer 1000 and a communication network via wired communication means or wireless communication means. User interface 1060 includes a display, such as a display. User interface 1060 also includes input units such as a keyboard, mouse, and touch panel.

The storage unit 1020 is an auxiliary storage device capable of holding various data. The storage unit 1020 is not necessarily a part of the computer 1000, and may be an external storage device or a cloud storage connected to the computer 1000 via a network.

ROM 1030 is a non-volatile storage device. For the ROM 1030, for example, a semiconductor storage device such as a flash memory having a relatively small capacity is used. Programs executed by processor 1010 may be stored in storage unit 1020 or ROM 1030 . Storage unit 1020 or ROM 1030 stores various programs for implementing the functions of each unit in each device included in the vehicle management system, for example.

The programs described above can be stored and provided to computer 1000 using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media such as flexible disks, magnetic tapes, or hard disks, magneto-optical recording media such as magneto-optical discs, compact discs (CDs), or digital versatile disks (DVDs). and semiconductor memory such as mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash ROM, or RAM. The program may also be delivered to the computer using various types of transitory computer readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can deliver the program to the computer via wired channels, such as wires and optical fibers, or wireless channels.

RAM 1040 is a volatile storage device. Various semiconductor memory devices such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory) are used for the RAM 1040 . RAM 1040 can be used as an internal buffer that temporarily stores data and the like. The processor 1010 develops the program stored in the memory|storage part 1020 or ROM1030 to RAM1040, and runs it. The processor 1010 may be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). Processor 1010 executes the program to implement the function of each unit in each device included in the vehicle management system. Processor 1010 may have internal buffers in which data and the like can be temporarily stored.

Although the present disclosure has been described with reference to the embodiments, the present disclosure is not limited to the above. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the invention.

Some or all of the above-described embodiments can also be described in the following supplementary remarks, but are not limited to the following.
(Appendix 1)
an image acquisition step of acquiring a captured image generated by capturing the scenery around the vehicle;
an identification information estimation step of estimating identification information of surrounding vehicles located around the vehicle based on the captured image;
a travel information generation step of acquiring first travel information about travel of the vehicle and generating second travel information about travel of the peripheral vehicle based on the first travel information;
A program for causing a computer to execute: an output control step for outputting the identification information and the second travel information in association with each other.
(Appendix 2)
the first travel information includes at least one of location information, traveling direction, and vehicle speed of the vehicle;
The program according to appendix 1, wherein the second travel information includes at least one of position information, traveling direction, and vehicle speed of the surrounding vehicle.
(Appendix 3)
3. The program according to appendix 1 or 2, wherein the driving information generating step includes generating the same information as the first driving information as the second driving information.
(Appendix 4)
3. The program according to appendix 1 or 2, wherein the driving information generating step includes generating the second driving information based on the captured image and the first driving information.
(Appendix 5)
5. The output control step includes a step of transmitting vehicle-related information in which the identification information of the surrounding vehicle and the second travel information are associated with each other to a management server. program.
(Appendix 6)
The identification information of the surrounding vehicle is license plate information of the surrounding vehicle,
The identification information estimation step includes:
a detection step of detecting license plates of the surrounding vehicles from the captured image;
an extracting step of extracting an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image;
6. The character recognition step of obtaining character information from the extracted image using a character recognition model, and generating license plate information of the surrounding vehicle based on the character information. program.
(Appendix 7)
Using a detection model that outputs the position of the image area showing the license plate included in the input image from the captured image generated by capturing the scenery around the vehicle, license plates of surrounding vehicles located around the vehicle a detection stage for detecting
an extracting step of extracting an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image;
a character recognition step of obtaining character information from the extracted image using a character recognition model and generating license plate information of the surrounding vehicle based on the character information;
A program for causing a computer to execute an output control step for outputting the license plate information of the peripheral vehicle.
(Appendix 8)
The character recognition model is learned using recognition teacher data,
The training data for recognition is a character image generated based on character information, and annotation information including the character information and the position of the figure for a character image including a figure corresponding to the character information. 8. The program according to appendix 6 or 7, which is data to which is assigned.
(Appendix 9)
9. The program according to appendix 8, wherein the character image is a character image obtained by correcting pixel values of a primary character image generated based on the character information.
(Appendix 10)
The character recognition model is
a first character recognition model that recognizes character information corresponding to the first identification information in the license plate information;
a second character recognition model that recognizes character information corresponding to the second identification information in the license plate information;
including
The first character recognition model is learned using first recognition teacher data corresponding to the first identification information,
10. The program according to any one of appendices 6 to 9, wherein the second character recognition model is trained using second recognition training data corresponding to the second identification information.
(Appendix 11)
The character recognition step includes:
selecting a predetermined number of character information from the plurality of character information based on the plurality of character information and the reliability of each of the plurality of character information output by the character recognition model;
and generating the license plate information based on the predetermined number of character information selected.
(Appendix 12)
The character recognition step includes:
selecting a predetermined number of pieces of character information from the plurality of pieces of character information based on the plurality of pieces of character information and the positions of graphics corresponding to each of the plurality of pieces of character information, which are output by the character recognition model;
and generating the license plate information based on the predetermined number of character information selected.
(Appendix 13)
13. The program according to any one of appendices 6 to 12, wherein the detecting step includes detecting the license plate from a predetermined detection target area included in the captured image.
(Appendix 14)
14. The program according to appendix 13, wherein the upper end of the detection target area is separated downward by a predetermined length from the upper end of the captured image.
(Appendix 15)
15. The program according to any one of appendices 6 to 14, wherein the extracting step does not include the step of generating the extracted image if the image area detected as the license plate of the surrounding vehicle is smaller than a predetermined size.
(Appendix 16)
The step of recognizing characters includes the step of generating one piece of license plate information based on license plate information generated for each of a plurality of captured images captured at different timings,
16. The program according to any one of appendices 6 to 15, wherein said output control step includes a step of outputting said 1 license plate information.
(Appendix 17)
an image acquisition unit that acquires a captured image of the scenery around the vehicle;
an identification information estimation unit for estimating identification information of surrounding vehicles located around the vehicle based on the captured image;
a travel information generation unit that acquires first travel information about travel of the vehicle and generates second travel information about travel of the surrounding vehicle based on the first travel information;
An information processing system comprising: an output control unit that associates and outputs the identification information and the second travel information.
(Appendix 18)
an image acquisition step of acquiring a photographed image of the scenery around the vehicle;
an identification information estimation step of estimating identification information of surrounding vehicles located around the vehicle based on the captured image;
a travel information generation step of acquiring first travel information about travel of the vehicle and generating second travel information about travel of the peripheral vehicle based on the first travel information;
An information processing method comprising: an output control step for outputting the identification information and the second travel information in association with each other.
(Appendix 19)
Detection that detects the license plates of surrounding vehicles located around the vehicle using a detection model that outputs the position of the image area showing the license plate included in the input image from the captured image of the scenery around the vehicle. Department and
an extraction unit that cuts out an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image;
a character recognition unit that acquires character information from the extracted image using a character recognition model and generates license plate information of the surrounding vehicle based on the character information;
An information processing system comprising: an output control unit for outputting the license plate information of the peripheral vehicle.
(Appendix 20)
Detection that detects the license plates of surrounding vehicles located around the vehicle using a detection model that outputs the position of the image area showing the license plate included in the input image from the captured image of the scenery around the vehicle. stages and
an extracting step of extracting an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image;
a character recognition step of obtaining character information from the extracted image using a character recognition model and generating license plate information of the surrounding vehicle based on the character information;
and an output control step for outputting the license plate information of the surrounding vehicle.

1, 1a, 1b vehicle management system 10 in-vehicle system 20 management server 30 detection teaching data generation device 40 detection learning device 50 recognition teaching data generation device 50b-1 to 4 first to fourth recognition teaching data generation device 60 Recognition learning devices 60b-1 to 4 First to fourth recognition learning devices 100, 900, 910 Information processing device (information processing system)
101 communication unit 102, 902 image acquisition unit 103, 903 identification information estimation unit 104, 914 detection unit 105, 915 extraction unit 106, 916 character recognition unit 107, 907 travel information generation unit 107b travel information generation device 108, 908, 918 output Control unit 108a First output control unit 108b Second output control unit 109 Storage unit 109a First storage unit 109b Second storage unit 110 I/O
120 Vehicle-mounted camera 140 Sensor 160 Image analysis device 180 Communication terminal 300, 330 Photographed image 310 Extracted image 320 Recognition result 500 Acquisition unit 501 Image generation unit 502 Correction unit 503 Annotation addition unit 504 Storage unit 505 Drawing point information DB
506 DB for learning
1000 computer 1010 processor 1020 storage unit 1030 ROM
1040 RAM
1050 Communication interface 1060 User interface NA License plate image area V Vehicle image area DA Detection target area T Land transport branch office C Classification number U Hiragana, etc. S Serial designation number CRM Character recognition model FM Format information DR Drawing point information

Claims (10)

an image acquisition step of acquiring a captured image generated by capturing the scenery around the vehicle;
an identification information estimation step of estimating identification information of surrounding vehicles located around the vehicle based on the captured image;
a travel information generation step of acquiring first travel information about travel of the vehicle and generating second travel information about travel of the peripheral vehicle based on the first travel information;
A program for causing a computer to execute: an output control step for outputting the identification information and the second travel information in association with each other. The identification information of the surrounding vehicle is license plate information of the surrounding vehicle,
The identification information estimation step includes:
a detection step of detecting license plates of the surrounding vehicles from the captured image;
an extracting step of extracting an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image;
The program according to claim 1, comprising a character recognition step of acquiring character information from the extracted image using a character recognition model, and generating license plate information of the surrounding vehicle based on the character information. Using a detection model that outputs the position of the image area showing the license plate included in the input image from the captured image generated by capturing the scenery around the vehicle, license plates of surrounding vehicles located around the vehicle a detection stage for detecting
an extracting step of extracting an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image;
a character recognition step of obtaining character information from the extracted image using a character recognition model and generating license plate information of the surrounding vehicle based on the character information;
A program for causing a computer to execute an output control step for outputting the license plate information of the peripheral vehicle. The character recognition model is learned using recognition teacher data,
The training data for recognition is a character image generated based on character information, and annotation information including the character information and the position of the figure for a character image including a figure corresponding to the character information. 4. The program according to claim 2 or 3, wherein the data is given with . 5. The program according to claim 4, wherein the character image is a character image obtained by correcting pixel values of a primary character image generated based on the character information. The character recognition model is
a first character recognition model that recognizes character information corresponding to the first identification information in the license plate information;
a second character recognition model that recognizes character information corresponding to the second identification information in the license plate information;
including
The first character recognition model is learned using first recognition teacher data corresponding to the first identification information,
6. The program according to any one of claims 2 to 5, wherein the second character recognition model is trained using second recognition training data corresponding to the second identification information. The program according to any one of claims 2 to 6, wherein the detecting step includes detecting the license plate from a predetermined detection target area included in the captured image. The step of recognizing characters includes the step of generating one piece of license plate information based on license plate information generated for each of a plurality of captured images captured at different timings,
8. The program according to any one of claims 2 to 7, wherein said output control step includes a step of outputting said one license plate information. an image acquisition unit that acquires a captured image of the scenery around the vehicle;
an identification information estimation unit for estimating identification information of surrounding vehicles located around the vehicle based on the captured image;
a travel information generation unit that acquires first travel information about travel of the vehicle and generates second travel information about travel of the surrounding vehicle based on the first travel information;
An information processing system comprising: an output control unit that associates and outputs the identification information and the second travel information. Detection that detects the license plates of surrounding vehicles located around the vehicle using a detection model that outputs the position of the image area showing the license plate included in the input image from the captured image of the scenery around the vehicle. Department and
an extraction unit that cuts out an extraction area corresponding to the image area detected as the license plate of the surrounding vehicle from the captured image to generate an extraction image;
a character recognition unit that acquires character information from the extracted image using a character recognition model and generates license plate information of the surrounding vehicle based on the character information;
An information processing system comprising: an output control unit for outputting the license plate information of the peripheral vehicle.

Images