JP2021022026A - Image recognition device and image recognition method - Google Patents

Abstract

To provide an image recognition device and an image recognition method that can suppress false recognition of a sign.SOLUTION: In an automatic driving control system 100 included in a vehicle 10, an image recognition device 110 includes: an image acquisition unit 111 that acquires the image photographed by a camera 122; a sign detection unit 112 that detects road sign candidates from images; a moving object recognition unit 113 that recognizes moving objects from the images; a calculation unit 114 that calculates degree of collation indicating degree to which the candidate is on an area of the moving object; and a determination unit 115 that determines, according to the degree of collation, whether or not the candidate is a road sign.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to an image recognition device and an image recognition method.

As an image recognition device, a technique for preventing erroneous recognition of a recognition target is known. For example, in Patent Document 1, it is possible to prevent a flat object other than a vehicle from being mistakenly recognized as a vehicle according to the position in real space of a feature point located above the horizontal edge extracted from the captured image. The technology is described.

Japanese Unexamined Patent Publication No. 2016-053867

A sticker with a design similar to a road sign may be attached to the rear of the vehicle. In this case, since the sticker has image features similar to the road sign, it may be mistakenly recognized as a road sign. In addition, since the positions of road signs and stickers in the image are not uniquely determined, there is a possibility that erroneous recognition cannot be prevented by the conventional technology. Therefore, a technique capable of suppressing erroneous recognition of road signs has been desired.

The present disclosure has been made to solve the above-mentioned problems, and can be realized in the following forms.

According to one embodiment of the present disclosure, an image recognition device (110) is provided. The image recognition device recognizes an image acquisition unit (111) that acquires an image taken by the camera (122), a sign detection unit (112) that detects a road sign candidate from the image, and a moving object from the image. Whether or not the candidate is a road sign according to the moving body recognition unit (113), the calculation unit (114) that calculates the degree of collation indicating the degree to which the candidate is on the area of the moving body, and the collation degree. It is provided with a determination unit (115) for determining whether or not.

According to the image recognition device of this form, the determination unit determines whether or not the road sign is a road sign according to the degree of collation indicating the degree to which the road sign candidate is on the area of the moving object. False recognition can be suppressed.

It is a schematic diagram which shows the structure of the automatic driving system. It is a flowchart which shows the image recognition process. It is a figure which shows an example of the image which contains a sign candidate. It is a figure which shows another example of the image which contains a marker candidate. It is a figure which shows another example of the image which contains a marker candidate.

A. First Embodiment:
As shown in FIG. 1, the vehicle 10 includes an automatic driving control system 100. In the present embodiment, the automatic driving control system 100 includes an image recognition device 110, a camera 122, an object sensor 124, a notification unit 140, an automatic driving control unit 210, and a driving force control ECU (Electronic Control Unit) 220. The braking force control ECU 230 and the steering control ECU 240 are provided. The image recognition device 110, the automatic driving control unit 210, the driving force control ECU 220, the braking force control ECU 230, and the steering control ECU 240 are connected via the vehicle-mounted network 250. The vehicle 10 is not limited to automatic driving, and may be manually driven by a driver.

The camera 122 takes an image of the surroundings of the own vehicle and acquires an image. The object sensor 124 detects the situation around the own vehicle. Examples of the object sensor 124 include an object sensor using reflected waves such as a laser radar, a millimeter wave radar, and an ultrasonic sensor.

The image recognition device 110 includes an image acquisition unit 111, a sign detection unit 112, a moving object recognition unit 113, a calculation unit 114, and a determination unit 115. The image recognition device 110 includes a central processing unit (CPU), a microcomputer composed of a RAM and a ROM, and the like, and the microcomputer executes a pre-installed program to realize the functions of each of these parts. However, some or all of the functions of each of these parts may be realized by a hardware circuit.

The image acquisition unit 111 acquires an image of the periphery of the vehicle 10 taken by the camera 122.

The sign detection unit 112 detects road sign candidates (hereinafter referred to as “sign candidates”) from the image acquired by the image acquisition unit 111 by using, for example, an image analysis technique such as a pattern matching method. In the present embodiment, the "road sign" includes a road sign and a road marking. A "sign candidate" has the same or similar appearance as a road sign, for example, a sticker similar to a road sign indicating a speed limit, which is affixed to a truck or a bus as an indication of traveling speed, or walking. Includes clothes that are worn by a person and have the same design as the road sign. The sign detection unit 112 recognizes the presence, position, size, distance, etc. of the sign candidate. In addition, the sign detection unit 112 recognizes the instruction content of the road sign indicated by the sign candidate.

The moving body recognition unit 113 recognizes a moving body from the image acquired by the image acquisition unit 111 by using an image analysis technique such as semantic segmentation. In the present embodiment, the "moving body" means a movable object, and includes a vehicle including a four-wheeled vehicle and a two-wheeled vehicle, and a person. More specifically, the moving body recognition unit 113 recognizes the existence, position, size, distance, and the like of the moving body. The mobile body recognition unit 113 may recognize a part or all of the information by the output signal of the object sensor 124 or the inter-vehicle communication.

The calculation unit 114 calculates a collation degree indicating the degree to which the sign candidate detected by the sign detection unit 112 is on the area of the mobile body recognized by the mobile body recognition unit 113. In the present embodiment, the calculation unit 114 calculates as a collation degree representing the degree to which the area of the sign candidate in the image overlaps the area of the moving body in the range of 0 to 1. The degree of collation increases as the degree to which the area of the sign candidate overlaps the area of the moving body is higher.

The determination unit 115 determines whether or not the sign candidate is a road sign according to the degree of collation calculated by the calculation unit 114.

The notification unit 140 notifies the driver of the vehicle 10 regarding the road sign determined by the determination unit 115 to be a road sign. The notification unit 140 notifies the information indicated by the sign by using, for example, a display device for displaying characters or pictures such as a car navigation system or a voice device such as a speaker.

The automatic operation control unit 210 is composed of a central processing unit (CPU), a microcomputer composed of a RAM and a ROM, and the like, and realizes an automatic operation function by executing a pre-installed program by the microcomputer. For example, the automatic driving control unit 210 controls the driving force control ECU 220, the braking force control ECU 230, and the steering control ECU 240 according to the road sign recognized by the image recognition device 110.

The driving force control ECU 220 is an electronic control device that controls an actuator that generates a driving force of a vehicle such as an engine. When the driver manually operates, the driving force control ECU 220 controls a power source such as an engine or an electric motor according to the amount of operation of the accelerator pedal. On the other hand, in the case of automatic operation, the driving force control ECU 220 controls the power source according to the required driving force calculated by the automatic operation control unit 210.

The braking force control ECU 230 is an electronic control device that controls a brake actuator that generates a braking force of the vehicle. When the driver manually operates, the braking force control ECU 230 controls the brake actuator according to the operation amount of the brake pedal. On the other hand, in the case of automatic driving, the braking force control ECU 230 controls the brake actuator according to the required braking force calculated by the automatic driving control unit 210.

The steering control ECU 240 is an electronic control device that controls a motor that generates steering torque of the vehicle. When the driver manually operates, the steering control ECU 240 controls the motor in response to the operation of the steering handle to generate an assist torque for the steering operation. As a result, the driver can operate the steering with a small amount of force, and the steering of the vehicle is realized. On the other hand, in the case of automatic driving, the steering control ECU 240 steers by controlling the motor according to the required steering angle calculated by the automatic driving control unit 210.

The image recognition process shown in FIG. 2 is a process in which the image recognition device 110 determines whether or not the sign candidate detected by the sign detection unit 112 is a road sign. This process is a process that is repeatedly executed by the image recognition device 110 while the vehicle 10 is traveling. First, the image acquisition unit 111 acquires the peripheral image of the vehicle 10 taken by the camera 122 in step S100.

Next, in step S110, the sign detection unit 112 detects a sign candidate from the image acquired in step S100. Subsequently, in step S120, the moving body recognition unit 113 recognizes the moving body from the image acquired in step S100. Note that steps S110 and S120 may be executed without limitation in this order, or may be executed in parallel.

Subsequently, in step S130, the calculation unit 114 calculates the degree to which the region with the labeling candidate detected in step S110 overlaps the region with the moving body recognized in step S120 as the collation degree.

Subsequently, the determination unit 115 determines in the process of step S140 whether or not the collation degree calculated in step S130 is equal to or greater than a predetermined threshold value. The threshold value is a collation degree that can be determined to be a region on the same object, and can be determined by performing a simulation or an experiment in advance. When the collation degree is equal to or higher than the threshold value, that is, when the sign candidate is on the moving body, the determination unit 115 proceeds to the process of step S150 and determines that the sign candidate is not a road sign. On the other hand, when the collation degree is smaller than the threshold value, that is, when the sign candidate is not on the moving body, the determination unit 115 proceeds to the process of step S155 and determines that the sign candidate is a road sign. After step S155, the image recognition device 110 may control the notification unit 140 to notify the road sign. The sign detection unit 112 may recognize the information indicated by the road sign after step S155, or may recognize the information indicated by the road sign at the time of step S110.

As shown in FIGS. 3 and 4, the image recognition device 110 determines whether or not the sign candidate detected from the image captured by the camera 122 is a sign. The threshold is, for example, 0.6. As shown in FIG. 3, when the region D1 where the other vehicle 20 which is a moving body is located and the region D2 where the sign candidate S1 is located all overlap, the collation degree is 1. Therefore, the collation degree is equal to or higher than the threshold value, and it is determined that the sign candidate S1 is not a road sign. As shown in FIG. 4, when the region D3 where the other vehicle 30 which is a moving body is located and the region D4 where the sign candidate S2 is located do not overlap at all, the collation degree is 0. Therefore, the collation degree is smaller than the threshold value, and it is determined that the sign candidate S2 is a road sign.

According to the image recognition device 110 of the present embodiment described above, whether or not the road sign candidate is a road sign is determined according to the degree of collation indicating the degree to which the road sign candidate is on the area of the moving object. Since the determination is made, erroneous recognition of road signs can be suppressed.

B. Second embodiment:
The second embodiment is different from the first embodiment in that the calculation unit 114 performs a calculation using the history of a plurality of past collations calculated within a predetermined period, and calculates the collation. Since the configuration of the image recognition device of the second embodiment is the same as the configuration of the image recognition device of the first embodiment, the description of the configuration of the image recognition device will be omitted.

In the second embodiment, in step S130 (FIG. 2), the calculation unit 114 performs a calculation using the history of a plurality of past collation degrees calculated within a predetermined period, and calculates the collation degree. The operation is, for example, an operation for obtaining a representative value such as an average value or a median value. Further, when the variance is obtained and the variance is large, the minimum value may be used as the collation degree.

As shown in FIG. 5, even when the area D5 where the other vehicle 40 is located and the area D6 where the sign candidate S3 is located overlap, the area D5 moves as the other vehicle 40 travels, and the area D5 moves. It will not overlap with D6. A more accurate collation degree can be calculated by the calculation unit 114 performing a calculation using the history of a plurality of past collation degrees calculated within a predetermined period and calculating the collation degree.

According to the image recognition device 110 of the present embodiment described above, the determination unit 115 performs a calculation using the history of a plurality of past collation degrees calculated without a predetermined period, and responds to the calculated collation degree. Therefore, since it is determined whether or not the candidate for the road sign is a road sign, erroneous recognition of the road sign can be further suppressed.

C. Other embodiments:
(C1) In the above-described embodiment, the calculation unit 114 calculates the degree to which the region with the labeling candidate overlaps the region of the moving body as the collation degree. Instead, the calculation unit 114 may calculate the degree of collation according to the degree to which the region with the sign candidate overlaps the region of the road surface on which the moving body exists. The road surface area is recognized from the image taken by the camera 122. For example, the calculation unit 114 may calculate the degree to which the area of the sign candidate on the image overlaps the area of the road surface, and subtract the calculated degree from 1 as the collation degree.

(C2) In the above-described embodiment, the determination unit 115 may set a threshold value according to the position where the labeling candidate is located. For example, when the sign candidate is located around the roadside, the sign candidate is likely to be a road sign, so the threshold value is set high.

(C3) In the above-described embodiment, the moving body recognition unit 113 recognizes the moving body from the image acquired by the image acquisition unit 111 by using semantic segmentation. Instead, the moving object may be recognized from the image acquired by the image acquisition unit 111 by using an image analysis technique such as pattern matching.

The present disclosure is not limited to the above-described embodiment, and can be realized by various configurations within a range not deviating from the gist thereof. For example, the technical features in the embodiments corresponding to the technical features in each form described in the column of the outline of the invention are for solving the above-mentioned problems or for achieving a part or all of the above-mentioned effects. In addition, it is possible to replace or combine them as appropriate. Further, if the technical feature is not described as essential in the present specification, it can be appropriately deleted.

10 ... Vehicle, 100 ... Automatic driving control system, 110 ... Image recognition device, 111 ... Image acquisition unit, 112 ... Sign detection unit, 113 ... Moving object recognition unit, 114 ... Calculation unit, 115 ... Judgment unit, 122 ... Camera

Claims (6)

It is an image recognition device
An image acquisition unit (111) that acquires an image taken by the camera (122), and
A sign detection unit (112) that detects road sign candidates from the image, and
A moving body recognition unit (113) that recognizes a moving body from the image,
A calculation unit (114) that calculates a collation degree indicating the degree to which the candidate is on the region of the moving body, and
An image recognition device including a determination unit (115) for determining whether or not the candidate is a road sign according to the degree of collation. The image recognition device according to claim 1.
The moving body recognition unit is an image recognition device that recognizes the moving body from the image by using semantic segmentation. The image recognition device according to claim 1 or 2.
The calculation unit is an image recognition device that calculates the degree to which the candidate region in the image overlaps the region of the moving body as the collation degree. The image recognition device according to any one of claims 1 to 3.
The calculation unit is an image recognition device that calculates the collation degree by performing a calculation using the history of a plurality of past collation degrees calculated within a predetermined period. The image recognition device according to claim 4.
The image recognition device is an operation for obtaining a representative value of the plurality of collation degrees. It is an image recognition method
The image acquisition process to acquire the image taken by the camera and
A sign detection step for detecting road sign candidates from the image, and
A moving body recognition step of recognizing a moving body from the image,
A calculation step of calculating a collation degree indicating the degree to which the candidate is on the region of the moving body, and
An image recognition method comprising a determination step of determining whether or not the candidate is a road sign according to the degree of collation.

Images