JP2018107582A - Shield determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent an error determination of a shield region.SOLUTION: A shield determination device 1 comprises: an image acquisition part that acquires a plurality of images imaged by an imaging device 3 at a different timing 11; a vehicle information acquisition part 13 that acquires a position of a vehicle 2 and an imaging direction of the imaging device 3; a map database 7 that stores the position of a material and external appearance information; a material estimation part 14 that estimates the material in a field angle in the imaging device 3 at an image ground point on the basis of the imaging direction of the position of the vehicle 2 and the imaging device 3 and the position of the material stored in the map database 7; an exclusion determination part 15 that determines whether the material in the field angle of the imaging device 3 estimated by the material estimation part 14 is a predetermined exclusion material on the basis of the external appearance information stored in the map database 7; and a state determination part 12 that determines whether the imaging device 3 is a shield state on the basis of a pixel value f a plurality of images excluding a region determined as the exclusion material by the exclusion determination part 15.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to a shielding determination device.

  Patent Document 1 discloses an apparatus that detects that a camera is shielded by using a plurality of captured images obtained by capturing images at different timings depending on a camera mounted on a vehicle, such as contamination. Have been described. This apparatus determines an area where the temporal change amount of the first predetermined type of pixel value is equal to or smaller than the predetermined amount and the distribution of the second predetermined type of pixel value is flat as a shielded portion.

Japanese Patent Laying-Open No. 2015-061163

  However, for a plain wall or a road without a road surface sign, the temporal change amount of the first predetermined type of pixel value is not more than a predetermined amount and the distribution of the second predetermined type of pixel value is flat on the image. It becomes. For this reason, there is a possibility that the device described in Patent Document 1 may erroneously determine a plain wall, a road without a road surface sign, or the like as a shielded part. In this technical field, a shielding determination device that can suppress erroneous determination of a shielding region is desired.

  The shielding determination device according to the present disclosure is a shielding determination device that determines whether an imaging device mounted on a vehicle is in a shielding state, and acquires a plurality of images captured by the imaging device at different timings. An acquisition unit, a vehicle information acquisition unit that acquires the position of the vehicle and the imaging direction of the imaging device, a map database that stores the position and appearance information of the object, and the vehicle position and the imaging direction of the imaging device are stored in the map database. Of the imaging device estimated by the object estimation unit based on the appearance information stored in the map database and the object estimation unit that estimates the object within the angle of view of the imaging device at the imaging point An exclusion determination unit that determines whether or not an object within the angle of view is a predetermined excluded object, and a plurality of image images excluding an area that is determined to be an excluded object by the exclusion determination unit Based on the value, it comprises an imaging device and determining the state determining unit that determines whether a blocking state, a.

  According to the present disclosure, erroneous determination of a shielding area can be suppressed.

3 is a functional block diagram of a vehicle 2 including a shielding determination device 1. FIG. 4 is a flowchart of a shielding determination process of the shielding determination device 1. It is a figure explaining the shielding determination process.

  Hereinafter, this embodiment will be described with reference to the drawings. In the following description, the same or equivalent elements will be denoted by the same reference numerals, and redundant description will be omitted.

  The shielding determination apparatus according to the present embodiment is an apparatus that determines whether an imaging device mounted on a vehicle is in a shielding state. The shielded state refers to a state in which a subject that is supposed to be imaged is shielded by a foreign object or the like. A specific example in which the imaging apparatus is in a shielding state is when water drops or dust adheres to the lens, or when water drops or bird droppings adhere to the windshield. Hereinafter, an example in which the shielding determination apparatus is mounted on a vehicle will be disclosed, but the present invention is not limited to this.

[Constitution]
FIG. 1 is a functional block diagram of a vehicle 2 including a shielding determination device 1. The vehicle 2 includes an imaging device 3, a positioning device 4, an internal sensor 5, an ECU (Electronic Control Unit) 6, and a map database 7.

  The imaging device 3 is a device that images the surroundings of the vehicle 2. Examples of the imaging device 3 include a stereo camera and a monocular camera. The imaging device 3 acquires frame images that are continuous in time series.

  The positioning device 4 is a device that acquires the position of the vehicle 2. The position of the vehicle 2 is expressed by position information such as latitude and longitude. Examples of the positioning device 4 include a GPS (Global Positioning System) receiver and a GPS / IMU (Inertial Measurement Unit).

  The internal sensor 5 is a detection device that detects the traveling state of the vehicle 2. The traveling state of the vehicle 2 includes the direction of the vehicle 2. The internal sensor 5 is a yaw rate sensor as an example. The internal sensor 5 may include a speed sensor.

  The ECU 6 is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a CAN (Controller Area Network) communication circuit, and the like. ECU6 is connected to the network which communicates using a CAN communication circuit, and is connected so that communication with the component of the vehicle 2 is possible. Details of the ECU 6 will be described later.

  The map database 7 stores the position and appearance information of the object. An object is a tangible object that can be recognized by a sensor, such as a three-dimensional object such as a wall or guardrail, or a road surface. Appearance information is information on the appearance of an object, such as presence / absence of road markings, edges, and textures. As an example, the map database 7 stores map information related to a map and the position and appearance information of an object in association with each other.

  The ECU 6 is connected to the imaging device 3, the positioning device 4, the internal sensor 5, and the map database 7. Functionally, the ECU 6 includes an image acquisition unit 11, a state determination unit 12, a vehicle information acquisition unit 13, an object estimation unit 14, and an exclusion determination unit 15.

  The image acquisition unit 11 acquires a plurality of images captured by the imaging device 3 at different timings. For example, the image acquisition unit 11 acquires at least two consecutive frame images.

  The vehicle information acquisition unit 13 acquires the position of the vehicle 2 and the imaging direction of the imaging device 3. The vehicle information acquisition unit 13 acquires the position of the vehicle 2 based on the information output from the positioning device 4. The vehicle information acquisition unit 13 acquires the direction of the vehicle 2 based on the detection result of the internal sensor 5, and acquires the imaging direction of the imaging device 3 based on the direction of the vehicle 2.

  The object estimation unit 14 estimates an object within the angle of view of the imaging device 3 at the imaging point based on the position of the vehicle 2 and the imaging direction of the imaging device 3 and the position of the object stored in the map database 7. For example, the object estimation unit 14 specifies the position of the vehicle 2 on the map, and reproduces an image directed from the position in the imaging direction based on the position of the object stored in the map database 7. Thereby, the object estimation unit 14 estimates an object within the angle of view of the imaging device 3 at the imaging point.

  Based on the appearance information stored in the map database 7, the exclusion determination unit 15 determines whether the object within the angle of view of the imaging device 3 estimated by the object estimation unit 14 is a predetermined excluded object. To do. An excluded object is an object that is excluded from the determination of the shielding state described later, and is a predetermined object. The excluded object is, for example, an object that has a small change in pixel value with time and is estimated to have no texture and no edges. An example of the excluded object is a wall without a texture and an edge, a road surface without a road marking, and the like.

  The state determination unit 12 determines whether or not the imaging device 3 is in a shielding state based on the pixel values of a plurality of images excluding the region determined to be an excluded object by the exclusion determination unit 15. In order to realize this function, the state determination unit 12 includes an area division unit 121, an edge extraction unit 122, a target region determination unit 123, a determination unit 124, and an output unit 125.

  The area dividing unit 121 divides an image acquired by the imaging device 3 into areas of a predetermined size. The size of the area is arbitrary. As an example, the region dividing unit 121 divides an image into 10 parts in the vertical direction and 6 parts in the horizontal direction.

  The target region determination unit 123 determines a target region to be determined by excluding a region determined to be an excluded object by the exclusion determination unit 15 from the regions divided by the region division unit 121.

  The edge extraction unit 122 extracts edges from the image that is the target of region division. The edge extraction unit 122 may acquire the entire range of edges, or may extract only the edges of the target region to be determined.

  The determination part 124 determines whether it is a shielding state. The determination unit 124 compares the pixel value and the temporal change amount of the edge, the dispersion of the pixel value, the score of the edge, and the intensity for each target region with a preset threshold value. It judges, and if it is not below a threshold value, it will judge with it not being a shielding area. Then, the determination unit 124 determines that it is in a shielding state when at least one target region is a shielding region, and determines that it is not in a shielding state when all the target regions are not shielding regions.

  The output unit 125 outputs the determination result of the determination unit 124. The output unit 125 may output the determination result to the display device, or may output and store the determination result in the storage unit. The output determination result is used, for example, for notification to the driver or used as additional information when the map database 7 is updated.

[Flowchart of shielding determination process]
FIG. 2 is a flowchart of the shielding determination process of the shielding determination device 1. FIG. 3 is a diagram illustrating the shielding determination process. The flowchart of FIG. 2 is started at the timing when the operation instruction of the shielding determination device 1 is received from the driver.

  As illustrated in FIG. 2, the image acquisition unit 11 of the shielding determination device 1 acquires a plurality of images captured by the imaging device 3 at different timings as an image acquisition process (S10). Next, the vehicle information acquisition part 13 of the shielding determination apparatus 1 acquires the position of the vehicle 2 and the imaging direction of the imaging device 3 as a vehicle information acquisition process (S12).

  Next, the object estimation unit 14 of the occlusion determination device 1 performs imaging based on the position of the vehicle 2, the imaging direction of the imaging device 3, and the position of the object stored in the map database 7 as object estimation processing (S <b> 14). An object within the angle of view of the imaging device 3 at the point is estimated. In the example illustrated in FIG. 3A, the image Gn (n is an integer) is an image captured by the imaging device 3. As illustrated in FIG. 3A, the object estimation unit 14 determines the road surface 30 and the guardrail 31 as objects existing within the angle of view of the imaging device 3 based on the position of the object stored in the map database 7. And the wall 32 is estimated.

  Subsequently, the area dividing unit 121 of the shielding determination apparatus 1 divides the image acquired by the imaging apparatus 3 into areas of a predetermined size as an area dividing process (S16). In the example shown in FIG. 3B, the region dividing unit 121 divides the image into 60 regions R.

  Subsequently, the exclusion determination unit 15 of the shielding determination apparatus 1 performs the excluded object determination process (S17), and the angle of view of the imaging device 3 estimated by the object estimation unit 14 based on the appearance information stored in the map database 7. It is determined whether or not the inside object is a predetermined excluded object. When it is determined that there is no road marking on the road surface 30 based on the appearance information, the exclusion determination unit 15 determines that the road surface 30 is an excluded object. The exclusion determination unit 15 determines that the wall 32 is an excluded object when it is determined that the wall 32 has no texture and no edges based on the appearance information.

  When it is determined in the excluded object determination process (S17) that an excluded object exists within the angle of view (S17: Yes), the target area determination unit 123 of the shielding determination apparatus 1 performs an exclusion determination unit as the area selection process (S18). The remaining area excluding the area determined to be an excluded object by 15 is selected as the target area. In the example shown in (C) of FIG. 3, the target region determination unit 123 selects the remaining region excluding the excluded region RB determined as an excluded object by the exclusion determination unit 15 as the target region RA.

  The edge extraction unit 122 of the occlusion determination apparatus 1 extracts an edge from the image subjected to region division as an edge extraction process (S20). In the shielding determination process (S22), the determination unit 124 of the shielding determination device 1 determines whether or not each target region RA is a shielding region, and determines the shielding state using the determination results of all regions. The output unit 125 outputs the determination result of the determination unit 124 as an output process (S24). This is the end of the flowchart shown in FIG. The flowchart is executed from the beginning at the next request timing.

[Effect of shielding determination device 1 according to embodiment]
In the case of a conventional device, there is a risk that a plain wall or a road without a road sign is erroneously determined as a shielded part. On the other hand, according to the shielding determination apparatus 1 according to the embodiment, the region where the object with the possibility of erroneous determination is copied is excluded from the determination target of the shielding state, and thus the erroneous determination of the shielding region is suppressed. can do. Thereby, according to the shielding determination apparatus 1 which concerns on embodiment, the precision of determination of a shielding area can be improved.

  The above-described embodiments can be implemented in various forms with various changes and improvements based on the knowledge of those skilled in the art.

  For example, the vehicle information acquisition unit 13 may acquire the position of the vehicle 2 using information other than the information output from the positioning device 4. For example, the vehicle information acquisition unit 13 may acquire the position of the vehicle 2 using the imaging result of the imaging device 3, the map database 7, and the internal sensor 5 (vehicle speed sensor). More specifically, the vehicle information acquisition unit 13 acquires the position at which the previous frame image was captured by collating the map database 7 and the previous frame image, and from the imaging time of the previous frame image to the imaging time of the rear frame image. The imaging position of the rear frame image is obtained by calculating the amount of movement of the vehicle 2 from the previous frame image to the rear frame image based on the time of the time and the detection result of the internal sensor 5 (vehicle speed sensor). May be.

  For example, the vehicle information acquisition unit 13 may acquire the imaging direction using a sensor that detects the angle of the imaging device 3, or may acquire the imaging direction using a gyro sensor provided in the imaging device 3. Good.

  The map database 7 may exist outside the vehicle 2.

DESCRIPTION OF SYMBOLS 1 ... Occlusion determination apparatus, 2 ... Vehicle, 3 ... Imaging apparatus, 4 ... Positioning apparatus, 5 ... Internal sensor, 6 ... ECU, 7 ... Map database, 11 ... Image acquisition part, 12 ... State determination part, 13 ... Vehicle information Acquisition unit, 14 ... object estimation unit, 15 ... exclusion determination unit.

Claims (1)

A shielding determination device that determines whether an imaging device mounted on a vehicle is in a shielding state,
An image acquisition unit that acquires a plurality of images captured by the imaging device at different timings;
A vehicle information acquisition unit that acquires the position of the vehicle and the imaging direction of the imaging device;
A map database for storing object position and appearance information;
An object estimation unit that estimates the object within the angle of view of the imaging device at an imaging point based on the position of the vehicle and the imaging direction of the imaging device and the position of the object stored in the map database;
An exclusion determination unit that determines whether or not the object within the angle of view of the imaging apparatus estimated by the object estimation unit is a predetermined excluded object based on the appearance information stored in the map database When,
A state determination unit that determines whether or not the imaging device is in a shielding state based on pixel values of the plurality of images excluding a region determined to be the excluded object by the exclusion determination unit;
A shielding determination device comprising:

Images