JP2019007744A - Object sensing device, program, and object sensing system - Google Patents

Abstract

To enhance accuracy of a sensing result of an object using imaging information.SOLUTION: An object sensing device 40 according to the present disclosure comprises: an imaging information acquisition unit 41 which acquires imaging information acquired by imaging a predetermined visual field range; a distance image acquisition unit 43 as a detection information acquisition unit, which acquires detection information of a reflected wave of an electromagnetic wave applied to at least one position in the predetermined visual field range; an imaging information processing unit 42 as an object sensing unit, which performs sensing processing of an object in the predetermined visual field range on the basis of the imaging information; and a determination unit 45 which determines correctness of a sensing result from the sensing processing on the basis of the detection information.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an object detection device, a program, and an object detection system.

  Patent Document 1 discloses a technique for detecting an object (such as another vehicle or a pedestrian) around the vehicle using a laser radar and an imaging device (camera) mounted on the vehicle. In this technique, an object having a high laser beam reflection intensity (such as a vehicle) and an object having a low reflection intensity (such as a pedestrian) are distinguished by laser radar. Then, when detecting an object by image processing on a captured image (imaging information) of the imaging apparatus, an object having a high reflection intensity and an object having a low reflection intensity are detected based on the detection result of the laser radar.

JP 2007-240314 A

  In the object detection process using the imaging information of the imaging apparatus as described above, further improvement in the accuracy of the detection result is required.

  An object of the present disclosure made in view of the above problems is to improve the accuracy of the detection result of an object using imaging information.

  In order to solve the above-described problem, an object detection device according to a first aspect is irradiated with an imaging information acquisition unit that acquires imaging information obtained by imaging a predetermined visual field range, and at least one position within the predetermined visual field range. A detection information acquisition unit that acquires detection information of the reflected wave of the electromagnetic wave, an object detection unit that performs detection processing of an object within the predetermined visual field range based on the imaging information, and the detection information based on the detection information, And a determination unit that determines whether the detection result by the detection process is correct or incorrect.

  The program according to the second aspect includes a step of acquiring imaging information obtained by imaging a predetermined visual field range in a computer, and detection information of a reflected wave of an electromagnetic wave irradiated to at least one position within the predetermined visual field range. Acquiring, based on the imaging information, performing detection processing of an object within the predetermined visual field range, and determining whether the detection result by the detection processing is correct based on the detection information. Let it run.

  An object detection system according to a third aspect includes an imaging device, an electromagnetic wave emission device that radiates electromagnetic waves, an electromagnetic wave detection device that detects reflected waves of the electromagnetic waves, and an object detection device, wherein the object detection device includes: An imaging information acquisition unit that acquires imaging information obtained by imaging the predetermined visual field range by the imaging device, and detection information that acquires detection information of reflected waves of electromagnetic waves irradiated to at least one position within the predetermined visual field range An acquisition unit, an object detection unit that performs detection processing of an object within the predetermined visual field range based on the imaging information, and a determination unit that determines whether the detection result by the detection processing is correct based on the detection information; .

  According to the present disclosure, it is possible to improve the accuracy of the detection result of an object using imaging information.

It is a figure showing an example of composition of an object detection system concerning one embodiment of this indication. It is a figure which shows an example of the captured image of the imaging device shown in FIG. It is a figure which shows an example of the detection result of the object by the imaging information processing part shown in FIG. It is a figure which shows an example of the detection result of the object by the imaging information processing part shown in FIG. It is a figure which shows an example of the distance image which the distance image sensor shown in FIG. 1 acquires. It is a flowchart which shows an example of an operation | movement in the case of determining whether the object area | region on the captured image is detected smaller than an actual object by the determination part shown in FIG. It is a flowchart which shows an example of operation | movement in the case of determining whether the object area | region on the captured image is detected larger than the actual object by the determination part shown in FIG. It is a figure which shows an example of the captured image of the imaging device shown in FIG. It is a flowchart which shows an example of operation | movement in the case of determining whether the misdetection by the background pattern etc. is contained in a detection result by the determination part shown in FIG. It is a figure which shows an example of the imaging state by the imaging device shown in FIG. It is a figure which shows the captured image of the imaging device in the state shown in FIG. It is a flowchart which shows an example of operation | movement in the case of determining whether the misdetection by the overlapping of an object is contained in a detection result by the determination part shown in FIG. It is a figure which shows the distance image obtained by a distance image sensor in the state shown in FIG.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each figure, the same numerals indicate the same or equivalent components.

  In recent years, studies on automatic driving of vehicles such as automobiles are underway. In automatic driving of a vehicle, it is necessary to detect an object (another vehicle, a pedestrian, etc.) around the vehicle. Therefore, for example, as disclosed in Patent Document 1, image recognition processing (object detection processing) for a captured image (imaging information) in a predetermined visual field range by an imaging device mounted on a vehicle is used. An object is detected.

  In the object detection process using the imaging information as described above, a portion having a large color change between pixels is extracted as a feature of an edge. The shape of the object is detected by connecting the boundary where the color change between the pixels is large to the adjacent boundary. In such processing, it may be difficult to accurately detect an object depending on the situation (such as how light strikes the object) at the time of acquisition of imaging information.

  For example, when a large object exists near the object to be detected (vehicle, person, etc.) and the shadow of the object hits the object to be detected, the object to be detected cannot be partially seen in the imaging information, or It may be difficult to see. In this case, it is difficult to detect a shadowed portion. When the light from the light source is strong, the entire captured image becomes whitish, and the color change between the object to be detected and the periphery of the object becomes small. In this case, the detection target object may be erroneously detected up to the peripheral part beyond the detection target object region.

  In addition, for example, when a plurality of similar-colored objects are arranged so as to overlap each other when viewed from the imaging device, the color change at the boundary between different objects is small on the captured image of the imaging device, and an edge is extracted. Difficult to do. Therefore, for example, a plurality of objects may be erroneously detected as one object.

  Further, when the background pattern other than the detection target object (vehicle, person, etc.) is similar to the shape of the detection target object, the background may be erroneously detected as the detection target object. .

  As described above, an error may be included in the detection result by the detection processing of the object within the predetermined visual field range using the imaging information. It is necessary to detect such an error and correct it to improve the accuracy of the detection result of the object. In the present disclosure, the accuracy of the detection result of the object is determined by determining whether the detection result of the detection process of the object using the imaging information is correct or not based on the detection information of the reflected wave of the irradiated electromagnetic wave within a predetermined visual field range. To improve.

  FIG. 1 is a diagram illustrating an example of a configuration of an object detection system 10 according to the present embodiment. The object detection system 10 according to the present embodiment detects an object within a predetermined visual field range and determines whether the detection result is correct or incorrect. The object detection system 10 is used, for example, to detect an object within a predetermined visual field range in front of or behind the vehicle.

  The object detection system 10 illustrated in FIG. 1 includes an imaging device 20, a distance image sensor 30, and an object detection device 40.

  The imaging device 20 is a camera, for example, and acquires imaging information (captured image) obtained by imaging a predetermined visual field range. The imaging information of the imaging device 20 indicates color information of objects and backgrounds within a predetermined visual field range. The imaging device 20 outputs the acquired imaging information to the object detection device 40.

  The distance image sensor 30 irradiates at least one position in a predetermined visual field range where the imaging device 20 captures an image such as ultraviolet rays, visible rays, and near infrared rays, and acquires detection information of reflected waves of the electromagnetic waves. Specifically, the distance image sensor 30 irradiates electromagnetic waves within a predetermined visual field range of the imaging device 20. The distance image sensor 30 detects the reflected wave reflected by the electromagnetic wave at the irradiation position (the object at the irradiation position), and acquires the detection information of the reflected wave.

  The reflected wave detection information includes, for example, information indicating distance information in the viewing direction to the irradiation position of the electromagnetic wave. The time difference from when the electromagnetic wave is radiated from the distance image sensor 30 to when the reflected wave of the electromagnetic wave is detected depends on the distance from the distance image sensor 30 to the irradiation position of the electromagnetic wave. Therefore, distance information to the irradiation position of the electromagnetic wave can be obtained by detecting the time difference. The distance information may be information indicating the above time difference, or may be information indicating the distance to the irradiation position of the electromagnetic wave obtained based on the time difference. The reflected wave detection information may include information indicating the intensity of the reflected wave.

  The distance image sensor 30 acquires detection information for a plurality of positions in the predetermined visual field range by irradiating the plural positions within the predetermined visual field range of the imaging device 20 and detecting a reflected wave of the electromagnetic waves. The distance image sensor 30 outputs detection information acquired for a plurality of positions within a predetermined visual field range to the object detection device 40. The output of the distance image sensor 30 includes detection information (distance information related to the distance in the viewing direction) for a plurality of positions within a predetermined visual field range. Therefore, if the detection information of the reflected wave from the irradiation position of each electromagnetic wave is the detection information in one pixel, the distance image sensor 30 has a plurality of pixels arranged, and the distance information in the viewing direction in the visual field range is displayed for each pixel. The distance image included in is output.

  The distance image sensor 30 includes an electromagnetic wave radiation device 31 and an electromagnetic wave detection device 32.

  The electromagnetic wave radiation device 31 radiates electromagnetic waves such as ultraviolet rays, visible rays, and near infrared rays. The electromagnetic wave radiation device 31 includes an LED (Light Emitting Diode), an LD (Laser Diode), and the like. The electromagnetic wave radiated from the electromagnetic wave radiation device 31 is irradiated within a predetermined visual field range that is imaged by the imaging device 20. The electromagnetic wave radiated from the electromagnetic wave radiation device 31 can be applied to a plurality of positions while being scanned within a predetermined visual field range, for example, by being reflected by a movable mirror. The above-described movable mirror includes, for example, a MEMS (Micro Electro Mechanical Systems) mirror, a polygon mirror, and a galvanometer mirror.

  The electromagnetic wave detection device 32 detects a reflected wave that is radiated from the electromagnetic wave radiation device 31 and radiated within a predetermined visual field range and reflected at the irradiation position. The electromagnetic wave detection device 32 includes, for example, an APD (Avalanche PhotoDiode), a PD (PhotoDiode), and the like. By detecting the reflected wave, it is possible to acquire detection information such as a time difference until the reflected image of the electromagnetic wave is detected from the distance image sensor 30 and the reflected wave of the electromagnetic wave is detected, and the intensity of the reflected wave.

  The electromagnetic wave emission device 31 may include a plurality of light emitting elements that emit electromagnetic waves, and the electromagnetic wave detection device 32 may include a plurality of light receiving elements that detect reflected waves of the electromagnetic waves. In this case, the distance image sensor 30 irradiates electromagnetic waves from a plurality of light emitting elements within a predetermined visual field range, and receives reflected waves of the electromagnetic waves by the plurality of light receiving elements, so that a plurality of positions within the predetermined visual field range are obtained. Detection information in can be acquired.

  The object detection device 40 detects an object within a predetermined visual field range by image recognition processing on the imaging information output from the imaging device 20, and outputs the detection result on the display device in FIG. . Here, the object detection device 40 determines whether the detection result of the object is correct based on the detection information detected by the distance image sensor 30. When the object detection device 40 determines that the object detection result is incorrect, for example, the object detection device 40 corrects the detection result based on the detection information, or performs object detection processing again based on the detection information. To go.

  The object detection device 40 includes an imaging information acquisition unit 41, an imaging information processing unit 42, a distance image acquisition unit 43 (detection information acquisition unit), a distance image processing unit 44, a determination unit 45, and an output unit 46. Prepare. Note that the imaging information processing unit 42, the distance image processing unit 44, and the determination unit 45 can be configured to include one or more processors and memories. The processor may include at least one of a general-purpose processor that reads a specific program and executes a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include an FPGA (Field-Programmable Gate Array). The imaging information processing unit 42, the distance image processing unit 44, and the determination unit 45 include at least one of SoC (System-a-Chip) in which one or more processors cooperate and SiP (System In a Package). You may comprise.

  The imaging information acquisition unit 41 acquires imaging information from the imaging device 20. The imaging information acquisition unit 41 is an interface with the imaging device 20. The imaging information acquisition unit 41 outputs the acquired imaging information to the imaging information processing unit 42.

  The imaging information processing unit 42 performs predetermined processing such as feature amount (for example, edge) extraction on the imaging information output from the imaging information acquisition unit 41. The imaging information processing unit 42 performs detection processing of an object within a predetermined visual field range based on the extracted feature amount, and outputs the detection result to the determination unit 45.

  The distance image acquisition unit 43 acquires a distance image (detection information) from the distance image sensor 30. The distance image acquisition unit 43 is an interface with the distance image sensor 30. The distance image acquisition unit 43 outputs the acquired distance image (detection information) to the distance image processing unit 44.

  The distance image processing unit 44 performs a predetermined process such as extraction of feature amounts (such as adjacent pixels in which the distance difference in the viewing direction is larger than a predetermined value) from the distance image (detection information) output from the distance image acquisition unit 43. Is output to the determination unit 45.

  The determination unit 45 determines whether the detection result of the object based on the imaging information output from the imaging information processing unit 42 is correct based on the detection information output from the distance image processing unit 44. If the determination unit 45 determines that the detection result is correct, the determination unit 45 outputs the detection result to the output unit 46. If the determination unit 45 determines that the detection result is incorrect, for example, the imaging information processing unit 42 corrects the detection result based on the detection information, or performs the object detection process again based on the detection result. Or do it.

  The output unit 46 outputs the detection result of the object output from the determination unit 45 by displaying it on the display device in FIG. The output unit 46 is an interface with an external device such as a display device.

  Next, the operation of the object detection device 40 according to the present embodiment will be described.

  FIG. 2 is a diagram illustrating an example of a captured image (imaging information) of the imaging device 20 that captures a predetermined visual field range. As shown in FIG. 2, a sideways vehicle exists in the center of the captured image, and a house or the like exists on the back side of the vehicle.

  When the object detection process is performed using the imaging information illustrated in FIG. 2, as illustrated in FIG. 3A, the object region 101 detected as an object (vehicle) on the captured image is more than the actual vehicle on the captured image. May be smaller (FIG. 3A) or larger than the actual vehicle on the captured image (FIG. 3B).

  FIG. 4 is a diagram showing a distance image acquired by the distance image sensor 30 for the predetermined visual field range shown in FIG. In FIG. 4, for convenience of explanation, the distance image is shown superimposed on the captured image of the imaging device 20.

  The distance image includes a plurality of pixels indicated by rectangular regions in FIG. Each pixel includes distance information with respect to an object included in the pixel in the viewing direction viewed from the distance image sensor 30. In FIG. 4, the distance information is shown by the density of dots in each pixel (each rectangular area). A pixel with a sparse dot indicates that the distance from the object included in the pixel is short, and a pixel with a dense dot indicates that the distance from the object included in the pixel is far. As shown in FIG. 4, a road near the vehicle exists in the lower area of the distance image. In the upper area of the area, there is an area farther away than the lower area. This area corresponds to the vehicle. In the upper region, there is a region farther away. This area is an area corresponding to a house on the back side of the vehicle and the sky.

  The determination unit 45 determines whether the detection result of the object is correct based on the distance image (detection information related to the distance) as illustrated in FIG.

  FIG. 5 illustrates an example of an operation when the determination unit 45 determines whether an area (object area) where an object is detected on the captured image is detected smaller than an actual object on the captured image. It is a flowchart which shows.

  When the determination unit 45 acquires the detection result of the object based on the imaging information from the imaging information processing unit 42, the determination unit 45 selects an unselected object region among the detected object regions on the captured image (step S11).

  Next, the determination unit 45 selects one pixel (hereinafter referred to as a peripheral pixel) around a region on the distance image (hereinafter referred to as an object region on the distance image) corresponding to the selected object region (hereinafter referred to as a peripheral pixel). Step S12). Since the positional relationship between the imaging device 20 and the distance image sensor 30 is known, the position on the distance image corresponding to an arbitrary position on the captured image can be specified based on the positional relationship. In addition, the peripheral pixels of the object area on the distance image indicate pixels adjacent to the pixels constituting the outer edge of the object area on the distance image.

  Next, the determination unit 45 determines whether or not the difference between the distance in the viewing direction at the selected peripheral pixel and the average value of the distance in the viewing direction at the pixels constituting the object region on the distance image is less than a predetermined value. Determination is made (step S13).

  When it is determined that the difference between the distance in the viewing direction in the peripheral pixels and the average value of the distance in the viewing direction in the pixels constituting the object region on the distance image is less than the predetermined value (step S13: Yes), the determination The unit 45 determines that the detection result is an error (step S14).

  When it is determined that the difference between the distance in the viewing direction in the peripheral pixels and the average value of the distance in the viewing direction in the pixels constituting the object region on the distance image is not less than a predetermined value (step S13: No), the determination unit 45 determines that the detection result is correct (step S15).

  After the process of step S14 or step S15, the determination unit 45 determines whether or not all peripheral pixels of the object region on the selected distance image have been selected (step S16).

  If it is determined that all the peripheral pixels of the object area on the distance image have not been selected (step S16: No), the determination unit 45 returns to the process of step S12.

  When it is determined that all the peripheral pixels of the object area on the distance image have been selected (step S16: Yes), the determination unit 45 determines whether all the object areas on the captured image have been selected ( Step S17). Here, for example, when the determination unit 45 determines that the detection result is correct for all the peripheral pixels of the object region on the distance image corresponding to the object region on the selected captured image, the object on the selected captured image It can be determined that the detection result of the region is correct. In addition, for example, the determination unit 45 has less than a predetermined number of peripheral pixels that have been determined that the detection result is incorrect among the peripheral pixels of the object region on the distance image corresponding to the object region on the selected captured image. In this case, it may be determined that the detection result of the selected object region is correct.

  If it is determined that not all object regions on the captured image have been selected (step S17: No), the determination unit 45 returns to the process of step S11.

  If it is determined that all object regions on the captured image have been selected (step S17: Yes), the determination unit 45 ends the process.

  As illustrated in FIG. 3A, when the object area 101 on the captured image is detected smaller than the actual object (vehicle) on the captured image, the same object exists in the area around the object area 101. Therefore, the distance in the viewing direction does not change significantly between the object region 101 and the surrounding region. That is, when the object area on the captured image is detected smaller than the actual object, the distance in the viewing direction in the peripheral pixels of the object area on the distance image corresponding to the object area on the captured image and the distance image The difference from the distance (average value) in the viewing direction in the pixels constituting the object region becomes small. Therefore, the determination unit 45 determines that the object area on the captured image is smaller than the actual object based on the difference between the distance in the viewing direction at the peripheral pixels and the distance in the viewing direction of the pixels constituting the object area on the distance image. It can be determined whether or not it has been detected.

  Note that the determination of whether or not the object region on the captured image is detected smaller than the actual object is not limited to the method described with reference to FIG. In other words, the determination unit 45 does not have to perform the processes in steps S13 to S15 for all the peripheral pixels around the object region. Further, the comparison with the distance in the viewing direction in the peripheral pixels does not have to be the average value of the distance in the viewing direction in the pixels constituting the object region on the distance image.

  The determination unit 45 corresponds to the detection information at the first position (detection information of the reflected wave of the electromagnetic wave irradiated to the first position) and the object detected by the detection process, unlike the first position. Based on the difference of the detection information at the second position (detection information of the reflected wave of the electromagnetic wave irradiated to the second position), whether the detection result is correct or not (whether the object region is detected smaller than the actual object) Can be determined. Here, the detection information indicates distance information to the first position and the second position, that is, a distance in the viewing direction to the first position and the second position. When determining whether or not the object area is detected smaller than the actual object, the first position is a position corresponding to the periphery of the object detected by the detection process (for example, the object area on the distance image). Is a position corresponding to the peripheral pixel of the image). In this case, the determination unit 45 determines that the detection result is incorrect when the difference between the distance in the viewing direction to the first position and the distance in the viewing direction at the second position is less than a predetermined value. Can do.

  When the determination unit 45 determines that the object area is detected smaller than the actual object, the imaging information processing unit 42 corrects the detection result or performs the detection process again. When correcting the detection result, for example, the imaging information processing unit 42 enlarges the object region so as to include the first position. When performing the detection process again, for example, the imaging information processing unit 42 performs the detection process again on the assumption that the first position is included in the object region.

  FIG. 6 is a flowchart illustrating an example of an operation when the determination unit 45 determines whether or not the object region on the captured image is detected to be larger than the actual object on the captured image.

  When the determination unit 45 acquires the detection result of the object from the imaging information processing unit 42 using the imaging information, the determination unit 45 selects an unselected object region from among the object regions on the detected captured image (step S21).

  Next, the determination unit 45 selects one pixel (hereinafter referred to as an area pixel) that constitutes the object area on the distance image corresponding to the selected object area on the captured image (step S22).

  Next, the determination unit 45 determines whether or not the difference between the distance in the viewing direction at the selected region pixel and the average value of the distance in the viewing direction at the pixels constituting the object region on the distance image is greater than or equal to a predetermined value. Determination is made (step S23).

  When it is determined that the difference between the distance in the viewing direction of the region pixel and the average value of the distance in the viewing direction in the pixels constituting the object region on the distance image is greater than or equal to a predetermined value (step S23: Yes), the determination The unit 45 determines that the detection result is an error (step S24).

  When it is determined that the difference group between the distance in the viewing direction of the area pixel and the average value of the distance in the viewing direction in the pixels constituting the object area on the distance image is not greater than or equal to the predetermined value (step S23: No), The unit 45 determines that the detection result is correct (step S25).

  After the process of step S24 or step S25, the determination unit 45 determines whether or not all region pixels of the object region on the distance image have been selected (step S26).

  When it is determined that not all region pixels of the object region on the distance image have been selected (step S26: No), the determination unit 45 returns to the process of step S22.

  When it is determined that all the region pixels of the object region on the distance image have been selected (step S26: Yes), the determination unit 45 determines whether or not all the object regions on the captured image have been selected ( Step S27). Here, for example, if the determination unit 45 determines that the detection result is correct for all of the pixels constituting the object region on the distance image corresponding to the object region on the selected captured image, It can be determined that the detection result is correct. In addition, for example, the determination unit 45 has a predetermined number or less of pixels that are determined to be erroneous among the pixels constituting the object region on the distance image corresponding to the selected object region on the captured image. In this case, it may be determined that the detection result of the selected object region is correct.

  If it is determined that not all object regions on the captured image have been selected (step S27: No), the determination unit 45 returns to the process of step S21.

  If it is determined that all the object regions on the captured image have been selected (step S27: Yes), the determination unit 45 ends the process.

  As illustrated in FIG. 3B, when the object area 101 on the captured image is detected to be larger than the actual object (vehicle) on the captured image, the object area 101 includes a road on the near side of the object (vehicle). Or the area | region equivalent to the house of the back | inner side and the sky from a vehicle etc. is also included. In this case, even within the object region 101, the distance in the viewing direction in the region corresponding to the vehicle is greatly different from the distance in the viewing direction in the region other than the vehicle. That is, when the object area on the captured image is detected larger than the actual object, the distance in the viewing direction between the pixels other than the detection target in the object area on the distance image and the pixels constituting the object area on the distance image The difference from (average value) increases. Therefore, the determination unit 45 determines that the object area on the captured image is larger than the actual object based on the difference between the distance in the viewing direction in the area pixel and the distance in the viewing direction in the pixels constituting the object area on the distance image. It can be determined whether or not it has been detected.

  Note that the determination of whether or not the object region on the captured image is detected larger than the actual object is not limited to the method described with reference to FIG. That is, it is not necessary to perform the processing from step S23 to step S25 for all the region pixels constituting the object region on the distance image. Further, the comparison with the distance in the viewing direction in the area pixel does not need to be the average value of the distance in the viewing direction in the pixels constituting the object area on the distance image.

  The determination unit 45 detects whether the detection result is correct based on the detection information at the first position and the difference between the detection information at the second position corresponding to the object detected by the detection process, unlike the first position. Whether or not the object region is detected larger than the actual object). Here, the detection information indicates distance information to the first position and the second position, that is, a distance in the viewing direction to the first position and the second position. When determining whether or not the object area is detected to be larger than the actual object, the first position corresponds to the object detected by the detection process (included in the object area), and the second position It is a position inside. In this case, the determination unit 45 determines that the detection result is an error when the difference between the distance in the viewing direction to the first position and the distance in the viewing direction to the second position is greater than or equal to a predetermined value. be able to.

  When the determination unit 45 determines that the object region is detected to be larger than the actual object, the imaging information processing unit 42 corrects the detection result or performs the detection process again. When correcting the detection result, the imaging information processing unit 42 reduces the object region to the first position, for example. When performing the detection process again, for example, the imaging information processing unit 42 performs the detection process again on the assumption that the second position is not included in the object region. Further, the imaging information processing unit 42 determines that the second position corresponds to the background. The background is an object other than an object to be detected by the detection process (other vehicle, pedestrian, etc.).

  Next, another method for determining whether the detection result is correct or incorrect will be described. As described above, if the background pattern other than the object to be detected (car, person, etc.) is similar to the shape of the object to be detected, the background is erroneously detected as the object to be detected. Sometimes.

  FIG. 7 is a diagram illustrating an example of a captured image of the imaging device 20. In FIG. 7, as a detection result, an area 102 that is detected as a person (area that is hatched diagonally upward to the right) and an area 103 that is detected to be a vehicle (hatched diagonally downward to the right) are added. And an area 104 that is a background and is erroneously detected as a person even though an object such as a person does not exist (an area that is hatched in a diagonally upward direction). Yes. In order to improve the accuracy of the detection result, it is necessary to exclude an erroneous detection result such as the region 104. Below, the method for determining the presence or absence of such a false detection is demonstrated.

  FIG. 8 is a flowchart illustrating an example of an operation in a case where the determination unit 45 determines whether or not erroneous detection due to a background pattern or the like is included in an object detection result.

  When the determination unit 45 acquires the detection result of the object using the imaging information from the imaging information processing unit 42, the determination unit 45 selects an unselected object region among the detected object regions on the captured image (Step S31).

  Next, the determination unit 45 determines whether or not the detection information of the region pixels constituting the object region on the distance image corresponding to the selected object region on the captured image is valid (step S32). For example, the determination unit 45 determines that the detection information of the region pixel is not valid when the distance in the viewing direction of the region pixel is infinity or a distance that cannot be measured.

  If it is determined that the detection information of the area pixel is valid (step S32: Yes), the determination unit 45 determines that the detection result of the selected object area is correct (step S33).

  If it is determined that the detection information of the region pixel is not valid (step S32: No), the determination unit 45 determines that the detection result of the selected object region is incorrect (step S34).

  After the process of step S33 or step S34, the determination unit 45 determines whether or not all object regions on the captured image have been selected (step S35).

  If it is determined that not all object regions on the captured image have been selected (step S35: No), the determination unit 45 returns to the process of step S31.

  If it is determined that all the object regions on the captured image have been selected (step S35: Yes), the determination unit 45 ends the process.

  As shown in FIG. 7, when an area 104 corresponding to a background in which no object such as a person exists is detected as the object area, no object such as a person exists in the area 104. The distance is infinite or cannot be measured. Therefore, the determination unit 45 can determine whether or not erroneous detection due to a background pattern or the like is included in the detection result by determining whether or not the detection information of the region pixel is valid. That is, the determination unit 45 can determine that the detection result is an error when the distance in the viewing direction to the position corresponding to the object detected by the detection process is greater than or equal to a predetermined value. In this case, the imaging information processing unit 42 detects that the object region determined to be erroneous detection is the background.

  Note that the determination of whether or not erroneous detection due to a background pattern or the like is included in the object detection result is not limited to the method described with reference to FIG. That is, it is not necessary to use the distance in the viewing direction to the position corresponding to the object detected by the detection process.

  For example, the determination unit 45 may determine whether or not the detection result is an error according to the intensity of the reflected wave of the electromagnetic wave applied to the position corresponding to the object detected by the detection process. In an area where an object such as a person is not present, the electromagnetic wave applied to the area is not reflected and a reflected wave is not detected, or the intensity of the reflected wave is smaller than when an object is actually present. Therefore, the determination unit 45 can determine that the detection result is incorrect when the intensity of the reflected wave of the electromagnetic wave applied to the position corresponding to the detected object is less than a predetermined value.

  Next, another method for determining whether the detection result is correct or incorrect will be described. As described above, when a plurality of similar-colored objects are arranged so as to overlap with each other when viewed from the imaging device 20, the color change at the boundary between different objects is small on the captured image of the imaging device 20, It becomes difficult to extract an edge. That is, as illustrated in FIG. 9, when the object 105 is similar in color to the object 105 and the object 106 that is larger than the object 105 overlaps when viewed from the image capturing apparatus 20, The object 105 and the object 106 overlap with each other, and the edge between the object 105 and the object 106 becomes difficult to distinguish. As a result, the object 105 may not be detected, and only the object 106 may be erroneously detected as one object. Below, the method for determining the presence or absence of such erroneous detection due to the overlap of a plurality of objects will be described.

  FIG. 11 is a flowchart illustrating an example of an operation when the determination unit 45 determines whether or not there is a false detection due to an overlap of a plurality of objects.

  The determination unit 45 acquires the detection result of the object using the imaging information from the imaging information processing unit 42. Further, the determination unit 45 acquires the detection result of the object using the distance image from the distance image processing unit 44. The determination unit 45 selects an unselected object area among the object areas on the distance image (step S41).

  FIG. 12 is a diagram illustrating a distance image obtained by the distance image sensor 30 in the state illustrated in FIG. 9 (a state in which the object 105 and the object 106 are overlapped). That is, FIG. 12 is a diagram illustrating a distance image obtained when the imaging device 20 illustrated in FIG. 9 is replaced with the distance image sensor 30. In FIG. 12, the distance information is indicated by the density of dots in each region. The sparse region of the dots indicates that the distance from the object in the region is near, and the region where the dots are dense indicates that the distance from the object in the region is long.

  As shown in FIG. 9, the object 105 is closer to the distance image sensor 30 than the object 106. Therefore, the distance in the viewing direction in the object region 107 corresponding to the position and shape of the object 105 corresponds to the position and shape of the object 106 and is closer than the distance in the viewing direction in the object region 108 surrounding the object region 107. Thus, in the distance image, even if a plurality of objects are overlapped, each object can be distinguished and detected.

  Referring to FIG. 11 again, the determination unit 45 determines whether or not the object region on the selected distance image overlaps the object region at the corresponding position on the captured image (the size and shape are the same or approximately the same). Determination is made (step S42).

  When it is determined that the object region on the distance image does not overlap with the object region at the corresponding position on the captured image (step S42: No), the determination unit 45 captures the object region on the selected distance image. It adds to the detection result of the object on the image (step S43).

  When it is determined that the object region on the distance image overlaps the object region at the corresponding position on the captured image (step S42: Yes), the determination unit 45 selects the object region on the selected distance image as the captured image. It is not added to the detection result of the upper object (step S44).

  After the process of step S43 or step S44, the determination unit 45 determines whether or not all object regions on the distance image have been selected (step S45).

  If it is determined that not all object regions on the distance image have been selected (step S45: No), the determination unit 45 returns to the process of step S41.

  If it is determined that all object regions on the distance image have been selected (step S45: Yes), the determination unit 45 ends the process.

  As described with reference to FIGS. 10 and 12, in the captured image of the imaging apparatus 20, when objects with similar colors overlap, it is difficult to distinguish and detect these objects. On the other hand, in the distance image of the distance image sensor 30, those objects can be distinguished and detected. Therefore, if the object area on the distance image does not overlap with the object area on the captured image, that is, if it is not detected by image recognition using the captured image, the object area on the distance image is By adding to the detection result, it is possible to improve the accuracy of the detection result.

  Note that the method for determining the presence or absence of erroneous detection due to overlapping of objects is not limited to the method described with reference to FIG. That is, it is not necessary to perform the processing of step S42 to step S44 for the object region on the distance image.

  For example, the determination unit 45 detects the detection result based on the detection information at the first position and the difference between the detection information at the second position corresponding to the object detected by the detection process, unlike the first position. Correct / incorrect (presence / absence of erroneous detection due to overlapping of objects) can be determined. Here, when determining the presence or absence of erroneous detection due to overlapping of objects, the first position corresponds to the object detected by the detection process, and is a position inside the second position. In this case, when the difference between the distance in the viewing direction at the first position and the distance in the viewing direction at the second position is greater than or equal to a predetermined value, the determination unit 45 determines that the detection result is an error. it can.

  As illustrated in FIG. 12, when a plurality of objects having different distances from the distance image sensor 30 are overlapped, the distances from the distance image sensor 30 are different in regions corresponding to the respective objects. Therefore, if the difference between the distance in the viewing direction at the first position and the distance in the viewing direction at the second position in the object region on the distance image is greater than or equal to a predetermined value, the object in the first position and the first There is a high possibility that the object is different from the object existing at the position 2. Therefore, the presence or absence of erroneous detection due to overlapping of objects can be determined according to the difference in the viewing direction distance between the first position and the second position.

  As described above, in the present embodiment, the object detection device 40 includes the imaging information acquisition unit 41 that acquires imaging information obtained by imaging a predetermined visual field range, and the reflection of electromagnetic waves irradiated to at least one position within the predetermined visual field range. A distance image acquisition unit 43 (detection information acquisition unit) that acquires wave detection information; an imaging information processing unit 42 (object detection unit) that performs detection processing of an object within a predetermined visual field range based on the imaging information; And a determination unit 45 that determines whether the detection result by the detection process is correct or not based on the detection information.

  The object detection device 40 can determine the shape of the object in the field of view, the size of the object, the presence or absence of the object, the overlap of the objects, and the like based on the detection information of the reflected wave of the electromagnetic wave irradiated in the predetermined field of view. Therefore, the object detection device 40 can further improve the accuracy of the detection result by determining whether the detection result of the object detection process using the imaging information is correct based on the detection information.

  In addition to the above-described embodiment, a program that causes a computer to execute each process performed by the object detection device 40 may be provided. The program may be recorded on a computer readable medium. If the computer uses a computer-readable medium, the program can be installed. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM.

  Alternatively, a chip configured to include a memory that stores a program for executing each process performed by the object detection device 40 and a processor that executes the program stored in the memory, and is mounted on the object detection device 40 is provided. Also good.

  Although one embodiment of the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various variations and modifications based on the present disclosure. Accordingly, it should be noted that these variations and modifications are included in the scope of the present disclosure.

DESCRIPTION OF SYMBOLS 10 Object detection system 20 Imaging device 30 Distance image sensor 31 Electromagnetic wave radiation device 32 Electromagnetic wave detection device 40 Object detection device 41 Imaging information acquisition part 42 Imaging information processing part (object detection part)
43 Distance image acquisition unit (detection information acquisition unit)
44 Distance image processing unit 45 Judgment unit 46 Output unit 101, 102, 103, 104 Object region 105, 106 Object 107, 108 Object region on the distance image

Claims (18)

An imaging information acquisition unit for acquiring imaging information obtained by imaging a predetermined visual field range;
A detection information acquisition unit for acquiring detection information of a reflected wave of an electromagnetic wave irradiated to at least one position within the predetermined visual field range;
An object detection unit that performs detection processing of an object within the predetermined visual field range based on the imaging information;
An object detection apparatus comprising: a determination unit that determines whether the detection result by the detection process is correct based on the detection information.   The object detection device according to claim 1, wherein the object detection unit corrects the detection result based on the detection information when it is determined that the detection result is an error.   The object detection device according to claim 1, wherein, when it is determined that the detection result is an error, the object detection unit performs the detection process again based on the detection information.   The determination unit is configured to detect the detection result based on a difference between detection information at a first position and detection information at a second position corresponding to the object detected by the detection process, which is different from the first position. The object detection device according to any one of claims 1 to 3, wherein the correctness of the error is determined. The detection information is a distance in the visual direction to the irradiation position of the electromagnetic wave,
5. The determination unit according to claim 4, wherein the determination unit determines whether the detection result is correct based on a difference between a distance in the viewing direction to the first position and a distance in the viewing direction to the second position. Object detection device.   The object detection apparatus according to claim 4, wherein the first position is a position corresponding to a periphery of the object detected by the detection process.   The object detection device according to claim 4, wherein the determination unit determines that the detection result is an error when the difference is less than a predetermined value.   8. The object detection unit according to claim 7, wherein when the difference is less than a predetermined value, the object detection unit expands the region of the object detected by the detection process so as to include the first position in the imaging information. Object detection device.   The object detection apparatus according to claim 4, wherein the first position corresponds to an object detected by the detection process and is a position inside the second position.   The object detection device according to claim 9, wherein the determination unit determines that the detection result is an error when the difference is equal to or greater than a predetermined value.   The object detection apparatus according to claim 10, wherein the object detection unit detects that a different object exists between the first position and the second position when the difference is equal to or greater than the predetermined value.   The object detection device according to claim 10, wherein the object detection unit reduces the area of the object detected by the detection process to the first position when the difference is equal to or greater than the predetermined value.   The object detection device according to any one of claims 10 to 12, wherein the object detection unit detects that the second position is a position corresponding to a background when the difference is equal to or greater than the predetermined value. .   The determination unit determines that the detection result is an error when the intensity of the reflected wave of the electromagnetic wave applied to the position corresponding to the object detected by the detection process is less than a predetermined value. The object detection apparatus according to any one of the above. The detection information is a distance in the visual direction to the irradiation position of the electromagnetic wave,
4. The determination unit according to claim 1, wherein the determination unit determines that the detection result is an error when a distance in a viewing direction to a position corresponding to the object detected by the detection process is a predetermined value or more. The object detection device according to one item.   The said object detection part detects that the position corresponding to the object detected by the said detection process is a position corresponding to a background, when it determines with the said detection result being an error. Object detection device. On the computer,
Acquiring imaging information obtained by imaging a predetermined visual field range;
Obtaining detection information of reflected waves of electromagnetic waves irradiated to at least one position within the predetermined visual field range;
Performing a process of detecting an object within the predetermined visual field range based on the imaging information;
Determining whether the detection result by the detection process is correct or not based on the detection information. An imaging device, an electromagnetic wave radiation device that radiates electromagnetic waves, an electromagnetic wave detection device that detects reflected waves of the electromagnetic waves, and an object detection device,
The object detection device includes:
An imaging information acquisition unit that acquires imaging information obtained by imaging the predetermined visual field range by the imaging device;
A detection information acquisition unit for acquiring detection information of a reflected wave of an electromagnetic wave irradiated to at least one position within the predetermined visual field range;
An object detection unit that performs detection processing of an object within the predetermined visual field range based on the imaging information;
An object detection system comprising: a determination unit that determines whether the detection result by the detection process is correct based on the detection information.

Images