JP4504959B2 - Three-dimensional object detection device - Google Patents

Description

  The present invention relates to a three-dimensional object detection device that detects a three-dimensional object around a vehicle, and more particularly to a three-dimensional object detection device suitable for detecting a three-dimensional object that moves at a relatively high speed.

  2. Description of the Related Art Conventionally, there has been known a camera that controls a camera and an illumination unit attached to an automobile to prevent misrecognition due to reflection such as a puddle and to detect a surrounding situation of the automobile (see, for example, Patent Document 1).

JP 2006-11671 A

  In Patent Document 1, image processing is performed by taking a difference between an image captured when the illumination unit is turned on and an image obtained when the illumination unit is turned off, and the surrounding situation is detected. Therefore, there is a time difference between the image when it is turned on and the image when it is turned off. Recognition may be difficult.

  An object of the present invention is to provide a three-dimensional object detection device capable of more accurately recognizing an object moving at a relatively high speed.

(1) In order to achieve the above object, the present invention provides a three-dimensional object detection device that detects a three-dimensional object around a vehicle, an imaging unit that images the vehicle periphery, and an illumination unit that illuminates the vehicle periphery. Lighting control means for controlling lighting on / off of the lighting means, imaging control means for controlling the imaging timing of the imaging means, and first and second images taken by the imaging means when the lighting means is lit Using the third image captured by the imaging unit between the image and the imaging timing of the first and second images and when the illumination unit is turned off, the first image and A change in the size and displacement of the high-intensity portion of the second image is calculated, and the second image is compared with the third image based on the change in size and the displacement of the high-luminance portion. Create an image at the imaging timing of as a corrected image, Serial wherein the second image to generate a difference image of the corrected image is obtained by so and an image processing means for detecting a three-dimensional object of this difference image to the image processing.
With this configuration, an object moving at a relatively high speed can be recognized more accurately.

  (2) In the above (1), preferably, the imaging timing of the imaging unit and the lighting / lighting timing of the illumination unit are synchronized, and the image processing unit is configured to perform the first and second operations at the time of lighting. A solid object is detected from the third image when the image and the first and second images are turned off.

  (3) In the above (1), preferably, the imaging timing of the imaging unit and the timing of turning on and off the illumination unit are asynchronous, and the image processing unit continuously captures 8 images taken by the imaging unit. The first image is extracted from an image acquired by the imaging unit when the illumination unit continues to be lit from one image, and the imaging unit is used when the illumination unit continues to be turned off after being lit. The third image is extracted from the acquired image, and the second image is extracted from the image acquired by the imaging unit when the illumination unit continues to be lit after being turned off.

  (4) In the above (1), preferably, the illuminating unit emits infrared light and visible light, and the imaging unit is configured to perform the first and second operations when the infrared light is turned on. And the third image is picked up when the infrared light is extinguished.

  (5) In the above (1), preferably, the imaging means is positioned below the illumination means.

  According to the present invention, an object moving at a relatively high speed can be recognized more accurately.

Hereinafter, the configuration and operation of a three-dimensional object detection device according to an embodiment of the present invention will be described with reference to FIGS.
First, the configuration of the three-dimensional object detection device according to the present embodiment will be described with reference to FIG.
FIG. 1 is a block diagram illustrating a configuration of a three-dimensional object detection device according to an embodiment of the present invention.

  The three-dimensional object detection device according to the present embodiment includes an imaging unit 10, an imaging control unit 20, an illumination unit 30, an illumination control unit 40, an image processing unit 50, and an image memory 60.

  The imaging means 10 is a camera provided with an imaging element such as a CCD. The imaging timing by the imaging unit 10 is controlled by an electronic shutter control signal from the imaging control unit 20. The electronic shutter control signal is also supplied to the illumination control means 40.

  The illumination unit 30 is, for example, an LED lamp that emits near infrared light. By using the LED lamp, it is possible to control turning on and off at high speed. Further, by using near infrared light, the oncoming vehicle or pedestrian does not feel dazzling.

  The illumination control unit 40 outputs an illumination control signal to the illumination unit 30 in synchronization with the electronic shutter control signal from the imaging control unit 20. That is, the timing when the illumination unit 30 is turned on and the timing when the illumination unit 30 is turned off are created in accordance with the timing when the imaging unit 10 captures an image using the electronic shutter control signal. Therefore, the imaging means 10 repeats taking a picture when the illumination means 30 is turned on, and then taking a picture when the illumination means 30 is turned off at regular intervals. Note that the synchronization of the illumination unit and the photographing unit may be performed by outputting a synchronization signal from the image processing unit 50.

  An image photographed by the imaging means 10 is captured by the image processing means 50 and temporarily stored in the image memory 60. Further, the image processing means 50 takes out image data stored in the image memory 60, performs image processing, and detects a three-dimensional object.

Next, the imaging operation of the three-dimensional object detection device according to the present embodiment will be described with reference to FIG.
FIG. 2 is a flowchart illustrating an imaging operation of the three-dimensional object detection device according to the embodiment of the present invention.

  In step S <b> 10, the illumination control unit 40 outputs an illumination control signal for turning on the illumination to the illumination unit 30 based on the electronic shutter control signal from the imaging control unit 20, and turns on the illumination unit 30. Since the electronic shutter control signal is also supplied to the image pickup means 10, the image pickup means 10 takes an image in synchronization with the lighting of the illumination means 30.

  Next, in step S <b> 20, the image processing unit 50 stores the image data captured by the imaging unit 10 in the image memory 60.

  Next, in step S30, the illumination control unit 40 outputs an illumination control signal for turning off the illumination to the illumination unit 30 based on the next electronic shutter control signal from the imaging control unit 20, and turns off the illumination unit 30. . Since the electronic shutter control signal is also supplied to the imaging unit 10, the imaging unit 10 captures an image in synchronization with the lighting unit 30 being turned off.

  Next, in step S <b> 40, the image processing unit 50 stores the image data captured by the imaging unit 10 in the image memory 60.

  That is, the image memory 60 repeatedly stores an image when the lighting unit 30 is turned on and an image at the time of the scratch.

Next, the image processing operation of the three-dimensional object detection device according to the present embodiment will be described with reference to FIGS.
FIG. 3 is a flowchart showing an image processing operation of the three-dimensional object detection device according to the embodiment of the present invention. 4-8 is explanatory drawing of the image processing operation | movement of the solid-object detection apparatus by one Embodiment of this invention.

  Here, the image data stored in the image memory 60 will be described with reference to FIGS. FIG. 4 was taken at time T0 and is an image when the illumination unit 30 is turned on. FIG. 5 is an image taken at time T0 + ΔT1, and is an image when the illumination unit 30 is turned off. FIG. 6 is an image taken at time T0 + 2 · ΔT1, and is an image when the illumination unit 30 is turned on. In each of FIGS. 4 to 6, for the sake of illustration, the portion shown in black is a portion with high luminance, and the portion shown in white is a portion with low luminance.

  As shown in FIG. 4, at the time T0, when the illumination means 30 is turned on, the bicycle B1 is photographed in addition to the center line CL1, the roadside band RSB1, and the road sign RTS1. As shown in FIG. 5, at time T0 + ΔT1, when the illumination unit 30 is turned off, the center line CL2, the roadside band RSB2, and the road sign RTS2 are photographed.

  That is, regardless of whether the illumination unit 30 is turned on or off, the center line, the roadside band, the road sign, and the like are always photographed with high brightness because of the high reflectance. On the other hand, when the bicycle is turned off, the reflectance is low, so that the bicycle is not photographed as shown in FIG. On the other hand, at the time of lighting, as shown in FIG.

  In FIG. 4 (time T0) to FIG. 6 (time T0 + 2 · ΔT1), the time interval is constant as ΔT1. During this time, the vehicle equipped with the three-dimensional object detection device of this embodiment is traveling in the direction of the road sign RTS1 in FIG. Accordingly, the state of FIGS. 5 and 6 is gradually larger than the state of FIG. 4 because it approaches the road sign, and it gradually moves from the center of the angle of view to the left side direction. The image processing means 50 performs the following image processing using the change in the captured image due to such a time change.

  That is, in step S100 of FIG. 3, the image processing unit 50 calculates the change in the size of the high-luminance portion and the amount of displacement from the two images when the illumination is on. Here, a road sign will be described as an example. The image of the circular road sign is determined by its size and the center position. For example, the road sign RTS1 in FIG. 4 has a radius R1 and a center coordinate (x1, y1). Further, the road sign RTS3 in FIG. 6 has a radius R3 and a center coordinate (x3, y3). Therefore, in FIGS. 4 and 6 when the illumination is ON, the size of the road sign changes from R1 → R3, and the displacement amount of the center position becomes (x1, y1) → (x3, y3). In addition, instead of the road sign, the size change and the displacement amount may be calculated from the roadside belt or the center line.

  Next, in step S110 of FIG. 3, the image processing unit 50 creates a corrected image when the illumination is OFF, from the change and the displacement calculated in step S100. Here, it demonstrates concretely. FIG. 5 is an image when the illumination is OFF. This image is taken at time T0 + ΔT1. On the other hand, the processing in step S110 is to estimate what the image of FIG. 5 is at the shooting timing (time T0 + 2 · ΔT1) of FIG.

  In step S110, between time 2 · ΔT1, the size of the road sign changes from R1 → R3, and the displacement amount of the center position becomes (x1, y1) → (x3, y3). ing. Since the time interval in FIGS. 5 to 6 is ΔT1, the change in size in FIGS. 4 to 6 and the change in size half and the amount of displacement in FIG. If corrected, a corrected image is obtained. That is, since the change in size is (R3 / R1) / 2 and the change in displacement is ((x3-x1) / 2, (y3-y1) / 2), the road sign in FIG. RTS2 (R2 (x2, y2)) has a magnitude of R2 · ((R3 / R1) / 2) after time ΔT1, and the center position is (x2 + (x3−x1) / 2, y2 + (y3). -Y1) / 2) is calculated. Similarly, a corrected image is calculated for the roadside band RSB1 and the road sign RTS1.

  FIG. 7 shows the corrected image calculated in step S110. As shown in FIG. 7, a center line CL2 ', a roadside band RSB2', and a road sign RTS2 ', which are corrected images at time T0 + 2 · ΔT1, are calculated.

  Next, in step S120 of FIG. 3, the image processing unit 50 generates a difference image from the image when the illumination is ON and the corrected image. That is, a difference image between the image when the illumination is turned on shown in FIG. 6 and the corrected image shown in FIG. 7 is generated. As a result, the road sign RTS2 in FIG. 6 and the road sign RTS2 ′ in FIG. 7 are canceled, the center line CL2 in FIG. 6 and the center line CL2 ′ in FIG. 7 are canceled, the roadside band RSB2 in FIG. Since the roadside band RSB2 ′ is canceled, an image in which only the bicycle B3 remains is generated as shown in FIG.

  Then, in step S130 of FIG. 3, the image processing means 50 executes an object recognition process from the difference image obtained in step S130. That is, since recognition processing is performed using the image of the bicycle B3 shown in FIG. 8, even if the bicycle is moving at a higher speed than a pedestrian or the like, it can be accurately recognized.

  As described above, according to this embodiment, an object moving at a relatively high speed can be recognized more accurately.

Next, the structure of the illumination means used for the three-dimensional object detection device according to the present embodiment will be described with reference to FIGS.
FIG. 9 is a front view showing the configuration of the illumination means used in the three-dimensional object detection device according to one embodiment of the present invention.

  As shown in FIG. 9, the illumination means 30 includes a plurality of visible light LEDs 32 that illuminate the periphery of the vehicle, such as a headlight, at night, and a plurality of infrared light LEDs 34 that repeatedly turn on and off. With this configuration, the size can be reduced.

FIG. 10 is a side view showing the configuration of the photographing means and the illumination means used in the three-dimensional object detection device according to one embodiment of the present invention.
As shown in FIG. 10, the illuminating means 30 and the imaging means 10 are integrated so that the illuminating means 30 and the imaging means 10 can be arranged close to each other, and the reflected light from the object is maximized in the imaging unit. This is advantageous because it is incident. The illumination means 30 and the imaging means 10 are installed in a light casing 70 that irradiates the periphery of the vehicle with a light shielding plate 72 interposed therebetween. The light shielding plate 72 prevents direct light from the illumination unit 30 from entering the imaging unit 10. Further, the light shielding plate 72 has an effect of heat insulation so that heat is not transferred from the illumination unit 30 to the imaging unit 10.

  Furthermore, by disposing the imaging unit 10 on the lower side (ground side) than the illumination unit 30, it is possible to minimize the influence of heat generated by the illumination unit 30.

FIG. 11 is a side view showing another configuration of the photographing means and the illumination means used in the three-dimensional object detection device according to one embodiment of the present invention.
As shown in FIG. 11, a plurality of illumination means 30A, 30B, and 30C and a plurality of imaging means 10A, 10B, and 10C are installed in a housing 70 such as a light box, and a light shielding plate 72 is sandwiched therebetween. A wider range can be covered by using a structure.

Next, the attachment position of the three-dimensional object detection device according to the present embodiment will be described with reference to FIGS.
FIG.12 and FIG.13 is a perspective view which shows the attachment position of the solid-object detection apparatus by one Embodiment of this invention.

  For example, as shown in FIG. 12, the three-dimensional object detection device of the present embodiment is attached to the headlights HL1, HL2, the room mirror RM1, and the side mirror SM1. Further, as shown in FIG. 13, it can be attached to the back lamps BL1 and BL2 and the license plate illumination means LA1.

Next, the configuration and operation of a three-dimensional object detection device according to another embodiment of the present invention will be described with reference to FIGS. 14 and 15.
FIG. 14 is a block diagram illustrating a configuration of a three-dimensional object detection device according to another embodiment of the present invention. FIG. 15 is an operation explanatory diagram of a three-dimensional object detection device according to another embodiment of the present invention. In FIG. 14, the same reference numerals as those in FIG. 1 denote the same parts.

  As shown in FIG. 14, the basic configuration of the three-dimensional object detection device of the present embodiment is the same as that of FIG. 1, but the control signal of the imaging means 20 is not input to the illumination control means 40A. The illumination control unit 40A outputs an illumination control signal to the illumination unit 30, and repeats turning on and off of the illumination unit 30 at a predetermined cycle. The imaging unit 10 also repeats imaging at a constant cycle. That is, the timing of repeated shooting by the imaging unit 10 and the timing of turning on and off by the illumination unit 30 are asynchronous. However, the cycle of turning on and off the illumination unit 30 is set to be twice or more the cycle of imaging by the imaging unit 10. Since the image processing unit 50A is asynchronous as described above, the image processing unit 50A selects an image to be processed by a method described later, and obtains an image of the object by the image processing of FIGS. 3 and 4 to 8. I am doing so.

  Next, image processing in the present embodiment will be described with reference to FIG. FIG. 15A shows captured image data, and FIG. 15B shows the lighting / lighting timing of the illumination means.

  The imaging means 10 captures images at a constant cycle. The shooting times of the shot images shown in FIG. 15A are T1, t2, t3, t4, t5, t6, t7, and t8, respectively.

  On the other hand, as shown in FIG. 15B, the illumination unit 30 is repeatedly turned on and off at a predetermined cycle. Although both are asynchronous, the cycle of turning on and off the illumination unit 30 is set to, for example, twice the cycle of imaging by the imaging unit 10.

  Then, images taken during the lighting timing shown in FIG. 15B are t1, t2, and t3. Since the images at t1 and t3 partially include the extinguishing period, the image at time t2 has the highest luminance. Becomes a high image. On the other hand, the image at time t4 is the image with the lowest luminance.

  Therefore, the image processing unit 50A acquires four consecutive pieces of image data (time t1 to t4). Among them, the image F having the highest luminance and the image G having the lowest luminance are selected. Subsequently, four images (time t5 to t8) are acquired, and the image H having the highest luminance and the image J having the lowest luminance are selected.

  Then, it is determined whether the image captured between the imaging time of the image F and the imaging time of the image H is the image G or the image J. For example, an image captured between the image F and the image H is determined. , Determined as an image G.

  Here, the image F corresponds to the image of FIG. 4, the image H corresponds to the image of FIG. 6, and the image G corresponds to the image of FIG. Then, a corrected image at the time of the image H is created from the image G, a difference image between the corrected image and the image H is obtained, and a three-dimensional object is detected.

  As described above, according to this embodiment, an object moving at a relatively high speed can be recognized more accurately.

  In addition, since the illumination unit and the imaging unit can be separated by making the illumination and the imaging asynchronous, the wiring can be reduced, and the cost can be reduced because a special illumination unit and illumination control become unnecessary.

The present invention can easily detect not only a three-dimensional object that moves at a relatively high speed around the vehicle, but also a three-dimensional object when the host vehicle is turning a curve.

It is a block diagram which shows the structure of the solid-object detection apparatus by one Embodiment of this invention. It is a flowchart which shows the imaging operation of the solid-object detection apparatus by one Embodiment of this invention. It is a flowchart which shows the image processing operation | movement of the solid-object detection apparatus by one Embodiment of this invention. It is explanatory drawing of the image processing operation | movement of the solid-object detection apparatus by one Embodiment of this invention. It is explanatory drawing of the image processing operation | movement of the solid-object detection apparatus by one Embodiment of this invention. It is explanatory drawing of the image processing operation | movement of the solid-object detection apparatus by one Embodiment of this invention. It is explanatory drawing of the image processing operation | movement of the solid-object detection apparatus by one Embodiment of this invention. It is explanatory drawing of the image processing operation | movement of the solid-object detection apparatus by one Embodiment of this invention. It is a front view which shows the structure of the illumination means used for the solid-object detection apparatus by one Embodiment of this invention. It is a side view which shows the structure of the imaging | photography means and illumination means used for the solid-object detection apparatus by one Embodiment of this invention. It is a side view which shows the other structure of the imaging | photography means and illumination means used for the solid-object detection apparatus by one Embodiment of this invention. It is a perspective view which shows the attachment position of the solid-object detection apparatus by one Embodiment of this invention. It is a perspective view which shows the attachment position of the solid-object detection apparatus by one Embodiment of this invention. It is a block diagram which shows the structure of the solid-object detection apparatus by other embodiment of this invention. It is operation | movement explanatory drawing of the solid-object detection apparatus by other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Imaging means 20 ... Imaging control means 30 ... Illumination means 40 ... Illumination control means 50 ... Image processing means 60 ... Image memory

Claims (5)

A three-dimensional object detection device for detecting a three-dimensional object around a vehicle,
Imaging means for imaging the periphery of the vehicle;
Illumination means for illuminating the vehicle periphery;
Illumination control means for controlling turning on and off the illumination means;
Imaging control means for controlling imaging timing of the imaging means;
Between the imaging timings of the first and second images captured by the imaging unit and the first and second images when the illumination unit is turned on, the illumination unit is turned off. Sometimes using a third image captured by the imaging means,
Calculating the change and displacement of the high-luminance portion of the first image and the second image;
Based on the change in the size of the high-luminance part and the amount of displacement, an image at the imaging timing of the second image is created as a corrected image for the third image,
Generating a difference image between the second image and the corrected image;
A three-dimensional object detection apparatus comprising: an image processing unit configured to detect a three-dimensional object by performing image processing on the difference image. The three-dimensional object detection device according to claim 1,
The imaging timing in the imaging means and the lighting / lighting timing in the illumination means are synchronized,
The three-dimensional object detection apparatus characterized in that the image processing means detects a three-dimensional object from the first and second images when turned on and the third image when the light is turned off between the first and second images. . The three-dimensional object detection device according to claim 1,
The imaging timing in the imaging means and the timing of turning on and off in the illumination means are asynchronous,
The image processing means, from eight consecutive images taken by the imaging means,
Extracting the first image from the image acquired by the imaging means when the illumination means continues to be lit;
Extracting the third image from the image acquired by the imaging means when the illumination means continues to be turned off after being turned on;
A three-dimensional object detection apparatus, wherein the second image is extracted from an image acquired by the imaging unit when the illumination unit continues to be turned on after being turned off. The three-dimensional object detection device according to claim 1,
The illumination means emits infrared light and visible light,
The three-dimensional object characterized in that the imaging means captures the first and second images when the infrared light is turned on and captures the third image when the infrared light is turned off. Detection device. The three-dimensional object detection device according to claim 1,
The three-dimensional object detection device, wherein the imaging unit is located below the illumination unit.

Images