JP2009281895A - Range image data generating device for vehicle and method for generating range image data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a range image data generating device for a vehicle and a method for generating the range image data for the vehicle capable of continuously obtaining the situation of the front of the vehicle. <P>SOLUTION: The range image data generating device includes a projector 5, an image intensifier 7b and a high speed camera 8, a timing controller 9, and an image processing part 10 for detecting an object in a plurality of the respective images with different target range obtained by the image intensifier 7b and high speed camera 8 and generating the range image data representing the range of the object based on the luminance of the object. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention belongs to the technical field of a vehicle distance image data generation device.

In Patent Literature 1, whether or not an object such as an obstacle exists at the target distance is based on an image that is projected in accordance with the timing of reflected light that is projected in front of the host vehicle and returned from the target distance. Techniques for detection are disclosed.
US Pat. No. 6,732,123

  However, in the above prior art, objects other than the target distance cannot be detected. That is, there is a problem that the situation is intermittently grasped and the situation ahead of the host vehicle cannot be grasped continuously.

  An object of the present invention is to provide a vehicular distance image data generation apparatus and a vehicular distance image data generation method capable of continuously grasping the situation ahead of the host vehicle.

  In order to achieve the above object, in the vehicle distance image data generation device of the present invention, a light projecting unit that projects pulsed light in front of the host vehicle at a predetermined cycle, and an imaging timing set according to the target distance. Imaging means for imaging reflected light returning from the distance, timing control means for controlling the imaging timing so that the target distance changes continuously, and a plurality of imagings with different target distances obtained by the imaging means Distance image data generating means for detecting an object for each image and generating distance image data representing a distance to the object based on the luminance of the object.

  Therefore, in this invention, the situation ahead of the own vehicle can be grasped | ascertained continuously.

  The best mode for realizing the vehicle distance image data generating apparatus and the vehicle distance image data generating method of the present invention will be described below with reference to the drawings.

First, the configuration will be described.
FIG. 1 is a block diagram illustrating a configuration of an obstacle detection apparatus 1 according to a first embodiment to which the vehicle distance image data generation apparatus according to the present invention is applied. The obstacle detection apparatus 1 according to the first embodiment generates distance image data. A device 2, an object recognition processing unit 3, and a determination unit 4 are provided.

The distance image data generation device 2 includes a projector (projecting unit) 5, an objective lens 6, a light multiplying unit 7, a high-speed camera (imaging unit) 8, a timing controller (timing control unit) 9, and an image. And a processing unit (distance image data generating means) 10.
The projector 5 is a near-infrared LED disposed at the front end of the vehicle, and outputs pulsed light for a predetermined light projecting time tL (for example, 5 ns) in accordance with the pulse signal output from the timing controller 9. The cycle of the pulse signal is the projection cycle tP of the projector 5, and the projection cycle tP is, for example, an interval of 1/100 s or less.
The objective lens 6 is for receiving reflected light from an object, and is disposed adjacent to the projector 5. For example, the optical system is set to have an angle of view capable of capturing a predetermined range in front of the host vehicle.

The light multiplication unit 7 includes a gate 7a and an image intensifier 7b.
The gate 7a opens and closes in response to an opening / closing command signal from the timing controller 9. Here, in Example 1, the opening time (gate time) tG of the gate 7a is set to 5 ns, which is the same as the light projection time tL. Here, the gate time tG corresponds to the imaging target width of the imaging area (target distance). The longer the gate time tG, the longer the imaging target width of the imaging area. In the first embodiment, since the gate time tG = 5 ns, the imaging target width is 1.5 m from the speed of light (about 3 × 10 ⁸ m / s) × gate time (5 ns).

The image intensifier 7b temporarily converts extremely weak light (reflected light from an object, etc.) into electrons and then electrically amplifies it, and returns it to a fluorescent image again, thereby doubling the amount of light and producing an image with contrast. It is a viewing device. Photoelectrons launched from the photocathode of the image intensifier 7b by a photoelectric phenomenon are accelerated at a high voltage of the order of kV, and are emitted into the phosphor screen on the anode side, thereby emitting fluorescence having a photon number of 100 times or more. The fluorescence generated on the phosphor screen is guided to the image sensor of the high-speed camera 8 by the fiber optic plate without being scattered while maintaining the same positional relationship.
The high-speed camera 8 captures an image emitted from the light doubling unit 7 in response to a command signal from the timing controller 9 and outputs a captured image (color image) to the image processing unit 10. In the first embodiment, a camera having a resolution of 640 × 480 (horizontal: vertical), a luminance value of 1 to 255 (256 levels), and 100 fps or more is used.

The timing controller 9 is the time from the light projection start time of the projector 5 until the gate 7a is opened so that the captured image captured by the high-speed camera 8 is the timing of the reflected light returning from the target imaging area. The imaging timing is controlled by setting a certain delay time tD and outputting an open / close command signal corresponding to the delay time. That is, the delay time tD is a value that determines the distance from the host vehicle to the imaging area (imaging target distance), and the relationship between the delay time tD and the imaging target distance is as follows.
Imaging distance = speed of light (approx. 3 x 10 ⁸ m / s) x delay time tD / 2
FIG. 2 shows a temporal relationship between the operation of the projector 5 (light projection operation) and the operation of the gate 7a (camera gate operation) when imaging one imaging area.

  The timing controller 9 increases the imaging range of the high-speed camera 8 by increasing the delay time tD by a predetermined interval (for example, 10 ns) so that the imaging area continuously moves from the front side of the vehicle to the front side. Change to the side. The timing controller 9 starts the imaging operation of the high-speed camera 8 immediately before the gate 7a is opened, and ends the imaging operation after the gate 7a is completely closed.

  Further, in the first embodiment, as illustrated in FIG. 3, when the imaging target distance is continuously changed from B1 → B2 → B3 →..., The imaging target distance is increased more than the imaging target width A of the imaging area. By increasing the amount (B2-B1), the increase amount of the imaging target distance is set so that a part of the imaging area changes while overlapping.

  FIG. 4 is a schematic diagram showing temporal luminance changes when the amount of increase in the imaging target distance is reduced to the limit, in other words, when imaging is performed with an infinite increase in the imaging area. By overlapping each other, the luminance value of the same pixel in a plurality of consecutive captured images gradually increases, and after the peak, the luminance value gradually decreases. In practice, the number of imaging areas is limited (1 to n), but by overlapping a part of continuous imaging areas, the temporal luminance change becomes close to the characteristics of FIG.

  The timing controller 9 outputs an image processing command signal to the image processing unit 10 when all the captured images for one frame, that is, the set predetermined range (area 1, area 2,..., Area n) are captured. To do.

  The image processing unit 10 detects an object (object) with distance information from a captured image of one frame imaged by the high-speed camera 8, and generates distance image data that represents the distance to the object by color, luminance, or the like. Then, the generated distance image data is output to the object recognition processing unit 3.

The object recognition processing unit 3 further identifies an object included in the distance image data by pattern matching or the like with respect to the object (object) included in the distance image data.
Based on the relationship (distance, relative speed, etc.) between the object (person, car, sign, etc.) identified by the object recognition processing unit 3 and the host vehicle, the determination unit 4 presents information to the driver by an alarm, Determine whether vehicle control such as automatic braking is necessary.

[Distance image data generation control processing]
FIG. 5 is a flowchart illustrating the flow of the distance image data generation control process executed by the image processing unit 10 according to the first embodiment. Each step will be described below. This process is repeatedly executed at a predetermined calculation cycle.

  In step S1, the image processing unit 10 stores the luminance value data having the lowest luminance of the captured image obtained by imaging the front of the host vehicle without performing the light projection by the projector 5 for later luminance determination, and proceeds to step S2. To do. This data is used when generating the distance image data.

  In step S2, the image processing unit 10 inputs a captured image, and proceeds to step S3.

  In step S3, the image processing unit 10 performs a process of binarizing the input captured image with a predetermined luminance threshold value for each pixel in the image to generate a binarized image. .

  In step S4, the image processing unit 10 performs a labeling process for assigning a label to a portion where a pixel constitutes an area area with luminance in the binarized image.

  In step S5, the image processing unit 10 determines whether image input for one frame (area 1, area 2,..., Area n) has been completed. If YES, the process proceeds to step S6. If NO, the process proceeds to step S2.

  In step S6, the image processing unit 10 detects an object that covers a plurality of images at a distance in the front and rear directions.

  In step S7, the image processing unit 10 determines whether or not an object that covers a plurality of images at the front and rear distances is detected. If detected, the process proceeds to step S8, and if not detected, the process proceeds to step S10.

  In step S8, the image processing unit 10 calculates the average luminance in each image for the portion where the detected objects overlap.

  In step S9, the image processing unit 10 determines the distance with the highest average luminance of the overlapping portion as the distance of the object.

  In step S10, distance image data is generated by using the object whose distance is determined as an image accompanied by distance information by color and brightness, and the process proceeds to return.

Next, the operation will be described.
[Distance image data generation function]
The timing controller 9 controls the imaging timing of the high-speed camera 8 by setting the delay time tD so that the captured image captured by the high-speed camera 8 becomes the timing of the reflected light returning from the target imaging area. To do. When an object is present in the target imaging area, the time for the light emitted from the projector 5 to return from the imaging area is the time for the light to reciprocate the distance between the host vehicle and the imaging area (imaging target distance). Therefore, the delay time tD can be obtained from the imaging target distance and the speed of light.

  In the captured image of the high-speed camera 8 obtained by the above method, when an object exists in the imaging area, the luminance value data of the pixel corresponding to the position of the object is affected by the reflected light, and the luminance of other pixels Indicates a value higher than the value data. Thereby, based on the luminance value data of each pixel, the distance from the object existing in the targeted imaging area can be obtained.

  Furthermore, in the first embodiment, captured images of the imaging areas 1 to n are acquired while changing the delay time tD. Subsequently, binarization (step S3) and labeling (step S4) are performed on each captured image, the luminance value data of the overlapping portions at the distance before and after the labeled object are compared, and the highest luminance value data is Data (distance image data) having distance information and distance information of the object in the imaging range (640 × 480) is generated.

  Conventional distance detection methods using laser radars and stereo cameras are susceptible to rain, fog, snow, etc., and the noise level with respect to the signal level is large (the SN ratio is small), so the reliability in bad weather is low. . When using a millimeter wave radar that is not easily affected by bad weather, the reliability of distance detection is high, but it is difficult to perform object recognition (object identification) from the millimeter wave radar signal. Is required. Further, since the camera image becomes unclear during bad weather, it is difficult to perform accurate object recognition.

On the other hand, in the first embodiment, only reflected waves returning from the target imaging area are reflected in the captured image, so that the light bent due to the influence of rain, fog, snow, or the like, that is, the noise mixing level is kept low. High signal-to-noise ratio can be obtained. That is, high distance detection accuracy can be obtained regardless of bad weather or night.
And since the distance of the object (object) detected from an image is known from the produced | generated distance image data, when performing object recognition using methods, such as pattern matching after that, the distance with an object can be grasped | ascertained instantaneously.

  Furthermore, in the first embodiment, the captured area is continuously changed to acquire a plurality of captured images, and the captured images are compared to detect the distance of each pixel. In addition, it can be grasped over a wide range. For example, even in the situation where a pedestrian has jumped between the host vehicle and the preceding vehicle, the distance between the preceding vehicle and the pedestrian can be grasped at the same time. Control can be performed.

FIG. 6 shows a situation in which four pedestrians A to D exist at different positions in front of the host vehicle, and the relationship between the distance between the host vehicle and each pedestrian is A <B <C <D. .
At this time, in the first embodiment, a part of the imaging area is overlapped so that the reflected light from one object is reflected on the pixels constituting the object of the captured image in the plurality of imaging areas in which the reflected light is continuous. For this reason, the temporal luminance change of the pixels constituting the object corresponding to each pedestrian shows a triangular characteristic that takes a peak at the position of the pedestrian as shown in FIG.

  Since the distance image data is data used for alarms and vehicle control, it is impossible to set an imaging area infinitely finely as long as a certain calculation speed is required. By making the reflected light from the plurality of captured images included, as shown in FIG. 8, the temporal luminance change of the pixels constituting the object is approximated to the above characteristics, and corresponds to the peak of the triangular portion. The detection accuracy can be increased by setting the imaging area to the distance of the object in the pixel.

[Object distance judgment]
In the distance image generating apparatus 2 of the first embodiment, as described above, binarization and labeling are performed on each distance image when generating the distance image data.
This point will be described in more detail with reference to the drawings.
FIG. 9 is a diagram illustrating an example of a captured image captured with a sufficiently long gate time (about 1/60 sec) in the same manner as a normal image. FIG. 10 is a diagram illustrating an example of captured images obtained by capturing the imaging range of FIG. 9 with eight distance images. FIG. 11 is a diagram illustrating an example of an overlapped portion of one detected object. FIG. 12 is a diagram illustrating an example of an object detection result for the imaging range of FIG.

Assume that the imaging range as shown in FIG. 9 is divided into eight distances as shown in FIG.
In this embodiment, binarization processing (step S3) and labeling processing (step S4) are performed for each image (FIGS. 10A to 10H).
As a result, a plurality of objects are detected as shown in FIGS. This object is indicated by reference numerals 101 to 106, for example, 101a by combining (a) to (h) in the figure. In the figure that is not detected, it is a missing number.

The distances in FIGS. 10 (a) to 10 (h) are assumed to increase in order, and in order to obtain the luminance data as shown in FIG. 8 as described above, the object is detected across the front and rear distances. (Steps S6 and S7).
In this case, for example, when the objects 101a to 101g are detected (see FIGS. 10A to 10G), they are overlapped as shown in FIG. Then, a portion 201 that overlaps all of the objects 101a to 101g is detected.

With respect to the overlapping portion 201, in each of the images of FIGS. 10A to 10G, the luminance values of the pixels constituting the overlapping portion 201 are summed and divided by the number of pixels to calculate an average luminance value. . Then, an average luminance value is calculated for each distance.
From this data, the distance of the highest average luminance value is set as the distance of this object.
In this way, the distance is determined for each object (see FIG. 12).

As the distance image data of this embodiment, the distance data for each pixel is determined from the highest luminance by looking at the pixels at the same position, that is, for example, the distance at each coordinate (1, 1), and the distance image data is created. It is possible to do. That is, the image has a distance for each pixel.
In this case, as the next processing, the object recognition processing unit 3 determines the object. However, since the pixels constituting the object do not have uniform distance data due to noise or the like, it is difficult to determine the object and specify the distance. .

  In this embodiment, in the processing of each distance image, the object is detected and the distance is determined in units of objects, so that the determination of the object and the distance are surely performed with easy processing.

Next, the effect will be described.
The distance image data generation device 2 according to the first embodiment has the following effects.

  (1) A projector 5 that projects pulsed light in front of the host vehicle at a predetermined period, an image intensifier 7b that captures reflected light returning from the target distance at an imaging timing set according to the target distance, and a high speed For each of a plurality of captured images with different target distances obtained by the camera 8, a timing controller 9 that controls the imaging timing so that the target distance changes continuously, the image intensifier 7b, and the high-speed camera 8. Since the image processing unit 10 that detects and generates distance image data representing the distance to the object based on the brightness of the object is provided, it is possible to continuously grasp the situation in front of the host vehicle. In addition, an object with distance information can be reliably detected with easy processing.

  (2) In the above (1), the image processing unit 10 uses the highest brightness distance of the same object existing in the preceding and succeeding distance images as the object distance, so that an object with distance information can be easily processed. Can be reliably detected.

(3) In the above (1) or (2), the image processing unit 10 calculates the average luminance value from the luminance data of the pixels constituting the overlapping portion with respect to the overlapping portion of the same object existing in the preceding and succeeding distance images. Since the processing of steps S7 to S9 is performed in which the distance of the highest luminance average value is set as the object distance, when overlapping portions are set in the consecutive distance images before and after, the same area (with the same number of pixels) is set. Configured). If the average luminance values in the same area are compared, a distance close to the peak in the state shown in FIG. 8 can be selected, and the distance of the object can be accurately determined.
And compared with judging an object from the distance image for every pixel, an object can be detected reliably with easy processing.
In the present specification, an object refers to an object that is recognized by pattern matching thereafter, but in a narrow sense, means an object that has been processed as not noise or the like when being widely labeled during processing. .

(Other examples)
The best mode for carrying out the present invention has been described above based on the embodiment. However, the specific configuration of the present invention is not limited to the configuration of the embodiment, and it relates to each claim of the claims. Design changes and additions are allowed without departing from the spirit of the invention.

For example, the light projection period, the light projection time, the gate time, the imaging target width, the amount of change in the imaging target distance, and the number of imaging areas in one frame are appropriately set according to the performance of the imaging means and the performance of the distance image data generation means can do.
In the first embodiment, binarization and labeling are performed by the image processing unit 10, and the object recognition processing unit 3 performs subsequent pattern matching. However, the image processing unit 10 performs pattern matching, An object (narrowly defined object) pattern-matched to the distance image data may be expressed.
In the distance image data according to the first embodiment, the distance image is determined based on the brightness of the pixel data at the same position in the distance image, and the distance image including the distance data for each pixel is included in the portion other than the object. It is good. When there is no need, a distance image composed only of detected objects may be used.

It is a block diagram which shows the structure of the obstruction detection apparatus 1 of Example 1. FIG. It is a figure which shows the temporal relationship between the operation | movement (light projection operation | movement) of the light projector 5, and the operation | movement (camera gate operation | movement) of the gate 7a at the time of imaging one imaging area. It is a figure which shows the state which a part of imaging area overlaps. It is a schematic diagram which shows a temporal luminance change at the time of imaging by increasing an imaging area infinitely. 3 is a flowchart illustrating a flow of distance image data generation control processing executed by the image processing unit 10 according to the first embodiment. It is a figure which shows the condition where four pedestrians AD exist in the different position ahead of the own vehicle. It is a schematic diagram which shows the temporal luminance change of the pixel corresponding to each pedestrian A-B. It is a figure which shows the distance image data generation effect | action of Example 1. FIG. It is a figure which shows the example of the captured image imaged with sufficiently long gate time (about 1/60 second) similarly to a normal image. It is a figure which shows the example of the captured image which each imaged the imaging range of FIG. 9 with eight distance images. It is a figure which shows the example of the part which has overlapped about one of the detected objects. It is a figure which shows the example of the object detection result about the imaging range of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Obstacle detection apparatus 2 Distance image data generation apparatus 3 Object recognition process part 4 Judgment part 5 Light projector (light projection means)
6 Objective lens 7 Photomultiplier 7a Gate 7b Image intensifier 8 High-speed camera (imaging means)
9 Timing controller (timing control means)
10 Image processing unit (distance image data generating means)
P (indicating the constant light emission location of the distance image data) 101a to 101g (detected) objects 102a to 102h (detected) objects 103a to 103e (detected) objects 104d to 104h (detected) objects 105e ~ 105h (detected) objects 106e to 106h (detected) objects 201 (overlapping objects with the same object but different distances)

Claims (3)

Light projecting means for projecting pulsed light in front of the host vehicle at a predetermined period;
Imaging means for imaging reflected light returning from the target distance at an imaging timing set according to the target distance;
Timing control means for controlling the imaging timing so that the target distance changes continuously;
Distance image data generating means for detecting an object for each of a plurality of captured images having different target distances obtained by the imaging means, and generating distance image data representing the distance to the object based on the luminance of the object;
With
A vehicular distance image data generating apparatus characterized by the above. The vehicle distance image data generation device according to claim 1,
The distance image data generation means includes
The distance of the highest brightness of the same object existing in the consecutive distance images before and after is set as the distance of the object.
A vehicular distance image data generating apparatus characterized by the above. In the vehicular distance image data generating device according to claim 1 or 2,
The distance image data generation means includes
For an overlapping portion of the same object existing in the preceding and following consecutive distance images, an average luminance value is calculated from the luminance data of the pixels constituting the overlapping portion, and the distance of the highest luminance average value is set as the distance of the object. ,
A vehicular distance image data generating apparatus characterized by the above.