JP2009282906A - Range image data generation device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a range image data generation device for a vehicle and a method of generating range image data of the vehicle for continuously obtaining situations ahead of a vehicle. <P>SOLUTION: The range image data generation device for a vehicle is provided with: an image processing part 10 for generating range image data showing a distance to an object for each pixel on the basis of brightness of the same pixel in a plurality of picked-up images different in target distance obtained by an image intensifier 7b and a high-speed camera 8; a short-range object sensor 11 for detecting an object in a short range of the vehicle; and a short-range object recognition part 12 and a short-range determination part 13 for reducing the projection amount of a projector 5 when an object in the short range is detected and outputting a command to a timing controller 9 so that image pickup timing corresponds to the reduced projection amount. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention belongs to the technical field of a vehicle distance image data generation device.

In Patent Literature 1, whether or not an object such as an obstacle exists at the target distance is based on an image that is projected in accordance with the timing of reflected light that is projected in front of the host vehicle and returned from the target distance. Techniques for detection are disclosed.
US Pat. No. 6,732,123

  However, in the above prior art, objects other than the target distance cannot be detected. That is, there is a problem that the situation is intermittently grasped and the situation ahead of the host vehicle cannot be grasped continuously.

  An object of the present invention is to provide a vehicular distance image data generation apparatus that can continuously grasp the situation ahead of the host vehicle.

  In order to achieve the above object, in the vehicle distance image data generation device of the present invention, a light projecting unit that projects pulsed light in front of the host vehicle at a predetermined cycle, and an imaging timing set according to the target distance. Imaging means for imaging reflected light returning from the distance, timing control means for controlling the imaging timing so that the target distance changes continuously, and a plurality of imagings with different target distances obtained by the imaging means A distance image data generating means for generating distance image data representing a distance to an object for each pixel based on the luminance of the same pixel in the image; a short distance detecting means for detecting a near object of the own vehicle; When the object is detected, the light projection amount of the light projecting means is weakened, and the light projection amount control is performed to output a command so that the imaging timing corresponds to the weak light projection amount. And stage, with a, characterized in that.

  Therefore, in the present invention, since the distance image data representing the distance to the object for each pixel is generated based on the luminance of the same pixel in the plurality of captured images having different target distances, the situation in front of the host vehicle is continuously displayed. Can be grasped.

  DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The best mode for realizing a vehicular distance image data generating apparatus according to the present invention will be described below with reference to the drawings.

First, the configuration will be described.
FIG. 1 is a block diagram illustrating a configuration of an obstacle detection apparatus 1 according to a first embodiment to which the vehicle distance image data generation apparatus according to the present invention is applied. The obstacle detection apparatus 1 according to the first embodiment generates distance image data. A device 2, an object recognition processing unit 3, and a determination unit 4 are provided.

The distance image data generation device 2 includes a projector (projecting unit) 5, an objective lens 6, a light multiplying unit 7, a high-speed camera (imaging unit) 8, a timing controller (timing control unit) 9, and an image. A processing unit (distance image data generation means) 10, a short-distance object sensor 11, a short-distance object recognition unit 12, and a short-distance determination unit 13 are provided.
The projector 5 is a near-infrared LED disposed at the front end of the vehicle, and outputs pulsed light for a predetermined light projecting time tL (for example, 5 ns) in accordance with the pulse signal output from the timing controller 9. The cycle of the pulse signal is the projection cycle tP of the projector 5, and the projection cycle tP is, for example, an interval of 1/100 s or less.
The objective lens 6 is for receiving reflected light from an object, and is disposed adjacent to the projector 5. For example, the optical system is set to have an angle of view capable of capturing a predetermined range in front of the host vehicle.

The light multiplication unit 7 includes a gate 7a and an image intensifier 7b.
The gate 7a opens and closes in response to an opening / closing command signal from the timing controller 9. Here, in Example 1, the opening time (gate time) tG of the gate 7a is set to 5 ns, which is the same as the light projection time tL. Here, the gate time tG corresponds to the imaging target width of the imaging area (target distance). The longer the gate time tG, the longer the imaging target width of the imaging area. In the first embodiment, since the gate time tG = 5 ns, the imaging target width is 1.5 m from the speed of light (about 3 × 10 ⁸ m / s) × gate time (5 ns).

The image intensifier 7b temporarily converts extremely weak light (reflected light from an object, etc.) into electrons and then electrically amplifies it, and returns it to a fluorescent image again, thereby doubling the amount of light and producing an image with contrast. It is a viewing device. Photoelectrons launched from the photocathode of the image intensifier 7b by a photoelectric phenomenon are accelerated at a high voltage of the order of kV, and are emitted into the phosphor screen on the anode side, thereby emitting fluorescence having a photon number of 100 times or more. The fluorescence generated on the phosphor screen is guided to the image sensor of the high-speed camera 8 by the fiber optic plate without being scattered while maintaining the same positional relationship.
The high-speed camera 8 captures an image emitted from the light doubling unit 7 in response to a command signal from the timing controller 9 and outputs a captured image (color image) to the image processing unit 10. In the first embodiment, a camera having a resolution of 640 × 480 (horizontal: vertical), a luminance value of 1 to 255 (256 levels), and 100 fps or more is used.

The timing controller 9 is the time from the light projection start time of the projector 5 until the gate 7a is opened so that the captured image captured by the high-speed camera 8 is the timing of the reflected light returning from the target imaging area. The imaging timing is controlled by setting a certain delay time tD and outputting an open / close command signal corresponding to the delay time. That is, the delay time tD is a value that determines the distance from the host vehicle to the imaging area (imaging target distance), and the relationship between the delay time tD and the imaging target distance is as follows.
Imaging distance = speed of light (approx. 3 x 10 ⁸ m / s) x delay time tD / 2
FIG. 2 shows a temporal relationship between the operation of the projector 5 (light projection operation) and the operation of the gate 7a (camera gate operation) when imaging one imaging area.

  The timing controller 9 increases the imaging range of the high-speed camera 8 by increasing the delay time tD by a predetermined interval (for example, 10 ns) so that the imaging area continuously moves from the front side of the vehicle to the front side. Change to the side. The timing controller 9 starts the imaging operation of the high-speed camera 8 immediately before the gate 7a is opened, and ends the imaging operation after the gate 7a is completely closed.

  Further, in the first embodiment, as illustrated in FIG. 3, when the imaging target distance is continuously changed from B1 → B2 → B3 →..., The imaging target distance is increased more than the imaging target width A of the imaging area. By increasing the amount (B2-B1), the increase amount of the imaging target distance is set so that a part of the imaging area changes while overlapping.

  FIG. 4 is a schematic diagram showing temporal luminance changes when the amount of increase in the imaging target distance is reduced to the limit, in other words, when imaging is performed with an infinite increase in the imaging area. By overlapping each other, the luminance value of the same pixel in a plurality of consecutive captured images gradually increases, and after the peak, the luminance value gradually decreases. In practice, the number of imaging areas is limited (1 to n), but by overlapping a part of continuous imaging areas, the temporal luminance change becomes close to the characteristics of FIG.

  The timing controller 9 outputs an image processing command signal to the image processing unit 10 when all the captured images for one frame, that is, the set predetermined range (area 1, area 2,..., Area n) are captured. To do.

  The image processing unit 10 generates distance image data representing the distance to the object for each pixel based on the luminance of the same pixel in the captured image for one frame captured by the high-speed camera 8, and the generated distance image data Is output to the object recognition processing unit 3.

The short distance object sensor 11 detects the presence or absence of a short distance object in the light projecting direction of the projector 5. For example, a photodiode or an ultrasonic sensor.
The short distance object recognizing unit 12 determines from the detection result of the short distance object sensor 11 whether there is an object at a short distance in the light projecting direction.
Based on information from the BCM 14 and information from the short-distance object recognition unit 12, the short distance determination unit 13 determines the optimum light amount and camera gate time based on the presence or absence of an obstacle while stopping, traveling, and to the timing controller 9. Outputs a command.

The object recognition processing unit 3 identifies an object included in the distance image data. A well-known technique such as pattern matching can be used as an object identification method.
Based on the relationship (distance, relative speed, etc.) between the object (person, car, sign, etc.) identified by the object recognition processing unit 3 and the host vehicle, the determination unit 4 presents information to the driver by an alarm, Determine whether vehicle control such as automatic braking is necessary.
The BCM 14 is provided in the vehicle as a vehicle-side controller, and performs various controls and communicates necessary information with a controller in another vehicle. Therefore, information handled by the BCM 14 and information acquired by the BCM 14 through communication can be output from the BCM 14 to the short distance determination unit 13.

[Distance image data generation control processing]
FIG. 5 is a flowchart illustrating the flow of the distance image data generation control process executed by the image processing unit 10 according to the first embodiment. Each step will be described below. This process is repeatedly executed at a predetermined calculation cycle.

  In step S1, the image processing unit 10 stores the luminance value data having the lowest luminance of the captured image obtained by imaging the front of the host vehicle without performing the light projection by the projector 5, as the minimum value data, and proceeds to step S2.

  In step S2, the image processing unit 10 inputs a captured image, and proceeds to step S3.

  In step S3, the image processing unit 10 compares the minimum value data stored in step S1 with the lowest luminance value data among the luminance value data of each pixel of the captured image input in step S2. Is lower than the minimum value data. If YES, the process proceeds to step S4. If NO, the process proceeds to step S5.

  In step S4, the image processing unit 10 stores luminance value data lower than the minimum value data as new minimum value data, and proceeds to step S5.

  In step S5, the image processing unit 10 calculates the difference between the luminance value data and the minimum value data for each pixel and sets it as current data in the pixel, and the process proceeds to step S6.

  In step S6, the image processing unit 10 calculates the difference between the previous data (previous control cycle) and the current data for each pixel, and proceeds to step S7.

  In step S7, the image processing unit 10 determines whether or not the difference obtained in step S7 is equal to or less than a threshold value (noise determination threshold value). If YES, the process proceeds to step S9. If NO, the process proceeds to step S8. In the first embodiment, the threshold value is set to 32.

  In step S8, the image processing unit 10 excludes data pixels whose difference from the previous data exceeds the threshold value (for example, sets the luminance value to 0), and proceeds to step S9.

  In step S9, the image processing unit 10 determines whether image input for one frame (area 1, area 2,..., Area n) has been completed. If YES, the process proceeds to step S10. If NO, the process proceeds to step S2.

  In step S10, the image processing unit 10 compares the luminance value data of the same pixel of n captured images, and the distance image data from the frame number (delay time tD) of the captured image corresponding to the highest luminance value data. And move to return.

[Light emission suppression processing]
FIG. 6 is a flowchart illustrating the flow of the light projection amount suppression process executed by the short distance determination unit 13 according to the first embodiment. Each step will be described below.

  In step S11, the short distance determination unit 13 determines whether or not the gate 7a is controlled based on the information from the timing controller 9. If the gate 7a is being controlled, the process proceeds to step S12 and the gate 7a is controlled. If not, the process is terminated.

  In step S12, the short distance determination unit 13 acquires vehicle speed information from the BCM 14.

  In step S13, the short distance determination unit 13 determines whether the vehicle is traveling based on the vehicle speed. If the vehicle is traveling, the process proceeds to step S14. If the vehicle is stopped, the process proceeds to step S17.

  In step S14, it is detected whether or not there is a person (object) within Nm of the light projecting direction from the vehicle. If there is a person (object), the process proceeds to step S17, and if not, the process proceeds to step S15.

  In step S15, the short distance determination unit 13 outputs a command to the timing controller 9 so as to set the power of the projector 5 high.

  In step S <b> 16, the short distance determination unit 13 sets the camera pulse time A (ns) so as to capture a longer distance, and outputs it to the timing controller 9.

  In step S17, the short distance determination unit 13 outputs a command to the timing controller 9 to set the power of the projector 5 low.

  In step S18, the short distance determination unit 13 sets the pulse time B (ns) of the camera so as to capture a closer distance, and outputs it to the timing controller 9.

Next, the operation will be described.
[Distance image data generation function]
The timing controller 9 controls the imaging timing of the high-speed camera 8 by setting the delay time tD so that the captured image captured by the high-speed camera 8 becomes the timing of the reflected light returning from the target imaging area. To do. When an object is present in the target imaging area, the time for the light emitted from the projector 5 to return from the imaging area is the time for the light to reciprocate the distance between the host vehicle and the imaging area (imaging target distance). Therefore, the delay time tD can be obtained from the imaging target distance and the speed of light.

  In the captured image of the high-speed camera 8 obtained by the above method, when an object exists in the imaging area, the luminance value data of the pixel corresponding to the position of the object is affected by the reflected light, and the luminance of other pixels Indicates a value higher than the value data. Thereby, based on the luminance value data of each pixel, the distance from the object existing in the targeted imaging area can be obtained.

  Furthermore, in the first embodiment, captured images of the imaging areas 1 to n are acquired while changing the delay time tD. Subsequently, the brightness value data of the same pixel in each captured image is compared, and the distance data (distance image data) of all the pixels (640 × 480) is generated using the highest brightness value data as the distance of the pixel.

  Conventional distance detection methods using laser radars and stereo cameras are susceptible to rain, fog, snow, etc., and the noise level with respect to the signal level is large (the SN ratio is small), so the reliability in bad weather is low. . When using a millimeter wave radar that is not easily affected by bad weather, the reliability of distance detection is high, but it is difficult to perform object recognition (object identification) from the millimeter wave radar signal. Is required. Further, since the camera image becomes unclear during bad weather, it is difficult to perform accurate object recognition.

On the other hand, in the first embodiment, only reflected waves returning from the target imaging area are reflected in the captured image, so that the light bent due to the influence of rain, fog, snow, or the like, that is, the noise mixing level is kept low. High signal-to-noise ratio can be obtained. That is, high distance detection accuracy can be obtained regardless of bad weather or night.
Since the distances of all the pixels are known from the generated distance image data, the distance to the object can be instantly grasped when performing object recognition using a technique such as pattern matching.

  Furthermore, in the first embodiment, the captured area is continuously changed to acquire a plurality of captured images, and the captured images are compared to detect the distance of each pixel. In addition, it can be grasped over a wide range. For example, even in the situation where a pedestrian has jumped between the host vehicle and the preceding vehicle, the distance between the preceding vehicle and the pedestrian can be grasped at the same time. Control can be performed.

FIG. 7 shows a situation in which four pedestrians A to D exist at different positions in front of the host vehicle, and the relationship between the distance between the host vehicle and each pedestrian is A <B <C <D. .
At this time, in Example 1, a part of the imaging area is overlapped so that the reflected light from one object is reflected in the pixels of the captured image in a plurality of continuous imaging areas. For this reason, the temporal luminance change of the pixel corresponding to each pedestrian shows a triangular characteristic that takes a peak at the position of the pedestrian as shown in FIG.

  Since the distance image data is data used for alarms and vehicle control, it is impossible to set an imaging area infinitely finely as long as a certain calculation speed is required. By making the reflected light from the plurality of captured images included, as shown in FIG. 9, the temporal luminance change of the pixels is approximated to the above characteristics, and the imaging area corresponding to the peak of the triangular portion is By setting the distance of the object in the pixel, detection accuracy can be increased.

[Avoiding glare]
In the imaging for generating the distance image data, the shutter time is short by the control of the gate 7a, and the projection is performed in a pulse shape.
In order to cope with an environment where the shutter time is short and the transmittance is low, such as long distance or fog, strong light is required as the light to be projected. For this reason, if the light from the projector 5 is looked into nearby, it becomes dazzling and uncomfortable.

In the distance image data generation device 2 according to the first embodiment, when the range gate function is on (according to the determination in step S11), the vehicle speed is acquired from the BCM 14, and the short distance determination unit 13 determines whether the vehicle is stopped or traveling. When the vehicle is stopped, the timing controller 9 reduces the light amount of the projector 5 to an output that is not dazzling.
For this reason, it is suppressed that a person feels dazzled and feels uncomfortable even when looking at the light of the projector 5 when the vehicle is stopped.

  Usually, in order to shoot far away, the intensity of light is needed, but it is only necessary to be able to determine the object in front of you when you are parked. Is possible. Further, in response to a command from the timing controller 9, the gate 7a also changes the gate time and operates so as to photograph only a short distance.

If it is determined that the vehicle is traveling from the vehicle speed information from the BCM 14 (step S13), normal distance image data generation processing is performed (steps S14 → S15 → S16). That is, it operates so as to photograph a relatively long distance.
Even when the vehicle is running, when a short distance object, such as a person, is detected by the short distance determination unit 13, the timing controller 9 reduces the light amount of the projector 5 to an unobtrusive output (steps S 14 → S 17 → S18).
For this reason, it is suppressed that a person feels dazzled and feels uncomfortable even when looking at the light of the projector 5 when the vehicle is stopped.

Furthermore, in the first embodiment, the image processing unit 10 generates distance image data as obstacle detection in front of the vehicle, and the object recognition processing unit 3 performs obstacle detection on this data.
Therefore, in the case where the image processing unit 10 and the object recognition processing unit 3 perform normal photographing, when the person is suddenly detected at a short distance or when approaching is detected, the projector 5 is similarly used. And the gate time of the high-speed camera 8 is also changed. When it can be determined that there is no person at a short distance from the image captured with the light intensity reduced, the original light intensity is restored, and the gate time of the high-speed camera 8 is returned to the normal time to perform imaging.

  In addition, when the vehicle moves from stop to travel, the light intensity is reduced during stop, but at the start of travel, the distance is gradually increased according to the vehicle speed while detecting obstacles ahead. In accordance with this, the amount of emitted light should be increased. In this way, when there is a person detected by remote detection, it is suppressed that the person feels dazzled by the intensity of sudden light projection and is uncomfortable.

Next, the effect will be described.
The distance image data generation device 2 according to the first embodiment has the following effects.

  (1) A projector 5 that projects pulsed light in front of the host vehicle at a predetermined period, an image intensifier 7b that captures reflected light returning from the target distance at an imaging timing set according to the target distance, and a high speed The same pixel in a plurality of captured images with different target distances obtained by the camera 8, the timing controller 9 that controls the imaging timing so that the target distance changes continuously, the image intensifier 7b, and the high-speed camera 8 Based on the luminance, the image processing unit 10 that generates distance image data representing the distance to the object for each pixel, the short-distance object sensor 11 that detects a short-distance object of the host vehicle, and the short-distance object are detected. In this case, the timing controller 9 is instructed to weaken the light projection amount of the projector 5 and to adjust the imaging timing to the weak light projection amount. Since the short-distance object recognition unit 12 and the short-distance determination unit 13 are provided, distance image data representing the distance to the object for each pixel is generated based on the luminance of the same pixel in a plurality of captured images having different target distances. The situation ahead of the host vehicle can be grasped continuously. In addition, it is possible to suppress feeling uncomfortable and feeling uncomfortable when there is a person in the light projection range.

  (2) For the above (1), an information input from the BCM 14 that acquires vehicle speed information is provided, and the short distance determination unit 13 weakens the light projection amount of the projector 5 when the vehicle is stopped, and sets the imaging timing. When the vehicle travels, the command output is made to respond to the weak light quantity, and the light intensity of the projector 5 is increased, and the command output is made to respond to the strong light quantity at the imaging timing. Even if a person suddenly looks into the screen or when the person is at a short distance in the light projecting direction, it can be prevented from being dull and uncomfortable.

(Other examples)
The best mode for carrying out the present invention has been described above based on the embodiment. However, the specific configuration of the present invention is not limited to the configuration of the embodiment, and it relates to each claim of the claims. Design changes and additions are allowed without departing from the spirit of the invention.

For example, the light projection period, the light projection time, the gate time, the imaging target width, the amount of change in the imaging target distance, and the number of imaging areas in one frame are appropriately set according to the performance of the imaging means and the performance of the distance image data generation means can do.
Further, for example, in the first embodiment, the short distance determination unit 12 and the short distance object recognition unit 13 are provided. However, the short distance determination unit 12 is also used as the determination unit 4, and the short distance determination unit 13 is used as the object recognition processing unit. 3 may also serve as a structure.

  Further, for example, in the first embodiment, when the stop time is determined, a process of reducing the light amount is performed. However, in the case of a temporary stop such as a right or left turn, there is an effect that a moving object from the front can be determined. The determination may be made by detecting an obstacle by the short-distance object sensor 11 instead of weakening the light amount so as to enable long-distance shooting instead of only short-distance shooting.

It is a block diagram which shows the structure of the obstruction detection apparatus of Example 1. FIG. It is a figure which shows the temporal relationship between the operation | movement (light projection operation | movement) of a light projector, and the operation | movement of a gate (camera gate operation | movement) at the time of imaging one imaging area. It is a figure which shows the state which a part of imaging area overlaps. It is a schematic diagram which shows a temporal luminance change at the time of imaging by increasing an imaging area infinitely. 6 is a flowchart illustrating a flow of distance image data generation control processing executed by the image processing unit according to the first embodiment. 6 is a flowchart illustrating a flow of a light projection amount suppression process executed by a short distance determination unit according to the first embodiment. It is a figure which shows the condition where four pedestrians AD exist in the different position ahead of the own vehicle. It is a schematic diagram which shows the temporal luminance change of the pixel corresponding to each pedestrian A-B. It is a figure which shows the distance image data generation effect | action of Example 1. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Obstacle detection apparatus 2 Distance image data generation apparatus 3 Object recognition process part 4 Judgment part 5 Light projector (light projection means)
6 Objective lens 7 Photomultiplier 7a Gate 7b Image intensifier 8 High-speed camera (imaging means)
9 Timing controller (timing control means)
10 Image processing unit (distance image data generating means)
11 Short-distance object sensor 12 Short-distance object recognition unit 13 Short-distance determination unit 14 BCM
P (Indicates the constant light emission location of distance image data)

Claims (2)

Light projecting means for projecting pulsed light in front of the host vehicle at a predetermined period;
Imaging means for imaging reflected light returning from the target distance at an imaging timing set according to the target distance;
Timing control means for controlling the imaging timing so that the target distance changes continuously;
Distance image data generation means for generating distance image data representing a distance to an object for each pixel based on the luminance of the same pixel in a plurality of captured images with different target distances obtained by the imaging means;
Short-range detection means for detecting a short-range object of the vehicle;
A light emission amount suppression control unit that outputs a command to decrease the light emission amount of the light projection unit and detect the imaging timing according to the weak light emission amount when an object at a short distance is detected;
With
A vehicular distance image data generating apparatus characterized by the above. The vehicle distance image data generation device according to claim 1,
A vehicle speed detecting means for acquiring vehicle speed information is provided,
The light emission amount suppression control means includes:
When the vehicle is stopped, the light projection amount of the light projecting means is weakened, and the command output is performed so that the imaging timing corresponds to the weak light projection amount,
When the vehicle travels, the light projection amount of the light projecting means is increased, and a command output is performed so that the imaging timing corresponds to the strong light projection amount.
A vehicular distance image data generating apparatus characterized by the above.