JP6693757B2 - Distance image generating apparatus and method - Google Patents

Description

  The present invention relates to a distance image generation device based on a time of flight (TOF) method.

  In the field of robot vision and the like, a distance image sensor that acquires three-dimensional spatial information in real time is becoming more important. One of such sensors is a TOF range image sensor (hereinafter, simply referred to as "TOF sensor"). This sensor generates a distance image by irradiating the target space with modulated light and detecting the phase delay of the reflected light received by the planar image sensor for each pixel. For example, Patent Documents 1 and 2 disclose TOF sensors.

Many methods of detecting the phase delay have been proposed. A known example will be described with reference to FIG. With respect to the modulation cycle T of the irradiation light, the light quantity of the reflected light received every T / 2 period as shown in FIG. 5A is A0 and A2, and the start point is delayed by T / 4 cycle as shown in FIG. 5B. Assuming that the amounts of reflected light received for each period are A1 and A3, the phase shift φ _d of the reflected light is
φ _d = arctan {(A1-A3) / (A0-A2)}
Required by. The time t required for the round trip to the reflecting surface that reflects the light is t = Tφ _d / 2π, and the distance z to the reflecting surface is z = ct / 2. Here, c is the speed of light.

  In many TOF sensors, electric charges generated in a photodiode (PD) are electronically distributed to a plurality of accumulating portions formed on the sides of the PD according to the amount of light, and the electric charges generated by a plurality of light receptions are collected and accumulated. Each is read as a signal. As a result, the components other than the sensor can be reduced, and the device can be downsized. Further, since the phase delay can be calculated only by calculating the difference and ratio of a plurality of signals, the calculation load is light and high-speed measurement is possible.

JP, 2003-051988, A JP, 2004-294420, A

  The problem of the current TOF sensor is to improve the accuracy of distance information. Since the TOF sensor obtains distance information by electronic processing, it is affected by various electronic errors. Random errors can be less affected by increasing the number of times of light reception under the same conditions and averaging. However, systematic errors cannot be removed by averaging.

  The present invention has been made in view of the above, and an object of the present invention is to provide an apparatus and method capable of generating a highly accurate range image by the TOF method.

  For the above purpose, the present invention corrects the original range image by the TOF sensor with the range information based on the principle of triangulation.

  A range image generating device of the present invention includes a first light source that irradiates a target space with modulated light, a second light source that irradiates a beam-shaped or sheet-shaped laser beam that traverses the target space, an imaging unit, and a correction unit. Have. The image pickup unit includes an image pickup device in which a plurality of photosensitive units are arranged in a matrix, and an original image is obtained from a phase difference between the modulated light emitted from the first light source and the reflected light of the modulated light from the target space. A distance image is generated and a brightness image including the reflected light of the laser light emitted from the second light source from the target space is generated. Then, the correction unit corrects the distance information of the original distance image based on the position of the pixel in which the reflected light of the laser light is recorded in the brightness image, and generates the corrected distance image.

  A range image generating method of the present invention irradiates a target space with modulated light, receives reflected light of the modulated light by an image pickup device in which a plurality of photosensitive portions are arranged in a matrix, and delays the phase of the reflected light. Generating an original distance image by detecting for each pixel, irradiating a beam-shaped or sheet-shaped laser light so as to traverse the target space, and by the image pickup device, a brightness image including reflected light of the laser light. And a step of generating the corrected distance image by correcting the distance information of the original distance image based on the position of the pixel where the reflected light of the laser light is recorded in the brightness image.

  Preferably, in the distance image generating method, the beam-shaped or sheet-shaped laser light is emitted from a second light source whose position and angle with respect to the image sensor are fixed.

  According to the distance image generating device or the distance image generating method of the present invention, the original distance image by the TOF method can be corrected by obtaining the distance from the position of the pixel recording the reflected light of the laser light to the object reflecting the laser light. .. Distance measurement using reflected light of laser light uses the principle of triangulation and is not affected by electronic error of the TOF method, so that a more accurate distance image can be obtained.

It is a figure which shows the structure of the distance image generation apparatus which is one Embodiment of this invention. It is a figure for demonstrating the effect | action of a beam-shaped laser. It is a figure for demonstrating the effect | action of a sheet-shaped laser. It is a figure which shows the flow of the distance image generation method which is one Embodiment of this invention. It is a figure for demonstrating the well-known method of distance measurement by a TOF system.

  An embodiment of the distance image generation apparatus of the present invention will be described based on FIGS.

  In FIG. 1, the range image generation device 10 of the present embodiment includes a first light source 11, a second light source 12, an image capturing section 13, a control section 16, and a correction section 18.

  The first light source 11 irradiates the entire target space to be measured with modulated light. The wavelength of the modulated light is not particularly limited as long as it is a wavelength that can be sensed by the image pickup device 14 described later, but near infrared rays are preferably used so that the irradiation light is not noticeable. The modulated light is intensity-modulated at high frequency into a pulse shape or a sine wave shape. The higher the modulation frequency, the better the distance resolution, which is preferable. The modulation frequency is typically suppressed to about several MHz to 10 MHz due to the limitation of manufacturing technology. As the light to be modulated, LED light or laser light that can be modulated at such a high frequency is used.

  The image pickup unit 13 includes an image pickup device 14 and an image generation unit 15. Light from the target space is focused on the image sensor by a lens or the like. Preferably, a bandpass filter that allows the modulated light emitted from the first light source to pass therethrough is arranged on the front surface of the image sensor. Thereby, the influence of ambient light can be suppressed.

  In the image pickup device 14, photodiodes (PD) that are photosensitive portions are arranged in a matrix, and each PD corresponds to each pixel of a distance image. The electric charge generated in the PD is distributed to a plurality of accumulating portions formed on the sides of the PD, and the electric charge generated in a plurality of light receptions is accumulated in the accumulating portion. The accumulated charge is transferred to the image generation unit as a signal for each storage unit of each PD.

  The image generation unit 15 carries out arithmetic processing on the signal transferred from the PD, and has distance information for each pixel from the phase difference between the modulated light emitted from the first light source and the reflected light received by the image sensor. Generate an original range image. The image generation unit also generates a brightness image including the received light intensity of each pixel.

  The second light source 12 irradiates the beam-shaped or sheet-shaped laser light so as to cross the target space, that is, so as to cross the visual field of the imaging unit 13. To irradiate the sheet-shaped laser light, it is preferable to use a line laser instead of scanning the beam-shaped laser. The line laser spreads the laser beam into a sheet by passing it through a cylindrical lens. This saves time for scanning. The wavelength of the laser light is not particularly limited as long as it can be sensed by the image sensor 14. If a bandpass filter is placed in front of the image sensor, the wavelength shall pass through the filter. The wavelength of the laser light can be, for example, the same wavelength as the modulated light emitted from the first light source 11.

  Here, the position and angle of the second light source 12 with respect to the image sensor 14 may be adjustable according to the distance and spread of the target space, but at least during a series of measurements on the same target space, they should not change. It is fixed.

  The control unit 16 controls the entire range image generation device 10. For example, it controls start and end of measurement, ON / OFF of the first light source and the second light source, modulation of the first light source, synchronization of the first light source and the image pickup unit, charge distribution of the image pickup device, and signal reading from the storage unit.

  The correction unit 18 receives the original distance image and the brightness image from the imaging unit 13. In the lightness image generated during the irradiation of the laser light from the second light source, the reflected light of the laser light from the target space is reflected. Referring to FIG. 2, when irradiating the beam-shaped laser light 31 from the second light source 12, the reflected light 33 from the surface of the object (target object) 22 in the target space 21 is received by the image sensor 14, and the brightness is increased. Appears as dots on the image. Referring to FIG. 3, when the sheet-shaped laser light 41 is emitted from the second light source 12, the reflected light is linearly reflected on the brightness image. The correction unit obtains the distance from the position of the pixel where the reflected light is recorded to the surface of the object that reflects the laser light. The correction unit corrects the original distance image based on the distance thus obtained. The corrected range image is output to the outside in an appropriate format according to the Ethernet (registered trademark) standard or the like.

  It should be noted that the above-mentioned parts do not necessarily have to be physically separated. For example, the correction unit may be configured integrally with the image generation unit and incorporated in the imaging unit.

  Next, an embodiment of the distance image generation method of the present invention will be described with reference to FIG.

  In FIG. 4, the distance image generating method of the present embodiment is performed using the distance image generating apparatus described above, and includes an original distance image generating step S1, a brightness image generating step S2, and an original distance image correcting step S3. Either the original distance image generating step S1 or the brightness image generating step S2 may be performed first.

  The original distance image generation step S1 can use the same method as a known TOF sensor. For example: First, modulated light is emitted from the first light source to the entire target space. While irradiating the modulated light, the imaging unit opens and closes the electronic shutter of the image sensor in synchronization with the modulation cycle of the modulated light according to the control of the control unit, and converts the charge amount accumulated at each pixel into a signal at a different timing. Then, it is transferred to the image generation unit. The image generation unit arithmetically processes a plurality of signals from the image sensor to generate an original distance image. In the original distance image generation step, the second light source is preferably turned off. This is to reduce background noise when generating the original range image.

  In the brightness image generation step S2, first, a laser light is emitted from the second light source so as to cross the target space. Next, while irradiating the laser light, the light from the target space is received by the image sensor at appropriate time intervals, and a brightness image is generated. The image generation unit may generate the brightness image by using the signal based on the charge amount accumulated at any timing in the original distance image generation step S1. This is because the laser light emitted from the second light source is not modulated and has nothing to do with the modulation cycle of the first light source. In the brightness image generation step, the first light source is preferably turned off. This is to reduce background noise when generating a brightness image.

  The laser light emitted from the second light source may be beam-shaped or sheet-shaped. Energy consumption can be reduced by irradiating a beam-shaped laser beam. On the other hand, the beam may hit a corner or a step of the surface of the object, and the reflected light may not be clearly discriminated on the brightness image. When the sheet-shaped laser light is irradiated, the reflected light is linearly reflected on the brightness image, so that it is possible to use the distance information of a plurality of points on the line.

  The second light source may be configured to be separable from the image sensor and set to an appropriate position each time the brightness image generating step S2 is performed. However, in that case, it is necessary to calibrate the position of the second light source and the emitting direction of the laser light each time. Preferably, the two are integrated so that the position and angle of the second light source with respect to the image sensor are constant. In order to integrate both, they may be housed in a common housing, or they may be housed in separate housings and then mechanically coupled. This eliminates the need for complicated calibration work.

In the original range image correction step S3, the original range image is corrected based on the brightness image. According to the brightness image, the distance from the position of the pixel where the reflected light of the laser light is recorded to the object surface reflected in the pixel is uniquely determined. The method of correcting the original distance image is not particularly limited. For example, assuming that the distance obtained from the brightness image is z _L and the distance information of the same pixel of the original distance image is z _T , z _L −z _T is added to the distance information of all pixels of the original distance image, or z _L / z You can multiply by _T. This method is particularly effective for removing systematic errors due to electronic factors contained in the original range image. Further, since the original distance image and the brightness image are generated using the same image sensor, there is no parallax between the two images, and there is no erroneous correspondence between the two images as in the stereoscopic vision method.

  In the above method, other images may be appropriately used in order to improve the SN ratio of the original distance image and the brightness image. For example, the distance image of only the object may be extracted by using the background image obtained by capturing the target space in the absence of the object whose distance information is to be measured.

  The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the technical idea thereof.

  For example, a plurality of second light sources may be used. For example, two line lasers may be arranged and used so that the sheet-like lights intersect each other. In that case, by sequentially switching the two second light sources to generate the brightness image, the distance information based on the brightness image can be used in a wider range.

10 Range Image Generation Device 11 First Light Source 12 Second Light Source 13 Imaging Unit 14 Imaging Device 15 Image Generation Unit 16 Control Unit 17 Input Unit 18 Correction Unit 19 Output Unit 21 Target Space 22 Object (Target)
31 beam-shaped laser light 33 reflected light of beam-shaped laser light 41 sheet-shaped laser light

Claims (5)

A first light source for emitting modulated light that spreads over the entire target space;
A second light source which is provided separately from the first light source and which is in the form of a beam or a sheet that crosses the target space and emits unmodulated laser light;
A plurality of photosensitive units are provided with an image sensor arranged in a matrix, and an original distance image is generated from a phase difference between the modulated light emitted from the first light source and the reflected light from the target space of the modulated light, An imaging unit that generates a brightness image including reflected light of the laser light emitted from the second light source from the target space;
A correction unit that corrects the distance information of the original distance image based on the position of a pixel in which the reflected light of the laser light is recorded in the brightness image, and that generates a corrected distance image,
Distance image generating device having. The position and angle of the second light source with respect to the image sensor are fixed,
The distance image generation device according to claim 1 . The first light source and the second light source are not illuminated at the same time,
The range image generating device according to claim 1 . The modulated light that spreads over the entire target space is emitted, the reflected light of the modulated light is received by the image sensor in which a plurality of photosensitive parts are arranged in a matrix, and the phase delay of the reflected light is detected for each pixel to obtain the original light. Generating a range image,
A laser beam that is not modulated in a beam shape or a sheet shape so as to traverse the target space, and is irradiated with light different from the modulated light, and a brightness image including reflected light of the laser light is emitted by the imaging device. The step of generating
Correcting the distance information of the original distance image based on the position of the pixel in which the reflected light of the laser light is recorded in the brightness image, and generating a corrected distance image,
And a range image generating method. The beam-shaped or sheet-shaped laser light is emitted from a second light source whose position and angle with respect to the image sensor are fixed.
The distance image generating method according to claim 4 .

Images