JP2006170744A - Three-dimensional distance measuring instrument - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To restore three-dimensional information on a flat unfigured floor, a wall, or a physical object having a repetitive pattern. <P>SOLUTION: This three-dimensional distance measuring instrument is equipped with one or more cameras and one or more light source devices. This instrument acquires distances from respective points to the object all over the areas photographed with the cameras, by using a light source device capable of continuously projecting a banded photo-pattern that continuously moves as time goes on, the camera for photographing the object with the pattern projected thereonto and capable of detecting a change in luminance of each point on an image plane, a means for detecting a difference in times of blinking from a change in the luminance of each point, and a means for converting the times of blinking into distance information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a three-dimensional distance device used for recognition of a work target and surrounding environment required by a robot, product shape measurement in a factory, and the like, and a manufacturing method thereof.

  Conventionally, as this type of apparatus, passive stereo vision, light cutting method, phase shift method, and the like have been used.

  Passive stereo vision is a widely used technique that arranges multiple cameras, detects corresponding points in the obtained images and detects the corresponding points, and is mainly used for triangulation. Get the distance to. However, the success or failure of corresponding point detection depends on the object. Corresponding points are not correctly detected on a surface without a pattern such as a white wall, or on a pattern with many repetitions such as a stitch.

  The light cutting method is also a well-known method, which is disclosed in Non-Patent Document 1, Patent Document 1, and the like. The object is irradiated with slit light, photographed with a camera from another direction, and three-dimensional information is obtained from the shape. Restore. However, the processing speed is limited because the number of slit lights is limited to one or several at the same time due to restrictions on shape detection. Moreover, the use is generally limited to a room in which lighting conditions such as a dark room are managed, due to restrictions on processing for recognizing slit light.

  As disclosed in Non-Patent Document 1, the phase shift method projects a striped light onto a target by slightly moving a plate on which fine sine wave horizontal stripes are formed in front of a light source, thereby producing several images. , And the phase of each point on the image is detected and converted to distance, but since a simple repeated fringe is used, if the ambiguity where different points have the same phase is not resolved reliably, the result is indeterminate. Further, there is a drawback that since the plate is moved in parallel, a return operation is required and continuous measurement is impossible.

Japanese Patent Laid-Open No. 5-46739 Yusuke Oike, Hiroaki Shinke, Makoto Ikeda, Kunihiro Asada: High-definition, real-time three-dimensional imaging system using the optical cutting method, Journal of the Institute of Image and Video Media Vol57-No9 1149/1151 V. Srinvasan, H, C, Liu, M. Halioua: Automated phase-measuring profilometry of 3-D diffuse objects, Applied Optics, Vol23-No18 3105/3108

  As described above, in the conventional technique, there are problems that the photographing target is limited for each method, the photographing environment is limited, the result may be uncertain, and continuous detection cannot be performed.

  The present invention reduces the uncertainty of the result regardless of the subject and the shooting environment, enables continuous detection, and can change the measurement time and measurement accuracy according to the situation. An object of the present invention is to provide a three-dimensional distance measuring apparatus.

  A light source device capable of continuously projecting a striped light pattern that continuously moves with time in a three-dimensional distance measuring device including one or more cameras and one or more light source devices; A camera capable of photographing the object on which the image is projected and detecting a change in luminance at each point on the screen, a means for detecting a difference in blinking timing from the change in luminance at each point, and distance information from the blinking timing There are provided a three-dimensional distance measuring device and a distance measuring method using the device for acquiring the distance to the object at each point over the entire region photographed by the camera by means of conversion.

  According to the present invention, in the three-dimensional distance measuring apparatus, the effect of continuously and reliably measuring the distance by changing the conditions such as the measurement time and the measurement accuracy according to the target without selecting the shooting target and the shooting environment. Is obtained.

  In order to realize the present invention, a light source device for projecting a light pattern, a camera for capturing the light pattern, and a processing device for processing the captured image are used.

  FIG. 1 shows the configuration of the entire apparatus. The light source device 1-1 projects light and dark stripes of light. This fringe is a fringe whose brightness changes continuously, and produces a striped light-dark pattern 1-3 on the object 1-2. Moreover, this stripe moves as time passes. As the stripes move, the brightness at any point on the illuminated object also changes. The stripes have a certain period, and the brightness of points on the object also changes periodically.

  This is photographed with the camera 1-4. The light source device is controlled by the light source control device 1-5 so as to be synchronized with the imaging of the camera. Based on the reference phase of the light source device and the photographed image, the phase detection device 1-6 can detect each of the images on the image. It is detected how much the luminance change for each point has advanced or retreated from the reference phase. This phase is converted into a distance by the distance converter 1-7. On the other hand, the captured images are averaged by the image averaging device 1-8 and converted to a normal image without stripes.

  FIG. 2 shows an implementation example of the light source device. The light source 2-1 is for illuminating the object. This light source is a point light source, a line light source parallel to the projected stripe, a parallel light source, or the like, but an equivalent light source may be prepared by combining an arbitrary light source and an optical system. Further, the wavelength of the light source can be arbitrarily selected to be suitable for illuminating the object.

  The shielding devices 2-2 to 2-4 include a cylinder 2-2 that is transparent to the wavelength of the light source, a shielding pattern 2-3 made of an opaque material formed on the cylinder, and a motor 2-4 that rotates the cylinder.

  The light source and the motor are controlled by the control device 2-5. The brightness of the light source is determined so as not to saturate the brightness of the captured image, and is controlled to reduce energy consumption by shutting off when not measured. The rotation of the motor is controlled so that the rotation is synchronized with the photographing. A synchronization signal is obtained from the video signal, and the rotation speed of the motor is controlled so that the luminance change period at each point is an integral multiple of the video frame interval.

  For example, as a specific configuration example, a transparent sheet on which stripes are printed is attached to an acrylic cylinder over one turn, and is rotated by a DC servo motor.

  In this example, the point light source and the shielding device are used as they are. However, the target is irradiated with the modulated light, and each point on the target is prevented from reaching the light from two or more paths from the same light source. As long as it is designed, the light path does not matter. That is, by using a lens system or a mirror, the radial optical path from the point light source can be converged in parallel or in a narrow range. Depending on the design of the optical system, it is possible to irradiate a very narrow area such as under a microscope.

  Further, the light source device used in the present invention is not limited to the light source device described in the claims, which is a method of rotating a striped cylinder, a disk, a prismatic cylinder, or a light source array. What is required of the light source device is to project light that periodically changes in time with different phases in different directions as viewed from the light source device. Any apparatus capable of projecting such light can be used in the present invention. For example, such a pattern can be projected by a device such as a liquid crystal projector.

  The camera indicated by 1-4 in FIG. 1 can be used as long as it can detect the wavelength of the light source, can continuously capture images at a constant time interval, and can obtain the image sequence. In the most general case, a normal video camera can be used. In this case, about 60 images are obtained continuously per second. An industrial high-speed video camera can be used, and a combination of a high-resolution digital still camera and an interval timer can be used although the measurement time is long. Further, there may be a method in which pattern light from the light source device is projected onto a very narrow range by an optical system and then observed with a microscope with a camera.

  The moving speed of the fringe is determined by the imaging interval of the video camera. However, if the daring is advanced a little faster than the imaging interval, specifically, for example, one fringe at one interval, sampling is performed. Folding distortion occurs according to the theorem, which is equivalent to the one advanced at low speed. With the conventional light pattern projection method, when the human eye sees the pattern, it tends to be offended and there are concerns about adverse effects on the physiological side, but when the stripes are advanced at a sufficiently high speed, the response speed of the human eye Therefore, it is perceived as normal light by humans and this problem is solved.

The following processing is performed on the obtained video. A periodically changing value always has a sinusoidal component of its fundamental frequency. This basic component is represented by the following formula.
Here, n is the number of images necessary for the fringe to advance by one period, and the rotation control of the cylinder is performed after designing to actually process with n sheets. Y _xy [i] is the luminance of the pixel at the coordinate (x, y) of the image sequence number i, Y _{0, xy} is a reference luminance constant, a, b are coefficients that determine the average value and amplitude of the luminance change, φ Is a phase value that depends on the irradiation direction of light from the light source device. Y _{B and xy} are background lights from other light sources.

The reference signals sin (2πi / n) and cos (2πi / n) are generated for the luminance change values at each point of the continuous image, multiplied and integrated.
Thereby, the ratio of the sine and cosine of the light and dark phases of each pixel is obtained. By using the arc tangent, the phase can be obtained.

In addition, by multiplying without multiplying the reference wave
Luminance is obtained when each pixel is illuminated with non-patterned light.

  In this embodiment, the luminance has been described as a representative. However, for example, components of three primary colors of red, green and blue can be obtained from a general video signal of a camera. Such components can be used as representatives, and numerical values obtained by these functions can also be used. Luminance is one of the function values of the three primary colors.

  FIG. 3 shows a configuration example of the phase detection device. The video signal becomes image data by the image acquisition circuit 3-1. When the phase detection device is configured using a computer, a video capture circuit is used. When the phase detection device is configured using a digital signal processing circuit, an AD converter is used.

  The number of captured images is counted by the counter 3-2, corrected by the control information of the light source device, and a sine wave and a cosine wave are generated by the reference signal generator 3-3. A multiplier 3-4 multiplies the luminance value of each pixel of the image data and two types of reference signals. This is integrated for each pixel in the integration buffer 3-5. After the integration period ends, the phase value is obtained from the integrated value of each pixel via the arctangent calculator 3-6.

  In this configuration, image data can be sent to an acquisition circuit, a multiplier, and a buffer for each screen or for each divided screen, and can also be sent sequentially pixel by pixel. In the former case, the integration buffer is composed of an array memory for storing pixels and an arithmetic unit for sequentially accessing the memory. In the latter case, the integration buffer can be configured as shown in FIG. Since the image data is composed of the same number of pixels for each screen, the image data is composed of a shift register 4-1 and an adder 4-2 having the length of the number of pixels. After addition processing of a certain pixel on a screen at a certain time point, the added value is sequentially sent by a shift register and output from the shift register at the time of addition processing of the same pixel on the next screen. Is input. The switch 4-3 is for clearing to zero at the start of the integration interval. Further, the integrated output is selected in the subsequent circuit so that only the final result is used. In the format of sending data for each screen, processing is intermittently performed, but in the format of sending one pixel at a time, each processing is performed in parallel and when the present invention is configured as a digital signal processing circuit. Available.

  The distance converter 1-7 is for converting the phase into distance information. The configuration is, for example, an arithmetic device that always performs triangulation calculation based on the geometric positional relationship between the light source device, the imaging device, and the pixel position, and a conversion table that has previously calculated the relationship between phase and distance. A method of storing and converting while referring to this, a method of using a table while interpolating, etc.

  Wave superposition will be described. The processing formula using the sine wave holds true even when a light source device that repeats an integer number of times of light and dark is used in the same processing period. Can be detected. Therefore, in the shielding device of one light source device, when setting the transparency, if two types of light and dark periods superimposed are prepared, it can be detected by one imaging device and each phase can be obtained independently. I can do it. One assumes that the entire measurement object falls within the range of one cycle, and the other projects finer stripes. By processing the phases detected by both at once, a rough and unambiguous phase can be obtained in the former, and a resolution can be obtained in the latter. In the phase shift method, only the latter is used, and in the measurement result, a lot of parts having the same phase are generated although they are different parts, which causes ambiguity. However, in the present invention, the former is added to eliminate ambiguity.

  Here, for the sake of simplicity, we used one that fits in one period. If two or more kinds of periods are used together and their wave numbers are relatively prime, the result is uniquely identified from the paired phases. it can.

  The use of the multiple light source device will be described. FIG. 5 shows an embodiment using a plurality of light source devices having different periods. In the present invention, when the shade of the light source is generated on the object due to the unevenness of the object, the phase cannot be detected at that portion, and the distance cannot be detected. Therefore, the shadow area is reduced by using a plurality of light source devices.

  For the target 5-1, the light source devices 5-2 and 5-3 that cause different numbers of brightness in the same processing section are placed in different directions, and patterns are projected from different directions and photographed with one camera. To do. The light source device 5-2 causes shadow 5-4 on the object, but the light from the light source 5-3 is irradiated. Two processing units 5-6 and 5-7 are used to distribute the output of the camera, and each processing unit performs processing with a reference wave corresponding to each light source device. Each processing device outputs the distance at each pixel of the camera and its reliability. Reliability is, for example, a function of the amplitude of periodic fluctuations in luminance. Considering this reliability, the distances obtained from the two systems of processing are synthesized by the distance integration processing device 5-9. In this example, there are two systems, but the same applies to three or more systems. A set of video averaging devices is sufficient. In the conventional light projection method, projection is performed from only one direction, and even when a plurality of light projections are used in combination, irradiation cannot be performed simultaneously. However, in this method, it is possible to irradiate simultaneously, measure simultaneously and synthesize.

  In addition to these embodiments, it is possible to parallelize more observations by simultaneously illuminating with a plurality of wavelengths that can be observed independently.

  Further, it is not necessary to have a single imaging device, and if imaging is synchronized, imaging can be performed simultaneously by a plurality of units, and processing can be performed based on the phase information of each light source device. In total, the phase detector is the product of the number of cycles used and the number of imaging devices.

  The three-dimensional distance measuring apparatus according to the present invention can be applied to various fields such as detection of a work target and surrounding environment for a robot, shape measurement of industrial products and agricultural products.

  The three-dimensional distance measuring device manufactured according to the present invention can be widely used as a robot state detecting device and an object shape measuring device.

1 is a schematic configuration diagram of a three-dimensional distance measuring device according to the present invention. It is an Example of the light source device used for this invention. 1 is a configuration example of a phase detection device and a video averaging device used in the present invention. It is a structural example of the circuit for performing a pixel integration device sequentially. It is the schematic for implementing the method of reducing the influence of a shadow using a two-system light source device.

Explanation of symbols

1-1 Light Source Device 1-2 Measurement Object 1-3 Bright and Dark Stripes of Light Projected by Light Source Device and Moves as Time Elapses 1-4 Imaging Device 1-5 Rotation of Light Source Device is Synchronized with Imaging Device 1-6 A device for detecting the phase at each pixel on the image 1-7 A device for converting the detected phase into a distance at each pixel 1-8 An image sequence is simply averaged, For obtaining accurate camera images 2-1 Light source (lamp etc.)
2-2 Transparent rotating body (transparent resin cylinder, etc.)
2-3 Light shielding pattern provided on the cylinder (printing of patterns, attachment of printed films, etc.)
2-4 Motor for Rotation Drive 2-5 Device for Controlling Motor Rotation and Luminance of Light Source 3-1 Circuit for Converting Video and Image Signals Input from Outside 3-2 The Number of Input Images Device for Measurement 3-3 Device for Generating Reference Signal with Progress of Image 3-4 Multiplier for Multiplying Pixel Brightness Value of Image and Reference Signal 3-5 Device for Accumulating Each Pixel (Memory for Integration) Including equipment)
3-6 Device for Dividing Integrated Value and Obtaining Phase by Inverse Tangent 3-7 Accumulating Device for Video Averaging 4-1 Shift register (for one screen) that sequentially sends the integrated value of pixel luminance and reference signal Length to store)
4-2 Adder 4-3 Switch 5-1 for Setting Holding Value to Zero at Start of Accumulation Object 5-2 with Unevenness that Can Shade 5-2 First System Light Source Device 5-3 Second System Light source device 5-4 Light from the light source device of the first system is not irradiated 5-5 Shadow by the object, etc. Light from the light source device of the second system is not irradiated 5-6 Shadow by the object, etc. 5-7 Second System Processing Device Group 5-8 Video Averaging Processing Device 5-9 for Obtaining Normal Images 5-9 Device that integrates distances of pixels obtained by two systems based on their reliability

Claims (5)

  A light source device capable of continuously projecting a striped light pattern that continuously moves with time in a three-dimensional distance measuring device including one or more cameras and one or more light source devices; A camera capable of photographing the object on which the image is projected and detecting a change in luminance at each point on the screen, a means for detecting a difference in blinking timing from the change in luminance at each point, and distance information from the blinking timing A three-dimensional distance measuring apparatus characterized in that, by means of conversion, a distance to an object at each point is acquired over the entire area photographed by a camera.   The light source device includes a light source having an arbitrary wavelength such as white light, single wavelength light, infrared light, and a shielding device having a stripe pattern, and the shielding device is represented by a cylinder, a disk, a prismatic cylinder, or the like. A fringe that changes continuously and periodically with a substance that can adjust the transmission of light with a stepwise brightness and darkness with respect to the wavelength, on an object that is transparent with respect to the wavelength and that consists of a transparent surface with the same center of gravity The three-dimensional distance measuring apparatus according to claim 1, wherein a pattern is rotated by a motor.   2. The light source device according to claim 1, wherein the light source device includes an array of a plurality of small light sources, and can project a striped light pattern onto an object by controlling blinking of the small light sources. 3D distance measuring device.   The distance measuring device is composed of two or more light source devices that illuminate from one or more cameras and different directions, and each light source device projects a fringe pattern that blinks in different periods, 4. The apparatus according to claim 1, further comprising means for reducing the influence of shadows by an object by simultaneously photographing a target, simultaneously detecting a distance for each pattern, and synthesizing the patterns. The three-dimensional distance measuring device described.   In a three-dimensional distance measuring device including one or more cameras and one or more light source devices, a light source device capable of projecting a striped light pattern that moves with the passage of time, and a target on which the light source device is projected. A camera capable of photographing and detecting a change in brightness of each point on the screen, a means for detecting a difference in blinking timing from the change in brightness of each point, and a means for converting the blinking timing into distance information A distance measuring method characterized in that the distance to the target at each point is acquired over the entire area imaged in (1).