TWI588508B - Stereoscopic depth measuring apparatus - Google Patents

Description

Stereo depth measuring device

The present invention relates to a depth distance measuring device; and more particularly to an optical non-contact stereo depth distance measuring device that uses an optical projector to project a pattern to a position to be tested and captures it with an image capturing device. The pattern is obtained by calculating the spatial frequency to obtain the depth information of the position to be tested.

Three-dimensional (3D) gesture capture is an emerging technology for human-computer interaction interface, and 3D sensor is the most important component of the technology. There have been many creations in the literature to explore this technology, and some of the creations have been developed into actual industrial or consumer products. In general, this technique is related to optical depth distance measurement and can be divided into three main categories by sensing mechanisms. They are: structured light method, time of flight method, and stereo vision method. Each architecture has its advantages and disadvantages and there are already corresponding products available. For example, the Kinect sensor used by Microsoft's Xbox system is designed with structured light. The SwissRanger product from MESA Imaging uses the time-of-flight method, and Leap Motion introduces stereo vision. The architecture of the sensor, these three products are the main representatives of the three types of technology.

The structured light method is a method of adopting active light projection to obtain a two-dimensional light intensity pattern that periodically changes in the object to be tested, and most of them are based on optical projectors. Now. The projected image reflected by the surface of the object to be tested is captured by the image capturing device, and the captured image is transmitted through a triangulation principle or a phase shift algorithm to obtain a surface of the object to be tested. In-depth information. Compared with the other two depth measurement techniques, structured light method requires a relatively complicated optical structure, so the cost is relatively high. This shortcoming was later improved by the Kinect sensor with the Xbox game console produced by Microsoft Corporation, which uses the technique disclosed in the contents of US Pat. No. 8,150,142 B2, US Pat. No. 8,493,496 B2 and US Pat. No. 8,878,682 B2, to apply the diffraction method. Achieve optical projection. The laser beam of infrared light is transmitted through two diffractive optical elements (DOE) to project a projected light field with an optical speckle distribution. One of the diffractive optical elements is designed by the computer generated hologram (CGH) method to generate the optical code of the spot, and the other diffractive optical element is designed as a two-dimensional periodic grating. The rectangular spot light field is reproduced as a 3 x 3 pattern to join a wider spot light field distribution. The two diffractive optical elements and the infrared laser diode are integrated into a projection module having a miniaturized size, and a visible light image capturing device and an infrared image capturing device respectively capture the image of the object to be tested and The structured light pattern reflected from the surface of the object to be tested can obtain the depth distance of the surface of the object to be tested through the off-axis geometric relationship between the object plane, the reference plane and the infrared image sensor. . The invention has the characteristics of simple architecture, compact size and low cost. However, the measurement resolution is a key point; since the depth calculation is based on the imaging position variation of the observed structural light pattern on the image pickup, the system usually uses a fixed focal length image capture device, so the structured light Pattern position variation is easily changed due to blurry images caused by factors such as defocus or distortion. The measurement resolution that the invention can provide is only about 1.5 cm, so it can only be used in system applications that do not require high measurement accuracy, such as gesture detection, object tracking or scene reconstruction.

Based on structured light and diffraction optical projections, the present invention proposes an innovative depth measurement architecture. The architecture uses an optical projector with a full-frame computer to project a periodic structured light field and form a coaxial optical triangulation relationship that allows the image capture device to capture the object under test. The spatial frequency of the surface structured light pattern changes with the object depth distance. By converting the captured image into a frequency domain, the corresponding depth distance can be calculated. The architecture is less sensitive to image quality and provides a higher measurement resolution.

The invention comprises a projection light field design for generating structured light, an optical structure between the projection and image capturing module, and a spatial frequency conversion and depth calculation of the captured image. A variety of different design architectures are proposed according to the content to achieve the purpose of stereo depth measurement. The detailed content and technology are as follows:

10‧‧‧Laser Projector

11‧‧‧Laser light source

12‧‧‧Computer full picture

13‧‧‧Projected beam

14‧‧‧Splitting lens

15‧‧‧ reference plane

16‧‧‧Determination

17‧‧‧Structural light pattern

18‧‧‧Imager imager

19‧‧‧Return beam

20‧‧‧Image Imager

21‧‧‧Image range taken

22‧‧‧ absorbing screen

23‧‧‧Image capture and arithmetic unit

30‧‧‧frequency distribution

31‧‧‧DC component

32‧‧‧Basic frequency

Figure 1 is a description of the basic inventive concepts and principles of the present invention.

Figure 2 illustrates the line density of the structured light pattern captured by the reflection planes of two different depth distances.

Figure 3 illustrates the spectral distribution of the periodic pattern after fast Fourier transform in the horizontal direction.

Figure 4 is a schematic view showing the structure of the first embodiment of the present invention.

Figure 5 is a schematic view showing the structure of a second embodiment of the present invention.

Figure 6 is a schematic view showing the structure of a third embodiment of the present invention.

Figure 7 is a schematic view showing the structure of a fourth embodiment of the present invention.

Please refer to "1st figure", which is a basic creative concept and principle description of the present invention. A laser projector 10 is placed on the optical axis 01. The laser projector 10 can project a projection beam 13 having a periodic structured light pattern 17 onto the reference surface 15 or the surface 16 to be tested. The diffraction angle of the beam is θ _p , and the distance between the laser projector 10 to the reference surface 15 and the surface 16 to be tested is L _o and L _{k , respectively} . An image pickup 20 having a focal length f is placed coaxially before the laser projector 10, and the angle of view of the return beam 19 reflected by the surface of the object is θ _c , which determines the structured light pattern 17 The image range 21 that is captured after being reflected by the reference surface 15 or the surface 16 to be tested. The distance between the image pickup 20 and the reference surface 15 and the surface 16 to be tested is D _o and D _{k , respectively} . Since θ _{c is} not equal to θ _p , the structured light pattern 17 captured by the image capture unit 20 will have different line periods and distributions for the reflection planes of different depth distances. Please refer to "Fig. 2", which illustrates the line density of the structured light pattern 17 taken by the reflection planes of two different depth distances, the left side is the structured light pattern 17 taken at the front reference surface 15, and the right side is The structured light pattern 17 at the later face 16 to be tested is taken. Depending on the geometric relationship, it is clear that the right pattern has a higher line density (i.e., a higher spatial frequency) than the left pattern. This feature constructs the theoretical basis of the present invention and can derive the mathematical relationship between the spatial frequency of the surface to be measured and the relative depth distance Δ z .

Where p _o and p _k are the line widths of the structured light pattern 17 projected on the reference surface 15 and the surface 16 to be tested, respectively. d _o and d _k are the line imaging widths respectively corresponding to the image capturing surface 18 of the image pickup. Since the projection beam 13 projected by the laser projector 10 has a fixed diffraction angle, there is a fixed proportional relationship between the line width corresponding to different projection plane positions and the projection distance.

Combining equations (1) to (3), the relationship between the line imaging widths do and dk can be obtained, If the line width on the image capturing surface 18 of the image pickup unit is directly reciprocal to obtain the spatial frequency, the function of the depth distance Δ z as a variable can be obtained as follows.

Wherein F _o and F _k represent the spatial frequency values of the reference surface 15 and the surface 16 to be tested, respectively, at the imaging surface 18 of the image capturer. By the formula (5) can be seen, when the laser projector 10, the image 15 from the imaging surface 20 and the reference configuration is fixed, changing the spatial frequency is only related to the variation in the depth distance △ z. Compared to other conventional depth measuring devices that employ the principle of triangulation, the measurement results of the present invention are completely unaffected by the focal length of the imager. The depth distance change value can be further expressed as

In practical applications, a fast Fourier transform (FFT) method can be used to calculate a digital image with a periodic distribution of image intensities. Please refer to "Fig. 3", which is an example of illustration. The right picture shows the spectral distribution of the left periodic pattern in the horizontal direction after fast Fourier transform. A few sharp frequency distributions 30 can be seen in the figure. The frequency of the main analysis of the present invention is the first lobe on the right side of the central DC component 31, which is the structured light pattern 17 within the captured image range 21. The basic frequency is 32.

Based on the above principles, we have achieved the creation of the present invention with the best optical architecture. Please refer to "FIG. 4", which illustrates a first embodiment of the present invention. a laser light Source 11 (the laser light source can be in the wavelength range of visible light or infrared light, and the laser is only the representative of the best light source, and other light sources other than any laser can also be used by the present invention) and the computer full image 12 A laser projector 10 that projects a projected beam 13 of a periodic structured light pattern 17 that is surrounded by a computer hologram 12. A beam splitting lens 14 is placed on the path of the projected beam 13 to prevent the projected beam 13 from being blocked by the image picker 20. A light absorbing screen 22 is placed on the path of the projected beam 13 reflected by the beam splitting lens 14 for absorbing reflected light that may cause background noise or image misjudgment. The projected beam 13 after passing through the spectroscopic lens 14 can be projected onto the surface of any object, such as the reference surface 15 for calibration or the actually measured surface 16 to be measured. An image finder 20 is placed on a path perpendicular to the optical axis 01 for collecting the return beam 19 reflected by the surface of the object. The image finder 20 is coupled to an image capture and operation unit 23, the unit 撷Taking the structural light pattern 17 captured by the image capturing device 20, after the preliminary image processing, the spatial frequency calculation and analysis are applied, and finally, the depth distance between the surface to be tested and the reference surface can be obtained according to the variation of the spatial frequency. The difference value is Δz. The image capturing and calculating unit 23 must previously select a reference surface 15 as a measurement reference, and obtain a true mathematical quantization relationship between the depth distance change amount and the spatial frequency change amount via appropriate zeroing and correction.

The foregoing first embodiment is only the main representative of the present invention, and the relationship between the position and the geometric position of the laser projector 10 and the image capturing device 20 can be variously changed. Please refer to FIG. 5, which is a second embodiment of the present invention, illustrating that the two-element position exchange does not affect the depth measurement effect disclosed in the present invention.

Although the present invention places the laser projector 10 and the image pickup 20 in a coaxial manner, the difference in field angle between the projected beam 13 and the return beam 19 is utilized to obtain the characteristic that the spatial frequency of the captured image changes with depth. However, this feature is not limited to the coaxial optical architecture, any The core architecture of the present invention can be achieved by an optical architecture that achieves a difference in field angle between the projected beam 13 and the returned beam 19. Please refer to FIG. 6 , which is a third embodiment of the present invention, which illustrates a non-coaxial optical architecture. The laser projector 10 and the image capturer 20 are not on the same optical axis, but by position. With the deployment, the effect of the spatial frequency of the captured image with the depth distance can still be obtained.

The foregoing three embodiments all employ a laser projector 10 in combination with an image capture unit 20. However, a wider field of view measurement range may be required in some applications, so a single laser projector 10 may not be sufficient to project a sufficient range of structured light patterns 17, two or more laser projectors 10 must be employed. . Please refer to FIG. 7 , which is a fourth embodiment of the present invention, which uses two laser projectors 10 and an image capturing device 20 to expand a wider projection range to achieve a wider field of view. The purpose of the measurement range.

According to the basic spirit of the present invention, in addition to exposing the design structure of the stereo depth measuring device, the design architecture can be promoted to various stereo sensing related applications without being limited by the use of distance measurement.

While the invention has been described above in terms of a preferred embodiment, it is not intended to limit the invention, and the invention may be modified and modified without departing from the spirit and scope of the invention. The scope of protection is subject to the definition of the scope of the patent application.

10‧‧‧Laser Projector

11‧‧‧Laser light source

12‧‧‧Computer full picture

13‧‧‧Projected beam

14‧‧‧Splitting lens

15‧‧‧ reference plane

16‧‧‧Determination

17‧‧‧Structural light pattern

19‧‧‧Return beam

20‧‧‧Image Imager

22‧‧‧ absorbing screen

23‧‧‧Image capture and arithmetic unit

Claims (8)

A measuring device capable of measuring a depth distance of a front object, comprising: a beam projection unit having at least a light source and a diffractive optical element, the diffractive optical element diverging and projecting the light beam emitted by the light source Going to the object to be tested in front, and causing the intensity of the projected beam to be periodically distributed, so that the projection projected onto the object to be tested has a structural pattern; a light splitting unit can penetrate the light beam emitted by the beam projecting unit, and The returning light beam reflected by the object to be tested is reflected by the light splitting unit; an image capturing unit is configured to receive the reflected light beam of the object to be tested reflected by the light splitting unit, and image the projection pattern on the object to be tested into the image capturing unit On the imaging surface, the image capturing unit and the beam projecting unit have a common optical axis but a geometric relationship of different light field angles; an image capturing and computing unit is used to capture the image collected by the image capturing unit, and the image is captured Calculate the spatial frequency. For example, in the measuring device described in claim 1, the light absorbing screen can be placed on the path of the reflected beam of the beam splitting unit to eliminate the interference caused by unnecessary stray light. The measuring device according to claim 1, wherein the beam of the beam emitted by the beam projecting unit can be in the visible or infrared band. The measuring device according to claim 1, wherein the light source of the beam projecting unit may be a laser element or a light emitting diode element. A measuring device that measures the depth distance of a front object, including: a beam projection unit having at least a light source and a diffractive optical element, the diffractive optical element diverging and projecting the light beam emitted from the light source to the object to be tested, and causing the intensity of the projected beam to be periodic Distributing so that the projection projected onto the object to be tested has a structural pattern; a light splitting unit can reflect the light beam emitted by the beam projecting unit, and cause the return beam reflected by the object to be tested to penetrate the light splitting unit; The image capturing unit is configured to receive the reflected light beam of the object to be tested that penetrates the light splitting unit, and image the projection pattern on the object to be tested on the imaging surface of the image capturing unit, the image capturing unit and the beam projecting unit have a common optical axis However, the geometric relationship of different light field angles; an image capturing and computing unit for capturing images collected by the image capturing unit and calculating the spatial frequency of the image. For example, in the measuring device described in claim 5, a light absorbing screen can be placed on the path of the beam splitting of the beam splitting unit to eliminate interference caused by unnecessary stray light. A measuring device capable of measuring a depth distance of a front object, comprising: a beam projection unit having at least a light source and a diffractive optical element, the diffractive optical element diverging and projecting the light beam emitted by the light source Going to the object to be tested in front, and causing the intensity of the projected beam to be periodically distributed, so that the projection projected onto the object to be tested has a structural pattern; an image capturing unit for receiving the reflected beam of the object to be tested, The projection pattern on the object to be tested is imaged on the imaging surface of the image capturing unit, the image The image capturing unit and the beam projecting unit exhibit geometric relations of different optical axes and different light field angles; an image capturing and calculating unit is used for capturing images collected by the image capturing unit, and calculating the spatial frequency of the image. A measuring device capable of measuring a depth distance of a front object, comprising: at least two beam projecting units, each beam projecting unit having at least one light source and a diffractive optical element, wherein the diffractive optical element can emit the light source The emitted beam is diverged and projected to the object to be tested in front, and the intensity of the projected beam is periodically distributed so that the projection projected onto the object to be tested has a structural pattern, and the at least two beam projection units are presented side by side. The geometrical arrangement relationship is such that the projected structural pattern has a large distribution range; an image capturing unit is configured to receive the reflected light beam of the object to be tested, and image the projection pattern on the object to be tested into the image capturing unit. The image capturing unit and the at least two beam projecting units respectively exhibit different optical axes and geometric relationships of different light field angles; and an image capturing and calculating unit for capturing images collected by the image capturing unit, The image is calculated as a spatial frequency.