TWI630431B - Device and system for capturing 3-d images - Google Patents

Abstract

The present invention provides an apparatus and system for capturing a stereoscopic image. The device for capturing a stereoscopic image comprises a time-of-flight image capturing module and a lighting module, wherein the lighting module generates a structural light source and an illumination light source, and the time-of-flight image capturing module captures the structure light source and the illumination light source projected onto the object. The first and second reflected light sources further obtain stereoscopic image information of the object. Through integrated time-of-flight (TOF) and structured light technology, the present invention can obtain stereoscopic image information with high depth accuracy and low noise, while reducing the overall volume and power consumption of the device and system.

Description

Device and system for capturing stereoscopic images

The present invention relates to an image information capturing device and system, and more particularly to an apparatus and system for capturing a stereoscopic image.

Stereoscopic scanning technology has been widely used in industrial fields and daily life, such as stereo scanning, monitoring and identification, and depth cameras. Stereoscopic scanning technology is mainly used to detect and analyze the geometric structure and appearance data of objects or environments, and use the detected data to perform three-dimensional operations to simulate and establish digital models of objects and environments.

The existing stereo scanning technology includes the use of time-of-flight (TOF), stereoscopic (Stereo Vision) or structured light (Structured Light) techniques for capturing and computing stereoscopic image data. In the foregoing stereo scanning technology, the time-of-flight method calculates the time between the object to be tested and the image capturing module by calculating the time when the light source is projected onto the surface of the object to be tested and then reflected from the surface of the object to be tested to reach the sensor. The distance is obtained, thereby obtaining stereoscopic image data of the object to be tested. The advantage of the time-of-flight method is that the time-of-flight method can be achieved with a system with low complexity compared to stereo vision technology and structured light technology using triangulation. For example, the time-of-flight image capturing module may include a light-emitting element disposed close to each other and an optical unit (eg, a lens) for focusing the reflected light without using triangulation, that is, no consideration is needed. The time-of-flight image captures the distance between the module and the structural light source (baseline), and the angle between the reflected light reflected by the structured light source and the reference line. In addition, the time-of-flight image capture module captures complete shots in a single shot (projection-shooting) Image information, so it is suitable for real-time applications. However, the depth information (Depth Noise) of the image information obtained by the time-of-flight method is large, and the accuracy of the generated image data is low.

On the other hand, the structured light technology is to project a specific figure to the object to be tested, and then draw a three-dimensional image of the surface of the object to be tested through the sensor, and finally calculate the three-dimensional coordinates of the object to be tested by using the principle of triangulation. Although structured light technology can obtain low depth noise and high image data accuracy, the calculation of 3D Cloud Point by structured light technology is relatively time consuming.

Therefore, there is still a need to provide a solution to combine the advantages of both the time-of-flight method and the structured light to provide a high-accuracy precision with a small volume and low power consumption for capturing stereoscopic images. system.

In order to solve the above technical problem, according to one embodiment of the present invention, an apparatus for capturing a stereoscopic image includes a time-of-flight image capturing module and a lighting module. The time-of-flight image capturing module is configured to capture a stereoscopic image information of at least one object. The light emitting module is adjacent to the time-of-flight image capturing module, and the light emitting module generates a structured light source and an illumination light source that are directed to the object. The structured light source is reflected by the object to form a first reflected light source. The illumination source is reflected by the object to form a second reflected source. The time-of-flight image capturing module captures the first reflective light source and the second reflective light source to obtain the stereoscopic image information of the object.

According to another embodiment of the present invention, a system for capturing a stereoscopic image is provided for capturing image information of at least one object, the system comprising a processor module electrically coupled to the processor module A flight time processing module, a lighting module, and a time-of-flight image capturing module electrically connected to the time-of-flight processing module. The illumination module is configured to generate a structured light source directed to the object and an illumination source directed to the object, and the structured light source generates a structured light information by reflection of the object, and the illumination source passes The inverse of the object Shoot to generate an illumination light information. The time-of-flight image capturing module is configured to capture the structured light information and the illumination light information. The structured light information captured by the time-of-flight image capturing module is calculated by the processor module to obtain a structured light stereo point cloud. The illumination light information captured by the time-of-flight image capturing module is calculated by the processor module to obtain a time-of-flight stereo point cloud. The structured light stereo point cloud and the time-of-flight stereo point cloud may be combined into a stereo depth map for providing the image information of the object.

The main technical means of the present invention is to generate a structured light source and an illumination source in a single image capture program, and use a time-of-flight image capture module to capture the surface of the object to be tested reflected by the structured light source and the illumination source. The first reflective light source and the second reflective light source can combine the advantages of both the time-of-flight method and the structured light technology to obtain stereoscopic image information of the object to be tested with low depth noise and high depth precision. In addition, the apparatus and system for capturing stereoscopic images of the present invention have a small overall volume and consume less energy for operating energy.

For a better understanding of the features and technical aspects of the present invention, reference should be made to the accompanying drawings.

D‧‧‧Device for capturing stereoscopic images

1‧‧‧Time-of-flight image capture module

11‧‧‧Light receiving surface

12‧‧‧Switch Module

13‧‧‧Lens module

14‧‧‧Time-of-flight sensing components

2‧‧‧Lighting module

21‧‧‧Light generating unit

211‧‧‧Laser Generator

2111‧‧ ‧collimator

212‧‧‧Optical components

2121‧‧‧Splitting components

2122‧‧‧Reflective components

213‧‧‧Lighting elements

22‧‧‧Light diffractive components

221‧‧‧Glossy

23‧‧‧Light diffusing elements

231‧‧‧Glossy

S‧‧‧System for capturing stereoscopic images

51‧‧‧ memory unit

52‧‧‧Processor Module

53‧‧‧Time-of-flight processing module

54‧‧‧First lighting module

55‧‧‧Second lighting module

56‧‧‧Time-of-flight image capture module

57‧‧‧Computation unit

58‧‧‧Lighting module

L11‧‧‧Laser light source

L12‧‧‧projection light source

SL‧‧‧Structural light source

IL‧‧‧Light source

R1‧‧‧first reflected light source

R2‧‧‧second reflected light source

O‧‧‧ objects

BA‧‧‧Base axis

BL‧‧‧ baseline

θ‧‧‧Triangulation angle

1 is a schematic diagram of an apparatus for capturing a stereoscopic image according to a first embodiment of the present invention; FIG. 2 is a schematic diagram of an apparatus for capturing a stereoscopic image according to a second embodiment of the present invention; 3 is a schematic diagram of a device for capturing a stereoscopic image provided by the embodiment; FIG. 4 is a flowchart of a method for capturing a stereoscopic image provided by the present invention; FIG. 5A and FIG. 5B are diagrams for capturing a stereoscopic image provided by the present invention. A schematic diagram of a system of images; and FIG. 6 is an operational flow of a system for capturing stereoscopic images provided by the present invention Figure.

The following is a description of embodiments of the present invention relating to "apparatus, method, and system for capturing stereoscopic images" by specific specific examples, and those skilled in the art can understand the advantages and effects of the present invention from the disclosure of the present specification. The present invention can be implemented or applied in various other specific embodiments, and various modifications and changes can be made without departing from the spirit and scope of the invention. In addition, the drawings of the present invention are merely illustrative and are not intended to be described in terms of actual dimensions. The following embodiments will further explain the related technical content of the present invention, but the disclosure is not intended to limit the technical scope of the present invention.

[First Embodiment]

First, please refer to Figure 1. FIG. 1 is a schematic diagram of an apparatus for capturing a stereoscopic image according to a first embodiment of the present invention. The device D for capturing a stereoscopic image according to the first embodiment of the present invention includes a time-of-flight image capturing module 1 and a lighting module 2. The time-of-flight image capturing module 1 includes at least a lens module 13 and a time-of-flight sensing unit 14. The time-of-flight image capturing module 1 is stereoscopic image information for capturing at least one object O. The lighting module 2 is adjacent to the time-of-flight image capturing module 1. The setting distance between the light-emitting module 2 and the time-of-flight image capturing module 1 depends on the image resolution to be achieved, and the distance between the device D for capturing the stereo image and the object O to be measured. The content will be explained later.

As described above, the light-emitting module 2 includes the light generating unit 21, the light diffraction element 22, and the light diffusing element 23. The light-emitting module 2 is a structure light source SL for generating the object O and the illumination light source IL light-generating unit 21 includes a light generator, and may further include other optical elements such as a lens. The kind of the light generator of the light generating unit 21 is not limited herein. The light generating unit 21 generates Coherent Light. Therefore, as long as the above object can be achieved, the selection and setting of the internal components of the light generating unit 21 can be added. To adjust. For example, the light generator of the light generating unit 21 is a laser generator 211 for generating the laser light source L11. For example, a light generator that produces infrared light can be used. In addition, other light-emitting elements may be used as the light generator of the light-generating unit 21, such as a light emitting diode (LED), and a "narrow bandwidth" bandpass filter ("Narrow Bandwidth" Band Pass Filter). The light source generated by the light-emitting diode is converted into the same dimming light. Preferably, the light generating unit 21 comprises a laser generator 211, so that a light source having a single wavelength can be directly generated without using a band pass filter, and is sufficient to provide sufficient energy for performing stereo image acquisition. . In the present embodiment, the light generating unit includes a laser generator 211. In addition, when the laser generator 211 is used, the laser generator 211 may have at least one collimator 2111.

As described above, the light diffractive element 22 and the light diffusing element 23 are respectively used to convert the coherent light generated by the light generating unit 21 into the structural light source SL and the illumination light source IL. In other words, the structured light source SL generated by the light-emitting module 2 is formed by the conversion of the light diffraction element 22, and the illumination light source IL generated by the light-emitting module 2 is formed by the conversion of the light diffusing element 23. . In addition, the optical diffractive element 22 and the light diffusing element 23 and the light generating unit 21 may further include other optical components. For example, the optical diffractive component 22 and the light diffusing component 23 and the light generating unit may include a lens. For example, a focusing lens.

Light diffractive element 22, also known as a Diffractive Optical Element (DOE), can be a full image, a grating, or other suitable optical component. The light diffractive element 22 is configured to generate a two-dimensional code using a light source through the microstructure of its surface. In the embodiment of the present invention, the light diffractive element 22 converts the coherent light generated by the light generating unit 21 into a structured light source SL projected onto the surface of the object O (or an environment composed of a plurality of objects O). The light diffusing element 23 is for diffusing the homology light generated by the light generating unit 21 on average, thereby generating an illumination light source IL projected onto the surface of the object O to be measured. In order to obtain complete stereoscopic image information of the object O or the environment to be tested, the light diffusing element 23 converts the homology light generated by the light generating unit 21 into a complete coverage. Measure the object O or the illumination source IL of the environment.

It is worth mentioning that the light generating unit 21 used in the embodiment of the present invention has a structural composition with a low complexity, which is small in size and volume and can be applied to small electronic products such as mobile phones and the like. In addition, the light generating unit 21 only needs to use a simple electronic control system to control the laser generator 211 to generate a pulsed light source signal. The embodiment of the present invention does not need to use a complicated panel and set peripheral circuits for use. The mode in which the light generating unit 21 emits light is controlled, and therefore, in addition to reducing the volume of itself, the process and production cost can be reduced.

In the first embodiment, the light generating unit 21 is composed of a single light generator (laser generator 211). Therefore, in order to convert the laser light source L1 generated by the laser generator 211 through the light diffractive element 22 and the light diffusing element 23 into the structural light source SL and the illumination light source IL, the light generating unit 21 further includes an optical component 212 for The laser light source L1 is split. In other words, as shown in FIG. 1, the light generating unit 21 includes a laser generator 211 and an optical component 212, and the laser light source L11 generated by the laser generator 211 sequentially passes through the optical component 212 and the light diffraction component 22, The structured light source SL is formed, and the laser light source L11 sequentially passes through the optical component 212 and the light diffusing element 23 to form the illumination light source IL.

Specifically, the optical component 212 includes a beam splitting element 2121 and a light reflecting element 2122. For example, the beam splitting element 2121 is a Half Mirror having a specific reflectance/transmittance ratio for splitting the laser light source L11. Reflective element 2122 is a light directing element, such as a mirror. After the laser light source L11 is split, a part of the light beam passes through the light diffraction element 22 to form the structured light source SL, and the other part of the light beam is guided to the light diffusing element 23 by the reflection of the light reflecting element 2122, thereby generating the illumination light source IL. After the structured light source SL is projected on the surface of the object O, the first reflective light source R1 is formed by the reflection of the surface of the object O, and the illumination light source IL is projected on the surface of the object O and is reflected by the surface of the object O to form a second reflection. Light source R2.

Referring also to FIG. 1, the time-of-flight image capturing module 1 captures the first reflected light source. R1 and the second reflected light source R2 are used to obtain stereoscopic image information of the object O. In other words, by using the lens module 13 in the time-of-flight image capturing module 1 to capture the first reflected light source R1 generated by the structured light source SL, the stereoscopic image information of the object O can be obtained by spatial modulation. By capturing the second reflected light source R2 generated by the illumination source IL by using the lens module 13 of the time-of-flight image capturing module 1, the stereoscopic image information of the object O can be obtained by time modulation.

The setting manners of the time-of-flight image capturing module 1 and the light-emitting module 2 in the first embodiment of the present invention will be described in detail below. In the present invention, the arrangement of the time-of-flight module 1 and the illumination module 2 depends on the resolution and accuracy of the stereoscopic image information to be obtained, and the device D for capturing the stereoscopic image and the object O to be measured. Depending on the distance between them. Specifically, when the stereoscopic image information is captured by the structured light technology, the triangulation method is used for calculation. The triangulation method includes a known parameter, that is, a lateral distance between the time-of-flight module 1 and the emission point of the structural light source SL (ie, Baseline in the triangulation method, the reference line BL in the x direction shown in FIG. 1), and The angle between the first reflected light source R1 and the reference line BL (the triangular ranging angle θ shown in Fig. 1) is used to estimate the depth distance (z direction shown in Fig. 1). Therefore, as shown in FIG. 1 , in the present invention, the time-of-flight image capturing module 1 , the light diffractive element 22 , and the light diffusing element 23 are linearly arranged, and the light receiving surface 11 of the time-of-flight image capturing module 1 is arranged. The light exit surface 221 of the light diffraction element 22 and the light exit surface 231 of the light diffusing element 23 are all arranged along the same reference axis BA.

In view of the above, based on the consideration of the triangulation, generally, the farther the distance between the optical diffractive element 22 and the time-of-flight image capturing module 1 is, the higher the accuracy of the obtained stereoscopic information is. For example, when using the Gray Code algorithm, the distance between the light diffracting element 22 and the time-of-flight image capturing module 1 for the 5 Mega Pixel time-of-flight image capturing module 1 (the reference line BL) is 14 cm, and the triangulation angle θ is 20 degrees, and a resolution of 0.1 mm (mm) is obtained at a depth of 20 cm. In addition, when using the Phase Shift algorithm, or using the Speckle algorithm plus the phase shift algorithm, the solution is based on the degree of contrast between light and dark. The degree of resolution can be further improved. However, as the distance between the light diffractive element 22 and the time-of-flight image capturing module 1 increases, the overall volume of the device D for capturing a stereoscopic image is also larger. On the other hand, since the illumination light source IL generated by the conversion of the light diffusing element 23 is calculated by the principle of time modulation, in the present invention, between the light diffusing element 23 and the time-of-flight image capturing module 1 The distance is not limited here. In other words, as long as the range of the illumination source IL projected by the light-emitting surface 231 of the light-diffusing element 23 on the surface of the object O is sufficient to cover the visible area of the time-of-flight image capturing module 1, the light diffusing element 23 and the time-of-flight image 撷The distance between the modules 1 can be adjusted according to other parameters.

In summary, the apparatus D for capturing a stereoscopic image provided by the first embodiment of the present invention can project a structured light source by a total of two projection-shooting (projection-photographing) programs, that is, by the light diffractive element 22. SL, the first reflected light source R1 generated by the structured light source SL is captured by the time-of-flight image capturing module 1 , and the illumination light source IL is projected by the light diffusing element 23, and then captured by the time-of-flight image capturing module 1 The second reflected light source R2 generated by the illumination source IL obtains stereoscopic image information obtained by spatial modulation and time modulation, respectively. In addition, the order of the aforementioned two shot-taking procedures may be reversed, and the present invention is not limited thereto.

By integrating two different stereo scanning technologies, it is possible to combine the advantages of these stereo scanning technologies, that is, to perform close-range and high-precision stereo image acquisition using structured light, and to perform medium and long distances by time-of-flight method. The stereo scanning has been used to improve the depth accuracy (Depth Accuracy) when performing close-range stereo scanning with structured light.

Referring to FIG. 1 , the time-of-flight image sensing module 1 of the first embodiment of the present invention further includes a switching module 12 . The switching module 12 is configured to perform color texturing on the obtained stereoscopic image information. Specifically, the switching module 12 includes an infrared light bandpass filter, a visible light bandpass filter, and an optical switcher. The optical switch is used to guide the first reflective light source R1 and the second reflective light source R2 to the infrared light band pass filter or to direct the ambient light source to the visible light band pass filter. The optical switch can be, for example, a piezoelectric motor (Piezoelectric Motor), a voice coil motor (Voice Coil) Motor, VCM) or electromagnetic switch.

In this way, different types of image information can be obtained according to the nature of the band pass filter passed by the light source received by the lens module 14. For example, in the infrared light mode, the first reflective light source R1 and the second reflective light source R2 are guided to the infrared light bandpass filter by switching of the optical switch, the first reflective light source R1 and the second reflective light source R2. The bands other than the mid-infrared light are filtered, and the 2D structured light source SL and the illumination source IL reflected image are converted into black-and-white stereoscopic image information of the object O through spatial modulation and time modulation algorithms, respectively. In addition, in the visible light mode, the reflected light generated by the ambient light source irradiated to the object O is guided to the visible light band pass filter by the switching of the optical switch, and the band other than the visible light in the reflected light is filtered to obtain the object. O color image information. Specifically, in the visible light mode, the light-emitting module 2 for generating the structural light source SL and the illumination light source IL may be in a closed state, and the light switcher is an ambient light such as external natural light (sunlight) or an indoor light source (for example The light source that is irradiated to the object O is guided to the visible light band pass filter, and the visible light band pass filter filters the band other than the visible light in the reflected light to reach the color image information of the object O. Effect. In other words, in the visible light mode, there is no need to perform a light source projection operation through the light emitting module.

In other words, after performing the aforementioned total two-shot projection-projection (achieved by the structured light source SL and the illumination source IL, respectively), the ambient light source can be illuminated by the switcher in the switching module 12. The reflected light generated by the object O is guided to the visible light band pass filter, thereby completing the third color shooting process. The third shooting procedure can obtain the color image information of the object O. Therefore, through the integration and calculation of the data, the black and white stereoscopic image information obtained by the previous two shooting and shooting procedures can be obtained through the third shooting procedure. Information is converted into color stereoscopic image information.

In summary, by further adding the switching module 12 to the structure of the time-of-flight image sensing module 1, the black and white stereo image information can be pasted, and the time-of-flight image sensing including the switching module 12 is included. Module 1 can be used as both a TOF Camera and a Color Camera. With the same pixels and a common world coordinate, it is possible to achieve fast color registration and seamless color registration without the need for additional calibration procedures, while eliminating the need to use additional color cameras.

[Second embodiment]

Please refer to Figure 2. 2 is a schematic diagram of an apparatus for capturing a stereoscopic image according to a second embodiment of the present invention. The biggest difference between the second embodiment of the present invention and the first embodiment is that the design of the light-emitting module 2 is different. The differences between the second embodiment and the first embodiment will be described below, and the portions of the second embodiment that are substantially the same as those of the first embodiment will not be described again.

As shown in FIG. 2, in the second embodiment, the light generating unit 21 includes a laser generator 211 and a light-emitting element 213, and the laser light source L11 generated by the laser light source 211 is converted into a structure light source SL by the light diffraction element 22. And the projection light source L12 generated by the light-emitting element 213 is converted into the illumination light source IL by the light diffusion element 23. In other words, unlike the first embodiment, the second embodiment utilizes two independent light generators to respectively generate a laser light source L11 and a projection light source L12 for generating the structured light source SL and the illumination light source IL.

Specifically, since the structural light source SL must be generated by the same dimming, in the second embodiment, the laser generator 211 is selected, and the collimator 2111 is used to generate the laser light source L11. On the other hand, the kind of the light-emitting element 231 for generating the illumination light source IL is not limited here. For example, the light emitting element 231 can be an LED light emitting element.

Referring also to FIG. 2, the light diffractive element 22 and the light diffusing element 23 are disposed on the same side of the time-of-flight image capturing module 1, and the light diffusing element 22 is disposed on the light diffractive element 23 and the time-of-flight image capturing mode. Between group 1. As described in the first embodiment, the time-of-flight image capturing module 1, the light diffractive element 22, and the light diffusing element 23 are linearly arranged, and the light receiving surface 111 of the time-of-flight image capturing module 11 and the light diffraction The light-emitting surface 221 of the element 22 and the light-emitting surface 231 of the light-diffusing element 23 are all arranged along the same reference axis BA, and the time-of-flight image capturing module 11 and the light diffraction The distance of the element 222 in the x direction (horizontal direction) is the reference line BL (Baseline) in the triangulation method.

[Third embodiment]

Please refer to Figure 3. FIG. 3 is a schematic diagram of an apparatus for capturing a stereoscopic image according to a third embodiment of the present invention. The biggest difference between the third embodiment of the present invention and the second embodiment is that the components of the light-emitting module 2 are arranged differently. Differences between the third embodiment and the second embodiment will be described below, and substantially the same portions of the third embodiment and the second embodiment will not be described again.

As shown in FIG. 3, the light generating unit 21 includes a laser generator 211 and a light-emitting element 213. The laser light source L11 generated by the laser light source 211 is converted into a structural light source SL by the light diffraction element 22, and the light-emitting element 213 is The generated projection light source L12 is converted into the illumination light source IL by the light diffusion element 23. In addition, the light diffractive element 22 and the light diffusing element 23 are respectively disposed on both sides of the time-of-flight image capturing module 1. In other words, the light diffractive element 22 and the light diffusing element 23 are respectively disposed on opposite sides of the time-of-flight image capturing module 1. It is to be noted that the relative positions between the light diffraction element 22, the light diffusing element 23, and the time-of-flight image capturing module 1 in the third embodiment and the foregoing second embodiment may be based on the desired stereoscopic image information. The accuracy, as well as other parameters in the process of fabricating the device D for capturing stereoscopic images, are adjusted.

In addition, the present invention also provides a method for capturing a stereoscopic image. Please refer to FIG. 4. FIG. 4 is a flowchart of a method for capturing a stereoscopic image provided by the present invention. The method for capturing a stereoscopic image provided by the present invention includes the following steps: generating a structured light source and an illumination light source directed to an object through a light emitting module (step S100); the structured light source and the illumination light source respectively pass through Reflecting the object to form a first reflected light source and a second reflected light source respectively (step S102); and capturing the first reflected light source by a time-of-flight image capturing module And the second reflective light source to obtain a stereoscopic image information of the object (step S104).

Please also refer to Figures 1 to 3. First, in step S100, the structured light source SL and the illumination light source IL that are directed to the object O are generated by the light-emitting module 2. The light-emitting module 2 can use the light-emitting modules 2 in the first to third embodiments described above, and the details thereof will not be described again.

Next, in step S102, the light source SL and the illumination light source IL are reflected by the surface of the object O, whereby the first reflection light source R1 and the second reflection light source R2 are formed, respectively. Finally, in step S104, the first reflected light source R1 and the second reflected light source R2 are captured by the time-of-flight image capturing module 1 to obtain stereoscopic image information of the object O. In addition, the method for capturing a stereoscopic image provided by the present invention may further comprise the step of coloring the stereoscopic image information. The manner in which the stereoscopic image information is pasted by the optical switch 12 is substantially the same as that described in the first embodiment.

Specifically, the time-of-flight image capturing module 1 may further include a switching module 12, where the switching module 12 includes an infrared light band pass filter, a visible light band pass filter, and an optical switch, and the optical switch is used for The first reflective light source R1 and the second reflective light source R2 are guided to the infrared light band pass filter, or the reflected light generated by the ambient light source is irradiated to the object O is guided to the visible light band pass filter. In the infrared light mode, both the first reflective light source R1 and the second reflective light source R2 are switched by the optical switch to be guided to the infrared light bandpass filter, and the first reflective light source R1 and the second reflective light source R2 pass the infrared light. The bandpass filter then reaches the time of flight sensing component 14 to produce black and white stereoscopic image information. In the visible light mode, the reflected light generated by the ambient light source illuminating the object O is switched by the optical switch to be guided to the visible light band pass filter, and the reflected light passes through the visible light band pass filter and then reaches the time-of-flight sensing element 14 to Get color image information. The color stereoscopic image information can be obtained by performing the calculation and processing using the black and white stereoscopic image information and the color image information.

The present invention further provides a system for capturing a stereoscopic image for capturing a stereoscopic image information of at least one object O. Please refer to FIG. 5A and FIG. 5B, and at the same time, as shown in FIG. 6. 5A and FIG. 5B are schematic diagrams of a system for capturing a stereoscopic image provided by the present invention, and FIG. 6 is a flowchart of operations of a system for capturing a stereoscopic image provided by the present invention. The system S for capturing stereoscopic images provided by the present invention comprises a memory unit 51, a processor module 52, a time-of-flight processing module 53, a lighting module 58, and a time-of-flight image capturing module 56.

First, please refer to FIG. 5A, which is also shown in FIG. The processor module 52 is electrically connected to the memory unit 51; the time-of-flight processing module 53 is electrically connected to the processor module 52; the light-emitting module 58 is electrically connected to the time-of-flight processing module 53; The module 56 is electrically connected to the time of flight processing module 53. For example, the light emitting module 58 corresponds to the light emitting module 2 in the first embodiment. The time-of-flight image capturing module 56 may correspond to the time-of-flight image capturing module 1 in the first embodiment.

As shown in FIG. 5A, the light-emitting module 58 can be controlled by the time-of-flight processing module 53 to generate a structured light source SL directed to the object O and an illumination source IL directed to the object O, respectively. The structured light source SL and the illumination light source IL respectively generate a structured light information and an illumination light information by reflection of the object O. The time-of-flight image capturing module 56 controls the flight time processing module 53 to extract structured light information and illumination light information.

In the above, the structured light information captured by the time-of-flight image capturing module 56 can be obtained by the processor module 52, and a structured light stereo point cloud of the object O can be obtained, and the time-of-flight image capturing module 56 is obtained. The acquired illumination light information can be obtained by the processor module 52, and a flight time stereo point cloud can be obtained, and the structured light stereo point cloud and the flight time stereo point cloud can be combined into a stereo depth map by operation. For example, the structured light stereo point cloud of the processor module 52 and the time-of-flight stereo point cloud are merged into a stereo depth map for providing image information of the object O by calculation by the microprocessor module 52. However, the invention is not limited thereto.

Specifically, the time-of-flight processing module 53 is used to modulate the illumination source IL generated by the illumination module 58 (ie, to provide a modulation signal for time modulation), and at the same time, according to the foregoing, for the modulation illumination module. The illumination source IL of 58 is used to de-modulate the sensor of the time-of-flight image capture module 56. The information captured by the time-of-flight image capturing module 56 and the information calculated by the processor module 52, including the structured light stereo point cloud, the flight time stereo point cloud, and other information, can be stored in the memory. Within the unit. In other words, the information captured by the time-of-flight image capturing module 56 can be temporarily stored in the memory unit and sent back to the processor module 52 for processing and calculation. In addition, the system S for capturing a stereoscopic image may further include a computing unit 57 electrically connected to the micro processing module 52 for correcting the stereoscopic depth map. The processor module 52 also controls the time-of-flight image capturing module 56 and the lighting module 58 as a whole to avoid interaction timing interference between the components.

Next, please refer to FIG. 5B, and refer to FIG. 2 and FIG. 3 at the same time. As shown in FIG. 5B, the system S for capturing a stereoscopic image includes a memory unit 51, a processor module 52, a time-of-flight processing module 53, a first illumination module 54, a second illumination module 55, and a time-of-flight image. The module 56 is captured. The processor module 52 is electrically connected to the memory unit 51; the time-of-flight processing module 53 is electrically connected to the processor module 52; the first lighting module 54 is electrically connected to the processor module 52; The group 55 is electrically connected to the time-of-flight processing module 53; and the time-of-flight image capturing module 56 is electrically connected to the time-of-flight processing module 53. The first lighting module 54 is controlled by the processor module 52 for generating the structural light source SL directed to the object O. The structured light source SL generates a structured light information by reflection of the object O. The second lighting module 55 is controlled by the time-of-flight processing module 53 for generating the illumination source IL directed to the object O. The illumination source IL generates an illumination light information by reflection of the object O. The time-of-flight image capturing module 56 controls the flight time processing module 53 to extract structured light information and illumination light information.

For example, the first light emitting module 54 and the second light emitting module 55 respectively correspond to The laser generator 211 and the light-emitting element 213 shown in FIGS. 2 and 3 are provided. The time-of-flight image capturing module 56 may correspond to the time-of-flight image capturing module 1 in the first embodiment to the third embodiment. In other words, the system for capturing stereoscopic images shown in FIG. 5B can employ the apparatus for capturing stereoscopic images of FIGS. 2 and 3. The difference between the system for capturing a stereoscopic image shown in FIG. 5B and FIG. 5A is that, in the system for capturing a stereoscopic image shown in FIG. 5B, the first illumination module 54 can directly pass through the processor module. Group 52 performs switching (On/Off switching). In other words, the first lighting module 54 does not need to be time modulated by the time of flight processing module 53. The other details of FIG. 5B are substantially the same as FIG. 5A and will not be described again here.

Please refer to FIG. 6 and refer to the contents of FIG. 1 to FIG. 3, FIG. 5A and FIG. 5B as needed. The program for capturing stereoscopic images can be performed in conjunction with the system for capturing stereoscopic images shown in FIGS. 5A and 5B. As shown in FIG. 6, first, the system S for capturing a stereoscopic image is operated in a black and white mode to obtain a black and white image information (step S200). In other words, step S200 is a system S for operating a stereoscopic image in an infrared light (IR) mode. Step S200 includes a program for obtaining a structured light stereoscopic point cloud through structured light (steps S2001, S2003, and S2005), and a program for obtaining a time-of-flight stereo point cloud by a time-of-flight method (steps S2002, S2004, and S2006). The order of the above two procedures can be adjusted as needed.

For example, the structured light source SL that is directed to the at least one object O may be generated first (step S2001), that is, the combination of the permeable light generating unit 21 and the light diffraction element 22 (including the light emitting module 58 or the first light emitting mode) Within group 54) a structured light source SL is produced. The structured light source SL is reflected by the object O to form the first reflected light source R1, and the time-of-flight image capturing module 56 captures the first reflected light source R1 to generate structured light information (step S2003). Next, the structured light information captured by the time-of-flight image capturing module 56 is calculated by the processor module 52 to obtain a structured light stereo point cloud (step S2005). So far, the program of capturing the stereoscopic image of the object O by the structured light has been completed.

Next, an illumination light source LL that is directed to at least one object O is generated (step S2002). That is, the combination of the light-transmitting unit 21 and the light diffusing element 23 (included in the light-emitting mode) The group 58 or the second lighting module 55 generates an illumination source IL. The illumination source IL is reflected by the object O to form a second reflected light source R2, and the time-of-flight image capturing module 56 captures the second reflected light source R2 to generate illumination light information (step S2004). Next, the illumination light information captured by the time-of-flight image capturing module 56 is calculated by the processor module 52 to obtain a time-of-flight stereo point cloud (step S2006). So far, the procedure for capturing the stereoscopic image of the object O by the flight time method has been completed.

Then, the structured light stereo point cloud and the flight time stereo point cloud are processed by the processor module 52 to be merged into a black and white stereo depth map (S2007). In this step, the information obtained by the close-range stereo scanning of the structured light and the information obtained by the time-of-flight and long-distance stereo scanning by the time-of-flight method are integrated. In this way, the structured light can be used to obtain high-precision stereo image acquisition at close range, and the medium-distance and long-distance stereo scanning can be performed by the time-of-flight method, and at the same time, the close-range stereo scanning with structured light is improved. Depth accuracy.

In the above, since the system S for capturing a stereoscopic image is operated in the infrared light mode, only the black and white stereoscopic depth map can be obtained. After performing the foregoing step S200, steps S202 and S204 can be performed, that is, operating in the color mode for capturing. The stereoscopic image system S obtains a color image information, and colors the black and white stereo depth map based on the Pixel information in the color image information to obtain a color stereo depth map. It is worth mentioning that since the pixel information in the color image information and the information obtained by using the structure light source SL or the illumination source IL are completely identical, the black and white stereo depth map can be accurately performed without external correction. Coloring. Step S202 can be accomplished by using the switching module 12 described in FIGS. 1 through 3. Specifically, step S202 includes projecting an object light source onto the object O, capturing the color reflected light reflected by the surface of the object O through the time-of-flight image capturing module 56, and guiding the light switch in the switching module 12. Color reflected light to visible light bandpass filter to obtain color image information of object O. Finally, in step S204, the black and white stereoscopic depth map obtained in step S200 is colored according to the color image information obtained in step S202 to obtain a color stereoscopic depth map.

In summary, the present invention has an advantageous effect that the present invention can obtain a stereo with high depth accuracy and low noise by integrating structured light and time-of-flight technology into a single device or system for stereoscopic image capture. Image information while reducing the overall size and power of the device and system.

Specifically, in the apparatus, method and system provided by the present invention, the stereoscopic image acquisition of the close-range and high-precision is performed by using the structured light, and the stereoscopic scanning of the intermediate distance and the long distance is performed by the time-of-flight method. In this way, the range of stereo scanning can be effectively increased, thereby increasing the application range of the product. In addition, since the apparatus and system of the present invention are small in size and low in structure complexity, they can be applied to smaller electronic products and can be operated under conditions of providing less energy. In detail, the device and system provided by the present invention can be combined with a simple component, that is, a small-volume laser module 211 with a collimator 2111 and a light diffraction module 22 (both are small-volume lenses). The structure light source SL is generated, wherein the laser module 211 only needs to provide a pulse signal without requiring a complicated control module for control, and the manufacturing cost can be greatly reduced. Furthermore, the time-of-flight method can be used to improve the depth accuracy (Depth Accuracy) when performing close-range stereo scanning with structured light.

In addition, the device, the method and the system provided by the present invention can realize the fast color registration by adding a switching module to the time-of-flight image capturing module without increasing the number of cameras and the size of the device and the system. And a seamless color effect.

The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Therefore, equivalent technical changes made by using the present specification and the contents of the drawings are included in the protection scope of the present invention. .

Claims (13)

An apparatus for capturing a stereoscopic image, comprising: a time-of-flight image capturing module for capturing a stereoscopic image information of at least one object; and a lighting module adjacent to the time-of-flight image capturing a module, wherein the light emitting module generates a structured light source and an illumination light source that are directed to the object; wherein the structured light source is reflected by the object to form a first reflective light source, and the illumination light source passes through the Reflecting the object to form a second reflected light source, and the time-of-flight image capturing module captures the first reflective light source and the second reflective light source to obtain the stereoscopic image information of the object . The device for capturing a stereoscopic image according to claim 1, wherein the light emitting module comprises a light generating unit, a light diffractive element and a light diffusing element, and the structure generated by the light emitting module The light source is formed by conversion of the light diffractive element, and the illumination light source generated by the light emitting module is formed by conversion of the light diffusing element. The apparatus for capturing a stereoscopic image according to claim 2, wherein the light generating unit comprises a laser generator for generating a laser light source and an optical component, the laser light source sequentially passing through The optical component and the light diffractive element are formed to form the structured light source, and the laser light source sequentially passes through the optical component and the light diffusing element to form the illumination light source. The apparatus for capturing a stereoscopic image according to claim 3, wherein the laser generator has at least one collimator, the optical component includes a beam splitting component and a light reflecting component, and the laser light source is sequentially Performing collimation of at least one of the collimators, splitting of the spectroscopic element, and conversion of the light diffracting element to form the structured light source, and the laser light source sequentially passes at least one of the collimating sources Collimation of the instrument, splitting of the spectroscopic element, reflection of the retroreflective element, and the light The conversion of the diffusing elements to form the illumination source. The apparatus for capturing a stereoscopic image according to claim 2, wherein the light generating unit comprises a laser generator and a light emitting element, and a laser light source generated by the laser generator passes the light A diffractive element is converted into the structured light source, and a projected light source generated by the light emitting element is converted into the illumination light source by the light diffusing element. The apparatus for capturing a stereoscopic image according to claim 2, wherein the time-of-flight image capturing module, the light diffractive element, and the light diffusing element are linearly arranged, and the time-of-flight image is A light receiving surface of the module, a light emitting surface of the light diffraction element, and a light emitting surface of the light diffusing element are all arranged along the same reference axis. The apparatus for capturing a stereoscopic image according to claim 6, wherein the light diffractive element and the light diffusing element are disposed on a same side of the time-of-flight image capturing module, and the light diffusing element And disposed between the light diffractive element and the time-of-flight image capturing module. The apparatus for capturing a stereoscopic image according to claim 6, wherein the light diffractive element and the light diffusing element are respectively disposed at two sides of the time-of-flight image capturing module. The apparatus for capturing a stereoscopic image according to claim 1, wherein the time-of-flight image sensing module further comprises a switching module, the switching module comprising: an infrared bandpass filter; a visible light a band pass filter; and an optical switch for guiding both the first reflected light source and the second reflected light source to the infrared light band pass filter or illuminating an ambient light source The reflected light generated by the object is guided to the visible light band pass filter; wherein, when the first reflective light source and the second reflective light source are switched by the optical switcher, the infrared light is guided to When the band pass filter is used, the first reflective light source and the second reflective light source pass through the infrared light bandpass filter to Generating a black and white stereoscopic image information; wherein, when the ambient light source illuminates the reflected light generated by the object through the switching of the optical switch to be guided to the visible light band pass filter, the environment The reflected light generated by the light source illuminating the object passes through the visible light band pass filter to obtain a color image information. A system for capturing a stereoscopic image for capturing image information of at least one object, the system comprising: a processor module; a time-of-flight processing module electrically coupled to the processor module a lighting module for generating a structured light source directed to the object and an illumination source directed to the object, wherein the structured light source is reflected by the object to generate a structured light information, and the illumination a light source is reflected by the object to generate an illumination light information; and a time-of-flight image capturing module is electrically connected to the time-of-flight processing module for capturing the structured light information and the The illumination light information; wherein the structured light information captured by the time-of-flight image capturing module is calculated by the processor module to obtain a structured light stereo point cloud; wherein the flight time The illumination light information captured by the image capture module is calculated by the processor module to obtain a time-of-flight stereo point cloud; wherein the structured light stereo point cloud and the fly Time can be combined into three-dimensional point cloud perspective providing the depth map for an image information. The system for capturing a stereoscopic image according to claim 10, wherein the light emitting module comprises a light generating unit, a light diffractive element and a light diffusing element, and the structure generated by the light emitting module The light source is formed by conversion of the light diffractive element, and the illumination source generated by the illumination module is formed by conversion of the light diffractive element. The system for capturing a stereoscopic image according to claim 10, further comprising a memory unit, wherein the structured light stereo point cloud, the time-of-flight stereo point The cloud and the stereo depth map are stored in the memory unit. The system for capturing a stereoscopic image according to claim 10, further comprising a computing unit electrically connected to the micro processing unit for correcting the stereoscopic depth map.