TW202119058A - Depth sensing device and method - Google Patents

Abstract

Depth sensing device and method are provided. The device includes a light emitting assembly, a light receiving assembly and a control unit. The light emitting assembly includes a laser source and a mirror, and the laser source includes one wavelength source or several sub-light sources to emit laser lights onto the mirror, so that the laser lights are projected to an object to be sensed. The light receiving assembly is optically coupled to the light emitting assembly to receive the reflected lights that the laser lights are reflected from the object. The control unit is electrically connected to the light emitting and light receiving assemblies and controls them according to the laser radar mode and/or structural light mode. The method includes steps of performing the laser radar mode and/or structural light mode. Therefore, in accordance with the distance of the object, the laser radar mode and/or structural light mode is selected to perform the depth sensing of the object.

Description

Depth sensing device and method

The present invention relates to a sensing device and method, and particularly to an object depth sensing device and method.

In applications such as face recognition, gesture recognition, object and environment modeling, depth sensing devices are used to sense the position and depth information of the object and the environment to obtain the corresponding point cloud data, thereby creating the object and the environment 3D model.

The depth sensing technology used by the device can be divided into three types: Stereo vision, Structured Light and Time of Flight (TOF). Each technology has its own advantages and disadvantages. For example, the advantage of stereo vision is that the hardware cost is low, but it is easily affected by ambient light to affect the depth accuracy of the measurement; the advantage of structured light is that it can obtain better depth accuracy, but it is limited by the structured light pattern projection The light intensity limits the working distance to be shorter; on the contrary, the advantage of the time-of-flight ranging is that the working distance is longer, but the depth accuracy at short distances is limited by the calculation resolution of the light flight time or the phase difference and is poor. Therefore, the industry will determine the appropriate depth sensing technology based on the characteristics and applications of the product.

In addition, some companies integrate different types of technologies into a product. For example, when the object to be measured is close, the product uses structured light technology for depth sensing, and when the object to be measured is far away, the product uses flight time Ranging technology to do depth sensing. In this way, the shortcomings between the technologies can be complementary, and better depth accuracy can be obtained at both short and long distances.

However, the above-mentioned product needs to include two different light emitting components to generate structured light patterns and laser light for time-of-flight distance measurement, respectively, so the volume and cost will increase. In addition, when the generated structured light pattern is a static speckle pattern, it is limited by the nano process technology that produces the speckle pattern element to achieve the laser light wavelength level, and it is difficult to obtain a better resolution; The generated laser light has a limited working distance due to energy divergence. Furthermore, this product usually uses a diffractive optical element (DOE) to generate a static structured light pattern, but if the process quality of the diffractive optical element is not good, it will easily affect the light uniformity of the structured light pattern, resulting in resolution Bad.

In view of this, how to improve the above shortcomings is a problem to be solved in the industry. It should also be noted that the above technical content is used to help the understanding of the problem to be solved by the present invention, and all or part of it is not necessarily disclosed or known in the art.

The purpose of the present invention is to provide a depth sensing device and a depth sensing method, which can select one of the structured light mode and the laser radar mode for depth sensing according to the distance between the object and the device. Moreover, the provided depth sensing device and method can use a single light emitting component composed of a laser light source and a galvanometer, that is, a structured light pattern in a structured light mode and a scanning light in a laser radar mode can be generated; It is known that compared with two light sources, the required volume of the device can be smaller, the cost is lower, and the power is saved. In addition, the depth sensing device and method can provide a dynamically changing structured light pattern to illuminate the object to improve the resolution of measurement

In one embodiment, the depth sensing device provided by the present invention may include: a light emitting component, including a laser light source, a light shaping element and a galvanometer that are optically coupled, and the laser light source can be controlled A continuous wave laser light and/or a pulsed laser light are emitted. The laser light source emits the laser light after passing through the light shaping member and then is reflected and scanned by the galvanometer, so that the laser light is projected to On an object under test; a light receiving component optically coupled with the light emitting component to receive the reflected light of the laser light reflected from the object under test; and a control unit with the light emitting component and the light receiving component The electrical connection is used to control the light emitting component and the light receiving component according to at least one of a structured light mode and a laser radar mode.

In one embodiment, the depth sensing device provided by the present invention may also include: a light emitting component, including a laser light source, a light shaping element and a galvanometer that are optically coupled, and the laser light source includes a plurality of The laser light source can be controlled to emit a continuous wave laser light or a pulsed laser light. The laser light source emits the laser light after passing through the light shaping element and then is reflected and scanned by the galvanometer to make the laser light Projected light onto an object under test; a light receiving component, including an image capture device and/or a light sensor optically coupled with the light emitting component, for receiving the laser lights from the object under test Reflected reflected light; and a control unit electrically connected to the light emitting component and the light receiving component for controlling the light emitting component and the light emitting component according to at least one of a structured light mode and a laser radar mode Light receiving components.

In an implementation aspect, the light receiving component may include an image capture device and/or a light sensor. When the light receiving component includes the image capturer and the light sensor, the light sensor can be disposed between the light emitting component and the image capturer.

In an implementation aspect, the light receiving component may include the image capturer; in the structured light mode, the control unit controls the image capturer to capture the structured light pattern on the object under test frame by frame ; In the laser radar mode, the control unit controls the image capture device to receive the reflected light reflected from the object to be measured line by line.

In an implementation aspect, the light-emitting time of the sub-light sources can be individually controlled.

In one embodiment, the depth sensing method provided by the present invention includes: Perform at least one of a laser radar mode and a structured light mode; wherein, when the laser radar mode is executed, a light emitting component is controlled to emit a pulsed laser light and/or a continuous wave laser light to an object to be measured Scan upward and control a light receiving component to receive the pulsed laser light and/or the reflected light of the continuous wave laser light reflected from the object to be measured to obtain the point cloud data of the object to be measured; wherein the structured light mode is executed When, control the light emitting component to emit the continuous wave laser light to form a set of structured light patterns on the object to be measured, and control the light receiving component to capture the structured light pattern to obtain another point cloud data of the object to be measured Wherein, the light emitting assembly includes a laser light source, a light shaping element and a galvanometer, wherein the light shaping element is disposed between the laser light source and the galvanometer, and the laser light source faces the light shaping element and The galvanometer emits the laser light.

In an implementation aspect, the laser radar mode can be executed first, and a distance between the object under test and the light receiving component is obtained; when the distance is judged to be less than a preset distance, the structured light mode is executed; When the distance is judged to be greater than the preset distance, the point cloud data obtained by the laser radar mode is used.

In an implementation aspect, the laser radar mode can be executed first, and multiple distances between the object to be measured and the light receiving component can be obtained; when one of the distances is less than the preset distance, and When the other is greater than the preset distance, the structured light mode is executed, and the point cloud data and the other point cloud data are merged.

In an implementation aspect, one of the laser radar mode and the structured light mode can be executed according to an input signal. The other of the laser radar mode and the structured light mode can also be executed according to another input signal. Then, the point cloud data and the other point cloud data can be merged.

In an implementation aspect, the laser radar mode and the structured light can be executed simultaneously Mode, and including: controlling the light emitting component to emit the pulsed laser light to scan on the object to be measured; and controlling the light receiving component to receive the reflected light of the pulsed laser light from the object to be measured to obtain the reflection The brightness of the light, the time when the reflected light comes back, and the structured light pattern of the reflected light.

In an implementation aspect, the light receiving component may include an image capture device and a light sensor; wherein, when the laser radar mode is executed, the light sensor is controlled to receive the reflected light, and the execution In the structured light mode, the image capture device is controlled to capture the structured light pattern.

In an implementation aspect, the light receiving component may include an image capture device; wherein, when the laser radar mode is executed, the image capture device is controlled to receive the reflected light line by line, and the structured light mode is executed When, control the image capture device to receive the structured light pattern frame by frame.

In an implementation aspect, the laser light source may include a plurality of sub-light sources, and the respective light-emitting times of the sub-light sources are controlled.

In order to let those skilled in the art know and apply the above-mentioned purposes, technical features and advantages, the following is a detailed description of several preferred embodiments of the invention and the accompanying drawings.

100: Depth sensing device

10: Light emitting components

11: Laser light source

111, 111A, 111B: sub-light source

12: Galvanometer

13: Light shaping parts

20: Optical receiving component

21: Image capture device

21S: Capture range

22: light sensor

30: control unit

200: Microprocessor

300: DUT

L, L1, L2: laser light

La: Laser beam

Ls: sub-beam

R: reflected light

D: distance

X: trace

S: structured light pattern

θ: tilt angle

S101~S107, S201~S209: steps

Figure 1 is a schematic diagram of a depth sensing device according to a preferred embodiment of the present invention.

FIG. 2A is a schematic diagram (side view) of the light emitting component of the depth sensing device according to the preferred embodiment of the present invention.

FIG. 2B is a schematic diagram (side view) of the light emitting component of the depth sensing device according to another preferred embodiment of the present invention.

Figure 2C is a schematic diagram (front view) of the laser light source shown in Figure 2B.

FIG. 3A and FIG. 3B are schematic diagrams of laser light of a light emitting device according to a preferred embodiment of the present invention, respectively.

4A and 4B are schematic diagrams of the laser radar mode executed by the depth sensing device according to the preferred embodiment of the present invention.

FIG. 5A is a schematic diagram of the structured light mode executed by the depth sensing device according to the preferred embodiment of the present invention.

5B to 5F are schematic diagrams of different structured light patterns, respectively.

Figures 6 and 7 are respectively a two-step flow chart of the depth sensing method according to the preferred embodiment of the present invention.

The following will specifically describe specific embodiments according to the present invention; however, without departing from the spirit of the present invention, the present invention can still be practiced in many different forms of embodiments, and the protection scope of the present invention should not be construed as being limited to what is stated in the specification By. In addition, the technical content of each implementation aspect in the above-mentioned invention content can also be used as the technical content of the embodiment or as a possible variation aspect of the embodiment. In addition, unless the context clearly indicates otherwise, the singular form "a" as used herein also includes the plural form. When the term "comprising" or "including" is used in this specification, it is used to indicate the existence of features, elements or components, etc. , Does not exclude the existence or addition of one or more other features, elements or components. In addition, the orientation (such as front, back, top, bottom, sides, etc.) are relative ones, which can be defined according to the state of use of the depth sensing device and method, rather than indicating or implying that the depth sensing device or method must be used. It has a specific orientation, is constructed or operated in a specific orientation; therefore, the orientation cannot be understood as a limitation of the present invention.

Please refer to Figure 1. In a preferred embodiment of the present invention, a depth sensing device 100 (hereinafter referred to as the device 100) is proposed, which can be installed in an electronic product (such as a mobile phone, a monitoring device, etc.) , As part of electronic products. The device 100 can execute a structured light mode and a laser radar mode to obtain depth information (ie point cloud data) of each part of the object to be measured (face, hand, environment, etc.) 300, and then generate (construct) A three-dimensional model (image) of the test object 300. The device 100 can be further electrically connected with other components of the electronic product, such as a microprocessor (chip) 200, so as to transmit the obtained distance or point cloud data to the microprocessor 200 to establish a three-dimensional model or make Identity recognition and other applications.

The device 100 may include a light emitting component 10, a light receiving component 20, and a control unit 30. The light receiving component 20 is adjacent to the light emitting component 10 (for example, located on the left and/or right side of the light emitting component 10), and both light Coupling, that is, the light emitted by the light emitting component 10 can be received by the light receiving component 20 after being reflected. Therefore, as long as this relationship is met, the light emitting component 10 and the light receiving component 20 are adjacent or optically coupled; in addition, the light emitting component 10 and the light receiving component 20 are not limited to be located on the same horizontal plane, and there is a step difference between the two. Optical coupling can still be achieved. The control unit 30 is electrically connected to the light emitting component 10 and the light receiving component 20, respectively, to control how the light emitting component 10 emits light and how the light receiving component 20 receives light; the control unit 30 can also be connected to the microprocessor 200 It is electrically connected to transmit the signal of the light receiving component 20 to the microprocessor 200 or receive the signal of the microprocessor 200 to control the light emitting component 10 and the light receiving component 20. The technical content of each element and how to sense the depth of the object to be measured 300 through these elements will be described in more detail below.

Please refer to FIGS. 2A to 2C. The light emitting assembly 10 may include a laser light source 11, a galvanometer 12, and a light shaping member 13. The laser light source 11 can emit laser light L, which preferably can be an infrared laser (invisible light), but is not limited to this. Laser light source can be one side EEL (Edge Emittine Laser) or Vertical Cavity Surface Emitting Laser (VCSEL: Vertical Cavity Surface Emitting Laser), etc. The laser light source 11 has two controllable output modes, namely continuous wave modulation (Continuous Wave Modulation) and pulse modulation (Pulse modulation), in other words, the laser light source 11 can be controlled to emit one of a continuous wave laser light and a pulse laser light, and the emission frequency of the continuous wave laser light can be adjusted.

Please refer to FIGS. 2A and 2B, where the laser light source 11 can use a single light source, which can emit a laser beam La with a specific wavelength, or use a plurality of sub-light sources 111, each sub-light source 111 The sub-beam Ls can be emitted, and then the laser beam La and the sub-beams Ls pass through the light shaping member 13 to form the laser light L; whether each sub-light source 111 emits light or not, and the light-emitting time can be individually controlled by the control unit 30 Therefore, the sub-light sources 111 can emit light in turn. The sub-light sources 111 can be arranged in a one-dimensional or two-dimensional array to emit sub-beams Ls from different positions. For the specific structure of the laser light source 11 and its sub-light sources 111, for example, please refer to the US Patent Application Publication No. US2019/0109436A1, but it is not limited thereto.

The galvanometer 12 is optically coupled with the laser light source 11, that is, it is arranged on the optical path of the laser light source 11. Therefore, the laser light beam La or the sub-beam Ls of the laser light source 11 can be reached by the laser light L shaped by the light shaping member 13 To the galvanometer 12. The galvanometer 12 is a microelectromechanical system (MEMS) scanning galvanometer, and can be a one-dimensional galvanometer that swings on one axis or a two-dimensional galvanometer that swings on two axes. The laser light L (laser beam La or sub-beam Ls) can be reflected on the galvanometer 12 to change the forward direction, and then projected out of the light emitting assembly 10. The swing angle of the galvanometer 12 can be controlled to make the laser light L project to a specific position. For the specific structure of the galvanometer 12, please refer to the US Patent Application Publication No. US2017/0044003A1, the US Patent No. 7,329,930, and the US Patent No. 9,219,219. The galvanometer 12 can also be a microelectromechanical scanning chip sold by the applicant. Not limited to this.

In other embodiments, the galvanometer 12 can be tilted with respect to the laser light source 11, and the tilt angle θ is greater than or equal to 45° and not greater than 60°, which enables the device 100 to have a larger sensing range and better Resolution and reduce the minimum detectable distance. At the same time, the light emitting assembly 10 as a whole is not tilted (only the inner galvanometer 12 is tilted), so the space occupied by the device 100 can be reduced, so that the device 100 can have a smaller overall volume.

The light shaping element 13 is arranged on the optical path of the laser light source 11 and between the laser light source 11 and the galvanometer 12, so that the laser beam La or sub-beam Ls emitted by the laser light source 11 needs to pass through the light shaping element 13 Arrive at the galvanometer 12. The light shaping element 13 may include optical elements such as lenses for adjusting the shape or angle of the laser beam La or sub-beam Ls of the laser light source 11, for example, collimating and shaping the laser beam La or the sub-beam Ls into a linear beam Or, the light shaping member 13 can be a diffractive element for adjusting the laser light L of the laser light source 11 to form multiple point emission or linear beams. If the laser beam La or the sub-beam Ls emitted by the laser light source 11 has the required shape and angle, the light shaping member 13 can be omitted.

On the other hand, as shown in Figures 3A and 3B, the laser light source 11 in this embodiment adopts a laser beam La emitted by a single light source, which forms a laser light L1 after passing through the light shaping member 13, and its energy is relatively concentrated The energy on both sides is lower than that of the center (that is, the energy on the outer side of the beam is smaller). In another embodiment, the laser light source 11 uses a plurality of sub-beams Ls emitted by the sub-light-emitting source 111. The light shaping member 13 forms a light beam L2, the energy distribution of the laser light L2 is relatively uniform, and a better resolution can be obtained. In addition, by firing a plurality of sub-beams Ls in turn, the number of emissions per unit time will be more than the conventional number of emissions, so more data can be obtained.

Please refer to Figure 1 again. The light receiving device 20 may include an image capturing device 21 and an optical sensor 22, both of which are located in the light emitting device. The two sides of the component 10 or the same side of the light emitting component 10. Preferably, since there is a need to maintain a basic distance between the light emitting component 10 and the image capturer 21, and the light sensor 22 is located between the light emitting component 10 and the image capturer 21, the two Maintaining a certain distance helps meet the above requirements and can reduce the size of the device 100.

The image capture device 21 may be a charge coupled device (CCD: Charge coupled device) or a complementary metal oxide semiconductor (CMOS) device. The image capture device 21 is mainly used to capture the structured light pattern (light spot) on the object 300 under the structured light mode. The light sensor 22 may include a silicon photomultiplier or a photodiode, etc., wherein the photodiode may be, for example, an avalanche photodiode (Avalanche Photodiode), a PIN photodiode, etc. (PIN Photodiode) or Single-Photon Avalanche Diode (SPAD) etc. The light sensor 22 is mainly used in the laser radar mode to receive the reflected light R reflected by the laser light L on the object 300 under test. In addition, the above-mentioned silicon photomultiplier tubes or photodiodes may be plural, which are arranged in a one-dimensional or two-dimensional array to increase the sensing resolution.

In another embodiment, the light receiving device 20 includes an image capture device 21 but does not include the photo sensor 22, and the image capture device 21 has two image capture methods. In the structured light mode, the image capturer 21 adopts a "frame by frame" capture method, using all or most of the pixel units of the image capturer 21 to capture one or more projections To the structured light pattern on the object under test 300; and in the laser radar mode, the image capturer 21 adopts a "line by line" capture method, using one of the lines of the image capturer 21 The pixels sense and receive the reflected light R reflected from the object 300 to be measured.

The control unit 30 may include a microcontroller and its corresponding peripheral components, which can execute a scanning mode to control the light emitting device 10 and the light receiving device 20. The scan mode can be The software (program) stored in the control unit 30 or software that can be read by the control unit 30 is executed.

The scanning mode includes a laser radar mode and a structured light mode.

As shown in Figures 2A, 2B, 4A and 4B, when the control unit 30 executes the laser radar mode, it can control the light emitting assembly 10 to emit pulsed laser light L (laser beam La or sub-beam Ls forming The laser light L) reaches the galvanometer 12, and is reflected and scanned by the galvanometer 12 to the object 300; the laser light L projected onto the object 300 may be a linear beam. As shown in Figure 4B, the control unit 30 can control the galvanometer 12 to swing along a single axis of rotation, so that the laser light L performs a one-dimensional scan on the object 300 (along the trace X), so that the laser light L The light spot moves to a different place on the object 300 to cover the object. In other implementations, the laser light L projected on the object to be measured 300 can be a concentrated light spot, and the galvanometer 12 can also be a two-dimensional galvanometer, by raster scan or Lissajous Lissajous scan is used to control the galvanometer 12 to swing along two mutually perpendicular rotating shafts, so that the laser light L performs a two-dimensional scan to cover the object 300 to be measured.

Next, the control unit 30 controls the light receiving assembly 20 to receive the reflected light R reflected from the laser light L from different places on the object 300 with the light sensor 22, and emits and receives the laser light L according to the brightness of the reflected light R and the laser light L By the time when the reflected light R returns, the distance between different parts of the object 300 to be measured relative to the light receiving assembly 30 (that is, TOF, Time of Flight) can be calculated, so as to obtain the point cloud and reflectance data of the object to be measured 300.

When the control unit 30 executes the laser radar mode, it may also adopt a frequency modulated continuous wave (FMCW: Frequency Modulated Continuous Wave) method to obtain point cloud data. That is, the sub-light source 111 controlled by the control unit 30 emits laser light L whose modulation frequency changes continuously and is reflected by the galvanometer 12 to the object under test 300 for scanning, and then the light sensor 22 receives the laser light L from the object under test 300 Reflected light R reflected back from different places on the upper surface, and changes according to the intensity and phase of the reflected light R, The distance, speed and reflectivity of the different parts of the object 300 relative to the light receiving assembly 30 are calculated.

Compared with the TOF method, FMCW can obtain the moving speed of the test object 300 at the same time, and is less affected by the interference of ambient light, which affects the measurement accuracy. In TOF, pulsed laser light L is used. The energy of pulsed laser light L is relatively concentrated and can be projected to a longer distance. Therefore, one of TOF and FMCW can be selected according to application conditions or requirements.

As shown in Figures 2A and 5A, when the control unit 30 executes the structured light mode, the laser light source 11 of the light emitting assembly 10 can be controlled to emit continuous-wave laser light L to the galvanometer 12, and to be reflected and scanned by the galvanometer 12 300 on the test object. The control unit 30 controls the galvanometer 12 to swing in a one-dimensional or two-dimensional scanning manner, so that the laser light L scans on the test object 300, and forms a set of structures on the test object 300 by controlling the brightness or intensity of the laser light L Light pattern S. As shown in Figures 5B to 5D, the structured light pattern S can be Binary code, Gray code, Fringe code, Dot code, etc., and these The structured light patterns S with different codes can be achieved by controlling the swing frequency of the galvanometer 12 and the light-emitting timing of the laser light source 11.

As shown in FIG. 5A, after the structured light pattern S is projected onto the test object 300, the control unit 30 controls the light receiving component 20 to use the image capture device 21 to capture the structured light pattern S within the capturing range 21S , And according to the algorithm corresponding to the structured light pattern S to calculate the depth of different places of the object 300 to be measured, so as to obtain the point cloud data of the object 300 to be measured. The control unit 30 can also enable the light emitting component 10 to project a set of structured light patterns S on the test object 300 according to the encoding type, that is, the structured light patterns S projected on the test object 300 will dynamically change, according to the figure The image capture device 21 sequentially captures the multiple groups of structured light patterns S, and obtains point cloud data with higher resolution according to the structured light patterns S.

On the other hand, as shown in Figures 2C, 5E, and 5F, when the light emitting assembly 10 includes a plurality of sub-light sources 111, the light-emitting positions of the sub-light sources 111 are designed to be in a specific row. Column mode, and the light-emitting time of the sub-light sources 111A and 111B can be individually controlled. For example, by controlling the alternate light-emitting timing of the sub-light sources 111A and 111B, structured light patterns S with different codes can be formed. For example, in the dot code pattern S, when the sub-light source 111A emits light, it can form a structured light pattern S1 arranged by dots D1 (white dots), and when the sub-light source 111B emits light, it can form a structure formed by dots D2 (black dots). The light pattern S2, when the sub-light sources 111A and 111B emit light at the same time, can form a structured light pattern S arranged by dots D1 and D2.

The main technical content of the depth sensing device 100 according to a preferred embodiment of the present invention has been described above, and then the depth sensing method according to another preferred embodiment of the present invention will be described. The depth sensing method can be implemented by the above-mentioned depth sensing device 100, so the depth sensing method and the technical content of the depth sensing device 100 can be referred to each other, and the repeated parts will be omitted or simplified. The depth sensing method can be executed by the control unit 30 of the depth sensing device 100, or the control unit 30 can cooperate with the microprocessor 200 of the electronic product. The depth sensing method includes at least three steps, which are explained in order as follows

Please refer to FIG. 6, the first process is to allow the control unit 30 to actively (self) determine and execute the appropriate scanning mode (laser radar mode and/or structured light mode) without the user's decision.

First, in step S101, the laser radar mode is first executed to obtain the point cloud data of the object 300 and the distance D between the object 300 and the light receiving component 20 (as shown in FIG. 4A). The distance D may be the distance between a certain part of the object 300 and the light receiving component 20 or the average value of the distance between multiple parts of the object 300 and the light receiving component 20.

Then, in step S103, it is determined whether the distance D is less than a preset distance. The preset distance is determined based on the measurement accuracy and range of the laser radar mode and structured light mode. For example, when it is set to 1 meter, it means that the point cloud data obtained by executing the laser radar mode over 1 meter is better, and the point cloud data obtained by executing the structured light mode under 1 meter is better.

If the distance D is greater than the preset distance, no other steps can be performed, and the point cloud data obtained in step S101 can be directly used to create a three-dimensional model or depth map of the object to be measured 300 or for other applications (step S107). If the distance D is less than the preset distance, the structured light mode is executed (step S105) to obtain another point cloud data of the object to be measured 300, and the other point cloud data is used to build a three-dimensional model of the object to be measured 300, etc. (step S105) S107); At this time, the point cloud data obtained in step S101 will not be used.

It can be seen that, according to the distance of the object 300 to be measured, the depth sensing method can actively select a suitable scanning mode.

On the other hand, if the distance D of a certain part of the test object 300 is less than the preset distance, but the distance D of another part is greater than the preset distance, that is, the test object 300 covers the range inside and outside the preset distance, then step S101 The point cloud data obtained in step S105 is merged with another point cloud data obtained in step S105. That is, in step S107, the two kinds of point cloud data are merged through a software algorithm, and the point cloud of the object under test 300 within the preset distance is obtained in the structured light mode, and the point cloud outside the preset distance is the one obtained in the structured light mode. The point cloud of the object 300 is obtained by the laser radar mode. The three-dimensional model established in this way can cover a more complete range, taking into account the advantages of high accuracy of short-distance point clouds and the ability to establish distant point cloud data.

Please refer to FIG. 7, the second process is that the control unit 30 needs to be executed according to the input signal input by the user, rather than actively executed, so it can be called passive depth sensing.

Specifically, first, in step S201, the control unit 30 receives an input signal input by the user. The input signal corresponds to three modes, including long range mode, close range mode and auto In the active mode, the user can select one of the modes through the input device of the electronic product, and then generate the input signal to the control unit 30. The user can determine the distance of the object 300 to select the long-range mode or the short-range mode. If it is difficult to determine the distance of the object 300, the user can select the automatic mode to let the device 100 determine.

Then, in step S203, according to the input signal, one of the laser radar mode and the structured light mode is executed. For example, when the input signal corresponds to the long-range mode, the laser radar mode is executed, and when the input signal corresponds to the short-range mode, the structured light mode is executed. If the input signal corresponds to the automatic mode, the first process shown in Figure 6 above is to execute the laser radar mode first, and then execute the structured light mode as appropriate.

After executing one of the laser radar mode and the structured light mode, step S205 is optionally performed. That is, the user can select another mode, and then generate another input signal to be received by the control unit 30. The control unit 30 will execute another mode according to another input signal (step S207). In this way, two kinds of point cloud data (point cloud data in the laser radar mode and point cloud data in the structured light mode) will be obtained, and then when the three-dimensional model is created (step S209), the two point cloud data can be optionally merged.

The third process of the depth sensing method is to allow the control unit 30 to execute the laser radar mode and the structured light mode at the same time. Specifically, first, the control unit 30 controls the sub-laser light 111 of the light emitting assembly 10 to emit pulsed laser light L (sub-beam Ls) on the object to be measured 300; the light-emitting position and time of the sub-laser light 111 are different, so The pulsed laser light L forms a structured light pattern with coded information on the object 300 to be tested. Next, the control unit 30 controls the image capturer 21 and the light sensor 22 to receive the reflected light R reflected by the pulsed laser light L from the object 300; the image capturer 21 can analyze the reflected light R The coded information of the reflected light R can be analyzed by the light sensor 22 between. Finally, the point cloud data of the object 300 under test is calculated based on the code information and the reflection time.

In summary, the depth sensing device and method of the present invention have at least the following technical effects:

1. The present invention can select a suitable scanning mode according to the distance of the object to be measured, and a device and method for three-dimensional depth sensing with short-distance high-precision and long-distance low power consumption. A single light emitting component is used to generate structured light The structured light pattern of the mode and the scanning light of the laser radar mode, compared with the conventional light emitting components required for each mode, the device requires a smaller volume and saves power.

2. Compared with the conventional static structured light pattern, the present invention can provide a dynamically changing structured light pattern on the object to be measured through a laser light source and a galvanometer, so as to improve the resolution of depth measurement. In addition, the structured light pattern of the present invention is generated by the reflection of a galvanometer and a control laser. The galvanometer can have a high-reflectivity mirror to reflect the laser light, so the light uniformity of the structured light pattern will not be poor.

3. In the present invention, a one-dimensional galvanometer can be used to scan the laser light to cover the object to be measured. Compared with a two-dimensional galvanometer, the one-dimensional galvanometer has simpler control and is easier to produce large-angle motion. Correspondingly, the dynamically changing structured light pattern can have a larger projection angle, so that the sensing device has a wider viewing angle.

4. The present invention can generate structured light patterns with better resolution by continuously changing laser light intensity, while structured light patterns generated by pulsed laser light usually have poor resolution.

5. The present invention can obtain the point cloud data of the laser radar mode and the structured light mode at the same time through one scan of the laser light on the object to be measured, which is more efficient.

6. Compared with the laser light by a single light source, the present invention uses multiple laser sub-light sources to project When laser light is emitted, more light is emitted per unit time, so better resolution can be obtained.

The above-mentioned embodiments are only used to exemplify the implementation of the new creation, and to explain the technical features of the new creation, and are not used to limit the protection scope of the new creation. Any changes or equivalence arrangements that can be easily completed by those familiar with this technology are within the scope of this new creation, and the scope of protection of the rights of this new creation shall be subject to the scope of the patent application.

100: Depth sensing device

10: Light emitting components

20: Optical receiving component

21: Image capture device

22: light sensor

30: control unit

200: Microprocessor

300: DUT

L: Laser light

Claims (18)

A depth sensing device, including: A light emitting component, including a laser light source, a light shaping element and a galvanometer coupled with light, the laser light source can be controlled to emit a continuous wave laser light and/or a pulsed laser light, the laser light source Emit the laser light to pass through the light shaping member and be reflected and scanned by the galvanometer, so that the laser light is projected onto an object to be measured; A light receiving component optically coupled with the light emitting component for receiving the reflected light of the laser light reflected from the object to be measured; and A control unit is electrically connected to the light emitting component and the light receiving component, and is used for controlling the light emitting component and the light receiving component according to at least one of a structured light mode and a laser radar mode. The depth sensing device according to claim 1, wherein the light receiving component includes an image capture device and/or a light sensor. The depth sensing device according to claim 2, wherein the light receiving element includes the image capturer and the light sensor, and the light sensor is disposed on the light emitting element and the image capture Between the devices. The depth sensing device according to claim 2, wherein the light receiving component includes the image capturer; in the structured light mode, the control unit controls the image capturer to capture the to-be-tested frame by frame The structured light pattern on the object; in the laser radar mode, the control unit controls the image capture device to receive the reflected light from the object to be measured line by line. A depth sensing device, including: A light emitting component includes a laser light source, a light shaping element and a galvanometer that are coupled with light, The laser light source includes a plurality of sub-light sources, and the laser light source can be controlled to emit a continuous wave laser light or a pulsed laser light, and the laser light source emits the laser light through the light-shaping member and then is reflected by the galvanometer Scan to project the laser light onto an object to be measured; A light receiving component, including an image capturer and/or a light sensor optically coupled with the light emitting component, for receiving the reflected light of the laser light from the object to be measured; and A control unit is electrically connected to the light emitting component and the light receiving component, and is used for controlling the light emitting component and the light receiving component according to at least one of a structured light mode and a laser radar mode. The depth sensing device according to claim 5, wherein the light-emitting time of the sub-light sources is individually controlled. The depth sensing device according to claim 5 or 6, wherein the light receiving component includes the image capturer and the light sensor, and the light sensor is disposed on the light emitting component and the image Between extractors. The depth sensing device according to claim 5 or 6, wherein the light receiving component includes the image capturer; in the structured light mode, the control unit controls the image capturer to capture the image frame by frame The structured light pattern on the object under test; in the laser radar mode, the control unit controls the image picker to receive the reflected light reflected from the object under test line by line. A depth sensing method, including: Execute at least one of a laser radar mode and a structured light mode; Wherein, when the laser radar mode is executed, a light emitting component is controlled to emit a pulsed laser light and/or a continuous wave laser light to scan on an object to be measured, and a light receiving component is controlled to receive the pulsed laser light and/or The continuous wave laser light reflects the reflected light from the object to be measured to obtain the Point cloud data of the measured object; Wherein, when the structured light mode is executed, the light emitting component is controlled to emit the continuous wave laser light to form a set of structured light patterns on the object under test, and the light receiving component is controlled to capture the structured light pattern to obtain the waiting Another point cloud data of the measured object; Wherein, the light emitting component includes a laser light source, a light shaping element and a galvanometer, wherein the light shaping element is disposed between the laser light source and the galvanometer, and the laser light source faces the light shaping element and the galvanometer mirror. The galvanometer emits the laser light. The depth sensing method according to claim 9, wherein the laser radar mode is executed first, and a distance between the object to be measured and the light receiving component is obtained; when the distance is judged to be less than a preset distance, The structured light mode is executed; when the distance is judged to be greater than the preset distance, the point cloud data obtained by the laser radar mode is used. The depth sensing method according to claim 9, wherein the laser radar mode is executed first, and a plurality of distances between the object to be measured and the light receiving component are obtained; when one of the distances is less than the When the preset distance is greater than the preset distance, the structured light mode is executed, and the point cloud data and the other point cloud data are merged. The depth sensing method according to claim 9, wherein one of the laser radar mode and the structured light mode is executed according to an input signal. The depth sensing method according to claim 12, wherein the other of the laser radar mode and the structured light mode is executed according to another input signal. The depth sensing method according to claim 13, wherein the point cloud data and the other point cloud data are merged. The depth sensing method according to claim 9, wherein the laser radar mode and the laser radar mode are executed simultaneously Structured light mode, which includes: Controlling the light emitting component to emit the pulsed laser light to scan the object under test; and The light receiving component is controlled to receive the reflected light reflected by the pulsed laser light from the object to be measured to obtain the brightness of the reflected light, the time for the reflected light to return, and the structured light pattern of the reflected light. The depth sensing method according to any one of claims 11 to 15, wherein the light receiving component includes an image capture device and a light sensor; wherein, when the laser radar mode is executed, the light is controlled The sensor receives the reflected light, and when the structured light mode is executed, the image capture device is controlled to capture the structured light pattern. The depth sensing method according to any one of claims 11 to 15, wherein the light receiving component includes an image capture device; wherein, when the laser radar mode is executed, the image capture device is controlled to be line by line When the reflected light is received and the structured light mode is executed, the image capturer is controlled to receive the structured light pattern frame by frame. The depth sensing method according to any one of claims 11 to 15, wherein the laser light source includes a plurality of sub-light sources, and the light-emitting time of the sub-light sources is controlled.

Images