CN110673150A

CN110673150A - Three-dimensional sensing system

Info

Publication number: CN110673150A
Application number: CN201810704958.7A
Authority: CN
Inventors: 林家宇; 陈志强
Original assignee: Acer Inc
Current assignee: Acer Inc
Priority date: 2018-07-02
Filing date: 2018-07-02
Publication date: 2020-01-10

Abstract

The invention discloses a three-dimensional sensing system, wherein a light source module comprises one or more light sources, and one or more MEMS scanning mirrors are arranged on a rotatable reflecting plate and are respectively used for reflecting light rays provided by the one or more light sources. The camera is used for recording one or more light points generated on a target object when light provided by one or more light sources is reflected to the target object. The microcontroller is used for calculating the distance between the target object and the movable reflecting plate according to a spacing between two adjacent light spots in the one or more light spots. Therefore, the three-dimensional sensing system of the invention can simplify the subsequent calculation.

Description

Three-dimensional sensing system

Technical Field

The present invention relates to a three-dimensional sensing system, and more particularly, to a three-dimensional sensing system using mems technology.

Background

With the development of science and technology, the three-dimensional Sensing (3D Sensing) technology is gradually introduced into new applications such as automatic driving and Advanced Driving Assistance Systems (ADAS), Virtual Reality (VR), Augmented Reality (AR), unmanned stores, and face recognition. The mainstream techniques adopted in the current three-dimensional sensing include triangulation (triangulation) and time delay. Applications of triangulation include stereo vision (stereo), structured light (structured light), and laser triangulation (laser triangulation). Applications of time delays include time-of-flight (ToF) and interferometry.

The triangulation system needs to identify and calculate the deformation of the grating, and the time-of-flight ranging system needs to record and calculate the round-trip time of the light, so that the subsequent calculation is quite complicated. Therefore, a three-dimensional sensing system that simplifies subsequent calculations is needed.

Disclosure of Invention

In view of the above-mentioned problems of the prior art, it is an object of the present invention to provide a three-dimensional sensing system capable of simplifying subsequent calculations.

To achieve the above object, the present invention discloses a three-dimensional sensing system, which includes a light source module including one or more light sources; a rotatable reflector plate, on which one or more MEMS scanning mirrors are disposed, for reflecting the light provided by the one or more light sources, respectively; a camera for recording one or more light points generated on a target when light provided by the one or more light sources is reflected to the target; and a microcontroller for determining the distance between the target and the rotatable reflector according to a spacing between two adjacent light spots of the one or more light spots.

Drawings

Fig. 1 is a functional block diagram of a three-dimensional sensing system according to an embodiment of the invention.

FIGS. 2 and 3 are schematic diagrams of an exemplary MEMS scanning mirror in operation.

Fig. 4 to 6 are schematic diagrams illustrating a three-dimensional sensing system determining a distance according to a light spot distance according to an embodiment of the present invention.

Fig. 7 to 10 are schematic diagrams illustrating a scanning manner of a three-dimensional sensing system according to an embodiment of the invention.

Fig. 11 and 12 are schematic diagrams illustrating a scanning range selected by the three-dimensional sensing system according to an embodiment of the invention.

Wherein the reference numerals are as follows:

10 light source module

20 rotatable reflecting plate

23 micro electronic coil

24 reflecting mirror

25 mirror flexure suspension

26 universal support

27 gimbal flexure suspension

30 Camera

32 initial picture

34 scan area

40 micro-electromechanical system scanning mirror controller

50 light source modulation controller

60 microcontroller

70 target object

100 three-dimensional sensing system

TX₁-TX_MLight source

MEMS₁-MEMS_MMicro-electromechanical system scanning mirror

Distance D1-D3

P1-P3 pitch

PHOTO1-PHOTO3 PHOTO

Theta 1-theta 3 light-emitting angle

Detailed Description

Fig. 1 is a functional block diagram of a three-dimensional sensing system 100 according to an embodiment of the invention. The three-dimensional sensing system 100 includes a light source module 10, a rotatable reflective plate 20, a camera 30, a Micro Electro Mechanical System (MEMS) scanning mirror controller 40, a light source modulation controller 50, and a Micro Controller Unit (MCU) 60. The light source module 10 includes one or more light sources TX₁-TX_MAnd one or more MEMS scanning mirrors MEMS are disposed on the rotatable reflective plate 20₁-MEMS_M(M is a positive integer) and can reflect the corresponding light sources TX in the light source module 10 respectively₁-TX_MThe light provided. The MEMS scanning mirror controller 40 can control the angle of the movable reflective plate 20, the light source modulation controller 50 can control the on/off of the light source module 10, and the microcontroller 60 can control the camera 30 and the MEMS scanning mirror controller 40 and a light source modulation controller 50 to enable the light source module 10, the rotatable reflective plate 20 and the camera 30 to operate synchronously, and to determine a distance to a target object according to a picture taken by the camera 30.

Mems devices are fabricated by micro-integration of three-dimensional fine structures, circuits, sensors, actuators, and other components on a silicon wafer, and operate using electromagnetic, electrostrictive (electrostrictive), thermoelectric, piezoelectric, piezoresistive (piezoresistance), and other effects. FIGS. 2 and 3 are schematic diagrams of an exemplary MEMS scanning mirror in operation. Each MEMS scanning mirror disposed on the rotatable mirror plate 20 can include a micro-electronic coil 23, a mirror 24, a mirror flexure 25, a gimbal frame 26, and a gimbal flexure 27. By supplying current to the miniature electrical coil 23, a magnetic moment is generated on the gimbal 26 and a component is generated along the desired axis of rotation. One of the magnetic moment components generated by the gimbal 26 causes the gimbal 26 to rotate about the gimbal flexure suspension 27, i.e., the mirror 24 to rotate in the direction indicated by arrow S1, as shown in fig. 2. Another magnetic moment component generated by the gimbal 26 may excite the mirror 24 to vibrate in a resonant mode and rotate about the mirror flexure suspension 25, i.e., rotate the mirror 24 in the direction shown by arrow S2, as shown in FIG. 3.

When the three-dimensional sensing system 100 of the embodiment of the invention is in operation, the mems scan mirror controller 40 controls the rotation angle of the reflective plate 20. During the rotation of the reflective plate 20, the light source modulation controller 50 controls the light source module 10 to emit light to the MEMS scanning mirror MEMS on the reflective plate 20₁-MEMS_MFurther, the corresponding light source TX in the light source module 10₁-TX_MThe provided light is reflected at a specific angle on the target object and photographed by the camera 30 to record the position of each light spot. The irradiation conditions of the target objects at different positions are different, and the closer the target object is to the reflector 20, the smaller the distance (pitch) between two light spots formed on the target object by two light beams emitted at a certain point and a specific angle; eyes of a userThe farther the subject is from the reflecting plate 20, the smaller the interval between two light spots formed on the subject by two light rays emitted at a certain point at a certain angle. The distance can be recorded by photographing with the camera 30, and the distance between the laser spot and the reflector 20 can be calculated by matching the calculated lookup table with the algorithm.

Fig. 4 to 6 are schematic diagrams illustrating the three-dimensional sensing system 100 according to the distance between light spots according to the embodiment of the invention. For illustration purposes, fig. 4 and 5 show embodiments when M is 2, wherein the light source module 10 includes two light sources TX₁And TX₂And two MEMS scanning mirrors MEMS are disposed on the rotatable reflective plate 20₁And MEMS₂. MEMS scanning mirror MEMS₁And MEMS₂Can reflect the light sources TX at specific angles respectively₁And TX₂Providing light to the target 70, FIG. 4 shows the distance between the target 70 and the reflective plate 20 from D1-D3 from top to bottom, wherein D1<D2<D3. On the other hand, the camera 30 may take a photograph to record the light spot caused by the light reaching the object 70, and fig. 5 shows photographs PHOTO1-PHOTO3 taken from the top to the bottom, respectively, when the object 70 and the reflective plate 20 are at a distance D1-D3, respectively, wherein the MEMS scanning mirror MEMS scans the light spot₁And MEMS₂Reflective light source TX₁And TX₂The two light points resulting from the light provided to the object 70 are represented by dots, and the pitch P1-P3 between the two light points is inversely proportional to the distance between the object 70 and the reflector 20, i.e., P1>P2>P3。

The distance between the object 70 and the reflector 20 at this time can be determined by the microcontroller 60 based on the spacing between the spots recorded when the camera 30 takes a picture. As shown in FIG. 6, the MEMS is a scanning mirror due to the MEMS₁And MEMS₂When the distances P1 to P3 of the object 70 at different distances D1 to D3 are recorded, the microcontroller 60 can determine the values of the distances D1 to D3 according to a trigonometric function, wherein D1 is cot θ 1/P1, D2 is cot θ 2/P2, and D3 is cot θ 3/P3.

In the embodiment of the present invention, the distances corresponding to different distances can be obtained in advance for different light-emitting angles, and the data can be stored in the microcontroller 60 in a look-up table manner, thereby shortening the calculation time. However, the embodiment of calculating the distance according to different pitches does not limit the scope of the present invention.

Fig. 7 to 10 are schematic diagrams illustrating a scanning manner of the three-dimensional sensing system 100 according to an embodiment of the invention. The present invention defines the vertical axis and the horizontal axis in time and sequence for a specific scan surface of the rotatable reflection plate 20. The arrow represents the moving manner of the rotatable reflective plate 20, wherein the solid arrow represents the actual scanning line when the light source emits light, and the dotted arrow represents that the rotatable reflective plate 20 is not scanned (the light source does not emit light) during the rotation process.

FIG. 7 shows a single laser source TX₁And a single MEMS scanning mirror MEMS₁In the embodiment of performing unidirectional scanning, after the distance of one scanning line in the horizontal direction is scanned, the rotatable reflective plate 20 does not scan (as shown by the dashed arrow) during the period of moving to the starting point of the next scanning line, and does not start scanning until moving to the starting point of the next scanning line. FIG. 8 shows a single laser source TX₁And a single MEMS scanning mirror MEMS₁In the embodiment of performing bidirectional scanning, after the movable reflective plate 20 scans the distance of one row of scanning lines in the horizontal direction, the movable reflective plate still scans (as shown by the solid line arrow) while moving to the starting point of the next row of scanning lines.

FIG. 9 shows a multi-laser light source TX₁-TX_MAnd multiple MEMS scanning mirror MEMS₁-MEMS_MIn the embodiment of performing unidirectional scanning, after the rotatable reflective plate 20 scans the distance between adjacent M rows of scanning lines in the horizontal direction, the rotatable reflective plate does not scan (as shown by the dashed arrow) during the period of moving to the starting point of the next adjacent M rows of scanning lines until moving to the starting point of the next adjacent M rows of scanning lines, and then starts scanning again. FIG. 8 shows a multi-laser TX₁-TX_MAnd multiple MEMS scanning mirror MEMS₁-MEMS_MIn the embodiment of bi-directional scanning, after the rotatable reflective plate 20 scans the distance between M adjacent scanning lines in the horizontal direction,the scan line is still scanned during the period of moving to the starting point of the next adjacent M rows of scan lines (as shown by the solid line arrows). For illustrative purposes, fig. 9 and 10 show an embodiment where M is 2, however, the value of M does not limit the scope of the present invention.

In the embodiments shown in fig. 7 to 10, the dots represent the light spots caused when the light provided by the light source module 10 is reflected on the target object at a specific angle, and the distance between the two light spots needs to be known to determine the value of the spacing. In FIGS. 7 and 8, a single laser source TX₁And a single MEMS scanning mirror MEMS₁In the scanning embodiment, the camera 30 can only record the position of a single light spot once a picture is taken, so the microcontroller 60 needs to synthesize the images of the pictures taken at multiple time points to determine the distance between two adjacent light spots. In the embodiment of fig. 9 and 10 in which multiple laser sources and multiple mems scanning mirrors are used for scanning, the camera 30 can record the positions of multiple light spots each time a picture is taken, so that the distance between two adjacent light spots can be directly determined from a single picture. Thus, the single laser source and single multi-MEMS scanning mirror architecture of FIGS. 7 and 8 may save hardware costs, while the multi-laser source and multi-MEMS scanning mirror architecture of FIGS. 9 and 10 may provide high speed and high resolution scanning.

In addition, in real world three-dimensional measurement, a background usually has a plurality of objects. Therefore, in the embodiment of the invention, the three-dimensional sensing system 100 can capture an initial image by the camera 30 before scanning, and the microcontroller 60 determines the scanning range in the actual three-dimensional measurement according to the initial image.

In one embodiment, the microcontroller 60 can perform image analysis on the initial frame to find all objects in the background, and then set the scanning range to the minimum range that can include one or more main objects, thereby shortening the three-dimensional scanning time; in another embodiment, the user can select one or more targets to be detected on the initial screen, and the microcontroller 60 sets the scanning range to the minimum range that can include the one or more targets, thereby shortening the three-dimensional scanning time.

After setting the scanning range, the micro-controller 60 can instruct the mems scanning mirror controller 40 to control the angle of the rotatable reflective plate 20, so as to perform three-dimensional scanning on the scanning range, and then the camera 30 takes a picture to record the position of each light spot during the three-dimensional scanning process. In one embodiment, the camera 30 takes the same resolution of the picture taken during the three-dimensional scan as the original picture; in another embodiment, the resolution of the pictures taken by the camera 30 during the three-dimensional scan may be higher than the resolution of the initial frame.

Fig. 11 is a schematic diagram illustrating the three-dimensional sensing system 100 selecting a scanning range according to an embodiment of the invention. Assuming that the original resolution of the camera 30 is 1920 × 1080, the microcontroller 60 determines a scanning area 34 according to the initial image 32 captured by the camera 30, so that the three-dimensional sensing system 100 performs three-dimensional scanning with the resolution of 1920 × 1080 on the scanning area 34.

Fig. 12 is a schematic diagram illustrating the three-dimensional sensing system 100 selecting a scanning range according to another embodiment of the invention. Assuming that the original resolution of the camera 30 is 1920 × 1080, the microcontroller 60 determines a scanning area 34 according to the initial image 32 captured by the camera 30, so that the three-dimensional sensing system 100 scans the scanning area 34 with a higher resolution (e.g., 2560 × 1440).

In the present invention, the light source TX included in the light source module 10₁-TX_MCan be a Light Emitting Diode (LED) or a Vertical Cavity Surface Emitting Laser (VCSEL). However, the light source TX₁-TX_MThe type of (c) does not limit the scope of the invention.

In summary, the present invention provides a three-dimensional sensing system using mems technology, which utilizes a scanning mirror of the mems to reflect light to a target object, and a camera to take a picture to record light spots, and finally calculate the distance between the target object according to the distance between the two light spots. The invention can calculate the distance corresponding to different distances under different light-emitting angles in advance, and then stores the data in a microcontroller of the three-dimensional sensing system in a lookup table mode, thereby simplifying the subsequent calculation steps.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional sensing system, comprising:

a light source module including one or more light sources;

a movable reflection plate, on which one or more MEMS scanning mirrors are disposed, for reflecting the light provided by the one or more light sources, respectively;

a camera for recording one or more light points generated on a target when light provided by the one or more light sources is reflected to the target; and

a microcontroller for determining a distance between the target object and the movable reflector according to a spacing between two adjacent light spots of the one or more light spots.

2. The three-dimensional sensing system of claim 1, wherein:

the light source module comprises a light source;

the movable reflecting plate is provided with a micro-electro-mechanical system scanning mirror for reflecting the light provided by the light source;

the camera is used for taking a first picture at a first time point to record a first light spot generated on the target object when the light provided by the light source is reflected to the target object at the first time point, and taking a second picture at a second time point to record a second light spot generated on the target object when the light provided by the light source is reflected to the target object at the second time point; and is

The microcontroller is further configured to synthesize the first and second photographs to find the separation distance between the first and second light spots.

3. The three-dimensional sensing system of claim 2, further comprising:

and a MEMS scanning mirror controller for controlling the angle of the movable reflector so that the MEMS scanning mirror sequentially scans from a start point to an end point of each of N parallel sequences included in a plane, wherein N is an integer greater than 1.

4. The three-dimensional sensing system of claim 2, further comprising:

a MEMS scanning mirror controller for controlling the angle of the movable reflective plate to make the MEMS scanning mirror scan from the starting point of the N-th parallel sequence to the end point of the N-th parallel sequence in the N parallel sequences included in a plane, and then scan from the end point of the N-th parallel sequence to the starting point of the (N +1) -th parallel sequence in the N parallel sequences, wherein N is an integer greater than M and N is a positive integer not greater than N.

5. The three-dimensional sensing system of claim 1, wherein:

the light source module includes first to Mth light sources;

the movable reflecting plate is provided with first to Mth MEMS scanning mirrors which are respectively used for reflecting light rays provided by the first to Mth light sources;

the camera is used for taking a picture at a specific time point so as to record first to Mth light spots generated at M positions on the target object when the light rays provided by the first to Mth light sources are reflected to the target object at the specific time point;

the microcontroller is further used for calculating first to Mth distances between M positions on the target object and the movable reflector according to first to Mth distances between every two adjacent light spots in the first to Mth light spots; and is

M is an integer greater than 1.

6. The three-dimensional sensing system of claim 5, further comprising:

and a MEMS scanning mirror controller for controlling the angle of the movable reflective plate to sequentially scan the first to Mth MEMS scanning mirrors from a start point to an end point of each of N parallel sequences included in a plane, wherein N is an integer greater than M.

7. The three-dimensional sensing system of claim 5, further comprising:

a MEMS scanning mirror controller for controlling the angle of the movable reflective plate to make the first to Mth MEMS scanning mirrors sequentially scan from a starting point of an nth parallel sequence to an end point of the nth parallel sequence in N parallel sequences included in a plane, and then scan from the end point of the nth parallel sequence to a starting point of an (N +1) th parallel sequence in the N parallel sequences, wherein N is an integer greater than M and N is a positive integer not greater than N.

8. The three-dimensional sensing system according to claim 5, further comprising a light modulation controller for controlling the light source module to turn on and off, wherein the microcontroller is further configured to synchronize operations of the camera, the MEMS scanning mirror controller, and the light modulation controller.

9. The three-dimensional sensing system of claim 1, wherein the microcontroller is further configured to:

instructing the camera to shoot an initial picture;

performing image analysis on the initial picture to obtain one or more objects in the initial picture;

determining a scan range for the target object selected from the one or more objects; and

and instructing the MEMS scanning mirror to control the angle of the movable reflecting plate so that the one or more MEMS scanning mirrors perform three-dimensional scanning on the scanning range.

10. The three-dimensional sensing system of claim 1, wherein the microcontroller is further configured to:

instructing the camera to shoot an initial picture;

determining a scan range based on the target selected by a user from the one or more objects; and

11. The three-dimensional sensing system according to claim 9 or 10, wherein the microcontroller is further configured to:

controlling the camera to shoot the initial picture at a first resolution; and

and controlling the camera to take pictures at a second resolution when the one or more MEMS scanning mirrors perform three-dimensional scanning on the scanning range, wherein the second resolution is higher than the first resolution.