WO2020022721A1 - Variable structured light generating device and 3d imaging system - Google Patents

Abstract

The present invention relates to a variable structured light device and a 3D imaging system. Specifically, the present invention can provide a variable structured light generating device which adjusts the modulation frequency, initial phase, and duty cycle of a modulation signal of laser light, and adjusts the drive frequency, initial phase, and amplitude of drive signals of both axes of a mirror for conducting Lissajous scanning to thereby generate a structured light in which the number of patterns, pattern shape, and the field of view have been changed. In addition, the present invention can provide a 3D imaging system for generating structured light in which the number of patterns, pattern shape, and the field of view have been changed, and for processing an image of a subject to which the structured light is irradiated thereby obtaining 3D information of the subject.

Description

Variable Structured Light Generation Device and 3D Imaging System

The present invention relates to a variable structured light generating device and a three-dimensional imaging system. More specifically, the structured light generating apparatus generates a structured light having a variable number of patterns, a pattern shape and a field of view, and generates a structured light having a variable number of patterns, a pattern shape and a field of view, and acquires three-dimensional information of a subject. A dimensional imaging system.

Recently, with increasing interest in 3D stereoscopic image services, devices for providing stereoscopic images have been continuously developed. Among the methods of implementing such stereoscopic images, there are a stereoscopic method, a time of flight (TOF) method, a structured-light method, and the like.

The basic principle of the stereoscopic method is to separate and input images arranged to be orthogonal to each other in the left eye and the right eye of a person, and to generate stereoscopic images by combining the images input to the left eye and the right eye in the human brain. In this case, the images arranged to be orthogonal to each other become a left view image and a right view image, respectively. Recently, a 3D camera is configured to capture a left eye image and a right eye image together in one device. For example, a stereo method using two identical cameras is often used. In the stereo system, two cameras are arranged at a baseline to acquire left and right images using two completely separate cameras (two lenses, two sensors, and two ISPs).

However, the stereo type 3D camera has a problem in that a quality problem due to an assembly error between two cameras degrades the 3D quality, resulting in a high precision assembly process and a low yield. In addition, there is a problem that the 3D depth measurement range is determined by a baseline, which is a distance between two fixed cameras. In addition, in the case of the 3D zoom lens, even if the alignment between the two cameras is well aligned at the beginning, an error occurs between the two cameras while zooming, causing a problem in that the image is degraded, causing viewer fatigue.

In addition, the time of flight (TOF) method obtains depth information of an object by directly irradiating light to the object and calculating a time of reflected light that is reflected and returned. This method has a problem of limited use due to the size and high power consumption of the TOF-only sensor.

In addition, the structured light method acquires depth information of an object by irradiating a laser beam having a specific pattern coded to the object and calculating a pattern shift amount of the reflected light. This method typically uses fixed-focus lenses and passive coding elements.

Therefore, because the structured light used in the conventional structured light 3D stereoscopic image providing system has to be divided into a large area, the light intensity must be large, and the number of patterns and the field of view can be changed according to various environments, and thus the image cannot be acquired. There is.

The present invention adjusts the modulation frequency, initial phase and duty cycle of the modulation signal of the laser light, and adjusts the driving frequency, initial phase and amplitude of the drive signal on both axes of the mirror for Lissajous scanning, so that the number of patterns, pattern shape and field of view can be adjusted. An object of the present invention is to provide a variable structured light generating device that generates structured light that changes.

In addition, the present invention provides a three-dimensional imaging system for generating a three-dimensional information of the subject by generating a structured light in which the number of patterns, pattern shape and the field of view is changed, and processing the image of the subject to which the structured light is irradiated It aims to do it.

According to an aspect of the present invention, there is provided a variable structured light generating apparatus including a light source unit for outputting modulated laser light, a mirror for reflecting the laser light, and driving signals of two axes orthogonal to each other being input as sinusoids to perform Lissajoe scanning; Information input unit for inputting the desired number of patterns, pattern shape and field of view of the structured light, the memory unit pre-stored values for the drive frequency, initial phase and amplitude of the biaxial drive signal of the mirror corresponding to the information And a controller configured to receive the information to generate structured light corresponding to the information, communicate with the memory unit, and generate and transmit a control signal of the mirror and a modulation signal of the light source unit.

In the apparatus for generating variable structured light according to an embodiment of the present invention, the controller may include: an information receiver configured to receive information about the pattern number, pattern shape, and field of view of the structured light input through the information input unit, corresponding to the information; Extracts values for driving frequency, initial phase, and amplitude of the two-axis driving signal of the mirror in communication with the memory unit, and calculates a LCM of the driving frequency of the extracted two-axis driving signal of the mirror as the laser modulation frequency of the light source unit; A signal generator for generating a sine wave control signal formed with values for a driving frequency, an initial phase, and an amplitude of the biaxial drive signal of the extracted mirror; and a square wave modulated signal formed with the calculated laser modulation frequency of the light source unit; Signal transmission for transmitting the control signal to the mirror and the modulation signal to the light source unit It may contain.

In the variable structured light generating apparatus according to an embodiment of the present invention, the mirror may be a MEMS mirror.

In addition, in the variable structured light generating apparatus according to an embodiment of the present invention, the MEMS mirror is rotated and oscillated with respect to two axes orthogonal to each other, and a Q-factor for the rotational vibration of the first axis is determined by the second axis. It can be designed lower than the quality factor for rotational vibration.

Further, in the variable structured light generating apparatus according to an embodiment of the present invention, the number of patterns of the structured light may be determined by the maximum common factor having a negative correlation with the greatest common factor of the driving frequencies of the driving signal.

In addition, in the variable structured light generating apparatus according to an embodiment of the present invention, the number of patterns of the structured light may vary according to an initial phase of the driving signal.

In addition, in the apparatus for generating variable structured light according to an embodiment of the present invention, the number of patterns of the structured light may vary according to a duty cycle of the modulated signal.

In addition, in the apparatus for generating variable structured light according to an embodiment of the present invention, the number of patterns of the structured light may vary according to an initial phase of the modulated signal.

In the variable structured light generating apparatus according to an embodiment of the present invention, the pattern shape of the structured light may include a driving frequency of the driving signal, an initial phase of the driving signal, a waveform of the modulation signal, and a duty of the modulation signal. It may be changed according to at least one of a cycle and an initial phase of the modulated signal.

In addition, in the variable structured light generating apparatus according to an embodiment of the present invention, the field of view of the structured light may have a positive correlation with the amplitude of the biaxial drive signal of the drive signal and may be determined by the amplitude.

In addition, in the variable structured light generating apparatus according to an embodiment of the present invention, the laser light output from the light source unit may be modulated by an ON / OFF square wave.

In the variable structured light generating apparatus according to an embodiment of the present invention, the frequency of the modulated signal may be a multiple of a minimum common multiple of a drive frequency of the biaxial drive signal of the mirror.

In addition, in the variable structured light generating apparatus according to an embodiment of the present invention, the light source unit further comprises a light modulator on one side of the laser to the mirror to modulate the laser light, ON / ON the light modulator A modulated signal that is an OFF type square wave is applied, and the modulated frequency of the square wave may be a minimum common multiple or a multiple of the minimum common multiple of a driving frequency of the biaxial drive signal of the mirror.

In addition, the three-dimensional imaging system according to an embodiment of the present invention is a variable structure lighting device for generating a structured light with a variable number of patterns, pattern shape and field of view, a camera module for photographing the image of the subject irradiated with the structured light; And an image processing module for restoring a three-dimensional shape of the subject by processing the image photographed by the camera module, wherein the variable structure lighting apparatus includes a light source for outputting laser light modulated by a modulation signal and the laser light. And a scanner for reflecting and orthogonally driving signals of two axes perpendicular to each other to perform Lissajous scanning, wherein the frequency of the modulated signal is a multiple of a least common multiple of the drive frequencies of the drive signal. It features.

In addition, in the 3D imaging system according to an embodiment of the present invention, the number of patterns of the structured light may be changed according to the driving frequencies with a negative correlation with the greatest common factor of the driving frequencies of the driving signal. .

In addition, in the three-dimensional imaging system according to an embodiment of the present invention, the number of patterns of the structured light may be changed according to the initial phase of the driving signal.

In addition, in the three-dimensional imaging system according to an embodiment of the present invention, the number of patterns of the structured light may vary according to the duty cycle of the modulated signal.

In addition, in the three-dimensional imaging system according to an embodiment of the present invention, the number of patterns of the structured light may be changed according to the initial phase of the modulated signal.

Further, in the three-dimensional imaging system according to an embodiment of the present invention, the pattern shape of the structured light, the driving frequency of the drive signal, the initial phase of the drive signal, the waveform of the modulation signal, the duty cycle of the modulation signal And an initial phase of the modulated signal.

In addition, in the 3D imaging system according to an embodiment of the present invention, the field of view of the structured light may be changed according to the amplitude with a positive correlation with the amplitude of the driving signal.

In addition, in the three-dimensional imaging system according to an embodiment of the present invention, the variable structure lighting device, the laser light output from the light source is disposed on the path toward the scanner, and the laser light (a collimated beam ( It may further include a collimator to make a collimated beam.

In addition, in the three-dimensional imaging system according to an embodiment of the present invention, the variable structured lighting device, the housing for receiving the optical fiber and the one end of the optical fiber, the collimator and the scanner to combine the laser light output from the light source It may further include a hole in which one end of the optical fiber can be inserted in one end of the housing, the inner surface of the other end of the housing may be provided with a scanner mount, the upper surface of the other end of the housing In the openings may be formed.

In addition, in the 3D imaging system according to an embodiment of the present invention, the scanner mount may be tiltable.

In addition, in the three-dimensional imaging system according to an embodiment of the present invention, the scanner may include a mirror and a driving unit for driving the mirror to Lissajou scanning.

In the three-dimensional imaging system according to an embodiment of the present invention, the mirror may be a MEMS mirror.

The variable structured light generating apparatus of the present invention controls the modulation frequency, initial phase and duty cycle of the laser light, and the driving frequency, initial phase and amplitude of the biaxial drive signal of the Lissajous scan of the mirror to control the pattern number, pattern shape and field of view of the structured light. Can be adjusted.

In addition, since the size of the mirror used in the variable structured light generating device of the present invention is approximately 1 mm x 1 mm, the device of the present invention can be miniaturized.

In addition, the variable structured light generating apparatus of the present invention is capable of generating structured light having a wide field of view with a laser of low power as compared to the structured light generating apparatus using a conventional vertical cavity surface emitting laser (VCSEL).

The three-dimensional imaging system of the present invention adjusts the frequency, initial phase and duty cycle of the laser light modulation signal, the driving frequency, the initial phase, and the amplitude of the biaxial drive signal of the scanner for scanning the LISA scan. Since the structured light whose pattern density is adjusted can be irradiated to the subject by changing the, high resolution 3D images can be obtained.

Further, since the size of the mirror used in the three-dimensional imaging system of the present invention is approximately 1 mm x 1 mm, the three-dimensional imaging system itself can be miniaturized.

In addition, the three-dimensional imaging system of the present invention is capable of generating structured light having a wide field of view with a low power laser.

1 is a schematic diagram illustrating a variable structured light generating apparatus according to an exemplary embodiment of the present invention.

2 is a block diagram illustrating a control unit of a variable structured light generating device according to an exemplary embodiment of the present invention.

3 is a perspective view illustrating a MEMS mirror used in the variable structured light generating apparatus according to the embodiment of the present invention.

4 is an enlarged view illustrating a portion A of FIG. 3.

5A through 5D are exemplary diagrams illustrating that the number of patterns of structured light changes according to a driving frequency of a driving signal of a mirror.

6A to 6E are exemplary views illustrating that the number of patterns of structured light changes according to an initial phase of a driving signal of a mirror.

7A to 7E are exemplary views illustrating that the number of patterns of structured light changes according to a duty cycle of a modulated signal of laser light.

8 is a flowchart illustrating a method for generating variable structured light according to an embodiment of the present invention.

9 is a schematic diagram illustrating a three-dimensional imaging system according to an embodiment of the present invention.

10 is a schematic diagram showing a variable structure lighting apparatus used in a three-dimensional imaging system according to an embodiment of the present invention.

11 is a perspective view of a variable structured lighting device used in a three-dimensional imaging system according to an embodiment of the present invention.

12 is a cross-sectional view of the variable structure lighting apparatus used in the three-dimensional imaging system according to an embodiment of the present invention.

13 is a schematic flowchart of a three-dimensional imaging method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention having the configuration as described above will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram illustrating a variable structured light generating apparatus 1000 according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the variable structured light generating apparatus 1000 according to an exemplary embodiment may include a light source unit 100, a mirror 200, an information input unit 300, a memory unit 400, and a controller 500. Include. In general, the variable structured light generating apparatus 1000 according to the present invention irradiates structured light to a target object, captures the reflected light reflected from the target object with an image capturing means such as a camera, and analyzes the acquired image. It is used in the system to obtain 3D stereoscopic information.

When the number of patterns irradiated by the structured light generating device is small, each pattern may be easily distinguished, but the accuracy of distance measurement of the target object may be lowered. On the other hand, if the number of patterns to be irradiated increases the calculation time and error for distinguishing each pattern, but the accuracy of the distance measurement with the target object can be improved. In addition, as the size of the pattern increases, the pattern may be easily distinguished, but the resolution of the stereoscopic image may be lowered. On the other hand, if the size of the pattern to be irradiated is difficult to distinguish the pattern, the resolution of the stereoscopic image may be increased. Therefore, it is necessary to determine and apply in real time the number and shape of the pattern of the irradiation light having the pattern of the desired condition.

The information input unit 300 is configured to input information on a pattern number, pattern shape, and field of view (FOV) of a desired irradiation light according to a condition for obtaining a 3D stereoscopic image. Information input to the information input unit 300 is received by the control unit 500, the control unit 500 is a drive signal of both axes of the mirror 200 corresponding to the input information to generate the structured light corresponding to the input information Information on the memory unit 400, the modulation frequency of the light source unit 100 is calculated based on the driving frequency of the driving signal of the mirror 200 received from the memory unit 400, and driving of both axes of the mirror 200. Control signal (V _x (t) = A _x sin (2πf _x t + φ _x ) and V _y (t) = A _y sin (2πf _y t + φ _y )) and modulation of the light source unit 100 The signal is transmitted to the mirror 200 and the light source unit 100, respectively. The laser light modulated by the light source unit 100 is output according to the transmitted signal, and the mirror 200 reflects the laser light and performs Lissajou scanning to have a desired number of patterns, pattern shapes, and fields of view on the target object. The light is irradiated.

The light source unit 200 of the variable structured light generating apparatus 1000 according to an exemplary embodiment of the present invention includes a laser 110, and the laser light is modulated by a square wave of an ON / OFF method and the mirror 200. Is irradiated on the surface of the. In this case, the modulated signal is transmitted from the control unit and is a least common multiple (LCM) or a multiple of the least common multiple of both axis driving frequencies f _x and f _y of the mirror 200. The wavelength band of the laser light may be a visible light or an infrared ray band.

The light source unit 100 of the variable structured light generating apparatus 1000 according to the exemplary embodiment of the present invention further includes a light modulator 120 on one side of the laser headed to the mirror 200 to modulate the laser 110 light. can do. The light modulator 120 may be formed of a Kerr cell made by using an electro-optic Kerr effect that modulates light or is used in a shutter of a high speed camera. In addition, the ON / OFF square wave signal is applied to the optical modulation element 120 and the frequency of the square wave is a multiple of the least common multiple or the least common multiple of the driving frequencies f _x and f _y of the biaxial drive signals of the mirror 200. .

2 is a block diagram illustrating a controller of the variable structured light generating apparatus 1000 according to an exemplary embodiment.

Referring to FIG. 2, the controller 500 includes an information receiver 510, a signal generator 520, and a signal transmitter 530. The controller 500 receives information on the pattern number, pattern shape, and field of view of the structured light inputted to the information input unit 300 to generate structured light corresponding to the information input to the information input unit 300. In communication with 400, a signal for the biaxial driving signal of the mirror 200 is received, and a signal for controlling the light source unit 100 and the mirror 200 is generated and transmitted.

The information receiver 510 receives information about the pattern number, pattern shape, and field of view of the structured light input through the information input unit 300. The signal generator 520 is a drive frequency (f _x , f _y ) and initial phase of the biaxial drive signal of the mirror 200 corresponding to the information on the pattern number, pattern shape, and field of view of the structured light received by the information receiver 510. Contact the memory unit 400 for the values of (φ _x , φ _y ) and amplitude (A _x , A _y ), and extract the corresponding values from the memory 400 if there are corresponding values. Generate a signal to control. In addition, the signal generator 520 calculates the least common multiple of the driving frequencies f _x and f _y of the extracted two-axis driving signals of the mirror 200 or the multiple of the minimum common multiple as the laser modulation frequency of the light source unit, and calculates the calculated modulation frequency. Generates a modulated signal of square waves formed by The signal transmitter 530 transmits the control signal generated by the signal generator to the mirror 200 and the modulation signal generated by the signal generator to the light source unit.

A MEMS mirror may be used as the mirror 200 of the variable structured light generating apparatus 1000 according to an exemplary embodiment of the present invention.

3 is a perspective view illustrating a MEMS mirror used in the variable structured light generating apparatus 1000 according to an exemplary embodiment, and FIG. 4 is an enlarged view illustrating portion A of FIG. 3.

3 and 4, the MEMS mirror includes a mirror plate 210, a first fixing end 221, a second fixing end 222, a first torsion bar 231, a second torsion bar 232, Frame 240. The mirror plate 210 reflects light and is rotatable, and the driving signals of both axes are respectively V _x (t) = A _x sin (2πf _x t + φ _x ) and V _y (t) = A _y. Lissajous scanning is performed by the light input into sin (2πf _y t + φ _y ) and reflected from the mirror plate. The first fixing end 221 is formed on both sides of the mirror plate, respectively, so that there is no displacement of the center of the mirror plate while the mirror plate is rotating and oscillating with a straight line connecting the first fixing end as an axis (here, X axis). It serves to fix it. The frame 240 is configured to surround the circumference of the mirror plate and the first fixed end, and the second fixed end 222 is formed on both sides of the frame, respectively, and is positioned on a line perpendicular to the straight line connecting the first fixed end. Therefore, the frame and the mirror plate serve to fix the center plate of the mirror plate during the rotational oscillation with the axis (here, the Y axis) as a straight line connecting the second fixing end. Four torsion bars are arranged on two straight lines perpendicular to the center of the mirror plate. The first torsion bar 231 elastically supports the mirror plate 210 to connect the mirror plate and the first fixing end. The second torsion bar 232 elastically supports the frame 240 to connect the frame and the second fixed end.

The light reflected by the rotational vibration of the mirror plate provides a rotational force in a first axis and a second axis that are perpendicular to each other to perform Lissajous scanning. The manner of providing the rotational force may be taken among electrostatic, electromagnetic, electrothermal and piezo methods. FIG. 3 illustrates a MEMS mirror that uses an electrostatic driving method, wherein the MEMS mirror has a comb-shaped comb shown in the enlarged view of FIG. 4, and the electric repulsive force of adjacent combs generated by applying a driving signal. Or the attraction force causes the mirror plate to rotate.

In order for the variable structured light generating apparatus 1000 according to an embodiment of the present invention to implement variable light having a variable number of patterns, it is necessary to variously select a driving frequency of a mirror that determines the number of patterns. The width of the resonance bandwidth should be wide. In other words, a wide resonance bandwidth means a low Q-factor of the vibration mode.

The quality factor of MEMS mirrors is determined by the relationship between the moment of rotational inertia of the mirror plate and frame and the elastic restoring force of the torsion bar. Therefore, the MEMS mirror is manufactured by changing the thickness of the first or second torsion bar in order to control the quality factor of the MEMS mirror vibration. That is, in the process of drawing a mask corresponding to the blueprint before the semiconductor process of manufacturing the MEMS mirror, the thickness of the torsion bar is adjusted to determine the quality factor for the vibration of both axes of the MEMS mirror.

When the MEMS mirror is used as the mirror 200 in the variable structured light generating apparatus 1000 according to the exemplary embodiment of the present invention, in order to generate the structured light having a variable number of patterns, the MEMS mirrors are based on both axes perpendicular to each other. It is preferable to design at least one of the quality factor for the rotational vibration of the first axis or the quality factor for the rotational vibration of the second axis, or to design one of the two quality factors lower, and the lower quality factor being 100 or less. More preferred.

5 to 7, a structured light generating apparatus 1000 according to an embodiment of the present invention, and a structured lighting device used in a 3D imaging system according to an embodiment of the present invention to be introduced later ( Note that the same applies to 1000 ' ).

5A to 5D are exemplary views illustrating that the number of patterns of structured light changes according to the driving frequency of the drive signal of the mirror, and FIGS. 6A to 6E illustrate the number of patterns of structured light varying according to the initial phase of the drive signal of the mirror. 7A to 7E are exemplary views illustrating a change in the number of patterns of structured light according to a duty cycle of a modulation signal of laser light. In addition, Table 1 below shows the number of patterns according to the driving frequency of the mirror.

X axis drive frequency f x	Y axis drive frequency f y	Greatest common factor	X-axis pattern count	Y-axis pattern count	Pattern count
5980	6800	20	340	299	101660
6000	6792	24	283	250	70750
6040	6800	40	170	151	25670
6006	6798	66	103	91	9373
5994	6804	162	42	37	1554
5984	6800	272	25	22	550
6000	6800	400	17	15	255
5957	6808	851	8	7	56

5A to 5D show that the greatest common divisor decreases from FIG. 5A to FIG. 5E. 5A shows the greatest common divisor (GCD) of the driving frequencies of the two axes of the mirror, 1684, FIG. 5B shows the greatest common divisor of the driving frequencies of the two axes of the mirror, and FIG. 5D shows a maximum common divisor of the driving frequencies of both axes of the mirror. That is, FIGS. 5A to 5D show that the number of patterns is changed according to the driving frequency of the driving signals on both axes of the mirror. In addition, it can be seen that the number of patterns has a negative correlation with the greatest common factor of the driving frequencies of the driving signals on both axes of the mirror. In other words, it can be seen that the larger the common factor of the driving frequencies of both axes is, the smaller the number of patterns of structured light is. This can also be seen with reference to Table 1. Therefore, it is possible to predict the number of patterns of structured light through the value of the greatest common divisor of the two driving frequencies.

6A to 6E are exemplary views illustrating that the number of patterns of structured light changes according to an initial phase of a driving signal of a mirror. 6A to 6E, when the driving frequencies of the mirrors are 5269 Hz and 6706 Hz, respectively, and the maximum common factor of the driving frequencies is 479, the driving signals of the mirrors V _x (t) = A _x sin (2πf _x t + φ _x ) and _{V y (t) = a y} sin (2πf y · t + φ y)) has shown that any of the initial phase from the Figure 6a structure, the light changes the increase gradually in Figure 6e in the. As shown in Figs. 6A to 6E, the number of patterns of structured light is changed according to the initial phase of the drive signal of the mirror. The pattern shape of the structured light also changes in accordance with the initial phase of the drive signal of the mirror.

7A to 7E are exemplary views illustrating that the number of patterns of structured light changes according to a duty cycle of a modulated signal of laser light. 7A to 7E, when the driving frequencies of the mirrors are 5269 Hz and 6706 Hz, respectively, and the maximum common factor of the driving frequency is 479, the change in the structured light is shown when the duty cycle of the modulation signal of the laser light is changed. In this case, the duty cycle refers to the ratio of time that is ON for one cycle T in the ON / OFF square wave signal. FIG. 7A shows a duty cycle of 10%, FIG. 7B shows a duty cycle of 30%, FIG. 7C shows a duty cycle of 50%, FIG. 7D shows a duty cycle of 70%, and FIG. 7E shows a duty cycle of 90%. As shown in Figs. 7A to 7E, the number of patterns of the structured light is changed in accordance with the duty cycle of the modulation signal of the laser light, and the pattern shape of the structured light is also changed in accordance with the duty cycle of the modulation signal of the laser light. In addition, the number of patterns of the structured light varies depending on the initial phase of the modulated signal.

5 to 7, the pattern shape of the structured light includes a drive frequency f _x , f _y of the drive signal of the mirror, an initial phase φ _x , φ _y of the drive signal, a waveform of a modulated signal of laser light, The at least one of the duty cycle of the modulated signal and the initial phase of the modulated signal is varied.

In addition, the field of view (FOV) of the structured light becomes wider as the angles of rotation of both axes of the mirror become larger, and the angle of rotation is determined by the amplitude (A _x , A _y ) of the drive signal of the mirror. It has a positive correlation with the amplitude of the drive signal of and changes according to the amplitude (A _x , A _y ) of the drive signal of the mirror.

In summary, the pattern number, pattern shape, and field of view of the structured light may vary depending on the driving frequency, initial phase and amplitude of the drive signal of both mirrors, and the duty cycle and initial phase of the modulated signal. On the basis of the above-described fact, the memory unit 400 may drive the drive frequency value, the initial phase value, and the amplitude value of the drive signals on both axes of the mirror corresponding to the specific number of patterns, the specific pattern shape, the specific field of view, and a combination thereof. This structured and stored in advance and communicates with the control unit 500 to transmit values corresponding to the information input by the user.

8 is a flowchart illustrating a method for generating variable structured light according to an embodiment of the present invention.

Referring to FIG. 8, a method for generating variable structured light using the variable structured light generating apparatus 1000, and the method for generating variable structured light according to an embodiment of the present invention may include a pattern number and pattern shape of desired structured light from a user. And receiving information about a field of view (S10), driving frequencies f _x and f _y , initial phases φ _x , φ _y , and amplitudes A _x , of the biaxial drive signal of the mirror corresponding to the received information. A _y ) is extracted in communication with the memory unit, and the least common multiple of the driving frequency (f _x , f _y ) or the multiple of the minimum common multiple of the extracted biaxial drive signals of the mirror is calculated as the laser modulation frequency of the light source unit. And a sine wave control signal formed as a value for driving frequencies f _x , f _y , initial phases φ _x , φ _y , and amplitudes A _x , A _y of the biaxial drive signals of the extracted mirrors, and A sphere formed at the calculated laser modulation frequency of the light source unit Generating a modulated signal of the wave (S20) and transmitting the control signal to the mirror and transmitting the modulated signal to the light source unit so that the modulated laser light is reflected by the mirror performing lisage scanning to generate structured light It includes a step (S30).

A detailed description of each step of the variable structured light generating method according to an embodiment of the present invention may be replaced with the description of the variable structured light generating apparatus 1000 according to an embodiment of the present invention.

9 is a schematic diagram illustrating a three-dimensional imaging system according to an embodiment of the present invention.

Referring to FIG. 9, a 3D imaging system according to an exemplary embodiment of the present invention includes a variable structure lighting device 1000 ′ , a camera module 2000, and an image processing module 3000. In the system, the variable structure lighting apparatus 1000 ′ generates structured light having a variable number of patterns, a pattern shape, and a field of view, and irradiates the object 10. The camera module 2000 captures an image of the subject to which the structured light is irradiated, and the image processing module 3000 processes the image captured by the camera module 2000 and generates a depth image of the surface of the subject 10. Restore the three-dimensional shape.

The camera module 2000 may be a general camera including lenses that collect light reflected from a subject and an image sensor that converts an optical signal into an electrical signal, and captures an image of the subject including a pattern of structured light irradiated to the subject. The image data is transferred to the image processing module 3000.

The image processing module 3000 receives an image captured from the camera module 2000, generates coordinates in the image, and shifts the amount of shift of the pattern reflected on the subject 10 with respect to the reference pattern at each coordinate. By calculating the depth information on the surface of the subject and to restore the three-dimensional shape of the subject. The image processing module 3000 may be configured to perform the above-described image processing function in a device such as a computer.

The variable structure lighting device 1000 ′ , the camera module 2000 and the image processing module 3000 may be individually configured in separate PC environments, or the variable structure lighting device 1000 ′ , the camera module (in an embedded form). 2000 and the image processing module 3000 may be configured to be driven by one board, or as shown in FIG. 9, only the variable structure lighting apparatus 1000 ′ and the camera module 2000 may be configured in an embedded form.

FIG. 10 is a schematic diagram illustrating a variable structure lighting apparatus 1000 ′ used in a 3D imaging system according to an exemplary embodiment of the present invention. Referring to FIG. 10, the variable structure lighting apparatus 1000 ′ is modulated. A light source 1100 for outputting laser light modulated by a signal, and a scanner 1200 for reflecting the laser light, and driving signals of both axes orthogonal to each other are inputted as sine waves to perform Lissajous scanning. And generate structured light having an arbitrary pattern and irradiate the subject.

The scanner 1200 includes a mirror 1210 and a driver 1220 for driving the mirror 1210 to Lissajou scanning. The mirror 1210 may be a micro-electro mechanical system (MEMS) mirror, and may include a mirror that reflects light, and if the scanner is capable of applying a sine wave to both axes orthogonal to each other on a mirror plane to enable Lissajou scanning, the MEMS It is not limited to the mirror and may be used for the variable structure lighting device 1000 ′ . For example, the scanner 1200 may be a galvano mirror scanner or the like. The driver 1220 transmits a driving signal to the mirror 1210, that is, V _x (t) = A _x sin (2πf _x t + φ _x ) and V _y (t) = A _y sin (2πf _y · t + φ _y ) When the driving signal is applied to the mirror 1210, the light reflected from the mirror surface performs Lissajous scanning.

The light source 1100 outputs laser light modulated so that the light reflected from the scanner 1200 is structured light having an arbitrary pattern. The light source 1100 includes a laser diode and outputs modulated laser light in a waveform of a modulation signal applied to the laser diode. At this time, the modulated signal may be a spherical pile of the ON / OFF method, as shown in Figure 10, it is not limited to this may have a variety of forms, such as sine wave, sawtooth wave. However, the frequency of the modulated signal is a multiple of the least common multiple of the biaxial driving frequencies f _x and f _y of the scanner 1200. In addition, it is preferable that the wavelength band of the laser light corresponds to the visible or infrared ray band, and the wavelength of the laser light is determined by the type of the subject.

The variable structure lighting apparatus 1000 ′ used in the 3D imaging system according to an exemplary embodiment of the present invention includes a light source 1100 and a scanner 1200, and the two axes of which the modulated laser light output from the light source is orthogonal to each other. Driving signals of V _x (t) = A _x sin (2πf _x t + φ _x ) and V _y (t) = A _y sin (2πf _y t + φ _y ) are applied to (X and Y axes). The light is incident on the mirror 1210 of the vibrating scanner 1200, and the scanner 1200 reflects the laser light to generate structured light having a pattern. The generated structured light is irradiated onto the subject to obtain three-dimensional depth information of the subject.

The variable structure lighting apparatus 1000 ′ used in the 3D imaging system according to an exemplary embodiment of the present invention is a dual axis of the mirror 1210 in the scanner 1200 that scans the frequency and duty cycle of the modulated signal of the laser light and the Lissajous. By changing the driving frequency, initial phase and amplitude of the driving signal, it is possible to change the number of patterns, pattern shape, and field of view of the structured light.

The variable structured light generating principle of the variable structured lighting device 1000 ′ used in the 3D imaging system according to the exemplary embodiment of the present invention is the variable structured light generating apparatus according to the embodiment of the present invention as described above. 1000).

In this case, the mirror 200 of the variable structured light generating apparatus 1000 may include the mirror 1210 of the variable structured lighting apparatus 1000 ′ used in the 3D imaging system according to the exemplary embodiment. The same configuration can be.

In addition, it is possible to change the pattern number, pattern shape, and field of view of the structured light by adjusting the frequency, duty cycle of the modulation signal of the laser light and the driving frequency, initial phase, and amplitude of the mirror when performing Lissajous scanning. 7 and Table 1 illustrate this. Since the same applies to the three-dimensional imaging system according to an embodiment of the present invention, a detailed description thereof will be omitted.

11 illustrates a perspective view of a variable structure lighting apparatus 1000 ′ used in a 3D imaging system according to an embodiment of the present invention, and FIG. 12 illustrates a variable used in a 3D imaging system according to an embodiment of the present invention. A cross-sectional view of the structural lighting device 1000 ′ is shown.

11 and 12, the variable structure lighting apparatus 1000 ′ used in the 3D imaging system according to an exemplary embodiment of the present invention may include a collimator 1260, an optical fiber 1270, a scanner mount 1280, and a housing. 1290 may be further included.

The collimator 1260 is disposed on a path where the laser light output from the light source 1100 is directed to the mirror 1210 of the scanner 1200 to make the laser light a collimated beam. A collimated beam refers to a parallel beam with very little dispersion or concentration in the laser. The collimator 1260 may be a set of lenses. The optical fiber 1270 is positioned between the light source 1100 and the collimator 1260, and combines with the laser light output from the light source 1100 to transmit the laser light to the collimator 1260 or the scanner 1200.

The housing 1290 accommodates one end of the optical fiber 1270, the collimator 1260, and the scanner 1200, and a life preserver capable of inserting one end of the optical fiber is formed at one end thereof, and a scanner mount 1280 is formed at the inner bottom of the other end thereof. Is provided, and an opening is formed in the upper surface of the other end part. The housing 1290 accommodates one end of the optical fiber 1270, the collimator 1260, and the scanner 1200 at fixed positions, so that the laser light output from the light source 1100 is accurately reflected by the scanner 1200 so that the object ( 10) to ensure that it is irradiated with structured light.

In addition, the housing 1290 may further include a driver 1220 of the scanner. The driver 1220 of the scanner is electrically connected to the mirror 1210 of the scanner and applies a driving signal to the mirror 1210 of the scanner so that the scanner executes Lissajous scanning, and the light reflected from the scanner results in the number of patterns. In order to form structured light having a variable pattern shape and a field of view, a sinusoidal wave having various frequencies, initial phases, and amplitudes may be applied to both orthogonal axes in the mirror surface of the scanner, as shown in FIG. It may be formed of a flexible printed circuit board and accommodated in the housing 1290. In addition, the driver 1220 of the scanner may be electrically connected to the light source 1100 to provide information for determining the frequency of the modulated signal. The scanner 1220 may be electrically connected to the camera module 2000 to perform irradiation of structured light and to capture an image. The operation may be synchronized, and may be electrically connected to the image processing module 3000 to provide the reference pattern to the image processing module. The reference pattern may be electrically connected to the user interface to have a desired number of patterns, pattern shapes, and views. Information about structured light can be received.

In addition, the housing 1290 may further accommodate a tilt unit (not shown) that enables tilting the scanner mount 1280. The tilt unit is mounted on the scanner mount 1280 to adjust the tilt of the scanner mount 1280. When the tilt of the scanner mount 1280 is adjusted, the tilt unit irradiates the structured light to the subject without moving the subject and the variable structure lighting device. can do. The tilt unit may be configured with an actuator or the like.

Since the size of the scanner included in the variable structure lighting apparatus 1000 ′ used in the 3D imaging system according to the exemplary embodiment of the present invention is approximately 1 mm × 1 mm, the apparatus can be miniaturized and embedded in the terminal. 3 It can also be used for medical devices such as endoscopes and the like which require dimensional shape information.

As described above, the variable structure lighting apparatus 1000 ′ used in the 3D imaging system according to the exemplary embodiment of the present invention includes a scanner, and the scanner includes a mirror 1210 and a driver 1220. May be a MEMS mirror.

A detailed embodiment of the MEMS mirror may be replaced with the above description and FIGS. 3 and 4.

The variable structure lighting apparatus 1000 ′ used in the 3D imaging system according to the exemplary embodiment of the present invention also has a variable structure variable light having a variable number of patterns like the variable structure lighting apparatus 1000 according to the exemplary embodiment of the present invention. To implement, the driving frequency of the mirror, which determines the number of patterns, should be variously selected, and the resonance bandwidth, which is the choice, should be wide. In other words, a wide resonance bandwidth means a low Q-factor of the vibration mode.

Therefore, when the MEMS mirror is used as the mirror 1210 in the variable structure lighting apparatus 1000 ′ used in the 3D imaging system according to an exemplary embodiment of the present invention, in order to generate the structured light having a variable number of patterns, MEMS mirrors are rotated about two axes perpendicular to each other and are designed to have at least one of a quality factor for rotational vibration of the first axis or a quality factor for rotational vibration of the second axis, or one of the two quality factors lower. It is preferred that the lower quality factor is less than or equal to 100.

Figure 13 shows a schematic flowchart of a three-dimensional imaging method according to an embodiment of the present invention. Referring to FIG. 13, in the three-dimensional imaging method according to an embodiment of the present invention, the method may include: irradiating structure light generated by the variable structure lighting apparatus to a subject (S100) and photographing an image of the subject to which the structure light is irradiated. And reconstructing the three-dimensional shape of the subject by processing the captured image (S200).

A detailed description of each step of the three-dimensional imaging method may be replaced with a description of the three-dimensional imaging system according to an embodiment of the present invention.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

[Description of the code]

1000: variable structured light generating device

100: light source

200, 1210: mirror

300: information input unit

400: memory

500: control unit

510: information receiver

520: signal generator

530: signal transmission unit

1000 ': Variable structure lighting device

1100: light source

1200: Scanner

1220: drive unit

1260: collimator

1270: optical fiber

1280: Scanner Mount

1290: Housing

2000: Camera Module

3000: Image Processing Module

Claims (19)

1. A light source unit outputting modulated laser light;

   A mirror for reflecting the laser light and receiving driving signals of two axes orthogonal to each other by a sine wave to perform LISA scanning;

   An information input unit configured to input information of a pattern number, pattern shape, and a field of view of a desired structure light;

   A memory unit in which values for driving frequency, initial phase, and amplitude of the biaxial drive signal of the mirror corresponding to the information are stored in advance; And

   And a control unit which receives the information to generate the structured light corresponding to the information and communicates with the memory unit to generate and transmit a control signal of the mirror and a modulation signal of the light source unit.
2. The method of claim 1,

   The control unit,

   An information receiver configured to receive information on the pattern number, pattern shape, and field of view of the structured light input through the information input unit;

   Values for driving frequency, initial phase, and amplitude of the two-axis driving signals of the mirror corresponding to the information are extracted by communicating with the memory unit, and the least common multiple of the driving frequencies of the two-axis driving signals of the mirror is extracted. Computes a modulation frequency, and generates a control signal of the sine wave formed by the values for the drive frequency, the initial phase and the amplitude of the two-axis drive signal of the extracted mirror and a modulated signal of the square wave formed by the calculated laser modulation frequency of the light source portion Signal generation unit to; And

   And a signal transmitter for transmitting the control signal to the mirror and transmitting the modulated signal to the light source.
3. The method of claim 1,

   And the mirror is a MEMS mirror.
4. The method of claim 3,

   The MEMS mirror is a variable structured light, characterized in that the rotational vibration on the basis of both axes perpendicular to each other and the quality factor (Q-factor) for the rotational vibration of the first axis is lower than the quality factor for the rotational vibration of the second axis Generating device.
5. The method of claim 1,

   The number of patterns of the structured light is

   Determined by the greatest common divisor having a negative correlation with the greatest common divisor of the driving frequencies of the driving signal,

   And varying according to an initial phase of the drive signal, a duty cycle of the modulated signal, or an initial phase of the modulated signal. A variable structured light generating device, characterized in that.
6. The method of claim 1,

   The pattern shape of the structured light is changed according to at least one of a driving frequency of the driving signal, an initial phase of the driving signal, a waveform of the modulation signal, a duty cycle of the modulation signal, and an initial phase of the modulation signal. A variable structured light generating device.
7. The method of claim 1,

   And the field of view of the structured light has a positive correlation with the amplitude of the biaxial drive signal of the drive signal and is determined by the amplitude.
8. The method of claim 1,

   And a laser beam output from the light source unit is modulated by an ON / OFF square wave.
9. The method of claim 1,

   And a frequency of the modulated signal is a multiple of a least common multiple of a drive frequency of both axis drive signals of the mirror.
10. The method of claim 1,

    The light source unit further includes a light modulator on one side of the laser toward the mirror to modulate the laser light, the modulated signal of the square wave of the ON / OFF method is applied to the light modulator, the modulation frequency of the square wave is And a minimum common multiple of a driving frequency of the two-axis driving signal of the mirror or a multiple of the minimum common multiple.
11. A variable structure lighting device for generating structure light having a variable number of patterns, a pattern shape, and a field of view;

    A camera module for capturing an image of the subject to which the structured light is irradiated; And

    And an image processing module for restoring the 3D shape of the subject by processing the image photographed by the camera module.

    The variable structure lighting apparatus includes a light source for outputting a laser light modulated by a modulated signal and the laser light, and driving signals of both axes orthogonal to each other are input as sine waves to perform Lissajous scanning. Includes a scanner,

    And the frequency of the modulated signal is a multiple of the least common multiple of the drive frequencies of the drive signal.
12. The method of claim 11,

    The number of patterns of the structured light is

    Determined by the greatest common divisor having a negative correlation with the greatest common divisor of the driving frequencies of the driving signal,

    And an initial phase of the drive signal, a duty cycle of the modulated signal, or an initial phase of the modulated signal.
13. The method of claim 11,

    The pattern shape of the structure light is changed according to at least one of a driving frequency of the driving signal, an initial phase of the driving signal, a waveform of the modulation signal, a duty cycle of the modulation signal, and an initial phase of the modulation signal. 3D imaging system.
14. The method of claim 11,

    And the field of view of the structured light is varied according to the amplitude with a positive correlation with the amplitude of the drive signal.
15. The method of claim 11,

    The variable structure lighting device,

    And a collimator disposed on a path toward which the laser light output from the light source is directed to the scanner to make the laser light into a collimated beam.
16. The method of claim 15,

    The variable structure lighting device,

    An optical fiber for coupling the laser light output from the light source; And

    One end of the optical fiber, the collimator and the housing for receiving the scanner further,

    A hole into which one end of the optical fiber is inserted at one end of the housing, a scanner mount is provided on an inner lower surface of the other end of the housing, and an opening is formed on an upper surface of the other end of the housing; 3D imaging system.
17. The method of claim 16,

    And the scanner mount is tiltable.
18. The method of claim 11,

    And the scanner comprises a mirror and a drive unit for driving the mirror to Lissajou scanning.
19. The method of claim 18,

    And the mirror is a MEMS mirror.

Images