KR20170076477A - Method and device for acquiring distance information - Google Patents

Abstract

There is provided a distance information acquiring method capable of efficiently reducing an error by controlling a projection light to be projected when a device acquires distance information with respect to a subject.

Description

[0001] The present invention relates to a method and device for acquiring distance information,

The disclosed subject matter relates to a method for obtaining distance information by a device that obtains distance information, and more particularly to a method for a device to obtain a three-dimensional image.

In recent years, the importance of 3D contents has been highlighted along with the development of 3D display devices capable of displaying deep-seated images and the increasing demand. Accordingly, a three-dimensional image acquiring device such as a 3D camera capable of directly producing 3D contents by a general user is being studied. Such a 3D camera should be able to acquire depth information together with existing two-dimensional color image information in one shot.

Depth information on the distance between the surface of the subject and the 3D camera can be obtained by using a stereoscopic vision method using two cameras or a triangulation method using a structured light and a camera have. However, in this method, it is difficult to obtain accurate depth information because the accuracy of the depth information is drastically deteriorated as the distance of the subject becomes farther away, and it depends on the surface condition of the subject.

Time-of-flight (TOF) has been introduced to solve this problem. TOF technology is a method of measuring the light flight time from the time when the light reflected from the subject is received by the light receiving unit after illuminating the subject with the illumination light. According to the TOF technique, light of a specific wavelength (for example, near-infrared light of 850 nm) is projected to a subject using an illumination optical system including a light emitting diode (LED) or a laser diode (LD), and light of the same wavelength After receiving the light by the light receiving unit, the received light is modulated by a modulator having a known gain waveform, and a series of processes are performed to obtain depth information. A variety of TOF technologies are introduced in this series of optical processes.

There is provided a distance information acquiring method capable of efficiently reducing an error by controlling a projection light to be projected when a device acquires distance information with respect to a subject.

According to a first aspect of the present invention, there is provided a method of acquiring information about a subject, the method comprising: acquiring N images sequentially projected on the subject, wherein N is a natural number of 3 or more ) Based on the position of the object and / or an external received input; Sequentially projecting the N different projection lights onto the subject according to a projection order of the projection light; Modulating N reflected light reflected from the subject to obtain N modulated reflected light; And obtaining distance information on the object using the N modulated reflected light.

The step of determining the projection order of the projection light on the basis of the position of the object and / or the external reception input may include: acquiring distance information between the device and the object; projecting the projection light on the basis of the obtained distance information; The order can be determined.

In addition, distance information between the device and the subject can be obtained in real time.

The step of determining the projection order of the projection light on the basis of the position of the subject and / or the external reception input may further include acquiring distance information between the device and the subject, and determining, based on the obtained distance information, The projection order of the projection light for minimizing the motion blur can be determined.

The step of determining the projection order of the projection light on the basis of the position of the object and / or the external reception input may include determining a projection order that minimizes motion blur for the object among a plurality of predetermined projection orders, Can be determined in order.

In addition, the N different projection lights may include periodic waves having the same period and at least one of a magnitude and a phase different from each other.

The step of modulating the N reflected light reflected from the subject to obtain the N modulated reflected light may include modulating the N reflected light using an optical modulated signal having a gain waveform to generate N The modulated reflected light can be obtained.

In addition, the optical modulation signal may include a periodic wave having the same period as the projected light.

In addition, the N different projected lights may have different phases from each other, and there may be a phase difference of N degrees of 360 degrees between the N different projected lights.

The step of determining the projection order of the projection light based on the position of the object and / or the external reception input may determine the projection order of the projection light based on the external reception input determined based on the input of the user.

According to a second aspect of the present disclosure, there is provided a device for acquiring information on a subject, the device comprising: a projecting sequence of N projected light beams to be projected sequentially to the subject (where N is a natural number of 3 or more) And / or an external reception input; A light source sequentially projecting the N different projection lights onto the subject according to a projection order of the projection lights; A modulator for modulating N reflected light reflected from the subject to obtain N modulated reflected light; And a controller for obtaining distance information on the object using the N modulated reflected lights.

The projection light control unit may acquire distance information between the device and the subject, and may determine a projection order of the projection light based on the obtained distance information.

In addition, distance information between the device and the subject can be obtained in real time.

In addition, the projection light control unit may acquire distance information between the device and the subject, and may determine a projection order of the projection light to minimize motion blur for the subject based on the obtained distance information .

In addition, the projection light control unit may determine a projection order that minimizes motion blur for the subject among a plurality of preset projection orders in the projection order of the projection light.

In addition, the N different projection lights may include periodic waves having the same period and at least one of a magnitude and a phase different from each other.

The modulator may modulate the N reflected light using the optical modulating signal having a gain waveform to obtain the N modulated reflected light.

In addition, the optical modulation signal may include a periodic wave having the same period as the projected light.

In addition, the N different projected lights may have different phases from each other, and there may be a phase difference of N degrees of 360 degrees between the N different projected lights.

In addition, the third aspect of the present disclosure can provide a computer-readable non-transitory medium in which a program for implementing the method of the first aspect is recorded.

In the case where the device obtains the distance information with respect to the object, the error can be effectively reduced by controlling the projection light to be projected.

Figure 1 schematically illustrates an exemplary structure of a device that can acquire distance information using optical time-of-flight (TOF).
FIG. 2 shows a process of modifying N different reflected lights and then generating N different images in the imaging device.
3 shows a process of generating N different images with one and the same projection light and N different optical modulation signals.
FIG. 4 shows a method of generating four different images with four projected lights having different phases and acquiring distance information using the generated four different images.
5 illustrates a method of obtaining Contact Image Sensors (CIS) images using reflected light and light modulated signals.
Fig. 6 shows an example of four projected lights whose phases are different from each other.
FIG. 7 is a diagram showing a change in brightness of an image obtained by four projected lights whose phases are different from each other when a subject moves. FIG.
Fig. 8 shows an example in which distance information is acquired by controlling the order of the projected light when the subject moves.
9 shows another example of obtaining distance information by controlling the order of the projected light when the subject moves.
10 is a diagram for explaining a method of determining the projection order of the projection light according to the distance between the subject and the device.
11 is a view showing an example of obtaining distance information by controlling the order of projection light according to the distance between a subject and a device.
12 is a block diagram illustrating a configuration of a device according to an embodiment.
Fig. 13 schematically shows an exemplary structure of a device for obtaining distance information through the control of projection light.
14 is a flowchart illustrating a method for a device according to an embodiment to acquire distance information using N projected lights.
15 is a flow diagram illustrating a method for a device according to an embodiment to acquire distance information using N different-phase projected lights.
16 is a flowchart showing a method of a device according to an embodiment for projecting different projection light according to a projection order of projection light determined based on an external reception input to acquire distance information.

Brief Description of the Drawings The advantages and features of the present invention, and how to accomplish them, will become apparent with reference to the embodiments described hereinafter with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

The terms used in this specification will be briefly described and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements, without departing from the spirit or scope of the present invention. Also, as used herein, the term "part " refers to a hardware component such as software, FPGA or ASIC, and" part " However, "part" is not meant to be limited to software or hardware. "Part" may be configured to reside on an addressable storage medium and may be configured to play back one or more processors. Thus, by way of example, and not limitation, "part (s) " refers to components such as software components, object oriented software components, class components and task components, and processes, Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and "parts " may be combined into a smaller number of components and" parts " or further separated into additional components and "parts ".

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

Throughout the specification, " distance " can mean a spatially separated length, and " depth " For example, the distance information may include depth information.

Throughout the specification, the image includes, but is not limited to, a CCD (Charge Coupled Device) image and a CIS (Contact Image Sensors or CMOS Image Sensors) image.

Hereinafter, an 'image' may represent a still image or an image of the video, or a moving image, that is, the video itself.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a method and apparatus for acquiring distance information will be described in detail with reference to the accompanying drawings. In the following drawings, like reference numerals refer to like elements, and the size of each element in the drawings may be exaggerated for clarity and convenience of explanation.

1 schematically illustrates an exemplary structure of a device 100 in accordance with one embodiment that can obtain distance information using optical time of flight (TOF). 1, a device 100 includes a light source 101 for generating light having a predetermined wavelength, a light source driving unit 102 for driving the light source 101, An optical modulator driving section 104 for driving the optical modulator 103, an imaging device 105 for generating an image from light optically modulated by the optical modulator 103, A distance image processor 107 for calculating distance information based on the output of the light source driving unit 102, the optical modulator driving unit 104, the image pickup device 105, And a control unit 106 for controlling the operation of the processing unit 107. [ A first lens 108 for condensing the reflected light in the region of the optical modulator 103 and a filter 109 for transmitting only light having a predetermined wavelength can be further disposed on the light incident surface of the optical modulator 103 have. Between the optical modulator 103 and the imaging device 105, a second lens 110 for condensing the optically modulated image into the area of the imaging device 105 may be further disposed.

The light source 101 may be, for example, a light emitting diode (LED) or a laser diode (LD) capable of emitting light having a near infrared (NIR) wavelength of about 800 nm to about 1100 nm which is invisible to the human eye for safety However, the wavelength band and the type of the light source are not limited. The light source driving unit 102 may drive the light source 101 according to a control signal received from the control unit 106, for example, by amplitude modulation or phase modulation. The projection light projected from the light source 101 to the subject 200 may have a form of a periodic continuous function having a predetermined period in accordance with the driving signal of the light source driving unit 102. [ For example, the projection light may have a specially defined waveform such as a sine wave, a ramp wave, a square wave, etc., but may have a waveform of a general type that is not defined.

The optical modulator 103 optically modulates the light reflected from the subject 200 under the control of the optical modulator driving unit 104. The optical modulator driving unit 104 drives the optical modulator 103 according to the control signal received from the control unit 106. [ For example, the optical modulator 103 can modulate the magnitude of the reflected light by changing the gain according to the optical modulated signal having the predetermined waveform provided by the optical modulator driver 104. [ To this end, the optical modulator 103 has a variable gain. The optical modulator 103 can operate at a high optical modulation rate of several tens to several hundreds of megahertz to identify the phase difference or travel time of light along the distance. As the optical modulator 103 in conformity with this, for example, an image intensifier having a multi-channel plate (MCP), a GaAs-based solid-state modulator device, and a thin-type modulator device using an electro- . Although FIG. 1 shows the optical modulator 103 as a transmission type, it is also possible to use a reflective optical modulator.

The image pickup element 105 serves to detect reflected light optically modulated by the optical modulator 103 under the control of the control unit 106 and generate an image. If only the distance to a point on the subject 200 is desired to be measured, the imaging device 105 may use one single optical sensor, for example a photodiode or an integrator. However, when it is desired to simultaneously measure the distances up to a plurality of points on the subject 200, the imaging device 105 may have a two-dimensional or one-dimensional array of photodiodes or other photodetectors. For example, the imaging device 105 may be a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor having a two-dimensional array. The distance information image processing unit 107 calculates the distance information based on the distance information acquisition algorithm on the basis of the output of the image pickup device 105. The distance information acquisition algorithm may be pre-set. The distance information image processing unit 107 may be implemented, for example, by a dedicated integrated circuit (IC) or software installed in the device 100. [ When implemented in software, the distance information image processing unit 107 may be stored in a separate removable storage medium.

Hereinafter, the operation of the device 100 having the above-described structure will be schematically described.

First, the light source 101 projects N different projected lights having a predetermined period and waveform to the subject 200 in succession under the control of the control unit 106 and the light source driving unit 102. Where N may be a natural number of 3 or greater. For example, in the case of using four different projected lights, the projection light 1 is generated for a time T1 and projected onto the subject 200. The projected light 2 is projected onto the subject 200 for the next time T2, The projection light 3 may be generated during the time T3 to be projected onto the subject 200 and then projected to the subject 200 during the time T4. The projected lights sequentially projected onto the subject 200 may have the form of a continuous function having a specific period such as a sinusoidal wave. For example, the projection light 1 to the projection light 4 may be periodic waves having the same period and waveform but different in magnitude or phase from each other. As another example, the projection light 1 to the projection light 4 may have periodic files having the same period, waveform, size, and different phases only. In this case, the phase of the projection light 1 is 90 degrees slower than that of the projection light 2, the phase of the projection light 2 is 90 degrees slower than the projection light 3, The phase of the projection light 3 may be 90 degrees slower than the projection light 4. [ Or in this case, the phase of the projection light 1 is 90 degrees faster than the projection light 2, the phase of the projection light 2 is 90 degrees faster than the projection light 3, and the phase of the projection light 3 is 90 degrees faster than the projection light 4.

The projection light projected on the subject 200 is reflected by the surface of the subject 200, and is then incident on the first lens 108. In general, the subject 200 has a number of surfaces with different distances, i.e. distances, from the device 100. In FIG. 1, for the sake of simplicity of description, an object 200 having five surfaces P1 to P5 having different distances is exemplarily shown. The projection light is reflected on each of the five surfaces P1 to P5 having different distances, and five reflected lights having different time delay (i.e., different phases) are generated. For example, when the projection light 1 is reflected on five surfaces P1 to P5 of the subject 200, five reflected lights 1 having different phases are generated, and the projection light 2 is reflected on five surfaces P1 To P5, and the five reflected lights 2 having different phases are generated. Likewise, the projected lights N are reflected by the five surfaces P1 to P5 of the subject 200, and five reflected lights N having different phases are generated . The reflected light reflected at the surface P1 that is the farthest from the device 100 reaches the first lens 108 after a time delay of PHI P1 and reaches the distance from the device 100 to the nearest surface P5 The reflected reflected light will reach the first lens 108 after a time delay of PHI P5 that is less than PHI P1.

The first lens 108 focuses the reflected light in the area of the optical modulator 103. A filter 109 for transmitting only light having a predetermined wavelength may be disposed between the first lens 108 and the optical modulator 103 in order to remove background light or other light other than the wavelength used. For example, when the light source 101 emits light having a near-infrared (NIR) wavelength of about 850 nm, the filter 109 may include a near-IR band pass filter for passing a near-infrared wavelength band of about 850 nm, Lt; / RTI > Accordingly, the light incident on the optical modulator 103 may be emitted from the light source 101, and the light reflected from the subject 200 may dominate. Although the filter 109 is shown in FIG. 1 as being disposed between the first lens 108 and the optical modulator 103, the positions of the first lens 108 and the filter 109 may be interchanged. For example, the near-infrared light having first passed through the filter 109 may be focused by the first lens 108 to the optical modulator 103.

Then, the optical modulator 103 modulates the reflected light with an optical modulated signal having a predetermined waveform. The period of the gain waveform in the optical modulator 103 may be the same as the waveform period of the projection light. 1, the optical modulator 103 optically modulates the five reflected light 1 reflected from the five surfaces P1 to P5 of the subject 200 to provide them to the imaging element 105, Two to five reflected light N can be sequentially optically modulated and provided to the image pickup device 105. [

The light modulated in size by the optical modulator 103 passes through the second lens 110 and is scaled and refocused and then reaches the imaging element 105. Thus, the modulated light is condensed by the second lens 110 in the region of the image pickup element 105. [ The image pickup device 105 receives the modulated light for a predetermined exposure time and generates an image. For example, as shown in (A) of Fig. 2, the image pickup element 105 may emit five reflected light 1 modulated after being respectively reflected by five surfaces P1 to P5 of the subject 200 to a predetermined exposure time The image 1 is generated. Subsequently, as shown by (B) in Fig. 2, the image pickup element 105 receives the five reflected light 2 modulated after being respectively reflected by the five surfaces P1 to P5 of the subject 200 for a predetermined exposure time And generates an image 2. 2, the image pickup element 105 picks up five reflected light beams N (N), which are respectively reflected by the five surfaces P1 to P5 of the subject 200 and then modulated, For a predetermined exposure time to generate an image N. In this way, as shown in (D) in Fig. 2, N different images can be sequentially obtained. Each of the images 1 to N thus obtained may be a sub-frame image for generating an image of one frame having distance information. For example, if the period of one frame is Td, the exposure time in the image pickup device 105 for obtaining each of the N images 1 to N may be approximately Td / N.

2 (A), in the first sub-frame, the projection light 1 projected from the light source 101 onto the subject 200 is reflected by the five surfaces P1 to P5 of the subject 200, 1 < / RTI > The five reflected lights 1 are modulated by the optical modulator 103 and then arrive at the imaging element 105. In Fig. 2, for ease of explanation, the imaging element 105 is shown as having only five pixels corresponding to five surfaces P1 to P5, respectively. Therefore, the five reflected light 1 can be incident on the corresponding five pixels, respectively. As shown in Fig. 2A, the five reflected light 1 reflected from each of the surfaces P1 to P5 has different phase delays? P1 to P5 depending on the distance from the device 100 to the surfaces P1 to P5, Respectively. The image pickup element 105 can generate the image 1 by photographing the reflected light 1 for an exposure time of, for example, approximately Td / N. In the same manner as above, the images 2 to N can be generated from the second sub-frame to the N-th sub-frame. As shown in Figs. 2 (B) and 2 (C), different phase delays? PH1 to? P5 are generated from the five surfaces P1 to P5 having different distances from the second to the Nth subframes.

In FIG. 2, N different images are generated using N different projected lights and reflected lights. However, it is also possible to use the same projection light in all the subframes, and to modulate the reflected light with different gain waveforms in the optical modulator 103 for each subframe. FIG. 3 illustrates a process of generating N different images using one and the same projection light and N different gain waveforms. Referring to FIG. 3, the reflected light reflected from the subject 200 has the same waveform and phase in all subframes. As described above, there are different phase delays? P1 to? P5 depending on the surfaces P1 to P5 of the subject 200 in the reflected light of each subframe. As shown in Figures 3A-3C, the optical modulator 103 modulates the reflected light in the first subframe and the optical modulator 103 modulates the reflected light in the second subframe, Modulates the reflected light with the modulated signal 1 and modulates the reflected light with the other optical modulated signal N in the Nth sub-frame. Here, the optical modulation signals 1 to N may be signals of totally different waveforms, but they may be signals having the same period and waveform but different phases only. Then, as shown in FIG. 3 (D), N different images 1 to N can be obtained.

The N images obtained in the above-described manner are transmitted to the distance information image processing unit 107. The distance information image processing unit 107 may obtain distance information according to a predetermined algorithm using the N images. For example, the device 100 may obtain distance information using an averaging algorithm.

FIG. 4 shows a method of generating four different images with four projected lights having different phases and acquiring distance information using the generated four different images.

, 위상차가 90도인 강도 이미지인

, 위상차가 180도인 강도 이미지인

및 위상차가 270도인 강도 이미지인

를 획득할 수 있다. 일 실시 예에 따른 디바이스(100)는 움직임 평균(moving average)를 이용하여 4개의 CIS 이미지를 획득할 수 있다. 본 명세서에서 변조된 영상의 표시를 CIS 이미지로 표시하였지만, 촬상 소자(105)는 CIS로 한정되지 않는다.As can be seen in the first portion 410, the device 100 according to one embodiment may acquire four intensity images through sequential shooting. For example, the device 100 according to one embodiment may be a intensity image with a phase difference of zero degrees

, Intensity image with a phase difference of 90 degrees

, Intensity image with a phase difference of 180 degrees

And intensity image with a phase difference of 270 degrees.

Can be obtained. The device 100 according to one embodiment may obtain four CIS images using a moving average. Although the display of the modulated image is represented by the CIS image in this specification, the image pickup element 105 is not limited to the CIS.

The device 100 according to an embodiment can acquire four images in the same order as in Equation (1).

The device 100 according to an embodiment may obtain a combination of four images as shown in Equation (2) by acquiring two new images and successively removing the two existing images.

  (Where p is an arbitrary number)

인 경우, 앞에 획득한 2개의 이미지를 순차적으로 제거하고 새로운 이미지 두개를 획득하여

의 4개 이미지 조합을 획득할 수 있다. 예를 들면, 제1 이미지(411)는 제거되고, 제 2 이미지(413)는 추가될 수 있다. 다른 예로, 일 실시 예에 따른 디바이스(100)는 현재 획득한 4개의 이미지가

인 경우, 앞에 획득한 2개의 이미지를 순차적으로 제거하고 새로운 이미지 두개를 획득하여

의 4개 이미지 조합을 획득할 수 있다.For example, the device 100 according to one embodiment may include four currently acquired images

, Two previously acquired images are sequentially removed and two new images are acquired

Of the image. For example, the first image 411 may be removed and the second image 413 may be added. As another example, a device 100 according to one embodiment may include four currently acquired images

, Two previously acquired images are sequentially removed and two new images are acquired

Of the image.

The device 100 according to an embodiment can acquire a depth image as in Equation (3) using the four images 412 currently acquired. The four images 412 currently acquired may include four intensity images.

-

로 나타내어지는 제1 중간 이미지(421) 및

-

로 나타내어지는 제2 중간 이미지(422)를 획득할 수 있다. 또한, 제3 부분(430)에서 확인할 수 있는 바와 같이 일 실시 예에 따른 디바이스(100)는 제1 중간 이미지(421) 및 제2 중간 이미지(422)를 이용하여 깊이 이미지(431)를 획득할 수 있다.Specifically, as can be seen in the second portion 422, the device 100 according to one embodiment

-

A first intermediate image 421 and a second intermediate image 422,

-

The second intermediate image 422 may be obtained. Also, as can be seen in the third part 430, the device 100 according to one embodiment obtains the depth image 431 using the first intermediate image 421 and the second intermediate image 422 .

Thus, the device 100 according to one embodiment can acquire one depth image at the time of acquiring two IR images.

5 illustrates a method of obtaining Contact Image Sensors (CIS) images using reflected light and light modulated signals. In Fig. 5, the light-irradiation process of the IR light can be explained.

는 S번째 투사광의 광출력(s-th emitting light optical power),

는 S번째 투사광의 위상차(phase shift of the s-th emitting light),

는 출력광의 DC 오프셋 (emitting light DC offset),

는 S번째 수신된 반사광의 광출력(s-th receiving light optical power),

는 수신된 외광(receiving ambient light), r은 피사체 표면의 광 감쇠(light attenuation of the object surface), G는 셔터 이득(shutter gain),

는 셔터 이득 DC 오프셋(shutter gain DC offset), w는 동작 주파수(operationg frequency),

는 TOF(Time of Flight)에 따른 위상 지연(phase delay due to TOF)을 의미할 수 있다.In the present specification,

(S-th emitting light optical power) of the S th projection light,

A phase shift of the S-th emission light,

Is a DC offset of the output light,

(S-th receiving light optical power) of the S-th received reflected light,

R is the light attenuation of the object surface, G is the shutter gain,

A shutter gain DC offset, w is an operation frequency,

May refer to a phase delay due to TOF (Time of Flight).

The light output of the S th projection light can be expressed as shown in Equation (4).

Where rect can be in the form of a square wave (AC) plus a DC component.

)은 [수학식 5]와 같이 표현될 수 있다. 필터를 통과한 후의 반사광은 피사체의 표면에서 반사되어 돌아오므로, 물체의 표면 반사도, 렌즈의 크기 등이 종합적으로 고려된 반사도인 r이 곱해진 형태이고, TOF에 의한 위상차가 존재하고, 외광이 존재할 수 있다.Reflected light after passing through filter

Can be expressed as: " (5) " Since the reflected light after passing through the filter is reflected and reflected on the surface of the object, the surface reflectance of the object, the size of the lens, etc. are multiplied by r, which is the reflectivity considered together, and there is a phase difference due to TOF, Can exist.

The modulated waveform Gain of the shutter can be expressed by the following equation (6).

For example, the modulation waveform G (t) of the shutter may be a form in which a DC component is added to a sine wave AC.

The light reaching the photographing element 105 can be expressed as shown in Equation (7).

The image obtained from the photographing element 105 can be expressed as shown in Equation (8).

The four images successively obtained from the above-described [Equation 8] can be expressed by [Equation 9] to [Equation 12].

Further, the equations (9) to (12) can satisfy the condition of (13).

 In the case where the light output of the S th projection light expressed by Equation (4) is realized by a triangular wave sine, the above equation expansion can be performed. As a result, A may be a different value A '. For example, A 'can be expressed as: " (14) "

The unknowns r, A, and B are canceled in the above-mentioned Equations (9) to (12) to solve the phase difference due to the depths as in the above-mentioned [Equation 3] or [Equation 15] Can be obtained.

The above-mentioned equations (9) to (15) can be applied to still images. For example, the above-mentioned Equations (9) to (15) are applicable to a subject without motion.

Further, Equation (16) can be obtained from Equations (3) and (15).

Fig. 6 shows an example of four projected lights whose phases are different from each other.

The four different projection lights may be first projection light 610, second projection light 620, third projection light 630, and fourth projection light 640 according to one embodiment. The first to fourth projected lights may differ in phase by 90 degrees. For example, the phase delay of the first projection light 610 is 0 degrees, the phase delay of the second projection light 620 is 90 degrees, and the phase difference of the third projection light 630 the phase delay of the fourth projection light 640 may be 180 degrees, and the phase delay of the fourth projection light 640 may be 270 degrees.

The modulated waveform G (t) 650 of the shutter according to one embodiment may be in the form of a sine wave (AC) plus a DC component.

FIG. 7 is a diagram showing a change in brightness of an image obtained by four projected lights whose phases are different from each other when a subject moves. FIG.

The brightness of the reflected light obtained by reflecting from the subject can be changed with the passage of time. For example, when the brightness of the i-th frame 710 is I, the brightness of the (i + 1) -th frame 720 may be I +?. The brightness of the i-th frame 710 and the brightness of the (i + 1) -th frame 720 may be different from each other.

The phase delay of the first projection light 610 is 0 degrees and the phase delay of the second projection light 620 is 90 degrees and the phase delay of the third projection light 630 is 180 degrees and the phase delay of the fourth projection light 640 is 270 degrees, the first projection light 610 to the fourth projection light 640 are sequentially projected onto the object.

According to one embodiment, when there is no change in the brightness of the reflected light obtained by reflection from the subject despite the lapse of time, the first frame 731, the second projection light (obtained by the first projection light 610) The third frame 733 obtained by the third projection light 630 and the fourth frame 734 obtained by the fourth projection light 640 obtained by the first projection light 620, Can be the same.

According to one embodiment, when there is a change in the brightness of the reflected light obtained by reflection from the subject over time, the first frame 741, the second projection light 620 (obtained by the first projection light 610) The brightness of the fourth frame 744 obtained by the second frame 742 obtained by the third projection light 630, the third frame 743 obtained by the third projection light 630 and the fourth projection light 640 is Can be different. For example, it may become brighter as it goes from the first frame 741 to the fourth frame 744. For example, if the brightness of the first frame 741 is I, the brightness of the second frame 742 is I +?, The brightness of the third frame 743 is I + 2 ?, and the brightness of the fourth frame 744 is I + Lt; RTI ID = 0.0 > I + 3. < / RTI > The change in brightness may occur, but not limited to, a change in light irradiated to the subject or movement of the subject.

According to one embodiment, when there is a change in the brightness of the reflected light obtained by reflection from the subject over time, the above-described Equations (9) to (12) can be expressed by Equation (17) .

When the equation (17) is reflected, the above-described equation (16) can be expressed by the following equation (18).

As described in the expression (18), the rate of change in brightness? Is interposed between the denominator and the numerator of the equation (18) with different signs, thereby causing an error in the depth calculation. Therefore, the larger the variation rate of the brightness? Or the greater the motion of the subject, the greater the motion blur.

The device 100 according to one embodiment may control the order of the projected projection light so that the motion blur may obtain reduced distance information.

Fig. 8 shows an example in which distance information is acquired by controlling the order of the projected light when the subject moves.

The device 100 according to one embodiment can control the order of the projected light. For example, the device 100 according to one embodiment may change the projection order between the projection lights. For example, the device 100 according to one embodiment may change the order of projection of the second projection light 620 and the fourth projection light 640, and may change the projection order of the first projection light 610, the fourth projection light 640, The third projection light 630 and the second projection light 620 can sequentially project the projection light onto the object.

According to one example, the device 100 controls the order of the projected light so that the order of the first projected light 610, the fourth projected light 640, the third projected light 630, and the second projected light 620 To the subject sequentially.

According to one embodiment, when there is no change in the brightness of the reflected light obtained by reflection from the subject despite the passage of time, the first frame 810 and the fourth projection light 810 obtained by the first projection light 610, The brightness of the fourth frame 840 obtained by the second frame 820 obtained by the first projection light 640, the third frame 830 obtained by the third projection light 630 and the second projection light 620, Can be the same.

The first frame 850 and the fourth projection light 640 obtained by the first projection light 610 can be obtained when the brightness of the reflected light obtained by reflection from the subject varies with the lapse of time, The brightness of the fourth frame 880 obtained by the second frame 860 obtained by the third projection light 630, the third frame 870 obtained by the third projection light 630 and the second projection light 620 is Can be different. For example, it may become brighter from the first frame 850 to the fourth frame 880. For example, if the brightness of the first frame 850 is I, the brightness of the second frame 860 is I +?, The brightness of the third frame 870 is I + 2 ?, and the brightness of the fourth frame 880 is I + Lt; RTI ID = 0.0 > I + 3. < / RTI > The change in brightness may occur, but not limited to, a change in light irradiated to the subject or movement of the subject.

According to one embodiment, when there is a change in the brightness of the reflected light obtained by reflection from the subject with the lapse of time, the above-mentioned Equations (9) to (12) can be expressed by Equation (19) .

Reflecting the expression (19), the above-described expression (16) can be expressed by the expression (20).

As can be seen from Equation (20), the brightness change rate? Intervenes in the denominator of Equation (20) with the same sign, so that the device 100 controls the projection order of the projected light, The motion blur effect can be reduced by making the error smaller than in the case of (18).

9 shows another example of obtaining distance information by controlling the order of the projected light when the subject moves.

The device 100 according to one embodiment can control the order of the projected light. For example, the device 100 according to one embodiment may shift the phase difference of the projected light as a whole. For example, the device 100 according to one embodiment may sequentially project the second projection light 620, the third projection light 630, the fourth projection light 640, and the first projection light 610 in this order, The light can be projected onto the object.

According to one example, the device 100 controls the order of the projected light so that the order of the second projected light 620, the third projected light 630, the fourth projected light 640, and the first projected light 610 To the subject sequentially.

According to one embodiment, when there is no change in the brightness of the reflected light obtained by reflection from the subject despite the passage of time, the first frame 910 and the third projection light 910 obtained by the second projection light 620, The third frame 930 obtained by the fourth projection light 640 and the fourth frame 940 obtained by the first projection light 610 obtained by the first projection light 630, Can be the same.

According to one embodiment, when there is a change in the brightness of the reflected light obtained from the subject as time elapses, the first frame 950 obtained by the second projection light 620, the third projected light 630 The third frame 970 obtained by the fourth projection light 640 and the fourth frame 980 acquired by the first projection light 610 are obtained by the following equation Can be different. For example, it may become brighter from the first frame 950 to the fourth frame 980. If the brightness of the first frame 950 is I, the brightness of the second frame 960 is I +?, The brightness of the third frame 970 is I + 2 ?, and the brightness of the fourth frame 980 is I + Lt; RTI ID = 0.0 > I + 3. < / RTI > The change in brightness may occur, but not limited to, a change in light irradiated to the subject or movement of the subject.

According to one embodiment, when there is a change in the brightness of the reflected light obtained by reflection from the subject over time, the above-mentioned Equations (9) to (12) can be expressed by Equation (21) .

When the following expression (21) is reflected, the above-described expression (16) can be expressed by the expression (22).

As can be seen from Equation (22), the brightness change rate? Intervenes in the denominator of Equation (22) with the same sign, so that the device 100 controls the projection order of the projected light, The motion blur effect can be reduced by making the error smaller than in the case of (18).

8 and 9 is only one embodiment, and the method by which the device 100 controls the order of the projected light is not limited to the embodiment disclosed in Figs. For example, the device 100 according to one embodiment may determine one of 24 projective orders of 4 in a projection order of projected light if there are four projected lights.

10 is a diagram for explaining a method of determining the projection order of the projection light according to the distance between the subject and the device.

The device 100 according to one embodiment may determine the projection order of the projection light according to the distance between the subject and the device 100. [

In the first embodiment, when the distance between the subject and the device 100 is 0 to 1 m or 4 to 5 m, the device 100 can transmit the first projection light 610, the second projection light 620, The first projection light 630, and the fourth projection light 640 in this order. The device 100 according to one embodiment can determine the projection order of the projection light to reduce the motion blur through the first graph 1010 when the distance between the subject and the device 100 is 0 to 1 m or 4 to 5 m have.

In the second embodiment, when the distance between the subject and the device 100 is 2 to 3 m or 6 to 7 m, the device 100 shifts the phase difference to the left by one shift to generate the second projection light 620, the third projection light 630 ), The fourth projection light 640, and the first projection light 610 in this order. The device 100 according to an embodiment can determine the projection order of the projection light to reduce the motion blur through the second graph 1020 when the distance between the subject and the device 100 is 2 to 3 m or 6 to 7 m have.

In the third embodiment, when the distance between the subject and the device 100 is 0 to 1 m or 4 to 5 m, the device 100 shifts the phase difference to the left by two times to output the third projection light 630, the fourth projection light 640 ), The first projection light 610, and the second projection light 620 in this order. The device 100 according to an exemplary embodiment can determine the projection order of the projection light for reducing the motion blur through the third graph 1030 when the distance between the subject and the device 100 is 0 to 1 m or 4 to 5 m have.

In the fourth embodiment, when the distance between the subject and the device 100 is 2 to 3 m or 6 to 7 m, the device 100 shifts the phase difference to the left by three times to output the fourth projection light 640, the first projection light 610 ), The second projection light 620, and the third projection light 630 in this order. The device 100 according to one embodiment can determine the projection order of the projection light to reduce the motion blur through the fourth graph 1040 when the distance between the subject and the device 100 is 2 to 3 m or 6 to 7 m have.

The projection order of the projection light according to the distance shown in Fig. 10 is only an embodiment, and the operation range of the device 100 is not limited to the embodiment of Fig.

11 is a view showing an example of obtaining distance information by controlling the order of projection light according to the distance between a subject and a device.

When the subject is at a distance of 2 m, the device 100 according to one embodiment may transmit the first projection light 610, the second projection light 620, the third projection light 630, and the fourth projection light 640, The projection light is sequentially projected onto the object in the following order. The variation of the distance difference between the subject and the device 100 in the first test region 1110 is small. However, if the subject moves, it can be seen that the variation of the distance difference is recognized in the first test area 1110. In this case, it can be confirmed that the standard deviation is 65.3 mm through the first graph 1130.

When the subject is at a distance of 2 m, the device 100 according to another embodiment may reflect the second projection light 620, the third projection light 630, the fourth projection light 640, and the first projection light 610 A description will be given of a case in which projection light is sequentially projected onto a subject in order. The variation of the distance difference between the subject and the device 100 in the second test region 1120 is small. However, if the subject moves, it can be seen that the variation of the distance difference is recognized in the first test area 1120. However, it can be confirmed that the change in the recognized distance difference is reduced by changing the projection order of the projection light. Also, in this case, it can be confirmed that the standard deviation is reduced to 38 mm through the second graph 1140.

Thus, the device 100 according to one embodiment can reduce the motion blur by controlling the projection order of the projected light.

For example, the device 100 according to one embodiment can determine the projection order of the projection light according to the distance between the subject and the device 100, and project the projection light onto the subject. In this case, the device 100 according to an embodiment obtains information on the distance between the subject and the device 100 in real time, determines the projection order of the projection light according to the distance information obtained in real time, It can be projected.

As another example, the device 100 according to one embodiment can determine the projection order of the projection light according to the reception input received from the outside, and project the projection light onto the object.

12 and 13 are one embodiment of a device 100 according to various embodiments. The device 100 is an apparatus capable of performing the above-described distance information acquisition method. It is possible to implement all the embodiments for performing the method of acquiring the distance information disclosed in Figs. 1 to 11 and Figs. 14 to 16.

12 is a block diagram illustrating a configuration of a device 100 according to an embodiment.

12, the device 100 according to one embodiment may include a light source 101, a control unit 1240, and a modulation unit 1230. [ However, the device 100 may be implemented by more components than the components shown, and the device 100 may be implemented by fewer components than the components shown. The device 100 according to another embodiment may include a light source 101, a control unit 1240, a modulation unit 1230, and an input device 1250. In addition, the controller 1240 may include a projection light controller 1210.

Hereinafter, the components will be described in order.

The projection light control unit 1210 according to an exemplary embodiment determines the projection order of N projected light beams that are sequentially projected onto a subject (where N is a natural number of 3 or more) based on the position of the subject and / or the external received input .

The N different projection lights may include periodic waves having the same period and different at least one of magnitude and phase.

The light source 101 according to an exemplary embodiment may project the projection light of four different phases sequentially to the object under the control of the controller 1240. [ For example, the projection light 1 to the projection light 4 may have periodic files having the same period, waveform, size, and different phases only. For example, the phase of the projection light 1 may be 90 degrees slower than the projection light 2, the phase of the projection light 2 may be 90 degrees slower than the projection light 3, and the phase of the projection light 3 may be 90 degrees slower than the projection light 4. In another example, the phase of the projection light 1 is 90 degrees faster than the projection light 2, the phase of the projection light 2 is 90 degrees faster than the projection light 3, and the phase of the projection light 3 is 90 degrees faster than the projection light 4.

The modulator 1230 according to an exemplary embodiment uses the three subframe images, three CIS images, or three CCD images to measure the depth of light using the phase difference of the light wave according to the optical time-flight method, the reflectivity of the surface of the object, a depth image can be obtained.

The modulator 1230 according to an embodiment can acquire a motion image with a reduced depth of motion using four subframe images, four CIS images, or four CCD images. When the modulator 1230 according to the embodiment acquires a depth image using four four CIS images, the four CIS images can be obtained by four projected lights having different phase differences. For example, the projection light 1 to the projection light 4 may be periodic files having the same period, waveform, and size, but differing in phase by 90 degrees.

The projection light control unit 1210 according to an exemplary embodiment may determine the projection order of N projection light beams sequentially projected to the object (where N is a natural number of 3 or more) based on the position of the object. For example, the projection light control unit 1210 according to an embodiment determines the projection order of the projection light according to the distance between the subject and the device 100, and outputs the projection light to the light source 101 Can be controlled.

The projection light control unit 1210 according to an exemplary embodiment of the present invention may be arranged such that the projection order corresponding to the distance between the device 100 and the object in the projection order of a predetermined plurality of projection lights is a projection order of the projection light for sequentially projecting the light source 101 to the object . For example, in the case of N = 4, all of the 24 projection sequences correspond to predetermined distances, and the projection sequence corresponding to the distances between the device 100 and the subject in the 24 projection sequences is obtained by the light source 101 And the projection order of the projection light to be sequentially projected. For example, when the distance between the device 100 and the subject is within 1 m, the first projection order is set. When the distance between the device 100 and the subject is 2 m to 3 m, the second projection order is set between the device 100 and the subject When the distance is 3m to 4m, the projection light control unit 1210 according to the embodiment can determine the third projection order by the projection order of the projection light.

The projection light controller 1210 according to an exemplary embodiment may determine a projection sequence that minimizes motion blur for a subject among a plurality of predetermined projection sequences as a projection order of projection light. For example, the projection light control unit 1210 according to an exemplary embodiment may set the projection order of the projection light to be sequentially projected on a subject to a subject in a predetermined plurality of projection orders, based on the position of the subject and / It is possible to determine the projection sequence that minimizes the motion blur for the projection light.

The light source 101 according to one embodiment sequentially projects N different projected lights onto a subject according to a projection order of the projection lights determined by the projection light controller 1210. [

For example, the light source 101 according to an exemplary embodiment may include a first projection light 610, a second projection light 620, a third projection light 630, and a third projection light 630 according to the projection order of the projection light determined by the projection light control unit 1210. [ And the fourth projection light 640 sequentially in this order.

The first projection light 640, the third projection light 640, and the third projection light 630 in accordance with the projection order of the projection light determined by the projection light control unit 1210. In other words, And the second projection light 620 in this order.

The second projection light 620, the first projection light 610, and the second projection light 610 according to the projection order of the projection light determined by the projection light control unit 1210. [ And the fourth projection light 640 sequentially in this order.

The second projection light 630, the fourth projection light 640, and the fourth projection light 640 according to the projection order of the projection light determined by the projection light control unit 1210. In other words, And the first projection light 610 can be sequentially projected onto the object in this order.

In another example, the light source 101 according to an exemplary embodiment may include a third projection light 630, a fourth projection light 640, a first projection light 610, and a second projection light 640 according to the projection order of the projection light determined by the projection light control unit 1210. [ And the second projection light 620 in this order.

The first projection light 610, the second projection light 620, and the second projection light 620 according to the projection order of the projection light determined by the projection light control unit 1210. In other words, And the third projection light 630 in this order.

The projection order of the projection light disclosed in Fig. 12 is one embodiment and is not limited to the embodiment disclosed in this drawing.

A modulator 1230 according to an embodiment modulates N reflected light 1205 reflected from an object to obtain N modulated reflected light.

The modulator 1230 according to one embodiment may obtain the reflected light 1205 in which N different projected lights projected from the light source 101 are reflected from the subject. The modulator 1230 according to an embodiment can acquire N modulated reflected light by modulating the obtained reflected light 1205. [

The modulator 1230 according to one embodiment can modulate N reflected light 1205 using an optical modulating signal having a gain waveform to obtain N modulated reflected light. The optical modulated signal may include a periodic wave having the same period as the projected light.

A specific method of acquiring and modulating the reflected light 1205 has been described above with reference to Figs.

The control unit 1240 according to an embodiment obtains distance information on the object using the N modulated reflected lights acquired by the modulator 1230. [

For example, the control unit 1240 according to an exemplary embodiment may acquire a three-dimensional image or a depth image of a subject using N modulated reflected light acquired by the modulator 1230. [

A specific method of acquiring a three-dimensional image or depth image with respect to the object using the modulated reflected light has been described above with reference to Figs.

When the subject moves, when the intensity of the external light reflected on the subject changes, or when the subject changes with time, such as when the reflectivity of the subject surface changes with time, motion blur may occur.

The controller 1240 according to an exemplary embodiment of the present invention may be configured such that when a plurality of projection lights are projected according to the projection order of the projection light determined by the projection light controller 1210, Can obtain a depth image with a small depth.

In addition, as described above with reference to FIGS. 4 to 11, the projection light control unit 1210 according to an exemplary embodiment may determine a projection sequence for reducing motion blur among a plurality of projection sequences. The light source 101 according to one embodiment projects the projection light according to the determined projection order, acquires the reflected light 1205 obtained by reflecting the projected projection light, modulates the obtained reflected light 1205, Can be obtained. In this case, the controller 1240 according to an exemplary embodiment can acquire a depth image having less motion blur than a case of projecting a plurality of projected lights according to a uniform projection order.

The control unit 1240 according to an exemplary embodiment receives information on the projection order of the projection light from the projection light control unit 1210, controls the LD (Laser Diode) using the received information, Images and depth images.

The control unit 1240 according to another embodiment receives the information on the projection order of the projection light from the projection light control unit 1210, controls the shutter using the received information, modulates the shutter, Images can be obtained.

Also, the projection light control unit 1210 can be used to receive the acquired depth image as feedback and determine the projection order of the projection light.

The control unit 1240 according to an embodiment receives the information on the projection order of the projection light from the external reception input, controls the LD using the received information, and obtains the phase image and the depth image by projecting light from the LD .

The control unit 1240 according to another embodiment receives the information on the projection order of the projection light from the external reception input, and controls the shutter using the received information to acquire the phase image and the depth image by modulating the shutter can do.

FIG. 13 schematically illustrates an exemplary structure of a device 100 for acquiring distance information through control of projected light.

12, the control unit 1240 includes a control unit 106, a projection light control unit 1210, a light source driving unit 102, an optical modulator driving unit 104, and a distance information image processing unit 107 can do. The modulator 1230 according to an embodiment may include a first lens 108, a filter 109, an optical modulator 103, a second lens 110, and an image pickup device 105.

The operation of each component in Fig. 13 can be referred to the contents of Fig. 1 and Fig.

14 is a flowchart illustrating a method for a device according to an embodiment to acquire distance information using N projected lights.

In step S1410, the device 100 according to the embodiment determines the projection order of N projected light beams (here, N is a natural number equal to or greater than 3) to be sequentially projected on the subject based on the position of the subject and / do.

The N different projection lights may include periodic waves having the same period and different at least one of magnitude and phase.

The device 100 according to an exemplary embodiment may project projection light sequentially different in four phases to a subject. For example, the projection light 1 to the projection light 4 may have periodic files having the same period, waveform, size, and different phases only. For example, the phase of the projection light 1 may be 90 degrees slower than the projection light 2, the phase of the projection light 2 may be 90 degrees slower than the projection light 3, and the phase of the projection light 3 may be 90 degrees slower than the projection light 4. In another example, the phase of the projection light 1 is 90 degrees faster than the projection light 2, the phase of the projection light 2 is 90 degrees faster than the projection light 3, and the phase of the projection light 3 is 90 degrees faster than the projection light 4.

The device 100 according to an exemplary embodiment uses the three subframe images, three CIS images, or three CCD images to measure the depth of a subject using the phase difference of the optical wave according to the optical time-flight method, the reflectivity of the subject's surface, depth image.

The device 100 according to one embodiment can obtain a depth-reduced image with motion blur using four subframe images, four CIS images, or four CCD images. When the device 100 according to an embodiment acquires a depth image using four four CIS images, the four CIS images can be obtained by four projected lights having different phase differences. For example, the projection light 1 to the projection light 4 may be periodic files having the same period, waveform, and size, but differing in phase by 90 degrees.

The device 100 according to the embodiment can determine the projection order of N projected light beams sequentially projected to the subject (where N is a natural number of 3 or more) based on the position of the subject. For example, the device 100 according to one embodiment can determine the projection order of the projection light according to the distance between the subject and the device 100, and project the projection light onto the subject.

The device 100 according to the embodiment may determine the projection order corresponding to the distance between the device 100 and the object in the projection order of a predetermined plurality of projection lights by the projection order of the projection light that the device 100 sequentially projects to the object . For example, when N is 4, the 24 projection sequences all correspond to predetermined distances, and the projection order corresponding to the distances between the device 100 and the subject in the 24 projection sequences is the device 100 And the projection order of the projection light to be sequentially projected. For example, when the distance between the device 100 and the subject is within 1 m, the first projection order is set. When the distance between the device 100 and the subject is 2 m to 3 m, the second projection order is set between the device 100 and the subject When the distance is 3m to 4m, the third projection order can be determined by the projection order of the projection light according to the embodiment.

The device 100 according to an exemplary embodiment may determine a projection sequence that minimizes motion blur for a subject among a predetermined plurality of projection sequences in a projection order of projection light. For example, the device 100 according to an embodiment may be arranged such that the projection order of the projection light to be sequentially projected on a subject is determined based on the position of the subject and / or the external reception input, The projection order that minimizes blur can be determined by the projection order of the projection light.

In step S1420, the device 100 according to one embodiment sequentially projects N different projection lights onto the subject in accordance with the projection order of the projection light determined in step S1410.

For example, the device 100 according to an exemplary embodiment may include a first projection light 610, a second projection light 620, a third projection light 630, and a fourth projection light 640 according to the projection order of the projection light determined in step S1410. And the projection light 640 can be sequentially projected onto the object in this order.

As another example, the device 100 according to an embodiment may calculate the first projection light 610, the fourth projection light 640, the third projection light 630, and the second projection 640 in accordance with the projection order of the projection light determined in step S1410. And the light 620 can be projected to the object sequentially.

As another example, the device 100 according to one embodiment may further include a third projection light 630, a second projection light 620, a first projection light 610, and a fourth projection light 620 according to the projection order of the projection light determined in step S1410. And the light 640 in this order.

As another example, the device 100 according to one embodiment may be configured such that the second projection light 620, the third projection light 630, the fourth projection light 640, and the first projection light 630, in accordance with the projection order of the projection light determined in step S1410, And the light 610 are sequentially projected onto the object.

In another example, the device 100 according to one embodiment may further include a third projection light 630, a fourth projection light 640, a first projection light 610, and a second projection light 640 according to the projection order of the projection light determined in step S1410. And the light 620 can be projected to the object sequentially.

As another example, the device 100 according to one embodiment may further include a fourth projection light 640, a first projection light 610, a second projection light 620, and a third projection light 610. In accordance with the projection order of the projection light determined in step S1410, The light 630 can be projected to the object sequentially in this order.

The projection order of the projection light started in step S1420 is one embodiment and is not limited to the embodiment disclosed in step S1420.

In step S1430, the device 100 according to one embodiment modulates N reflected light reflected from the subject to obtain N modulated reflected light.

The device 100 according to one embodiment can obtain the reflected light in which the N different projected lights projected in step S1420 are reflected from the subject. The device 100 according to one embodiment can obtain the N modulated reflected light by modulating the obtained reflected light.

The device 100 according to one embodiment can modulate N reflected light using an optical modulated signal having a gain waveform to obtain N modulated reflected light. The optical modulated signal may include a periodic wave having the same period as the projected light.

A specific method of acquiring and modulating reflected light has been described above with reference to Figs.

In step S1440, the device 100 according to one embodiment acquires distance information on the object using the N modulated reflected light obtained in step S1430.

For example, the device 100 according to one embodiment may obtain a three-dimensional image or a depth image for a subject using the N modulated reflected light obtained in step S1430.

A specific method of acquiring a three-dimensional image or depth image with respect to the object using the modulated reflected light has been described above with reference to Figs.

When the subject moves, when the intensity of the external light reflected on the subject changes, or when the subject changes with time, such as when the reflectivity of the subject surface changes with time, motion blur may occur.

When projecting a plurality of projected lights in accordance with the projection order of the projection light determined in step S1410, the device 100 according to the embodiment may display a depth image having less motion blur than a case of projecting a plurality of projected lights according to a uniform projection order Can be obtained.

In addition, as described above with reference to FIGS. 4 through 11, the device 100 according to an embodiment may determine a projection order for reducing motion blur among a plurality of projection orders. The device 100 according to an embodiment can project the projection light according to the determined projection order, obtain the reflected light obtained by reflecting the projected projection light, and modulate the obtained reflected light to obtain the depth image. In this case, the device 100 according to an exemplary embodiment can acquire a depth image having less motion blur than a case of projecting a plurality of projected lights according to a uniform projection order.

15 is a flow diagram illustrating a method for a device according to an embodiment to acquire distance information using N different-phase projected lights.

In step S1510, the device 100 according to one embodiment obtains distance information between the device 100 and the subject.

For example, the device 100 according to one embodiment may obtain the distance between the device 100 and the subject using the depth image or the three-dimensional image that has already been obtained.

In another example, the device 100 according to one embodiment may obtain information about the distance between the subject and the device 100 in real time. For example, the device 100 according to an exemplary embodiment receives a newly acquired depth image or a three-dimensional image as a feedback in real time and receives a depth image or a three-dimensional image to receive a distance between the device 100 and a subject Can be obtained. As another example, the device 100 according to one embodiment may acquire the distance information obtained in step S1560, which will be described later, as distance information between the device and the object.

In step S1520, the device 100 according to one embodiment determines the projection order of N projected light beams (here, N is a natural number equal to or greater than 3) to be projected sequentially on the basis of the distance information obtained.

The device 100 according to one embodiment may obtain the distance information between the device 100 and the subject and may determine the projection order of the projection light to minimize motion blur for the subject based on the obtained distance information.

The device 100 according to one embodiment may determine the projection order corresponding to the distance obtained in step S1510 as the projection order of the projection light. For example, the device 100 according to one embodiment may determine the projection order that minimizes motion blur in the distance obtained in step S1510, in terms of the projection order of the projection light. The projection order, which can minimize motion blur according to distance, may be predetermined and calculated. A specific example of determining the projection order of the projection light according to the distance between the subject and the device 100 has been described above with reference to FIG.

In step S1530, the device 100 according to an embodiment sequentially projects projection light of N different phases to the subject in accordance with the projection order of the projection light determined in step S1520. There may be a phase difference N divided by 360 degrees between the N projected lights having different phases.

The specific method by which the device 100 according to one embodiment sequentially projects N different projection lights to the subject in accordance with the projection order of the projection light determined in step S1520 has been described above in step S1420.

In step S1540, the device 100 according to one embodiment obtains N reflected light reflected from the subject.

The device 100 according to an embodiment can acquire N reflected light projected from the object in step S1530. The reflected light can be determined by being influenced by the reflectivity of the subject. A specific method of obtaining reflected light has been described in Figs.

Steps S1550 and S1560 correspond to steps S1430 and S1440, respectively, and therefore, a detailed description thereof will be omitted for the sake of simplicity.

16 is a flowchart showing a method of a device according to an embodiment to project distance-based information by projecting different projection light according to a projection order of projection light determined based on an external reception input.

Since steps S1630 to S1650 correspond to steps S1420 to S1440, a detailed description will be omitted for the sake of simplicity.

In step S1610, the device 100 according to one embodiment receives input from the outside.

The device 100 according to one embodiment may receive user input. For example, device 100 according to one embodiment may receive an external receive input determined based on a received user input. The user input may include at least one of a touch input, a keyboard input, a voice input, a sound input, a button input, a gesture input, and a multimodal input, but is not limited thereto.

The device 100 according to one embodiment may receive an external receive input. For example, the device 100 according to one embodiment may receive an external receive input via a wireless or wired manner. In another example, a device 100 according to one embodiment may receive an external receive input from a server external to the device 100 or from an external device external to the device 100.

In step S1620, the device 100 according to the embodiment determines the projection order of N projected light beams (here, N is a natural number equal to or greater than 3) to be projected on the object sequentially based on the external reception input.

For example, the device 100 according to an exemplary embodiment may be arranged such that a projection order of projection light to be sequentially projected on a subject is determined based on an external reception input, a projection order in which motion blur for a subject is minimized among a predetermined plurality of projection orders As the projection order of the projection light.

The device 100 according to the embodiment may determine the projection order corresponding to the external reception input among the projection orders of the predetermined plurality of projection lights by the projection order of the projection light that the device 100 sequentially projects to the object. For example, when N is 4, the projection order determined according to the external reception input among the 24 projection orders can be determined in accordance with the projection order of the projection light that the device 100 sequentially projects to the object.

The information obtaining method and apparatus according to the embodiment described above can be recorded in a computer-readable recording medium and executed by a computer so that the above-described functions can be executed.

In addition, such code may further include memory reference related code as to what additional information or media needed to cause the processor of the computer to execute the aforementioned functions should be referenced at any location (address) of the internal or external memory of the computer .

The computer-readable recording medium on which the above-described program is recorded includes ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical media storage, and the like.

A computer capable of reading a recording medium on which an application, which is a program for executing the information obtaining method and apparatus according to each embodiment, can be read is not limited to a general PC such as a general desktop or a notebook computer, but also a smart phone, a tablet PC, Assistants, and mobile terminals. In addition, it should be interpreted as all devices capable of computing.

It is to be understood that the terms "comprises", "comprising", or "having" as used in the foregoing description mean that the constituent element can be implanted unless specifically stated to the contrary, But should be construed as further including other elements.

The above description is merely illustrative of technical ideas, and various modifications and changes may be made without departing from the essential characteristics of those skilled in the art. Therefore, the disclosed embodiments are intended to be illustrative rather than limiting, and the scope of the technical idea is not limited by these embodiments. The scope of protection is to be interpreted by the following claims, and all technical ideas within the scope of equivalents should be interpreted as being included in the scope of the right.

Claims (20)

A method for a device to obtain information about a subject,
Determining a projection sequence of N projected light beams to be sequentially projected onto the subject (where N is a natural number of 3 or more) based on the position of the subject and / or an external received input;
Sequentially projecting the N different projection lights onto the subject according to a projection order of the projection light;
Modulating N reflected light reflected from the subject to obtain N modulated reflected light; And
And obtaining distance information on the object using the N modulated reflected light. The method according to claim 1,
Wherein the step of determining the projection order of the projection light based on the position of the subject and /
Acquiring distance information between the device and the subject, and determining a projection order of the projection light based on the obtained distance information. The method according to claim 1,
Wherein the distance information between the device and the subject is obtained in real time. The method according to claim 1,
Wherein the step of determining the projection order of the projection light based on the position of the subject and /
Acquiring distance information between the device and the subject, and determining a projection order of the projection light for minimizing motion blur for the subject based on the obtained distance information. The method according to claim 1,
Wherein the step of determining the projection order of the projection light based on the position of the subject and /
And a projection order in which a motion blur for the subject is minimized among a plurality of predetermined projection orders is determined as a projection order of the projection light. The method according to claim 1,
Wherein the N different projection lights have the same period and at least one of a magnitude and a phase includes different periodic waves. The method according to claim 1,
Wherein the step of modulating the N reflected light reflected from the subject to obtain the N modulated reflected light comprises:
And modulating the N reflected light by using an optical modulated signal having a gain waveform to obtain the N modulated reflected light. 8. The method of claim 7,
Wherein the optical modulating signal includes a periodic wave having the same period as the projected light. The method according to claim 1,
Wherein the N different projected lights have different phases from each other and a phase difference of 360 degrees divided by N is present between the N different projected lights. The method according to claim 1,
Wherein the step of determining the projection order of the projection light based on the position of the subject and /
And a projection order of the projection light is determined based on an external reception input determined based on a user's input.
A device for acquiring information on a subject, the device comprising:
A projection light control unit for determining a projection sequence of N different projection light sequentially projected onto the object (where N is a natural number of 3 or more) based on the position of the object and / or an external reception input;
A light source sequentially projecting the N different projection lights onto the subject according to a projection order of the projection lights;
A modulator for modulating N reflected light reflected from the subject to obtain N modulated reflected light; And
And a controller for obtaining distance information on the object using the N modulated reflected lights. 12. The method of claim 11,
The projection-
Obtain distance information between the device and the subject, and determine a projection order of the projection light based on the obtained distance information. 12. The method of claim 11,
Wherein distance information between the device and the subject is obtained in real time. 12. The method of claim 11,
The projection-
Obtains distance information between the device and the subject, and determines a projection order of the projection light for minimizing motion blur for the subject based on the obtained distance information. 12. The method of claim 11,
The projection-
And a projection order in which a motion blur for the subject is minimized among a plurality of predetermined projection orders is determined as a projection order of the projection light. 12. The method of claim 11,
Wherein the N different projected lights include periodic waves having the same period and at least one of a magnitude and a phase being different from each other. 12. The method of claim 11,
The modulator
A device for modulating the N reflected light using an optical modulated signal having a gain waveform to obtain the N modulated reflected light. 18. The method of claim 17,
Wherein the optical modulating signal comprises a periodic wave having the same period as the projected light. 12. The method of claim 11,
Wherein the N different projected lights have different phases from each other and a phase difference of 360 degrees divided by N is present between the N different projected lights. A computer readable nonvolatile recording medium on which a program for implementing the method of any one of claims 1 to 10 is recorded.

Images