KR20130077330A - 3d image sensor and 3d image processing system having the same - Google Patents

Abstract

PURPOSE: A 3D image processing system is provided to produce color information and distance information by using wavelengths of infrared area and visible light area by controlling selective opening of a third region and a fourth region in a shutter controller. CONSTITUTION: A filter (40) comprises a first region for passing wavelengths of infrared area and a second region for blocking wavelengths of the infrared area. A shutter (50) comprises a third region corresponding to the first area and a forth region corresponding to the second region. An image sensor (10) produces distance information by using the wavelengths of the infrared area when a shutter controller (51) opens the third region. The image sensor produces color information by using visible light area when the shutter controller opens the forth region.

Description

3D image sensor and 3D image processing system having the same}

An embodiment according to the concept of the present invention relates to a 3D image processing system, and more particularly, to a 3D image processing system for generating a 3D image including distance information as well as color information.

A sensor is a device that detects a state or position of an object and converts the detection result into an electrical signal and transmits the result. Examples of the sensor include a light sensor, a temperature sensor, a pressure sensor, a magnetic sensor, or a 3D image sensor.

The 3D image sensor generates a 3D image including color information of the object as well as distance information between the 3D image sensor and the object.

As a signal output from the source of the 3D image sensor, microwave, light wave or ultrasonic wave may be used.

The 3D image sensor generates distance information using a time-of-flight measurement method. That is, the 3D image sensor measures a delay time until a signal in the form of pulses radiated from a source is reflected by a target object (or a measurement object) and returns to the 3D image sensor and the object. The distance information (or depth information) is calculated.

The 3D image sensor uses the wavelength of the infrared region to generate the distance information, and uses the wavelength of the visible region to generate the color information.

Accordingly, the 3D image sensor must receive not only the wavelength of the visible light region but also the wavelength of the infrared region to generate the 3D image.

An object of the present invention is to provide a 3D image processing system for generating a 3D image including color information and distance information.

According to an embodiment of the present invention, a 3D image processing system includes a filter including a first region for passing a wavelength in an infrared region and a second region for blocking a wavelength in the infrared region, and a filter corresponding to the first region. And a shutter including three regions and a fourth region corresponding to the second region.

The 3D image processing system further includes a shutter controller to control the shutter to selectively open the third region and the fourth region.

The shutter controller controls the shutter to open the third region to generate distance information using the wavelength of the infrared region.

The shutter controller controls the shutter to open the fourth region to generate color information by excluding the wavelength of the infrared region.

The shutter is implemented by liquid crystal.

The 3D image processing system includes a pixel array including a depth pixel for generating a depth pixel signal corresponding to wavelengths of the infrared light and a plurality of color pixels for generating a color pixel signal corresponding to wavelengths of a visible light region. It includes more.

The 3D image processing system is a digital camera.

The 3D image processing system according to an embodiment of the present invention has the effect of generating a 3D image including color information and distance information.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In order to more fully understand the drawings recited in the detailed description of the present invention, a detailed description of each drawing is provided.
1 is a block diagram of a 3D image processing system according to an exemplary embodiment.
FIG. 2 shows an embodiment of the filter shown in FIG. 1.
3 illustrates an embodiment of the shutter illustrated in FIG. 1.
4 illustrates a block diagram of the integrated circuit shown in FIG. 1.
FIG. 5 illustrates an embodiment of a unit pixel array constituting a part of the pixel array illustrated in FIG. 4.
6 illustrates another embodiment of a unit pixel array constituting a part of the pixel array illustrated in FIG. 4.
FIG. 7 is a flowchart illustrating a method of operating the 3D image processing system illustrated in FIG. 1.

It is to be understood that the specific structural or functional descriptions of embodiments of the present invention disclosed herein are only for the purpose of illustrating embodiments of the inventive concept, But may be embodied in many different forms and is not limited to the embodiments set forth herein.

Embodiments in accordance with the concepts of the present invention are capable of various modifications and may take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. It should be understood, however, that it is not intended to limit the embodiments according to the concepts of the present invention to the particular forms disclosed, but includes all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are intended to distinguish one element from another, for example, without departing from the scope of the invention in accordance with the concepts of the present invention, the first element may be termed the second element, The second component may also be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions describing the relationship between components, such as "between" and "immediately between," or "neighboring to," and "directly neighboring to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "comprises ", or" having ", or the like, specify that there is a stated feature, number, step, operation, , Steps, operations, components, parts, or combinations thereof, as a matter of principle.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings.

1 is a block diagram of a 3D image processing system according to an exemplary embodiment.

Referring to FIG. 1, the 3D image processing system 100 may be a 3D range finder, a game controller, a digital camera, or a gesture sensing apparatus.

The 3D image processing system 100 may include a 3D image sensor 10 and a processor 110.

The 3D image sensor 10 includes a light source 32, a lens 34, a filter 40, a shutter 50, a shutter controller 51, and an integrated circuit 20.

The light source 32 emits the modulated light signal EL to the target object 60. As the light source 32, a light emitting diode (LED) or a laser diode may be used. The modulated optical signal EL may be a sine wave or a square file.

The modulated light signal EL output from the light source 32 is reflected at the target object 60.

The reflected optical signal RL1 is incident to the integrated circuit 20 through the lens 34, the filter 40, and the shutter 50.

The optical signal RL1 incident to the integrated circuit 20 has a wavelength in the infrared region.

The 3D image sensor 10 includes a plurality of light sources arranged in a circle around the lens 34, but only one light source 32 is shown for convenience of description.

The optical signal RL1 incident to the integrated circuit 20 through the lens 34, the filter 40, and the shutter 50 may be demodulated by the integrated circuit 20. That is, distance information may be generated by the optical signal RL1 incident to the integrated circuit 20 through the lens 34, the filter 40, and the shutter 50.

The optical signal AL output from the ambient light 61 is reflected by the target object 60.

The optical signal RL2 reflected by the target object 60 has a wavelength in the visible light region.

The optical signal RL2 incident to the integrated circuit 20 through the lens 34, the filter 40, and the shutter 50 may generate color information by the integrated circuit 20.

The shutter controller 51 controls the shutter 50 to pass the optical signal RL1 having the wavelength in the infrared region or to pass the optical signal RL2 having the wavelength in the visible region.

The processor 110 may control the operation of the 3D image sensor 10. For example, the processor 110 may access the memory 120 in which a program for controlling the operation of the 3D image sensor 10 is stored and execute the program stored in the memory 120.

The 3D image sensor 10 may generate 3D image information based on the color information and the distance information under the control of the processor 110. The generated 3D image information may be displayed through a display (not shown) connected to the interface 130.

The 3D image information generated by the 3D image sensor 10 may be stored in the memory 120 through the bus 101 under the control of the processor 110. The memory 120 may be implemented as a nonvolatile memory device.

The interface 130 may be implemented as an interface for inputting and outputting 3D image information. According to an embodiment, the interface 130 may be implemented as a wireless interface.

FIG. 2 shows an embodiment of the filter shown in FIG. 1.

1 and 2, the filter 40 includes a first region 40-1 for passing the wavelength of the infrared region and a second region 40-2 for blocking the wavelength of the infrared region. do. The filter 40 may be implemented radially or latticely.

The first region 40-1 may be implemented as an infrared pass filter, and the second region 40-2 may be implemented as an infrared cut filter.

According to an embodiment, the filter 40 may be implemented in various patterns.

3 illustrates an embodiment of the shutter illustrated in FIG. 1.

1 to 3, the shutter 50 includes a third region 50-1 corresponding to the first region 40-1 and a fourth region 50 corresponding to the second region 40-2. -2).

The shutter 50 may be implemented in a radial or lattice form.

The shutter 50 is a device for transmitting infrared or visible light to the image sensor 10 for a predetermined time.

The shutter 50 is implemented by liquid crystal.

The shutter controller 51 controls the shutter 50 to selectively open the third region 50-1 and the fourth region 50-2.

The shutter controller 51 controls the shutter 50 to open the third region 50-1 to generate distance information using the wavelength of the infrared region.

When the shutter controller 51 controls the third region 50-1 to be opened, the image sensor 10 generates the distance information by using the wavelength of the infrared region.

The shutter controller 51 controls the shutter 50 to open the fourth region 50-2 to generate color information by excluding the wavelength of the infrared region (using the wavelength of the visible ray region).

When the shutter controller 51 controls the fourth region 50-2 to be opened, the image sensor 10 generates the color information by using the wavelength of the visible light region.

4 illustrates a block diagram of the integrated circuit shown in FIG. 1.

1 and 4, the integrated circuit 20 generates distance information using a wavelength in an infrared region and generates color information using a wavelength in a visible ray region.

The integrated circuit 20 includes a pixel array 22 including depth pixels capable of generating the distance information and RGB color pixels capable of generating the color information.

The pixel array 22 will be described in detail in FIG. 5 or FIG. 6.

The depth pixel is a 1-tap depth pixel or a 2-tap depth pixel.

The row decoder 24 selects any one of the plurality of rows in response to a row address output from the timing controller 26. Here, a row refers to a set of a plurality of 1-tap depth pixels arranged in the X-direction in the array 22.

The photo gate controller 28 may generate and supply the plurality of photo gate control signals Ga, Gb, Gc, and Gd to the array 22 under the control of the timing controller 26.

Each photo gate control signal Ga, Gb, Gc, and Gd is a square wave.

The difference between the phase of the first port gate control signal Ga and the phase of the third port gate control signal Gc is 90 °, and the phase of the first port gate control signal Ga and the second photo gate control signal Gb. ) Is 180 °, and the difference between the phase of the first port gate control signal Ga and the phase of the fourth photo gate control signal Gd is 270 °.

The light source driver 30 may generate a clock signal capable of driving the light source 32 under the control of the timing controller 26.

The light source driver 30 supplies the clock signal MLS or the information about the clock signal MLS to the photo gate controller 28. Accordingly, the photo gate controller 28 may control the first port gate control signal Ga having the same phase as that of the modulated optical signal EL under the control of the timing controller 26 and the modulated optical signal EL. A second port gate control signal Gb having a phase difference of 180 ° with a phase is generated. In addition, the photo gate controller 28 may include a third port gate control signal Gc having a phase difference of 90 ° and a phase of the modulated optical signal EL, and a fourth port gate control signal having a phase difference of 270 °. Produces (Gd).

The photo gate controller 28 sequentially controls the first photo gate control signal Ga, the second photo gate control signal Gb, the third photo gate control signal Gc, and under the control of the timing controller 26. The fourth photo gate control signal Gd may be supplied to the array 22.

According to an embodiment, when the depth pixel is a 2-tap depth pixel, the photo gate controller 28 may control the first photo gate control signal Ga and the third photo gate control signal under the control of the timing controller 26. Gc may be simultaneously supplied to the array 22, and the second photo gate control signal Gb and the fourth photo gate control signal Gd may be simultaneously supplied to the array 22.

For example, the photo gate controller 28 and the light source driver 30 may operate in synchronization with each other.

The photo gate 110 may be implemented with transparent poly silicon. In some embodiments, the photo gate 110 may be formed of indium tin oxide (ITO) or tin-doped indium oxide (ITO), indium zinc oxide (IZO), zinc oxide (ZnO), or the like.

The depth pixel accumulates photo electrons (or photo charges) of the incident optical signal for a predetermined time, for example, integration time, and generates pixel signals A0 ', A1', and A2 'according to the accumulation result. , And A3 '). Each pixel signal Ak 'generated by the depth pixel may be represented by Equation 1 below.

[Equation 1]

Here, k is 0 when the signal input to the photo gate 110 of the depth pixel is the first photo gate control signal Ga, and k is 1 when the third photo gate control signal Gc is present. K is 2 when the second photo gate control signal Gb is, and k is 3 when the phase difference of the gate signal Gd is 270 degrees.

a _{k and n} represent the number of photoelectrons (or photocharges) detected at the depth pixel when the n (n is a natural number) gate signal is applied with a phase difference corresponding to k, and N = fm * Tint. Here, fm represents the frequency of the modulated optical signal EL, and Tint represents the integration time.

Under the control of the timing controller 26, the CDS / ADC circuit 36 performs correlated double sampling (CDS) operation on each pixel signal A0 ', A1', A2 ', and A3' detected from the depth pixel. An analog to digital converting (ADC) operation is performed to output each digital pixel signal A0, A1, A2, and A3.

The integrated circuit 20 may further include active load circuits (not shown) for transmitting pixel signals output from the plurality of column lines implemented in the pixel array 22 to the CDS / ADC circuit 36. .

The memory 38 receives and stores each of the digital pixel signals A0, A1, A2, and A3 output from the CDS / ADC circuit 36.

Hereinafter, each of the digital pixel signals A0, A1, A2, and A3 is referred to as each pixel signal for convenience of description.

The microprocessor 39 calculates distance information to the target object 40 using the plurality of pixel signals A0, A1, A2, and A3.

For example, the modulated optical signal EL is A (1 + cosωt) / 2 and the optical signal RL1 incident on the 3D image sensor 10 or the optical signal RL1 detected by the 3D image sensor 10. Is A0 = A '(1 + cos (ωt + θ)) + B, A1 = A' (1 + sin (ωt + θ)) + B, A2 = A '(1-cos (ωt + θ)) + When B, A3 = A '(1-sin (ωt + θ)) + B, the phase shift due to TOF is represented by Equation 2.

&Quot; (2) "

θ = arctan ((A3-A1) / (A2-A0))

Accordingly, the modulated light signal EL output from the light source 32 is reflected by the target object 60, and the distance information between the 3D image processing system 100 and the target object 60 is calculated as in Equation 3 below. do.

&Quot; (3) "

Z = θ * C / (2 * ω) = θ * C / (2 * (2πf))

Here, C represents the luminous flux.

According to an embodiment, the processor 110 may calculate the distance information to the target object 40 using the plurality of pixel signals A0, A1, A2, and A3 instead of the microprocessor 39.

According to another embodiment, the microprocessor 39 and the processor 110 may be implemented as one processor.

FIG. 5 illustrates an embodiment of a unit pixel array constituting a part of the pixel array illustrated in FIG. 4.

Referring to FIG. 5, the unit pixel array 522-1 constituting a part of the pixel array 22 of FIG. 4 may include a red pixel R, a green pixel G, a blue pixel B, and a depth pixel (B). D). Red pixel R, green pixel G, and blue pixel B may be referred to as RGB color pixels.

The red pixel R generates a red pixel signal corresponding to the wavelengths belonging to the red region of the visible region, and the green pixel G generates the green pixel signal corresponding to the wavelengths belonging to the green region of the visible region. The blue pixel B generates a blue pixel signal corresponding to wavelengths belonging to a blue region of the visible light region. The depth pixel D generates a depth pixel signal corresponding to the wavelengths belonging to the infrared region.

6 illustrates another embodiment of a unit pixel array constituting a part of the pixel array illustrated in FIG. 4. Referring to FIG. 6, the unit pixel array 22-2 constituting a part of the pixel array 22 of FIG. 4 includes a red pixel R, a green pixel G, a blue pixel B, and two depths. Pixels D may be included.

The unit pixel arrays 22-1 and 22-2 illustrated in FIGS. 5 and 6 are exemplarily illustrated for convenience of description, and the pattern of the unit pixel array and the pixels constituting the pattern may vary. Modifications are possible. For example, each pixel R, G, and B shown in FIGS. 5 and 6 may be replaced with a magenta pixel, cyan pixel, and yellow pixel.

FIG. 7 is a flowchart illustrating a method of operating the 3D image processing system illustrated in FIG. 1.

1 to 7, the 3D image sensor 10 receives an optical signal RL1 having a wavelength in an infrared region (S10).

At this time, the first region 40-1 of the filter 40 passes the optical signal RL1 having the wavelength of the infrared region. The shutter controller 51 controls the shutter 50 to open the third region 50-1 of the shutter 50.

The 3D image sensor 10 generates distance information by using the optical signal RL1 having the wavelength of the infrared region (S20).

The 3D image sensor 10 receives an optical signal RL2 having a wavelength in the visible light region (S30).

At this time, the second region 40-2 of the filter 40 passes the optical signal RL2 having the wavelength of the visible ray region. The shutter controller 51 controls the shutter 50 to open the fourth region 50-2 of the shutter 50.

The 3D image sensor 10 generates color information by using the optical signal RL2 having the wavelength of the visible ultraviolet region (S40).

The processor 110 generates a 3D image by using the distance information and the color information (S50).

Although the present invention has been described with reference to one embodiment shown in the drawings, this is merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

10; 3D image sensor
20; integrated circuit
32; Light source
34; lens
40; filter
40-1; First area
40-2; Second area
50; shutter
50-1; Third area
50-2; Fourth area
51; Shutter controller
100; 3D image processing system
110; Processor

Claims (7)

A filter comprising a first region for passing the wavelength in the infrared region and a second region for blocking the wavelength in the infrared region; And
And a shutter including a third region corresponding to the first region and a fourth region corresponding to the second region. The system of claim 1, wherein the 3D image processing system comprises:
And a shutter controller to control the shutter to selectively open the third region and the fourth region. The method of claim 2, wherein the shutter controller,
And control the shutter to open the third region to generate distance information using the wavelength of the infrared region. The method of claim 2, wherein the shutter controller,
And control the shutter to open the fourth region to generate color information excluding the wavelength of the infrared region. The 3D image processing system of claim 1, wherein the shutter is implemented by liquid crystal. The system of claim 1, wherein the 3D image processing system comprises:
3D image processing further comprising a pixel array comprising a depth pixel for generating a depth pixel signal corresponding to the wavelengths of the infrared light, and a plurality of color pixels for generating a color pixel signal corresponding to wavelengths of a visible light region system. The system of claim 1, wherein the 3D image processing system comprises:
3D image processing system that is a digital camera.

Images