KR20100081195A - Image pickup apparatus and image control method thereof - Google Patents

Abstract

PURPOSE: An image photographing device and an image control method thereof are provided to use 2D image sensor and sense a reflected signal from a signal, thereby displaying 3D image. CONSTITUTION: A control unit(130) controls transmitting/receiving timing of a light about a light source and an image sensor. The control unit receives pixel data from the image sensor. An image processor(132) obtains delay phase of transmitting/receiving signal from pixel data by unit pixel of a first pixel group and a second pixel group. The image processor obtains distance with an object. A display unit displays 3D image by the image processor.

Description

Image capturing device and image control method thereof {IMAGE PICKUP APPARATUS AND IMAGE CONTROL METHOD THEREOF}

The embodiment relates to a photographing apparatus and an image control method thereof.

Since the 3D imaging apparatus provides 3D image data about an object, it is used in various fields such as a general camera, a home service robot, a medical imaging device, a security surveillance system, and the like.

The embodiment provides a photographing apparatus and a method of controlling an image using the same, which enable a 2D image sensor to detect a signal reflected from an object and display the 3D image.

According to an embodiment of the present invention, a phase retardation plate is disposed on one of a plurality of adjacent pixels of an image sensor, and an image pickup device and a distance difference of a transmission / reception signal can be calculated from a correlation function of a plurality of pixel data. Provides an image control method.

According to an embodiment, an image photographing apparatus includes a light source for transmitting modulated light; A lens collecting the reflected light from the object; An image sensor arranged to intersect the pixels of the first pixel group and the second pixel group having the phase delay means, for receiving the reflected light received through the lens and outputting a plurality of pixel data for each pixel in the transmission period of the light source ; A control unit controlling transmission / reception timing of light to the light source and the image sensor and receiving pixel data from the image sensor; An image processor obtaining a delay phase of a transmission / reception signal and a distance between an object from pixel data of unit pixels of a first pixel group and a second pixel group among pixel data input through the controller; And a display unit displaying the 3D image by the image processor.

An image control method of an imaging apparatus according to an embodiment includes transmitting modulated light; Receiving light reflected from the object through pixels of a first pixel group of an image sensor and a second pixel group having a phase delay means, and outputting pixel data of the received first pixel group and the second pixel group; Obtaining a delay phase and a distance difference of a transmission / reception signal using pixel data of each unit pixel of a first pixel group of the image sensor and a second pixel group adjacent thereto; And displaying a 3D image by using a distance difference from the pixel data of the image sensor.

The embodiment has the effect of expressing a 3D image using a 2D image sensor.

The embodiment has the effect of expressing a 3D image without using a 3D camera.

Hereinafter, exemplary embodiments will be described with reference to the accompanying drawings.

1 is a diagram illustrating a photographing apparatus according to an exemplary embodiment.

Referring to FIG. 1, the photographing apparatus 100 includes a light source 110, a shutter 120, a lens 122, an image sensor 124, a first analog digital conversion unit 126, and a second analog digital conversion unit ( 128, a controller 130, an image processor 132, a memory 134, and a display unit 136.

The light source 110 includes a laser or a light emitting device (LED) array and is driven under the control of the controller 130. The light irradiated from the light source 110 is reflected by the object 115 and is incident on the lens 120 through the shutter 120. The light source 110 may be provided in a single unit on one side of the image sensor 124 or a plurality of light sources 110 on both sides.

The light source 110 generates an optical signal modulated at a predetermined frequency. For example, the light source 110 may output a modulation frequency of 20 MHz. The modulation frequency may vary with distance, but is not limited to the frequency.

Under the control of the controller 130, the shutter 120 may adjust the opening and closing time of the shutter 120 to be long or short, and to adjust the amount of light incident through the lens 122. In addition, the shutter 120 may be disposed at a front end and / or a rear end of the lens 122, but is not limited thereto. The lens 122 provides the light reflected from the object to the image sensor 124.

1 and 2, the image sensor 124 is a single two-dimensional image sensor, which may be implemented as a complementary metal-oxide semiconductor (CMOS), and a plurality of pixels are arranged in a matrix. Each pixel P includes a photodetector (eg, a photodiode) and a switch. The photodetector is turned on and off by the switch and converts light into an electrical signal.

The image sensor 124 may be implemented as a charge-coupled device (CCD), and may be implemented with a resolution of N * N pixels (N ≧ 2), but is not limited thereto.

The pixel array of the image sensor 124 includes an odd number field arranged in the order of the first pixel P1 and the second pixel P2, and the array of the second pixel P2 and the first pixel P1 in the order. The odd field and the even field pixels are arranged to cross each other. The phase retardation plate W1 is disposed on the second pixel P2, and the first pixel P1 is disposed at left, right, top, and bottom positions of the second pixel P2.

Here, the phase retardation plate W1 may be disposed on the second pixel P2 or the photodetector at a position where light is incident, and the phase retardation plate W1 may have a phase retardation value of λ / 2 or λ. It will have a / 4 phase delay value. Accordingly, the phase of the signal received at the second pixel P2 may be delayed by at least λ / 2 or λ / 4 relative to the signal received at the first pixel P1.

The group of the first pixels P1 without the phase retardation plate W1 is a first pixel group 124A, and the group of the second pixels P2 with the phase retardation plate W1 is a second pixel group ( 124B).

In addition, each pixel P of the image sensor 124 may define a general first pixel P1 and a second pixel P2 for detecting a signal having a predetermined phase delay as the unit pixel P3.

The image sensor 124 is turned on / off by the control signal (S1, S2) of the control unit 130, the photodetector is an electrical signal to the light incident through the lens 122 The output will be converted to. In this case, the image sensor 124 is turned on by the driver (driver) arranged in the row and column to turn on the switch element of each pixel (P) to drive the light detection element (for example, Photo Diode).

The image sensor 124 drives all pixels at regular cycles to output pixel signals of the first pixel group 124A and the second pixel group 124B.

The first analog digital converter 126 converts the signal of the first pixel group 124A into digital data in the image sensor 124 and outputs the digital data. The second analog digital converter 128 converts the signal of the second pixel group 124B into digital data in the image sensor 124 and outputs the digital data.

That is, the pixel signal of the image sensor 124 is data of the first pixel group 124A and data of the second pixel group 124B, and is converted into digital data and then transmitted to the controller 130. The digital data is input to the controller 130 in a serial manner. In this case, a parity check bit may be added to each pixel data to distinguish a group and a pixel of each pixel data.

The first analog digital converter 126 and the second analog digital converter 128 may be implemented in the image sensor 124, but are not limited thereto.

The controller 130 controls operations of the light source 110, the shutter 120, the image sensor 124, the image processor 132, the memory 134, and the like, and the light source 110 and the image sensor 124. ), The timing of operation is controlled to synchronize the transmission / reception signal.

In addition, the controller 130 is controlled by the image processor 132 to control the scanning timing of reading data from the image sensor 124. The controller 130 scans each pixel P of the image sensor 124 in a two-tap pixel architecture, which outputs two pixel data every clock, that is, every modulation period.

The controller 130 transmits the digital data of the first analog digital converter 126 and the second analog digital converter 128 to the image processor 132.

The controller 130 controls the scanning of the image sensor 124 under the control of the image processor 132. That is, data of the first pixel group 124A and data of the second pixel group 124B are received. For example, the data of the first pixel P1 group 124A and the data of the second pixel P2 group 124B are scanned at every half period of the transmission signal (or modulated signal), and each scanned pixel is scanned. Receive data by group. Accordingly, two data are received from each pixel during one modulation period.

The image processor 132 stores the received data of the first pixel P1 group 124A and the data of the second pixel P2 group 124B in the memory 134 and uses a two-tap pixel architecture. The correlation function of four pieces of sampling data is calculated, with two pieces of data per unit pixel per pixel group.

That is, the image processor 132 obtains, from the image sensor 124, the frequency and transmission time of the modulated transmission signal and sampling data of the reception signal reflected through the object at each period of the transmission signal. The convolution sum between the transmit signal and the receive signal makes it possible to calculate the delay phase between the transmit and receive signals. The delay phase is a phase difference between the transmission signal and the reception signal. The delay phase is a distance from the object 115 through the delay phase of the unit pixel. The delay phase is applied to the entire pixel area to the display unit 136 as three-dimensional image data. It becomes possible to display.

The image processor 132 processes the image data and displays the 3D image on the display unit 136. The image processor 132 may be connected to an information processing terminal such as a computer, a PDA, etc. using an interface means (eg, a cable, a USB cable, etc.).

The photographing apparatus 100 uses a time-of-flight (TOF) method using a reflection time of the light source 110. The TOF method sequentially scans the light source 110 to the object 115, records the arrival speed of the light reflected by the object 110, and stores the bending information of the object. That is, since the concave portion of the object 115 has a relatively longer reach and reflection distance than the convex portion, the arrival and reflection times become relatively long, so that the detected phase for each point is detected through each pixel of the image sensor. The delay phase and distance data between pixels can be obtained and displayed in a three-dimensional image.

More specifically, the sampling data of each pixel for measuring the delay phase and the distance difference from the received signal can be obtained as follows.

As shown in FIG. 2, the pixel data of the first pixel group 124A and the second pixel group 124B is read in a plurality of pixel structures of the image sensor, for example, one pixel (eg, one pixel of the first pixel group 124A). P1) and one pixel P2 of the second pixel group 124B are defined as one unit pixel P3, and the unit pixel P3 is defined twice in one cycle of the transmission signal (that is, 1/2 cycle). Is read through).

2 and 3, the transmission signal Se is output at a modulation frequency having a predetermined level Po, and the reception signal Sd is a signal received by reflecting the transmission signal Se from an object. Each pixel P of the image sensor may be detected as a predetermined delayed signal. The reference level D1 of the received signal includes brightness information.

The transmission signal Se and the reception signal Sd have a predetermined delay phase φ corresponding to the distance from the object. The delay phase φ is obtained through the correlation function of the information of the transmission signal and the pixel data detected from the unit pixel P3, that is, the sampling data.

During the first period 1T of the transmission signal Se, the pixels of the first pixel group 124A and the second pixel group 124B of the image sensor are scanned twice at intervals of a half period (T / 2), Four sampling data A0, A1, A2, and A3 which are pixel data of the scanned first pixel group 124A and the second pixel group 124B are detected. In other words, A0 is obtained from the first pixel P1 during the first scanning, A1 is obtained from the second pixel P2, A2 is obtained from the first pixel P1 during the second scanning, and A3 from the second pixel P2. Will be obtained. Accordingly, as shown in FIG. 3B, the first pixel P1 outputs A0 and A2, and the second pixel P2 outputs A1 and A3. The phase difference between the A0 and A1 and the A2 and A3 may differ by the delay phase of the phase delay plate.

As shown in (c) of FIG. 3, the sampling data A0, A1, A2, and A3 are scanned in half a period of the transmission signal to be detected in the data order of A0, A1, A2, A3 from the adjacent unit pixel P3. Can be.

Referring to FIG. 4, since the light reflected from the object is normally received in the first pixel P1 of the unit pixel P3, the sampling data A0 and A2 may be detected. Since the light reflected from the object is received by the phase delay means W1 in the second pixel P2 of the unit pixel P3, sampling data A1 and A3 can be detected. Here, the sampling data A0 and A1, A2 and A3 have a predetermined phase difference according to the phase delay means W1 and the reflected light.

Hereinafter, the delay phase and the distance measuring method will be described.

When four pieces of sampling data are obtained, a convolution sum of the transmission / reception signals is obtained to obtain a correlation function according to the phase of each sampling data to obtain a delay phase between the transmission signal and the reception signal.

The phase of the received signal can be detected by demodulating the light received within each pixel of the image sensor simultaneously, and the demodulation can be obtained by correlation or cross correlation values within the period of the modulated signal.

)은 복조 신호 또는 상관 신호(g(t))와 수신 신호(s(t))의 컨볼루션 합으로 구해진다. 이는 수학식 1과 같이 구해질 수 있다.The cross-correlation value (

Is obtained by the convolution sum of the demodulated signal or correlation signal g (t) and the received signal s (t). This may be obtained as in Equation 1.

The correlation signal g (t) and the received signal s (t) may be given by Equation 2.

은 지연 위상을 나타낸다. Here, s (t) is a received optical signal, g (t) is a demodulation signal (correlation signal), a is a modulation amplitude (modulation amplitude),

Denotes the delay phase.

)이 되며, 상기 수학식 2의 상기 수신된 광 신호와 상기 복조 신호를 대입하여 구하게 되며, 이는 수학식 3과 같이 구해진다.The correlation function C (τ) is the cross-correlation value (

) Is obtained by substituting the received optical signal and the demodulation signal of Equation (2).

)을 갖는 함수로 계산될 수 있으며, 상기 4개의 샘플링 데이터의 위상은

= 0°,

=90°,

=180°,

=270° 이 적용될 수 있으며, 수신된 신호의 위상과 증폭도를 재 계산할 수 있다. 또한 상기 상관 함수(C(τ))에는 실제 측정된 값을 획득하기 위해 오프셋(K) 값을 더해진다.The correlation function C (τ) is a phase of four sampling data (

And the phase of the four sampling data

= 0 °,

= 90 °,

= 180 °,

= 270 ° can be applied and the phase and amplification of the received signal can be recalculated. The correlation function C (τ) is also added with an offset K value to obtain the actual measured value.

The correlation function of each sampling data (AO, A1, A2, A3) can be obtained as shown in Equation 4. The correlation function for each phase of each sampling data is actually four sampling data values.

C (τ ₀ ) is a correlation function of A ₀ , C (τ ₁ ) is a correlation function of A 0, C (τ ₂ ) is a correlation function of A 0, and C (τ ₃ ) is a correlation function of A 0.

 The offset value K may be obtained as shown in Equation 5.

)과 증폭도(a)는 수학식 6과 같이 구해질 수 있다.When the correlation function of the four sampling data is obtained, the delay phase of the received signal s (t)

And amplification degree (a) can be obtained as shown in Equation 6.

)은 각 단위 픽셀에서의 물체와의 위상 차를 나타내는 값이며, 상기 증폭도 a는 물체의 깊이 해상도의 직접 측정치 또는 이미지 센서의 자동 이득 제어 값을 선정하기 위한 기준 값으로 이용될 수 있다.The delay phase (

) Is a value representing a phase difference with an object in each unit pixel, and the amplification degree a may be used as a reference value for selecting a direct measurement of the depth resolution of the object or an automatic gain control value of the image sensor.

)과 증폭도(a)가 구해지면, 물체와의 거리(D)를 구할 수 있게 된다. 이러한 물체와의 거리(D)는 물체와의 깊이 데이터로 표현될 수 있으며, 상기 깊이 데이터는 깊이 차이에 따라 서로 다른 색으로 처리함으로써, 3차원 이미지 데이터로 구현할 수 있게 된다. The delay phase (

) And the amplification degree (a), the distance (D) from the object can be obtained. The distance D to the object may be expressed as depth data with the object, and the depth data may be processed into different colors according to the depth difference, thereby realizing the 3D image data.

The distance from the object may be obtained as shown in Equation 7.

Where D is the distance to the object, L is the non-ambiguity distance range, c is the luminous flux, and fm is the RF modulation frequency.

In this way, the delay phase and distance difference for the unit pixels can be calculated. The embodiment scans a first pixel for receiving normal light in the image sensor and a second pixel for receiving light whose phase is delayed by the phase delay means in a 2-tap pixel architecture, obtains four sampling data, and then applies a TOF method. The delay phase and the distance difference of the signal reflected from the object can be calculated, and the distance difference can be applied to the entire pixel area to implement 3D image data that can accurately express the bending information of the object. That is, it can be expressed as a three-dimensional image using a two-dimensional image sensor.

5 is a flowchart illustrating an image control method using a photographing apparatus according to an exemplary embodiment.

Referring to FIG. 5, light modulated by a light source is reflected from an object, and the reflected light is received by the image sensor through a lens (S101).

The pixel array of the image sensor includes an odd field arranged in an order of a first pixel and a second pixel in which a phase delay plate is disposed, and an even field arranged in an order different from the pixel order of the odd field, and the even field and the even field Identical pixels in the field are arranged offset from each other. That is, the first pixel is disposed at left, right, up, and down positions of the second pixel.

The image sensor scans two pixel data in the modulation period of the transmission signal in each pixel by using two adjacent pixels, that is, the first pixel and the second pixel as unit pixels (S103). At this time, the first pixel of the unit pixel receives the normal light reflected from the object, the second pixel receives the light delayed by the phase delay plate. The pixel data scanned by the first and second pixels is output to the image processor through the control unit.

The image processor obtains pixel data input from a first pixel and a second pixel of a unit pixel (S105), obtains a correlation function of the obtained four pixel data (ie, sampling data), and delays a transmission / reception signal. The difference in phase and distance is obtained (S107). In this case, the correlation function of the four sampling data may be obtained through Equation 1 to Equation 4, and description thereof will be omitted. In addition, through Equation 5, the delay phase of the transmission / reception signal and the distance difference between the object using the delay phase are obtained by using a correlation function of four sampling data. By reflecting the difference between the delay phase and the distance of each unit pixel using the two pixels to the entire pixel, it is possible to implement a three-dimensional image (S109).

Although the present invention has been described above with reference to the embodiments, these are merely examples and are not intended to limit the present invention. It will be appreciated that various modifications and applications are not illustrated.

For example, each component shown in detail in the embodiment of the present invention may be modified. And differences relating to such modifications and applications will have to be construed as being included in the scope of the invention defined in the appended claims.

1 is a block diagram illustrating a photographing apparatus according to an exemplary embodiment.

2 is a diagram illustrating an image sensor according to an exemplary embodiment.

3A is a waveform diagram of a transmission / reception signal, (B) shows sampling data obtained from each pixel, and (C) shows arrangement of sampling data of a unit pixel.

4 is a diagram illustrating an example of a unit pixel and sampling data output therefrom according to an exemplary embodiment.

5 is a flowchart illustrating an image control method of a photographing apparatus according to an exemplary embodiment.

Claims (15)

A light source for transmitting modulated light; A lens collecting the reflected light from the object; An image sensor arranged to intersect the pixels of the first pixel group and the second pixel group having the phase delay means, for receiving the reflected light received through the lens and outputting a plurality of pixel data for each pixel in the transmission period of the light source ; A control unit controlling transmission / reception timing of light to the light source and the image sensor and receiving pixel data from the image sensor; An image processor obtaining a delay phase of a transmission / reception signal and a distance between an object from pixel data of unit pixels of a first pixel group and a second pixel group among pixel data input through the controller; And a display unit to display a 3D image by the image processor. The method of claim 1, The phase delay means is a phase delay plate disposed over each pixel of the second pixel group, And the phase retardation plate has a phase retardation value of λ / 2 or λ / 4 wavelength. The method of claim 1, And the image sensor is a single two-dimensional image sensor. The method of claim 1, A first analog digital converting unit converting the pixel signal of the first pixel group of the image sensor into digital data and outputting the digital signal to the controller; And a second analog digital converter configured to convert pixel signals of the first pixel group of the image sensor into digital data and output the digital signals to the controller. The method of claim 1, And the image processor scans the pixels of the image sensor at predetermined intervals through the control unit to obtain two data for each pixel in the modulation period of the transmission signal. The method of claim 1, And the unit pixel includes one pixel of a first pixel group and one pixel of a second pixel group adjacent thereto. The method of claim 6, And the image processor calculates a delay phase and a distance difference of a transmission / reception signal using four pixel data scanned per unit pixel every 1/2 cycle of a modulation period of the transmission signal. The method of claim 1, The light source includes a laser or LED. The method of claim 1, And a memory for storing the pixel data. The method of claim 7, wherein The image processor obtains a convolution sum of a transmission signal and a received signal, obtains a correlation function for four pixel data obtained from a unit pixel using the convolution sum, and calculates a difference of a correlation function of each pixel of the unit pixel. Photographing device for calculating the delay phase of the transmission and reception signals through. Transmitting the modulated light; Receiving light reflected from the object through pixels of a first pixel group of an image sensor and a second pixel group having a phase delay means, and outputting pixel data of the received first pixel group and the second pixel group; Obtaining a delay phase and a distance difference of a transmit / receive signal using pixel data of unit pixels of a first pixel group of the image sensor and a second pixel group adjacent thereto; And displaying a 3D image by using a distance difference from the pixel data of the image sensor. The method of claim 11, And the image sensor scans pixels of the image sensor at regular intervals and outputs two pixel data for each pixel in a modulation period of the transmission signal. The method of claim 11, The image sensor is a single two-dimensional image sensor, And said phase delay means is a phase delay plate disposed on each pixel of said second pixel group and having a phase delay value of? / 2 or? / 4 wavelength. The method of claim 11, The delay phase is obtained by calculating a correlation function of four pixel data per unit pixel through a convolutional sum of transmission and reception signals, and calculating a difference between correlation functions of the four pixel data and delay phase of the transmission / reception signal for each pixel. Image control method of the photographing apparatus to calculate the. The method of claim 11,  And the image sensor outputs pixel data to a 2-tap pixel architecture.

Images