WO2013100328A1

WO2013100328A1 - Image sensing device and image sensing method

Info

Publication number: WO2013100328A1
Application number: PCT/KR2012/007914
Authority: WO
Inventors: 이병욱
Original assignee: 이화여자대학교 산학협력단
Priority date: 2011-12-29
Filing date: 2012-09-28
Publication date: 2013-07-04
Also published as: KR20130077718A; KR101313968B1

Abstract

The present invention relates to an image sensing device and to an image sensing method capable of extending the dynamic range representing the brightness of light in an image capturing device such as a digital camera. The image sensing device according to one embodiment of the present invention includes: a light sensing unit including a plurality of pixels which convert the light reflected by an input subject into an electrical signal to create an output value; a comparing unit for determining whether the output value from each of the plurality of pixels reaches a preset reference value for a preset first capturing time; a clock unit for measuring the attainment time when the output value is determined to attain the preset reference value for the preset first capturing time by the comparing unit; and a memory unit for calculating the brightness of an image by using the determined and measured values by the comparing unit and the clock unit, and storing the brightness of the image. The image sensing method according to one embodiment of the present invention includes the steps of: creating an output value by converting the light reflected by an input subject into an electrical signal; determining whether the output value from each of the plurality of pixels attains a preset reference value for a preset first capturing time; measuring the time of attainment when the output value is determined to attain the preset reference value for the preset first capturing time; and calculating brightness of the image by using the measured time of attainment.

Description

Image Sensing Device and Image Sensing Method

The present invention relates to an image sensing device and an image sensing method, and more particularly, to an image sensing device and an image sensing method capable of extending a dynamic range expressing brightness of light in an image capturing device such as a digital camera. .

Dynamic range is one of the determinants of image sensor performance. It is an index that indicates how weak and strong light signals can be processed into images. That is, it means the saturation level of the pixel with respect to the signal noise level of the pixel, and is given by the following equation.

Equation 1

Here, D represents the dynamic range of the image sensor, Noise represents the signal noise, Saturation-level represents the saturation level of the pixel.

For example, if the image sensor detects about 200,000 electrons at saturation and about 40 electrons at noise, the dynamic range is about 5,000 and the dB is about 75 dB.

On the other hand, when a dark portion and a bright portion are mixed in one screen, the exposure time for the input light can be adjusted to distinguish each of them. However, there is a limit in distinguishing between dark and bright areas by adjusting the exposure time, and therefore, it is required to widen the dynamic range.

To this end, a method of producing a high dynamic range image by combining a plurality of images taken while changing the exposure time in a conventional imaging device has been proposed. That is, the information of the bright part is obtained by shortening the exposure time, and the information of the dark area is obtained from the image with the long exposure time, and then the information is combined to obtain an image of high dynamic range. However, this method has the advantage of using an existing imaging device, but there is a problem in that the shooting time is excessively long because it is necessary to take several times while changing the exposure time.

In order to overcome this problem, a method of acquiring various ranges of information at the same time by different gains of adjacent image elements has been proposed. However, this method has a problem of reducing the spatial resolution in order to increase the dynamic range.

On the other hand, in order to reduce the shooting time, ① a method of obtaining multiple exposure image information by sampling several times during a long shooting time, ② adjusting the exposure time electronically according to the brightness, ③ resetting each time the brightness is saturated And save the number of times. However, ① method has a problem that the number of shooting time is limited, ② method has a problem that the adjustment of the exposure time is complicated, and ③ method has a problem that the circuit is complicated and the error is likely to occur due to the reset time, etc. have.

[Preceding technical literature]

[Patent Documents]

(Patent Document 1) Korean Patent Publication No. 10-2008-0043635

(Patent Document 2) Japanese Patent Application Laid-Open No. 2011-066795

(Patent Document 3) US Patent No. 7079178

An object of the present invention is to provide an image sensing device and an image sensing method that can solve the above problems and extend the dynamic range.

An image sensing device according to an embodiment of the present invention,

An optical sensor unit comprising a plurality of pixels for converting light reflected by a subject into an electrical signal and generating an output value; A comparison unit determining whether an output value of each of the plurality of pixels reaches a preset reference value during a preset maximum shooting time; A clock unit configured to measure the arrival time when the output value reaches the preset reference value during the preset first photographing time; And a memory unit for calculating and storing the brightness of the image by using the measured value determined by the comparison unit and the clock unit.

When the output value does not reach the preset reference value during the preset maximum shooting time, the clock unit generates a correction value by lowering the preset reference value, but the output value corresponding to the correction value and the preset reference value does not correspond. And a reference value correction unit for generating the correction value until such time as possible.

The optical sensor unit may include a micro lens that focuses the input light; A color filter extracting a specific color signal from signals input from the micro lens; And a substrate configured to convert the extracted color signal into the electrical signal and output the converted electrical signal. The color filter may include at least one of a primary color filter, a complementary color filter, an infrared filter, and a low pass filter.

The preset reference value may be a saturation output value of the optical sensor.

The comparison unit may include at least one capacitor, and compares a voltage value or a current value generated by the charge accumulated in the optical sensor unit with a preset reference value.

It may include an image brightness calculation unit for calculating the brightness of the image by the following equation (1) using the arrival time measured by the clock unit.

Equation (1): B _i = B _th * T _B / t _i

Here, B _i is the brightness of the image, B _th is the preset reference value, T _B is the preset secondary shooting time, t _i is the time to reach the preset reference value.

The apparatus may further include an image brightness calculator configured to calculate the brightness of the image by the following equation (2) using the reference value corrected by the reference value correction unit and the time to reach the corrected reference value.

Equation (2): B _i '= (B _th ' -B _th ) / (t _i '-T _A ) * t _i ' + α

Here, B _i 'is the brightness of the image, B _th is the preset reference value, B _th ' is the corrected reference value, T _A is the preset primary shooting time, t _i 'is the time to reach the corrected reference value, α is a constant to be.

Α may be ((B _th −B _th ') / (T _A -T _B )) * T _B , where T _B is a preset secondary imaging time.

In addition, the image sensing method according to an embodiment of the present invention,

Converting the light reflected and input by the subject into an electrical signal to generate an output value; Determining whether the output value reaches a preset reference value during a preset maximum shooting time; Measuring the arrival time when it is determined that the output value reaches a preset reference value during the preset first photographing time; And calculating brightness of the image using the measured arrival time.

On the other hand, when the output value does not reach the preset reference value during the preset maximum shooting time, the preset reference value is lowered to generate a correction value until the output value does not reach the preset reference value. Generating a correction value.

In calculating the brightness of the image, the brightness of the image may be calculated by Equation (1) below.

Equation (1): B _i = B _th * T _B / t _i

When the output value does not reach the preset reference value during the preset first shooting time, the brightness of the image may be calculated by the following equation (2) using the corrected reference value and the time to reach the corrected reference value. have.

Equation (2): B _i '= (B _th ' -B _th ) / (t _i '-T _A ) * t _i ' + α

According to the embodiment of the present invention as described above, the image brightness of the relatively dark portion can be measured together while preventing the image of the relatively bright portion from being saturated at the maximum brightness during the maximum shooting time. This has the same effect as taking the shooting time of a relatively dark part longer than the shooting time of a relatively bright part, and having the same effect as opening the camera shutter until both the bright part and the dark part have the same brightness. This has the effect of equalizing image noise relative to image brightness.

1 is a block diagram illustrating an image sensing device according to an embodiment of the present invention;

2 is a view for explaining the extension of the dynamic range by the image sensing device according to an embodiment of the present invention,

3 is a flowchart illustrating an image sensing method according to an embodiment of the present invention.

According to an aspect of the present invention, there is provided an image sensing device including: an optical sensor unit configured to convert an input light reflected by a subject into an electrical signal to generate an output value; A comparison unit determining whether an output value of each of the plurality of pixels reaches a preset reference value during a preset maximum shooting time; A clock unit configured to measure the arrival time when the output value reaches the preset reference value during the preset first photographing time; And a memory unit for calculating and storing the brightness of the image by using the measured value determined by the comparison unit and the clock unit.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention; First, it should be noted that the same components or parts in the drawings represent the same reference numerals as much as possible. In describing the present invention, detailed descriptions of related well-known functions or configurations are omitted in order not to obscure the gist of the present invention.

FIG. 1 is a block diagram illustrating an image sensing device according to an embodiment of the present invention, FIG. 2 is a diagram illustrating an extension of a dynamic range by an image sensing device according to an embodiment of the present invention, and FIG. 3. Is a flowchart illustrating an image sensing method according to an embodiment of the present invention.

In the present invention, the time at which the brightness of the image sensing device reaches a predetermined reference value is recorded. More specifically, the time at which the brightness of each of the plurality of pixels constituting the image sensing element reaches a predetermined reference value is measured and recorded. In the present invention, the exposure time measurement error is within the quantization error range of the clock frequency, which is a reference for time measurement, to be very accurate, and the measured value is excellent in linearity in proportion to the brightness of the image. The linearity of the image is a very important factor in improving and reconstructing the image quality.

To this end, the image sensing device according to an embodiment of the present invention, as shown in Figure 1, the optical sensor unit 100, the comparison unit 200, the clock unit 300, the image brightness calculation unit 400, and a memory unit 500.

The optical sensor unit 100 is formed of a plurality of pixels that convert the light reflected by the subject into an electrical signal to generate an output value. Here, the optical sensor unit 100 may include a micro lens, a color filter, and a substrate. The color filter may include at least one of a primary color filter, a complementary color filter, an infrared filter, and a low pass filter. The primary color filter uses red, green, and blue (RGB), which are three primary colors of light, and has excellent color reproducibility, but has a disadvantage in that the boundary part is somewhat unclear due to the sharpness of the image. Complementary color filters use cyan, magenta, yellow, and green (CMYK), and the image has excellent clarity, but it has a disadvantage of lack of color and shows a slightly flat color. However, the detail power is superior to the complementary color filter. Infrared filters are also useful for filtering out invisible light rays in those areas because of their ability to respond to light in an infrared region that is invisible to the human eye. The low pass filter has the advantage of removing high-frequency components such as fine branches of trees or black letters on white paper and preventing color changes. In the present invention, at least one of the above filters may be included. Here, the image sensing device of the present invention relates to an image sensing device for a high dynamic range, and does not necessarily include a color filter, but may be applied to a black and white image.

The micro lens (not shown) improves light efficiency by condensing the light input to the pixel to receive an optical signal input to an area other than the pixel. The color filter (not shown) extracts a specific color signal from the signals input from the micro lens. The substrate (not shown) includes a photodiode and a transfer electrode, and photoelectrically converts a signal input from a color filter and then transfers the converted electrical signal to the outside.

Here, the optical sensor unit 100 may be a CCD (Charge Coupled Device) image sensor that moves the electrons generated by the input light to the output unit using a gate pulse. Alternatively, the present invention may be a CMOS image sensor that converts electrons generated by the input light into voltage in each pixel and then outputs them through a plurality of complementary metal oxide semiconductor (CMOS) switches.

The comparison unit 200 determines whether the output value of each of the plurality of pixels reaches a preset reference value during the preset first photographing time. The reference value is set by the reference value correction unit 210. The comparison unit 200 may be configured to include at least one capacitor for accumulating charges, and compares a voltage value (or an equivalent current value) generated by the charge accumulated in the optical sensor unit with a preset reference value. Compare.

Here, the preset reference value may be a saturation output value of the optical sensor unit 100, but it is preferably a value smaller than the saturation output value for accurate measurement. The optical sensor unit 100 accumulates charges according to the light intensity, and outputs a voltage corresponding to the accumulated charge amount to determine the brightness of the image. However, the optical sensor unit 100 cannot accumulate charges having a constant light intensity or more, and the maximum accumulation value is referred to as a saturation output value.

Conventionally, the light intensity above the saturation output value is determined by the brightness of the same image. In the present invention, the saturation output value or a smaller value thereof is set as a reference value in advance, and the brightness of the actual image can be expressed according to the time to reach the reference value. . That is, the faster the time to reach the reference value, the brighter the brightness can be determined, so as to determine whether the preset reference value reaches by the comparison unit 200, the clock unit 300 to determine the arrival time Measure it. To this end, the clock unit 300 includes a reference clock 310 for setting the first shooting time and the second shooting time, a counter 320 for measuring the time for reaching the preset reference value, and the start of the measurement. It may include a start control unit 330 for controlling the. The smaller the reference value is, the faster the determination is possible, but too small has the disadvantage of inferior reliability. Therefore, the reference value selects a saturation output value or a predetermined value smaller than it, but is not too small.

The clock unit 300 measures the arrival time when it is determined by the comparator 200 that the output value reaches a preset reference value during the preset first shooting time, and does not measure time when it is determined that the output value is not reached. Do not. For example, referring to FIG. 2, when the output value of the first pixel (see dotted line B1, hereinafter referred to as B1) reaches the reference value Bth, the clock unit 300 measures the arrival time t1. Similarly, when the output value of the second pixel (see dotted line B2, hereinafter referred to as B2) reaches the reference value, the clock unit 300 measures the arrival time t2. When the arrival time is measured in this way, the image brightness calculator 400 calculates the image brightness of the pixel according to Equation (1) below and stores it in the memory unit 500.

Equation (1): B _i = B _th * T _B / t _i

(B _i is the brightness of the image, B _th is the preset reference value, T _B is the preset second shooting time, t _i is the time to reach the preset reference value)

That is, as shown in FIG. 2, the image brightness of the corresponding pixel is linearly calculated using the time ti at which the preset reference value is reached within the preset first photographing time T _A as a parameter. See B2)

However, the output value of the third pixel (see dotted line B3, hereinafter referred to as B3) does not reach the reference value within the preset primary imaging time, and thus the arrival time is not measured. Therefore, ti cannot be determined and the brightness of the pixel cannot be calculated by Equation (1). In order to compensate for this, the reference value correction unit 210 may further include a reference value correction unit 210 that generates a correction value (for example, B _th ', B _th ") by lowering the preset reference value B _th over time. When it is determined that the output value B3 of the third pixel does not reach the preset reference value Bth, the reference value correction unit 210 compensates while lowering the preset reference value, but the corrected value and the output value are in an error range. Correction is performed until it coincides within the circuit, which will be described with reference to Fig. 3, and the output value of the third pixel does not reach the preset reference value B _th during the preset first photographing time T _A. hayeoteumeuro, fails to measure the arrival time. Thus, expression not possible to calculate image brightness according to (1), and corrects the reference value (B _th) predetermined to a value that is smaller than B _th _1. output value (B3 of the third pixel ) Does not reach the calibrated reference value B _th _1 Therefore, the corrected reference value B _th 'is recalibrated to a smaller value, in which case B _th can be regarded as a function that decreases with time, and can be expressed as B _th (t). The time t ₃ at the point where the output value of the pixel meets the straight line formed by B _th (t) is measured The image brightness calculator 400 uses t ₃ measured as described above and the corrected reference value B _th ′. The brightness of the image is calculated by equation (2).

Equation (2): B _i '= (B _th ' -B _th ) / (t _i '-T _A ) * t _i ' + α

Referring to FIG. 3, the α may be calculated, and since B _th becomes 0 at the second photographing time T _B , 0 = (B _th '-B _th ) / (T _B -T _A ) * T _B + From α, it can be found that α is ((B _th -B _th ') / (T _A -T _B )) * T _B. The output values B1, B2, and B3 of the first to third pixels are for illustrative purposes only and do not refer to the order or number of pixels or to a specific pixel. In addition, we define "corresponds" to "matches within tolerance."

In this way, the image brightness of the relatively dark portion can be measured together while preventing the image of the relatively bright portion from being saturated at the maximum brightness during the second photographing time T _B. This has the same effect as taking the shooting time of a relatively dark part longer than the shooting time of a relatively bright part, and having the same effect as opening the camera shutter until both the bright part and the dark part have the same brightness. This has the effect of equalizing image noise relative to image brightness.

Next, an image sensing method according to an embodiment of the present invention will be described with reference to FIG. 3.

First, the brightness reference value B _th , the first photographing time T _A , and the second photographing time T _B of the pixel (pixel) are set. (S10) The brightness reference value is saturated in the optical sensor unit 100. It is desirable to set it to an output value or smaller.

Next, the light intensity B _ti of the pixel is measured (S20). That is, the light intensity of each of the plurality of pixels is measured. For example, the output values B1 to B3 of the first to third pixels are measured.

Next, it is determined whether the measured light intensity B _ti reaches a preset reference value B _th . (S30) At this time, the measured light intensity B _ti is a preset reference value B _th . When is reached, the time t _{i at} that time is measured (S40), and the brightness (B _i ) of the corresponding pixel is calculated by substituting it into Equation (1) below (S50). (S60)

Equation (1): B _i = B _th * T _B / t _i

(B _i is the brightness of the image, B _th is the preset reference value, T _B is the preset second shooting time, t _i is the time when the preset reference value is reached)

As a result of the determination in step S30, if the measured light intensity B _ti does not reach the preset reference value B _th , it is determined whether the first photographing time T _A has elapsed (S70). If the shooting time has not elapsed, the process is repeated again from the step S20.

The first recording time (T _A) is the light intensity (B _ti) measurements were elapsed does not reach a previously set reference value (B _th), while reducing the B _th measure the B _th '= B _ti that t _i is (S80)

Using the measured t _i and the corrected reference value B _th ′, the brightness of the image is calculated using Equation (2) below (S90), and the brightness of the corresponding pixel is determined (S100).

Equation (2): B _i '= (B _th ' -B _th ) / (t _i '-T _A ) * t _i ' + α

Here, B _i 'is the brightness of the image, B _th is the preset reference value, B _th ' is the corrected reference value, T _A is the preset primary shooting time, t _i 'is the time to reach the corrected reference value, α is a constant to be. Referring to FIG. 3, the α may be calculated, and since B _th becomes 0 at the second photographing time T _B , 0 = (B _th '-B _th ) / (T _B -T _A ) * T _B + From α, it can be found that α is ((B _th -B _th ') / (T _A -T _B )) * T _B.

As described above with reference to the drawings illustrating an image sensing device and an image sensing method according to the present invention, the present invention is not limited by the embodiments and drawings disclosed herein, it is within the technical scope of the present invention Of course, various modifications can be made by those skilled in the art.

Claims

An optical sensor unit comprising a plurality of pixels for converting light reflected by a subject into an electrical signal and generating an output value;

A comparison unit determining whether an output value of each of the plurality of pixels reaches a preset reference value during a preset maximum shooting time;

A clock unit configured to measure the arrival time when the output value reaches the preset reference value during the preset first photographing time; And,

Memory unit for calculating and storing the brightness of the image by using the measured value determined by the comparison unit and the clock unit

Image sensing device comprising a.
The method according to claim 1,

When the output value does not reach the preset reference value during the preset maximum shooting time, the clock unit generates a correction value by lowering the preset reference value, but the output value corresponding to the correction value and the preset reference value does not correspond. And a reference value correction unit for generating the correction value until the image sensing element.
The method according to claim 1,

The optical sensor unit,

A micro lens for focusing the input light;

A color filter extracting a specific color signal from signals input from the micro lens; And,

A substrate converting the extracted color signal into the electrical signal and outputting the converted electrical signal

Image sensing device comprising a.
The method according to claim 3,

The color filter includes at least one of a primary color filter, a complementary color filter, an infrared filter, and a low pass filter.
The method according to any one of claims 1 to 3,

And the preset reference value is equal to or less than the saturation output value of the optical sensor unit.
The method according to any one of claims 1 to 3,

And the comparator comprises at least one capacitor, and compares a voltage value or a current value generated by the charge accumulated in the optical sensor part with a preset reference value.
The method according to any one of claims 1 to 3,

An image sensing device including an image brightness calculator configured to calculate brightness of an image by the following equation (1) using the arrival time measured by the clock unit;

Equation (1): B i = B th * T B / t i

(B i is the brightness of the image, B th is the preset reference value, T B is the preset second shooting time, and t i is the time when the preset reference value is reached).
The method according to claim 2, by using the following arrival time measured by the clock unit by the following equation (1),

An image sensing element including an image brightness calculator configured to calculate brightness of an image by the following equation (2) by using the reference value corrected by the reference value correction unit and the time of reaching the corrected reference value

Equation (2): B i '= (B th ' -B th ) / (t i '-T A ) * t i ' + α

(B i 'is the brightness of the image, B th is the preset reference value, B th ' is the corrected reference value, T A is the preset primary shooting time, t i 'is the time to reach the corrected reference value, α is a constant).
The method according to claim 8,

Wherein α is ((B th -B th ') / (T A -T B )) * T B (where T B is a preset secondary shooting time).
Converting the light reflected and input by the subject into an electrical signal to generate an output value;

Determining whether the output value reaches a preset reference value during a preset maximum shooting time;

Measuring the arrival time when it is determined that the output value reaches a preset reference value during the preset first photographing time; And,

Calculating brightness of the image using the measured arrival time

Image sensing method comprising a.
The method according to claim 10,

If the output value does not reach the preset reference value during the preset maximum shooting time, the correction value is generated by lowering the preset reference value, but the correction value until the output value does not correspond to the corrected reference value Image sensing method comprising the step of generating.
The method according to claim 10,

And the preset reference value is equal to or less than the saturation output value of the optical sensor.
The method according to any one of claims 10 to 12,

Computing the brightness of the image, Image sensing method for calculating the brightness of the image by the following equation (1)

Equation (1): B i = B th * T B / t i

(B i is the brightness of the image, B th is the preset reference value, T B is the preset second shooting time, and t i is the time when the preset reference value is reached).
The method according to claim 11,

Image sensing method for calculating the brightness of the image by the following equation (2) using the corrected reference value and the time to reach the corrected reference value

Equation (2): B i '= (B th ' -B th ) / (t i '-T A ) * t i ' + α

(B i 'is the brightness of the image, B th is the preset reference value, B th ' is the corrected reference value, T A is the preset primary shooting time, t i 'is the time to reach the corrected reference value, α is a constant).
The method according to claim 14,

Wherein α is ((B th -B th ') / (T A -T B )) * T B (where T B is a preset second shooting time).