WO2013136817A1

WO2013136817A1 - Image capture device

Info

Publication number: WO2013136817A1
Application number: PCT/JP2013/001795
Authority: WO
Inventors: 慎矢藤原; 森　吉造
Original assignee: 株式会社ニコン
Priority date: 2012-03-15
Filing date: 2013-03-15
Publication date: 2013-09-19

Abstract

[Solution] An image capture device comprises an image capture unit, a generator unit, and a control unit. The image capture unit further comprises a first photoreceptor element which generates a first image signal, and a second photoreceptor element which generates a second image signal with a higher sensitivity than the first image signal. On the basis of the second image signal and the first image signal, the generator unit generates a third image signal with an SNR which is higher than the first image signal and lower than the second image signal. The control unit selects as an output signal from a pixel an image signal among the first image signal, the second image signal, and the third image signal with the highest SNR with a set image capture sensitivity.

Description

Imaging device

The present invention relates to an imaging apparatus.

2. Description of the Related Art Conventionally, an electronic camera that captures an image with a wide dynamic range by applying an image sensor that generates image signals having different sensitivities with one set of light receiving elements per pixel is known.

JP 2005-72965 A

In a general electronic camera, the signal-to-noise ratio of the image signal is reduced as the imaging sensitivity increases. On the other hand, in the above electronic camera, two types of outputs, a low-sensitivity light-receiving element and a high-sensitivity light-receiving element, are switched, but the S / N ratio can be greatly increased despite increasing the imaging sensitivity before and after switching. There is a possibility, it can give the user a sense of incongruity.

In view of the above circumstances, there is provided an imaging apparatus that enables imaging in a relatively wide dynamic range and that does not give a sense of discomfort to the user when the imaging sensitivity is changed.

The imaging device according to one aspect includes an imaging unit, a generation unit, and a control unit. The imaging unit includes a first light receiving element that generates a first image signal and a second light receiving element that generates a second image signal with higher sensitivity than the first image signal. The generation unit generates a third image signal having an SN ratio that is larger than the first image signal and lower than the second image signal, based on the second image signal and the first image signal. The control unit selects, as an output signal from the pixel, an image signal having the highest S / N ratio with the set imaging sensitivity among the first image signal, the second image signal, and the third image signal.

1 is a block diagram illustrating a configuration example of an electronic camera according to an embodiment. The figure which shows the example of the 1st light receiving element and 2nd light receiving element of one Embodiment The figure which shows the circuit structural example of the imaging part in one Embodiment (A), (b): The schematic diagram which shows the operation example of the estimation part in one Embodiment. Schematic diagram showing an example of a data table in one embodiment

<Description of Embodiment>
FIG. 1 is a block diagram illustrating a configuration example of an electronic camera 11 according to an embodiment that is an example of an imaging apparatus. The electronic camera 11 includes an imaging lens 12, an imaging unit 13, a signal processing unit 14, an image processing unit 15, a first memory 16, a second memory 17, a recording I / F 18, a CPU 19, and an operation unit. 20 and a bus 21. Here, the image processing unit 15, the first memory 16, the second memory 17, the recording I / F 18, and the CPU 19 are respectively connected via a bus 21. Further, the imaging unit 13, the signal processing unit 14, and the operation unit 20 are each connected to the CPU 19.

The imaging unit 13 is an imaging device that images an image formed by the imaging lens 12. The imaging unit 13 according to the embodiment is a CMOS solid-state imaging device that can read an image signal of an arbitrary light receiving device by random access. The output of the imaging unit 13 is connected to the signal processing unit 14.

A plurality of pixels are arranged in a matrix on the light receiving surface of the imaging unit 13. Further, red (R), green (Gr, Gb), and blue (B) color filters are arranged in a known Bayer array in each pixel of the imaging unit 13.

Each pixel of the imaging unit 13 has two light receiving elements, a first light receiving element PD1 and a second light receiving element PD2. Note that, under the same conditions, the first light receiving element PD1 generates a relatively low sensitivity image signal (first image signal), and the second light receiving element PD2 has a relatively high sensitivity image signal (second image signal). ) Is generated. As an example, the sensitivity of the second light receiving element PD2 in one embodiment is set to about four times that of the first light receiving element PD1.

FIG. 2 is a diagram illustrating an example of the first light receiving element PD1 and the second light receiving element PD2 according to an embodiment. Although 2 × 2 pixels (PX) are partially shown in FIG. 2, it goes without saying that a larger number of pixels are arranged on the light receiving surface of the actual imaging unit 13.

In each pixel of one embodiment, the light receiving area of the second light receiving element PD2 is set larger than the light receiving area of the first light receiving element PD1. Therefore, under the same conditions, the second light receiving element PD2 has a higher light receiving amount per unit time than the first light receiving element PD1, and the second image signal is relatively more sensitive than the first image signal.

FIG. 3 is a diagram illustrating a circuit configuration example of the imaging unit 13 according to an embodiment. The imaging unit 13 is provided for each of the plurality of pixels PX, the vertical scanning unit 13a, the horizontal scanning unit 13b, the signal storage unit 13c, the plurality of scanning lines 13d extending in the horizontal direction, and the columns of the pixels PX. And a signal readout line 13e.

One end of each scanning line 13d is connected to the vertical scanning unit 13a. The vertical scanning unit 13a gives a pulse (selection pulse, reset pulse, transfer pulse, etc.) for designating a row to be read from the scanning line 13d to each pixel. The horizontal scanning unit 13b outputs a pulse for designating a readout target column to the signal storage unit 13c. The lower end of the signal readout line 13e is connected to the signal storage unit 13c. The signal storage unit 13 c reads out image signals from the respective pixels via the signal readout line 13 e and sequentially outputs the read image signals to the signal processing unit 14.

3 includes a first light receiving element PD1, a transfer transistor TX1 connected to the first light receiving element PD1, a second light receiving element PD2, and a transfer transistor TX2 connected to the second light receiving element PD2. A floating diffusion FD, a reset transistor RES, an amplification transistor AMP, and a selection transistor SEL. In the example of FIG. 3, the floating diffusion FD, the reset transistor RES, the amplification transistor AMP, and the selection transistor SEL are shared for the two sets of light receiving elements and transfer transistors.

The first light receiving element PD1 and the second light receiving element PD2 each generate a signal charge according to the amount of incident light. The transfer transistors TX1 and TX2 transfer the signal charges accumulated in the connected light receiving elements to the floating diffusion FD.

The reset transistor RES resets the floating diffusion FD to the power supply voltage. The amplification transistor AMP outputs a read current to the output terminal via the selection transistor SEL according to the voltage value of the floating diffusion FD. The selection transistor SEL connects the source of the amplification transistor AMP to the output terminal.

Here, the imaging unit 13 of the embodiment can read out the first image signal or the second image signal as the output signal of the pixel by operating any one of the transfer transistors TX1 and TX2. In the subtraction readout mode described later, the imaging unit 13 outputs the first image signal and the second image signal separately for each pixel.

In addition, the imaging unit 13 according to an embodiment can read the added image signal of the first image signal and the second image signal as the output signal of the pixel. When the first image signal and the second image signal are added and read out, the imaging unit 13 once accumulates the signal charges of the first light receiving element PD1 and the second light receiving element PD2 in the floating diffusion FD and then outputs them. Good.

Referring back to FIG. 1, the signal processing unit 14 includes an AFE 14a, a subtraction unit 14b, and an estimation unit 14c. The AFE 14a is an analog front-end circuit that performs analog signal processing on the output of the signal storage unit 13c. The AFE 14a performs correlated double sampling, image signal gain adjustment, and image signal A / D conversion. As an example of the sensitivity changing unit, the AFE 14a adjusts the gain of the image signal in accordance with an instruction from the CPU 19, and changes the imaging sensitivity corresponding to the ISO sensitivity.

The subtraction unit 14b is an example of a generation unit, and subtracts the first image signal from the second image signal for each pixel in accordance with an instruction from the CPU 19 to subtract an image signal (an example of a third image signal) for each pixel. Is a circuit that generates

Here, in the following description, an operation mode in which the first image signal is output from each pixel of the imaging unit 13 is referred to as a “first readout mode”. An operation mode in which the second image signal is output from each pixel of the imaging unit 13 is referred to as a “second readout mode”. An operation mode in which an added image signal is output from each pixel of the imaging unit 13 is referred to as an “addition readout mode”. An operation mode in which the subtraction unit 14b is operated to generate a subtraction image signal for each pixel is referred to as a “subtraction readout mode”. Each operation mode is switched according to an instruction from the CPU 19 which is an example of a control unit.

The estimation unit 14c is a circuit that outputs an estimated value of the subtraction image signal as an output signal when the second image signal of the pixel to be processed is at a saturation level in the subtraction readout mode. When the second image signal is saturated, the signal level of the second image signal becomes lower than the original level, and if the first image signal is subtracted from the second image signal as it is, it is excessively pulled. Therefore, when the second image signal is saturated in the subtraction readout mode, the CPU 19 outputs the estimated value obtained by the estimation unit 14c as a subtraction image signal. Thereby, the subtraction image signal can be used even in a scene where the second image signal is saturated.

FIG. 4 is a schematic diagram illustrating an operation example of the estimation unit according to one embodiment. As an example, the estimation unit 14c may obtain an estimated value by multiplying the first image signal by a gain corresponding to the sensitivity ratio between the first image signal and the subtracted image signal (see FIG. 4a). As another example, the estimation unit 14c may estimate the value of the second image signal by multiplying the first image signal by a gain corresponding to the sensitivity ratio between the first image signal and the second image signal. Thereafter, the estimation unit 14c may obtain an estimated value by subtracting the value of the first image signal from the estimated value of the second image signal (see FIG. 4b).

The image processing unit 15 performs various types of image processing (color interpolation processing, gradation conversion processing, white balance adjustment, etc.) on the digital image signal output from the signal processing unit 14.

The first memory 16 temporarily stores image data in the pre-process and post-process of image processing. For example, the first memory 16 is an SDRAM that is a volatile storage medium. The second memory 17 is a non-volatile memory that stores a program executed by the CPU 19 and a data table indicating a correspondence relationship (see FIG. 4) between the SN ratio and the imaging sensitivity value in each image signal. .

The recording I / F 18 has a connector for connecting the nonvolatile storage medium 22. The recording I / F 18 writes / reads image data to / from the storage medium 22 connected to the connector. The storage medium 22 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 22.

The CPU 19 is a processor that comprehensively controls the operation of the electronic camera 11. For example, the CPU 19 drives the imaging unit 13 in accordance with a user's imaging instruction input in a shooting mode for shooting a subject, and executes a still image imaging process that involves recording in the nonvolatile storage medium 22. To do.

In addition, the CPU 19 in the shooting mode executes a known automatic exposure calculation prior to capturing a still image, and sets the imaging sensitivity that is one parameter of the imaging conditions. Further, the CPU 19 may set the above-described imaging sensitivity based on a user input. Then, the CPU 19 in the shooting mode selects the operation mode (first readout mode, second readout mode, addition readout mode, subtraction readout mode) of the imaging unit 13 and the signal processing unit 14 according to the value of the imaging sensitivity. select.

The operation unit 20 has a plurality of switches that accept user operations. The operation unit 20 includes, for example, a release button that accepts an instruction to capture a main image, a cross-shaped cursor key, a determination button, and the like.

Next, an operation example of the electronic camera 11 in one embodiment will be described. The CPU 19 in the shooting mode refers to the data table in the second memory 17 and changes the operation mode of the imaging unit 13 and the signal processing unit 14 according to the value of the imaging sensitivity. At this time, the CPU 19 selects the above operation mode so that the SN ratio of the image signal output from the pixel is the highest. And the imaging part 13 switches the image signal output in imaging | photography mode according to the operation mode which CPU19 selected.

Here, the SN ratio of each image signal in one embodiment will be described. The noise of the light receiving element includes a component of light shot noise and a component of baseline noise (hereinafter referred to as “B”) generated in the readout circuit in the dark. The light shot noise can be obtained from the square root of photons incident on the light receiving element.

For example, consider a case where the sensitivity of the second light receiving element PD2 is about four times that of the first light receiving element PD1. The charge generation amount per lux in the first light receiving element PD1 is “A [e / lx · s]”, and the charge generation amount per lux in the second light receiving element PD2 is “4A [e / lx · s]. ] "And the baseline noise B is assumed to be sufficiently small to be considered zero. Under such conditions, the SN ratio (S ₁ / N ₁ ) of the first image signal can be obtained by the following equation (1). Similarly, the SN ratio (S ₂ / N ₂ ) of the second image signal can be obtained by the following equation (2).

Comparing the above formulas (1) and (2), S ₂ / N ₂ has a larger value than S ₁ / N ₁ . Therefore, it can be seen that, under the same conditions, the image signal acquired from the relatively high sensitivity light receiving element is superior in the SN ratio.

When the first image signal and the second image signal are added, the amount of charge generated per unit time in the added image signal becomes larger than that of the first image signal and the second image signal. Therefore, the SN ratio of the added image signal is higher than that of the second image signal under the same conditions. Further, when the first image signal is subtracted from the second image signal under the conditions of the embodiment, the amount of charge generated per unit time in the subtracted image signal is larger than that of the first image signal, but smaller than that of the second image signal. Become.

FIG. 5 is a schematic diagram showing an example of a data table in one embodiment. The data table of FIG. 5 shows each image signal (first image signal) when the signal value of the same brightness (for example, 100LSB in a 12-bit gradation range) is acquired while changing the value of the imaging sensitivity (ISO sensitivity). , The second image signal, and the added image signal). For convenience, in FIG. 5, imaging sensitivity values are indicated by symbols A to I in ascending order.

In FIG. 5, the higher the imaging sensitivity value, the shorter the charge accumulation time during imaging. Further, when the imaging sensitivity is increased with the same type of image signal, the charge accumulation time is shortened, the signal component is reduced, and the influence of the baseline noise is relatively increased. As a result, in FIG. 5, the SN ratio decreases as the imaging sensitivity is increased even for the same type of image signal.

In addition, since the charge accumulation time at the time of photographing becomes longer when the value of the imaging sensitivity is lowered, the light receiving element of the imaging unit 13 becomes saturated with a lower imaging sensitivity as the sensitivity becomes higher, and white spots are likely to occur. Therefore, in FIG. 5, there is no value of the subtraction image signal in the range where the ISO sensitivity is less than C, and the selection of the subtraction readout mode in this range (A to B) is disabled. Further, there is no value of the second image signal in a range where the ISO sensitivity is less than D, and selection of the second readout mode in the range (A to C) is disabled. Further, in FIG. 5, there is no value of the added image signal in the range where the ISO sensitivity is less than E, and the selection of the addition reading mode in the range (A to D) is disabled.

The CPU 19 in the shooting mode refers to the data table of FIG. 5 and selects the first readout mode as the operation mode of the imaging unit 13 and the signal processing unit 14 when the ISO sensitivity is the lowest. Then, the CPU 19 changes the operation mode stepwise in the order of the subtraction readout mode, the second readout mode, and the addition readout mode in accordance with the increase in ISO sensitivity.

That is, when the value of the imaging sensitivity is A to B, only the first image signal can be selected as the output signal in the data table. At this time, the CPU 19 selects the first readout mode corresponding to the first image signal as the operation mode of the imaging unit 13 and the signal processing unit 14.

When the value of the imaging sensitivity is C, the first image signal and the subtracted image signal can be selected as output signals in the data table. At this time, the CPU 19 selects the subtraction readout mode corresponding to the subtraction image signal having the highest SN ratio from the two types of image signals as the operation mode of the imaging unit 13 and the signal processing unit 14.

In the subtraction readout mode, both the first image signal and the second image signal are required. In the above example, when the ISO sensitivity is C, the selection of the second readout mode is disabled. However, when the second image signal is saturated, the estimated value of the estimation unit 14c is used to reduce the ISO sensitivity. Even in the case of C, it is possible to output a subtraction image signal.

When the value of the imaging sensitivity is D, the first image signal, the subtraction image signal, and the second image signal can be selected as output signals in the data table. At this time, the CPU 19 selects the second readout mode corresponding to the second image signal having the highest SN ratio from among the three types of image signals as the operation modes of the imaging unit 13 and the signal processing unit 14.

When the value of the imaging sensitivity is E or more, the first image signal, the subtraction image signal, the second image signal, and the addition image signal can be selected as output signals in the data table. At this time, the CPU 19 selects an addition readout mode corresponding to the addition image signal having the highest SN ratio from the four types of image signals as the operation modes of the imaging unit 13 and the signal processing unit 14.

Hereinafter, the operational effects of the electronic camera of one embodiment will be described. According to the electronic camera 11 of one embodiment, an image can be acquired with low imaging sensitivity by performing imaging using the relatively light-sensitive first light receiving element PD1. In addition, the electronic camera 11 according to an embodiment can perform imaging using the relatively high-sensitivity second light receiving element PD2 according to the setting of the imaging sensitivity. Then, the electronic camera 11 according to the embodiment selects an image signal having the highest SN ratio with the current imaging sensitivity among the first image signal, the second image signal, the added image signal, and the subtracted image signal, Images with good S / N ratios can be acquired in a wide range of imaging sensitivity.

In the electronic camera of one embodiment, since the output image signal is selected from four types of image signals, the image signal is switched compared to the case where only the first image signal and the second image signal are selectively used. The change width of the SN ratio becomes small. Therefore, the discontinuity between the change in the imaging sensitivity and the change in the SN ratio is alleviated, so that the user's uncomfortable feeling can be reduced.

<Modification of Embodiment>
(1) In the above embodiment, the difference in sensitivity is caused by the difference in the light receiving area between the first light receiving element PD1 and the second light receiving element PD2. However, the first light receiving element and the second light receiving element of the present invention are the above-described ones. The configuration is not limited.

For example, microlenses having different light collection rates may be arranged on the first light receiving element and the second light receiving element, respectively, to cause a difference in sensitivity between the two light receiving elements. In this case, the same effect as that of the embodiment can be obtained, and the first light receiving element and the second light receiving element can be manufactured by the same semiconductor process.

Alternatively, the same color filters having different light transmittances may be arranged on the first light receiving element and the second light receiving element, respectively, to cause a difference in sensitivity between the two light receiving elements. In this case, the same effect as that of the embodiment can be obtained, and the first light receiving element and the second light receiving element can be manufactured by the same semiconductor process.

Furthermore, two light receiving elements having different photoelectric conversion efficiencies may be arranged in one pixel to serve as the first light receiving element and the second light receiving element. In this case, the same effect as that of the embodiment can be obtained.

It should be noted that the configurations of the first light receiving element and the second light receiving element of the present invention may be realized by a combination of two or more of the techniques disclosed in the above-described one embodiment and the modified example of the one embodiment.

(2) In the above embodiment, when the sensitivity difference between the first light receiving element PD1 and the second light receiving element PD2 is small, the control unit selects the subtracted image signal as the output signal when the imaging sensitivity is the lowest, and the imaging sensitivity The output signal may be changed stepwise in the order of the first image signal, the second image signal, and the added image signal.

(3) In the above embodiment, the subtraction image signal may be generated inside the solid-state imaging device 14. For example, the first image signal and the second image signal may be subtracted using a CDS circuit of the image sensor.

(4) In the above embodiment, the example in which the imaging device is an electronic camera has been described. However, the imaging device of the present invention integrates an imaging unit, an addition unit, a subtraction unit, a sensitivity change unit, a control unit, and an estimation unit. The solid-state imaging device may be used.

(5) In the above embodiment, the estimation unit 14c may obtain an image signal (estimated value) having an SN ratio larger than that of the first image signal and lower than that of the second image signal as follows. For example, the estimation unit 14c calculates a difference value between the second image signal and the first image signal, and multiplies the calculated difference value by a predetermined coefficient. Then, the estimation unit 14c may obtain an estimated value by adding a value obtained by multiplying the difference value by a predetermined coefficient to the first image signal.

From the above detailed description, the features and advantages of the embodiment will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 11 ... Electronic camera, 13 ... Imaging part, 13a ... Vertical scanning part, 13b ... Horizontal scanning part, 14 ... Signal processing part, 14a ... AFE, 14b ... Subtraction part, 14c ... Estimation part, 17 ... 2nd memory, 19 ... CPU, PD1 ... first light receiving element, PD2 ... second light receiving element, TX1 ... transfer transistor, TX2 ... transfer transistor, FD ... floating diffusion, RES ... reset transistor, AMP ... amplification transistor, SEL ... selection transistor

Claims

An imaging unit including a first light receiving element that generates a first image signal and a second light receiving element that generates a second image signal with higher sensitivity than the first image signal;
Based on the second image signal and the first image signal, a generation unit that generates a third image signal having an SN ratio that is larger than the first image signal and lower than the second image signal;
A control unit that selects, as an output signal from a pixel, an image signal having the highest S / N ratio at a set imaging sensitivity among the first image signal, the second image signal, and the third image signal;
An imaging apparatus comprising:
The imaging device according to claim 1,
The control unit selects the first image signal as the output signal when the imaging sensitivity is the lowest, and the third image signal and the second image signal in this order in accordance with an increase in the imaging sensitivity. An imaging device that changes output signals in stages.
In the imaging device according to claim 1 or 2,
An imaging apparatus further comprising: an estimation unit that outputs an estimated value of the third image signal as the output signal when the second image signal is at a saturation level.
The imaging device according to claim 3.
The estimation apparatus is an imaging apparatus that obtains the estimated value by multiplying the first image signal by a gain.
The imaging device according to claim 3.
The estimation unit estimates a value of the second image signal from a sensitivity ratio between the first image signal and the second image signal and a value of the first image signal, and the estimated second image signal An imaging apparatus that obtains the estimated value by subtracting the value of the first image signal from the value.