JP2020141412A - Image pickup device and imaging apparatus - Google Patents

Abstract

To provide an image pickup device and an imaging apparatus capable of effectively using electric charge generated in a photodiode.SOLUTION: The image pickup device includes: a photoelectric conversion unit for generating electric charges by photoelectric conversion of incident light; and a plurality of pixels having first and second reading units connected with the photoelectric conversion unit. The first reading unit includes: a first charge/voltage conversion unit for converting the electric charges, transferred from the photoelectric conversion unit, to a voltage; and a first output unit for outputting an output signal corresponding to the voltage of the first charge/voltage conversion unit. The second reading unit comprises a second charge/voltage conversion unit for converting the charges, transferred from the photoelectric conversion unit, to a voltage; and a second output unit for outputting an output signal corresponding to the voltage of the second charge/voltage conversion unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image sensor and an image pickup device.

A global shutter type image sensor is known (for example, Patent Document 1). The conventional technique has a problem that the electric charge generated by the photodiode cannot be effectively used.

JP-A-2004-297089

The imaging device according to claim 1 includes a plurality of pixels having a photoelectric conversion unit that photoelectrically converts incident light to generate electric charges, and first and second reading units connected to the photoelectric conversion unit. The first reading unit includes a first charge-voltage conversion unit that converts the charge transferred from the photoelectric conversion unit into a voltage, and a first output unit that outputs an output signal corresponding to the voltage of the first charge-voltage conversion unit. And the second
The reading unit includes a second charge-voltage conversion unit that converts the charge transferred from the photoelectric conversion unit into a voltage, and a second charge-voltage conversion unit.
It has a second output unit that outputs an output signal corresponding to the voltage of the second charge-voltage conversion unit.
The image pickup apparatus according to claim 7 creates a first display image based on the image pickup element according to any one of claims 1 to 6 and an output signal output from the first output unit. At the same time, an image creation unit that creates a second display image based on the output signal output from the second output unit, and an image display unit that alternately displays the first display image and the second display image. Be prepared.
The image pickup apparatus according to claim 8 includes the image pickup device according to any one of claims 2 to 6, an open state that does not block incident light on the image pickup device, and an incident light on the image pickup device. A shutter device capable of switching between a light blocking state that blocks light is provided, and the first reading unit uses the charge generated by the photoelectric conversion unit during the first period when the shutter device is in the open state as the first charge. The light is transferred to the holding unit, and then an output signal corresponding to the charge is output from the first output unit. In the second reading unit, the shutter device is in the open state and does not overlap with the first period. 2
The electric charge generated by the photoelectric conversion unit during the period is transferred to the second charge holding unit, and then an output signal corresponding to the electric charge is output from the second output unit, and the shutter device is the shutter device. The electric charge generated by the photoelectric conversion unit in the open state and in the first period and the third period after the second period is switched to the light-shielding state and held in the photoelectric conversion unit, and the first reading is performed. The unit receives from the first output unit an output signal based on the electric charge held by the photoelectric conversion unit when the shutter device is in the light-shielding state and the electric charge generated by the photoelectric conversion unit during the first period. After the output is completed, the electric charge is transferred to the first charge holding unit.
The image pickup apparatus according to claim 11 includes the image pickup device according to any one of claims 2 to 6, and a projection unit that projects a predetermined modulated light onto a subject, and the first reading unit is The photoelectric conversion unit transfers a part of the reflected light of the modulated light by photoelectric conversion to the first charge holding unit, and outputs an output signal based on the charge from the first output unit. In the second reading unit, the photoelectric conversion unit transfers the charge generated by photoelectric conversion of the rest of the reflected light of the modulated light to the second charge holding unit, and outputs the light based on the charge. A distance calculation unit that outputs a signal from the second output unit and calculates the distance to the subject based on the output signal output from the first output unit and the output signal output from the second output unit. To be equipped with.
The imaging device according to claim 12 includes a photoelectric conversion unit that photoelectrically converts incident light to generate electric charges.
A plurality of pixels having a first and second reading unit connected to the photoelectric conversion unit are provided, and the first
The reading unit has a first charge holding unit that holds the electric charge transferred from the photoelectric conversion unit, and a first output unit that outputs an output signal corresponding to the electric charge held in the first charge holding unit. The second reading unit is a second charge holding unit that holds the electric charge transferred from the photoelectric conversion unit, and a second output unit that outputs an output signal corresponding to the electric charge held in the second charge holding unit. And have.

It is a schematic diagram which shows the structure of the image pickup apparatus which concerns on 1st Embodiment of this invention. It is a top view which shows typically the image pickup surface 10 of the image pickup element 120. It is a circuit diagram of the image pickup pixel 20. It is a schematic diagram which enlarged the image pickup pixel 20 on the image pickup surface 10. It is sectional drawing and potential figure of the image pickup pixel 20. It is a timing chart of the live view operation. It is a timing chart of the live view operation. It is a timing chart of the main shooting. It is a timing chart of the main shooting. It is a timing chart of the live view operation. It is a timing chart of the live view operation. It is a timing chart of the main shooting. It is a timing chart of the main shooting. It is a timing chart of the exposure auto bracketing shooting operation. It is a figure which shows typically the exposure time of high dynamic range composition. It is a schematic diagram which shows the structure of the image pickup apparatus which concerns on 4th Embodiment of this invention. It is a timing chart of the distance measurement operation.

(First Embodiment)
FIG. 1 is a schematic view showing a configuration of an image pickup apparatus according to a first embodiment of the present invention. The image pickup device 100 includes an imaging optical system 110, an image pickup element 120, a control circuit 130, a mechanical shutter 140, a display device 150, and a recording medium 160.

The imaging optical system 110 forms a subject image on the imaging surface of the image pickup device 120. Control circuit 13
0 controls the entire image pickup apparatus 100. The mechanical shutter 140 is a so-called focal plane shutter, and is provided in the vicinity of the image pickup surface of the image pickup element 120. The display device 150 is a display device such as a liquid crystal display. The recording medium 160 is a portable recording medium such as a memory card.

FIG. 2 is a plan view schematically showing the imaging surface 10 of the imaging element 120. On the imaging surface 10,
A large number of a plurality of imaging pixels 20 are arranged two-dimensionally. The imaging pixels 20 are each red (R).
), Green (G), and blue (B). For example, the imaging pixel 20 corresponding to the red (R) color component outputs a photoelectric conversion signal (imaging signal) obtained by photoelectrically converting the light of the red color component of the incident light. These three types of imaging pixels 20 form a so-called Bayer array.

When the control circuit 130 captures a recording image (so-called main imaging), the control circuit 130 reads an imaging signal from all the imaging pixels. On the other hand, when capturing a live view image, the control circuit 130 thins out and reads out the imaging pixels 20 according to the number of display pixels of the display device 150. That is, instead of reading the imaging signals from all the imaging pixels 20, the display device 1
According to the number of display pixels of 50, for example, the imaging signal is read only from the imaging pixels 20 every three rows (only one of the three rows is read and the remaining two rows are skipped).

Next, the configuration of the imaging pixel 20 will be described in detail with reference to FIGS. 3 to 5. FIG. 3 is a circuit diagram of the imaging pixel 20. The imaging pixel 20 includes a photoelectric conversion unit PD, a first reading unit 21, a second reading unit 22, and a reset transistor R.

The photoelectric conversion unit PD is a photodiode, which performs photoelectric conversion of incident light to generate an electric charge.
The image pickup pixel 20 is provided with a microlens (not shown), and the incident light on the image pickup pixel 20 is focused on the photoelectric conversion unit PD by the microlens. A first reading unit 21 and a second reading unit 22 are connected in parallel to the cathode terminal of the photoelectric conversion unit PD.

The first reading unit 21 includes a first charge holding unit SG1, a first transfer transistor TX1, a first charge-voltage conversion unit FD1, a first amplification transistor SF1, a first output transistor S1 and the like.
It has a first output unit 23. A first constant current source 25 is connected between the first output transistor S1 and the first output unit 23. There is one first constant current source 25 for each row, and 1
The imaging pixels 20 for all rows existing in one column are connected in parallel.

The first charge holding unit SG1 is an N-type MOSFET. Although the details will be described later, the first charge holding unit SG1 has a large capacity so that the charge generated by the photoelectric conversion unit PD can be temporarily held. The first transfer transistor TX1 is an N-type MOSFET, and has a function of transferring the charge held in the first charge holding unit SG1 to the first charge-voltage conversion unit FD1.

The first charge-voltage conversion unit FD1 is a so-called floating diffusion, and is the first.
By temporarily holding the charge transferred from the first charge holding unit SG1 by the transfer transistor TX1, a potential corresponding to the amount of charge held in the first charge holding unit SG1 is taken. That is, the charge held in the first charge holding unit SG1 is converted into a voltage. The first amplification transistor SF1 is an N-type MOSFET, and outputs a signal corresponding to the amount of charge held in the first charge-voltage conversion unit FD1 (potential of the first charge-voltage conversion unit FD1).
Output to 1. The first output transistor S1 is an N-type MOSFET, and outputs an output signal output from the first amplification transistor SF1 to the first output unit 23. That is, an output signal corresponding to the amount of charge held in the first charge-voltage conversion unit FD1 is output to the first output unit 23.

The drain terminal of the first charge holding unit SG1 is connected to the cathode terminal of the photoelectric conversion unit PD. The source terminal of the first charge holding unit SG1 is connected to the drain terminal of the first transfer transistor TX1. The source terminal of the first transfer transistor TX1 is connected to the source terminal of the reset transistor R, the gate terminal of the first amplification transistor SF1, and the first charge-voltage conversion unit FD1. The drain terminal of the reset transistor R and the drain terminal of the first amplification transistor SF1 are connected to the power supply VDD. First amplification transistor SF
The source terminal of 1 is connected to the drain terminal of the first output transistor S1. The source terminal of the first output transistor S1 is connected to the output terminal of the first constant current source 25 and the first output unit 23. The remaining terminals, that is, the gate terminal of the first charge holding unit SG1, the gate terminal of the first transfer transistor TX1, and the gate terminal of the first output transistor S1 are each connected to a scanning circuit (not shown), and scanning (not shown). It is controlled by the circuit.

In the following description, the control operation originally performed by the scanning circuit in the image sensor 120 is described as being performed by the control circuit 130 for convenience. The various operations described below as being performed by the control circuit 130 may be performed by a circuit inside the image sensor 120 (for example, a scanning circuit (not shown)), or may be performed by a circuit outside the image sensor 120 (for example, a scanning circuit (not shown)). For example, the control circuit 130 or other circuits may be used.

The second reading unit 22 includes a second charge holding unit SG2, a second transfer transistor TX2, a second charge / voltage conversion unit FD2, a second amplification transistor SF2, a second output transistor S2, and the like.
It has a second output unit 24. A second constant current source 26 is connected between the second output transistor S2 and the second output unit 24. There is one second constant current source 26 per row, and one
The imaging pixels 20 for all rows existing in one column are connected in parallel. Since each of these units is the same as the first reading unit 21, description thereof will be omitted. The reset transistor R is an N-type MOSFET, and has a photoelectric conversion unit PD, a first charge holding unit SG1, a second charge holding unit SG2, and the like.
It has a function of resetting the first charge-voltage conversion unit FD1 and the second charge-voltage conversion unit FD2.

FIG. 4 is an enlarged schematic view of the imaging pixels 20 on the imaging surface 10. The photoelectric conversion unit PD is arranged in the upper right corner of the imaging pixel 20, and occupies a relatively large area with respect to the entire imaging pixel 20 so that the generated charge can be held to a certain extent. 1st reading unit 21 and 2nd reading unit 22
Are arranged on the lower side and the left side of the photoelectric conversion unit PD, respectively.

The first charge holding unit SG1 and the second charge holding unit SG2 are arranged adjacent to the lower end and the left end of the photoelectric conversion unit PD, respectively. The first charge holding unit SG1 and the second charge holding unit SG2 have a relatively large capacitance (for example, a larger capacitance than the photoelectric conversion unit PD) so as to hold the charge generated in the photoelectric conversion unit PD. You need to have it. Therefore, the first charge holding unit SG1 and the second charge holding unit SG2 have an area close to that of the photoelectric conversion unit PD. The first charge-voltage conversion unit FD1 and the second charge-voltage conversion unit FD2 have a larger capacitance than the first charge holding unit SG1 and the second charge holding unit SG2.

In the remaining area of about a quarter of the entire imaging pixel 20, the remaining portion, that is, the first transfer transistor TX1, the second transfer transistor TX2, and the first charge-voltage conversion unit F.
D1, a second charge-voltage conversion unit FD2, a first amplification transistor SF1, a second amplification transistor SF2, a first output transistor S1, a second output transistor S2, and a reset transistor R are arranged.

FIG. 5A is a diagram schematically showing a cross section taken along the line AA'of FIG. 3, and FIG. 5B is a potential diagram thereof. As shown in FIG. 5A, the photoelectric conversion unit PD is composed of a p + type region 31 on the light receiving surface side (insulating film 30 side) and an n-type region 32 on the substrate side.

The first charge holding portion SG1 is composed of a gate 33 formed on the insulating film 30 and an n-type region 34 and an n-type region 35 formed under the insulating film 30. First charge holding unit S
An n-type region 35 is formed in most of the area of G1, and the n-type region 34 is formed only in a part adjacent to the photoelectric conversion unit PD. Although omitted in FIG. 5A, it is desirable to cover the surface of the gate 33 with a light-shielding layer formed of metal or the like to prevent light leakage.

The first transfer transistor TX1 includes a gate 36 formed on the insulating film 30 and the insulating film 30.
It is composed of an n-type region 37 formed below. An n-type region 38 is formed between the first charge holding unit SG1 and the first transfer transistor TX1. The first charge-voltage conversion unit FD1 adjacent to the first transfer transistor TX1 is an n + type region 3 formed under the insulating film 30.
It is composed of 9. Next to it is a gate 40 that functions as a reset transistor R. When a voltage equal to or higher than a predetermined level is applied to the gate 40, that is, the gate terminal of the reset transistor R, an n + type region formed on the opposite side of the gate 40 from the n + type region 39 which is the first charge voltage conversion unit FD1. A current flows through 41.

Next, the operation of each of these parts will be described with reference to the potential diagram of FIG. 5 (b). In the potential diagram of FIG. 5B, the potential becomes higher toward the lower side of the paper surface.

A predetermined high (H) level voltage or a predetermined low (L) level voltage is applied to the gate 33 of the first charge holding unit SG1. When an L-level voltage is applied to the gate 33 of the first charge holding unit SG1, the potential of the n-type region 34 of the first charge holding unit SG1 is higher than the potential of the n-type region 32 of the photoelectric conversion unit PD. Because it is low, the charge generated in the photoelectric conversion unit PD is
It is accumulated in the n-type region 32 of the photoelectric conversion unit PD. When an H-level voltage is applied to the gate 33 of the first charge holding unit SG1, the n-type region 34 and the n-type region 35 of the first charge holding unit SG1
Since the potential of is higher than the potential of the n-type region 32 of the photoelectric conversion unit PD, n of the photoelectric conversion unit PD
The charge accumulated in the mold region 32 is transferred to the n-type region 35 of the first charge holding unit SG1.

When an L-level voltage is applied to the gate 36 of the first transfer transistor TX1, the potential of the n-type region 35 of the first charge holding unit SG1 is the n-type region 3 of the first transfer transistor TX1.
Since the potential is higher than the potential of 7, the charge transferred to the n-type region 35 of the first charge holding unit SG1 is held in the n-type region 35 of the first charge holding unit SG1.

When an L level voltage is applied to the gate 33 of the first charge holding unit SG1 and an H level voltage is applied to the gate 36 of the first transfer transistor TX1, the n-type region of the first transfer transistor TX1 The potential of 37 is higher than the potential of the n-type region 35 of the first charge holding unit SG1. As a result, the charge held in the n-type region 35 of the first charge holding unit SG1 is reduced.
It is transferred to the first charge-voltage conversion unit FD1. The potential of the first charge-voltage conversion unit FD1 is determined according to the amount of transferred charge. At this time, when an H-level voltage is applied to the gate of the first output transistor S1, the amount of charge held by the first charge-voltage conversion unit FD1 (the first charge-voltage conversion unit FD1) is applied to the first output unit 23. An output signal of a magnitude corresponding to the electric charge appears.

The image pickup pixel 20 configured as described above has a first charge holding unit SG1 and a second charge holding unit SG2 that temporarily hold the charges generated by the photoelectric conversion unit PD. Therefore,
When performing the so-called global shutter operation, it is possible to simultaneously hold charges corresponding to three captured images. Here, the global shutter operation is an operation in which all the imaging pixels 20 simultaneously perform both the start of charge generation (that is, the shutter open operation) and the end of charge generation (that is, the shutter close operation) by the photoelectric conversion unit PD. Point to.

Hereinafter, a method of simultaneously holding charges corresponding to three captured images will be specifically described. In the following, the mechanical shutter 140 will be described without being included in the configuration. First, when the first global shutter operation is completed, the charges accumulated in the photoelectric conversion unit PD are transferred to the first charge holding unit SG1 in all the imaging pixels 20. In the following description, this operation is referred to as global transfer. Next, a second global shutter operation is performed before the output signals are read from the first charge holding unit SG1 of all the imaging pixels 20. The electric charge generated by the photoelectric conversion unit PD in the second global shutter operation is second by global transfer.
Transfer to the charge holding unit SG2. After that, before the output signals are read from the first charge holding unit SG1 and the second charge holding unit SG2 of all the imaging pixels 20, that is, the first charge holding unit S
While both G1 and the second charge holding unit SG2 hold the transferred charge, a third global shutter operation is further performed. By this third global shutter operation, the photoelectric conversion unit P
The electric charge generated by D can be retained in the photoelectric conversion unit PD. As described above, the image pickup pixels 20 charge the charges corresponding to the three captured images to the first charge holding unit SG1 and the second charge, respectively.
It is possible to simultaneously hold the charge at three locations, the charge holding unit SG2 and the photoelectric conversion unit PD.

Next, the live view function of the image pickup apparatus 100 will be described. FIG. 6 is a timing chart of the live view operation. In the live view operation described below, "all the image pickup pixels 20 used for creating the live view image" means the image pickup pixels 20 to be thinned out and read out. That is, when thinning out reading is performed, there are imaging pixels 20 that are not involved in the live view image, but such imaging pixels 20 are excluded from the following processing.

The control circuit 130 first resets the photoelectric conversion unit PD, the first charge holding unit SG1 and the second charge holding unit SG2 at the same time at time t1 in all the imaging pixels 20 used for creating the live view image (shutter). Equivalent to open operation). At the subsequent time t2, the control circuit 130 simultaneously transfers the charge of the photoelectric conversion unit PD to the first charge holding unit SG1 in all the imaging pixels 20 used for creating the live view image (shutter closing operation and also). Global transfer,
Also, it corresponds to the shutter opening operation in the next imaging).

The charge transferred here is generated by the photoelectric conversion unit PD by photoelectric conversion during the period from time t1 to time t2. That is, it corresponds to an imaging signal in which the period from time t1 to time t2 is the exposure time. In other words, the above operation corresponds to a global shutter operation in which the shutter is opened at time t1 and the shutter is closed at time t2.

After that, the control circuit 130 reads out the image pickup signal line by line in order from the first reading unit 21 of the image pickup pixel 20 in the first line. When the imaging signals have been read from all the lines, the control circuit 130 creates a live view image based on the read imaging signals and displays it on the display device 150.

The read time T1 is required to read the image pickup signal from all the rows to be read. For example, when it is desired to display a live view image at 60 fps based on the image pickup signal read from the first read unit 21, the read time T1 needs to be set to 1/60 second or less.

At time t4, which is only the read time T1 from time t2, the control circuit 130 again simultaneously transfers the charge of the photoelectric conversion unit PD to the first charge holding unit SG1 in all the imaging pixels 20 used for creating the live view image. Transfer (shutter closing operation, combined, global transfer, and equivalent to shutter opening operation in the next imaging). The control circuit 130 is then subjected to the imaging pixel 2 in the first row.
The image pickup signal is read out line by line in order from the first reading unit 21 of 0, and the live view image is created and displayed.

The control circuit 130 repeats the above operation every read time T1. This will
A live view image based on the image pickup signal read from the first reading unit 21 is created and displayed, for example, every 1 / 60th of a second.

In parallel with the above operation, the control circuit 130 charges the photoelectric conversion unit PD in all the imaging pixels 20 used for creating the live view image at the time t3, which is half the read time T1 from the time t2. Simultaneously transfers to the second charge holding unit SG2 (shutter closing operation, combined with global transfer, and corresponding to shutter opening operation in the next imaging). The electric charge transferred here is generated by the photoelectric conversion unit PD by photoelectric conversion during the period from time t2 to time t3. That is, it corresponds to an imaging signal in which the period from time t2 to time t3 is the exposure time. In other words, the above operation corresponds to a global shutter operation in which the shutter is opened at time t2 and the shutter is closed at time t3.

After that, the control circuit 130 reads out the image pickup signal line by line in order from the second reading unit 22 of the image pickup pixel 20 in the first line. When the imaging signals have been read from all the lines, the control circuit 130 creates a live view image based on the read imaging signals and displays it on the display device 150.

The control circuit 130 repeats the above operation every read time T1. This will
A live view image based on the image pickup signal read from the second reading unit 22 is created and displayed, for example, every 1 / 60th of a second.

As described above, the control circuit 130 reads out the imaging signal from the first reading unit 21 and creates and displays the live view image, and reads out the imaging signal from the second reading unit 22 and creates and displays the live view image. And are performed in parallel at the timing shifted by half a cycle. Therefore, the live view image is displayed at a frame rate twice as high as when only one of the first reading unit 21 and the second reading unit 22 is used.

Further, the photoelectric conversion unit PD is not reset except when the live view display is started. That is, it can be said that all the electric charges generated by the photoelectric conversion unit PD during the live view display are fully utilized.

FIG. 7 is a timing chart of the live view operation focusing on one imaging pixel 20. The control circuit 130 first arrives at the first charge holding unit SG1 and the second charge holding unit S at time t11.
By turning on / off the G2, the first transfer transistor TX1, the second transfer transistor TX2, and the reset transistor R (switching the voltage applied to the gate between the H level and the L level), the photoelectric conversion unit PD and the first charge holding unit The SG1 and the second charge holding unit SG2 are reset at the same time. At a subsequent time t12, the control circuit 130 transfers the charge from the photoelectric conversion unit PD to the first charge holding unit SG1 by turning on and off the first charge holding unit SG1. In addition, it should be noted
As illustrated in FIG. 7, the timing of turning off the first transfer transistor TX1, the second transfer transistor TX2, and the reset transistor R is delayed from the timing of turning off the first charge holding unit SG1 and the second charge holding unit SG2. Is desirable.

At time t13, the control circuit 130 reads an imaging signal from the first reading unit 21. The control circuit 130 first turns on the first output transistor S1. Then, the first output transistor S
With 1 turned on, the reset transistor R is turned on and off, and then the first transfer transistor TX1 is turned on and off. As a result, the charge held in the first charge holding unit SG1 is transferred to the first charge-voltage conversion unit FD1, and an output signal having a size corresponding to the amount of the charge is generated.
It is output to the first output unit 23. After that, the control circuit 130 turns off the first output transistor S1.

At time t14 after time t13, the control circuit 130 transfers the charge from the photoelectric conversion unit PD to the second charge holding unit SG2 by turning on and off the second charge holding unit SG2. Time t
Fifteenth, the control circuit 130 reads an image pickup signal from the second reading unit 22. The control circuit 130 first turns on the second output transistor S2. Then, with the second output transistor S2 turned on, the reset transistor R is turned on and off, and then the second transfer transistor TX2 is turned on and off. As a result, the charge held in the second charge holding unit SG2 is transferred to the second charge-voltage conversion unit FD2, and an output signal having a size corresponding to the amount of the charge is output to the second output unit 24. .. After that, the second output transistor S2 is turned off.

By repeating the above operation, the control circuit 130 alternately reads the imaging signal from the first reading unit 21 and the second reading unit 22.

Next, the main shooting (shooting for creating a recorded image) during the live view operation will be described. FIG. 8 is a timing chart of the main shooting. It is assumed that a predetermined main shooting operation (for example, a full release operation of the release switch) is performed at time t21.

When the control circuit 130 finishes reading the image pickup signal from the first reading unit 21 currently being executed at time t23, the control circuit 130 transfers the charge to the second charge holding unit SG2 at the latest time t2.
At the time t24 when the predetermined exposure time T2 elapses from 2, the charges of the photoelectric conversion unit PD are simultaneously transferred to the first charge holding unit SG1 in all the imaging pixels 20.

Further, the control circuit 130 transfers the charge of the photoelectric conversion unit PD to the second charge holding unit SG2 in all the imaging pixels 20 at the time t25 when a predetermined exposure time T3 (a time different from the exposure time T2) elapses from the time t24. Transfer to at the same time.

After that, the control circuit 130 reads out the image pickup signal line by line in order from the first reading unit 21 of the image pickup pixel 20 in the first line. In parallel with this, the control circuit 130 is the second of the imaging pixels 20 in the first row.
The image pickup signal is read out line by line in order from the reading unit 22. As a result, from the image sensor 120,
An imaging signal corresponding to the exposure time T2 and an imaging signal corresponding to the exposure time T3 can be obtained at the same time.

When the imaging signals have been read from all the rows, the control circuit 130 creates a recorded image based on the read imaging signals and records them on the recording medium 160. For example, the first recorded image based on the image pickup signal read from the first reading unit 21 and the second recorded image based on the image pickup signal read from the second reading unit 22 may be recorded on the recording medium 160, or they may be recorded. The two imaging signals may be combined to create a single recorded image and recorded on the recording medium 160. When the recording of the recorded image is completed, the control circuit 130 restarts the operation from the time t1 of FIG. 6 and resumes the live view display.

FIG. 9 is a timing chart of the main shooting focusing on one imaging pixel 20. At time t31, it is assumed that the electric charge is transferred from the photoelectric conversion unit PD to the second charge holding unit SG2 for the live view display. The control circuit 130 transfers the charge from the photoelectric conversion unit PD to the first charge holding unit SG1 by turning on and off the first charge holding unit SG1 at the time t32 when the predetermined exposure time T2 has elapsed from the time t31. The control circuit 130 transfers the electric charge from the photoelectric conversion unit PD to the second charge holding unit SG2 by turning on and off the second charge holding unit SG2 at the time t33 when the predetermined exposure time T3 has elapsed from the time t32.

At a subsequent time t34, the control circuit 130 simultaneously reads an imaging signal from the first reading unit 21 and the second reading unit 22. The control circuit 130 first turns on the first output transistor S1 and the second output transistor S2 at the same time. The control circuit 130 turns on / off the reset transistor R while keeping the first output transistor S1 and the second output transistor S2 on, and then turns on / off the first transfer transistor TX1 and the second transfer transistor TX2.
As a result, the charge held in the first charge holding unit SG1 is transferred to the first charge-voltage conversion unit FD1, and the charge held in the second charge holding unit SG2 is transferred to the second charge-voltage conversion unit F.
It is transferred to D2, and output signals having a magnitude corresponding to each charge amount are output to the first output unit 23 and the second output unit 24. The control circuit 130 then turns off the first output transistor S1 and the second output transistor S2.

According to the image pickup apparatus according to the first embodiment described above, the following effects can be obtained.
(1) The image pickup element 120 is an image pickup pixel having a photoelectric conversion unit PD that photoelectrically converts incident light to generate an electric charge, and a first reading unit 21 and a second reading unit 22 connected to the photoelectric conversion unit PD. 20
It is provided with a plurality of. The first reading unit 21 has a first charge holding unit SG1 that temporarily holds the charge transferred from the photoelectric conversion unit PD and a first charge voltage that converts the charge transferred from the first charge holding unit SG1 into a voltage. It has a conversion unit FD1 and a first output unit 23 that outputs an output signal corresponding to the voltage of the first charge-voltage conversion unit FD1. Similarly, the second reading unit 22 converts the charge transferred from the photoelectric conversion unit PD into the voltage of the second charge holding unit SG2 that temporarily holds the charge and the charge transferred from the second charge holding unit SG2. It has a two-charge-voltage conversion unit FD2 and a second output unit 24 that outputs an output signal corresponding to the voltage of the second charge-voltage conversion unit FD2. Since this is done, the electric charge generated by the photodiode (photoelectric conversion unit PD) can be effectively used.

(2) The capacitance of the first charge-voltage conversion unit FD1 is larger than the capacitance of the first charge holding unit SG1. Similarly, the capacitance of the second charge-voltage conversion unit FD2 is larger than the capacitance of the second charge-holding unit SG2. That is, according to the order in which the electric charges generated by the photoelectric conversion unit PD are transferred.
The capacitance is high. Since this is done, the electric charge generated by the photoelectric conversion unit PD can be handled without being damaged.

(3) The first reading unit 21 outputs an output signal corresponding to the electric charge from the second output unit 24 from the first time when the electric charge is transferred from the photoelectric conversion unit PD to the second electric charge holding unit SG2. The electric charge generated by the photoelectric conversion unit PD is transferred to the first charge holding unit SG1 within the period up to 2 hours. The second reading unit 22 is from the third time when the electric charge is transferred from the photoelectric conversion unit PD to the first charge holding unit SG1 to the fourth time when the output signal corresponding to the electric charge is output from the first output unit 23. The electric charge generated by the photoelectric conversion unit PD is transferred to the second charge holding unit SG2 within the period of. Since this is done, the electric charge generated by the photoelectric conversion unit PD while the reading operation is being performed in one reading unit can be transferred to the other reading unit for utilization.

(4) The control circuit 130 creates a first display image based on the output signal output from the first output unit 23, and creates a second display image based on the output signal output from the second output unit 24. Functions as an image creation unit. The display device 150 functions as an image display unit that alternately displays the first display image and the second display image. Since this is done, the live view image can be displayed at a frame rate that is twice the frame rate defined by the reading speed of the imaging signal (output signal) and the like.

(5) The first charge holding unit SG1 and the second charge holding unit SG2 have a so-called embedded channel structure. As a result, unlike the usual so-called floating diffusion, the charges generated by the photoelectric conversion unit PD are accumulated in the first charge holding unit SG1 and the second charge holding unit SG2 for a relatively long period of time without being damaged. It will be possible to keep it.

(Second Embodiment)
The image pickup device according to the second embodiment has the same configuration as the image pickup device 100 according to the first embodiment, but the drive method of the image pickup element 120 is different from that of the first embodiment. ing.
Hereinafter, the imaging apparatus according to the second embodiment will be described with particular focus on the differences from the first embodiment.

In the first embodiment, the live view image is taken with an exposure time (for example, about 1/60 second) corresponding to the frame rate (for example, 60 fps) of the live view display. In the present embodiment, the live view is displayed at double speed as in the first embodiment, but the exposure time of each live view image is arbitrarily set.

FIG. 10 is a timing chart of the live view operation. The control circuit 130 first resets the photoelectric conversion unit PD and the first charge holding unit SG1 at the same time at time t41 in all the imaging pixels 20 used for creating the live view image. The control circuit 130 is at time t
At time t42, which is only a predetermined exposure time T4 after 41, the charges of the photoelectric conversion unit PD are simultaneously transferred to the first charge holding unit SG1 in all the imaging pixels 20 used for creating the live view image.

After that, the control circuit 130 reads out the image pickup signal line by line in order from the first reading unit 21 of the image pickup pixel 20 in the first line. A read time T1 is required to read the imaging signals from all the rows. At the time t45 when the read time T1 elapses from the time t42, the reading of the imaging signals from all the rows is completed. The control circuit 130 creates a live view image based on the read image pickup signal and displays it on the display device 150.

At the time t45 when the image pickup signal is read out, the control circuit 130 resets the photoelectric conversion unit PD and the first charge holding unit SG1. At time t46, which is only the exposure time T4 after time t45, the control circuit 130 simultaneously transfers the charge of the photoelectric conversion unit PD to the first charge holding unit SG1 for all the imaging pixels 20 used for creating the live view image. .. Control circuit 13
After that, 0 reads the imaging signal line by line in order from the first reading unit 21 of the imaging pixel 20 in the first row to create and display a live view image.

The control circuit 130 repeats the above operation every read time T1 + exposure time T4. As a result, the live view image based on the image pickup signal read from the first reading unit 21 becomes
Created and displayed every T1 + T4 time (for example, every 1 / 60th of a second). This live view image is an image taken with an exposure time T4.

In parallel with the above operation, the control circuit 130 also creates and displays a live view image in the second reading unit 22 as well. Now consider time t43, which corresponds to the center of the period from time t41 to t45. At this time t43, the control circuit 130 resets the photoelectric conversion unit PD and the second charge holding unit SG2. At time t44, which is only the exposure time T4 after time t43, the control circuit 130 simultaneously transfers the charge of the photoelectric conversion unit PD to the second charge holding unit SG2 for all the imaging pixels 20 used for creating the live view image. .. After that, the control circuit 130 reads out the image pickup signal line by line in order from the second reading unit 22 of the image pickup pixel 20 in the first line, and creates and displays the live view image.

The control circuit 130 repeats the above operation every read time T1 + exposure time T4. As a result, the live view image based on the image pickup signal read from the second reading unit 22 becomes
Created and displayed, for example, every 1 / 60th of a second. This live view image has an exposure time of T.
It is an image taken in 4.

As described above, the control circuit 130 reads out the imaging signal from the first reading unit 21 and creates and displays the live view image, and reads out the imaging signal from the second reading unit 22 and creates and displays the live view image. And are performed in parallel at a timing shifted by half a cycle ((T1 + T4) / 2). Therefore, the live view image is the first reading unit 21 and the second reading unit 2.
It will be displayed at twice the frame rate when only one of 2 is used.

FIG. 11 is a timing chart of the live view operation focusing on one imaging pixel 20. The control circuit 130 first turns on the first charge holding unit SG1, the first transfer transistor TX1, and the reset transistor R at time t51 (switches the voltage applied to the gate from L level to H level) to perform photoelectric conversion. The unit PD and the first charge holding unit SG1 are reset at the same time.

At time t52, which is only the exposure time T4 after the time t51, the control circuit 130 transfers the charge from the photoelectric conversion unit PD to the first charge holding unit SG1 by turning on the first charge holding unit SG1.

At time t55, the control circuit 130 reads an imaging signal from the first reading unit 21. The control circuit 130 first turns on the first output transistor S1. The control circuit 130 turns on / off the reset transistor R while keeping the first output transistor S1 on, and then turns on / off the first transfer transistor TX1. As a result, the charge held in the first charge holding unit SG1 is transferred to the first charge-voltage conversion unit FD1, and an output signal having a size corresponding to the amount of the charge is output to the first output unit 23. .. After that, the control circuit 130 turns off the first output transistor S1.

At time t53 after time t52, the control circuit 130 turns on the second charge holding unit SG2, the second transfer transistor TX2, and the reset transistor R (switches the voltage applied to the gate from L level to H level). The photoelectric conversion unit PD and the second charge holding unit SG2 are reset at the same time.

At time t54, which is only the exposure time T4 after the time t53, the control circuit 130 transfers the charge from the photoelectric conversion unit PD to the second charge holding unit SG2 by turning on the second charge holding unit SG2.

At time t56, the control circuit 130 reads an image pickup signal from the second reading unit 22. The control circuit 130 first turns on the second output transistor S2. The control circuit 130 turns on / off the reset transistor R while keeping the second output transistor S2 on, and then turns on / off the second transfer transistor TX2. As a result, the charge held in the second charge holding unit SG2 is transferred to the second charge-voltage conversion unit FD2, and an output signal having a size corresponding to the amount of the charge is output to the second output unit 24. .. After that, the control circuit 130 turns off the second output transistor S2.

By repeating the above operation, the control circuit 130 alternately reads out the imaging signals corresponding to the exposure time T4 from the first reading unit 21 and the second reading unit 22.

Next, the main shooting during the live view operation will be described. FIG. 12 is a timing chart of the main shooting. It is assumed that a predetermined main shooting operation (for example, a full release operation of the release switch) is performed at time t61.

When the reading of the imaging signal from the first reading unit 21 currently being executed at time t62 is completed, the control circuit 130 simultaneously resets the photoelectric conversion unit PD and the first charge holding unit SG1 for all the imaging pixels 20. .. The control circuit 130 has a predetermined exposure time T5 from time t62.
At time t63, the charge of the photoelectric conversion unit PD is simultaneously transferred to the first charge holding unit SG1 for all the imaging pixels 20.

The control circuit 130 simultaneously resets the photoelectric conversion unit PD and the second charge holding unit SG2 for all the imaging pixels 20 at a time t64 after the time t63. The control circuit 130 simultaneously transfers the charge of the photoelectric conversion unit PD to the second charge holding unit SG2 for all the imaging pixels 20 at the time t65 after the predetermined exposure time T6 from the time t64.

After that, the control circuit 130 reads out the imaging signal line by line in order from the first reading unit 21 of the imaging pixel 20 in the first row. In parallel with this, the control circuit 130 is the second of the imaging pixels 20 in the first row.
The image pickup signal is read out line by line in order from the reading unit 22. As a result, from the image sensor 120,
An imaging signal corresponding to the exposure time T5 and an imaging signal corresponding to the exposure time T6 can be obtained at the same time.

When the imaging signals have been read from all the rows, the control circuit 130 creates a recorded image based on the read imaging signals and records them on the recording medium 160. For example, the first recorded image based on the image pickup signal read from the first reading unit 21 and the second recorded image based on the image pickup signal read from the second reading unit 22 may be recorded on the recording medium 160, or they may be recorded. The two imaging signals may be combined to create a single recorded image and recorded on the recording medium 160.

When the recording of the recorded image is completed, the control circuit 130 restarts the operation from the time t51 of FIG. 11 and resumes the live view display.

FIG. 13 is a timing chart of the main shooting focusing on one imaging pixel 20. The control circuit 130 includes the reset transistor R and the first charge holding unit SG1 at time t71.
And the first transfer transistor TX1 are turned on to reset the photoelectric conversion unit PD and the first charge holding unit SG1.

The control circuit 130 turns on the first charge holding unit SG1 at the time t72 when the predetermined exposure time T5 has elapsed from the time t71, so that the photoelectric conversion unit PD to the first charge holding unit SG
Transfer the charge to 1.

The control circuit 130 has a reset transistor R at a time t73 after the time t72.
And the second charge holding unit SG2 and the second transfer transistor TX2 are turned on to reset the photoelectric conversion unit PD and the second charge holding unit SG2.

The control circuit 130 turns on the second charge holding unit SG2 at the time t74 when the predetermined exposure time T6 has elapsed from the time t73, so that the photoelectric conversion unit PD to the second charge holding unit SG
Transfer the charge to 2.

At a subsequent time t75, the control circuit 130 simultaneously reads an imaging signal from the first reading unit 21 and the second reading unit 22. The control circuit 130 first turns on the first output transistor S1 and the second output transistor S2 at the same time. The control circuit 130 turns on / off the reset transistor R while keeping the first output transistor S1 and the second output transistor S2 on, and then turns on / off the first transfer transistor TX1 and the second transfer transistor TX2.
As a result, the charge held in the first charge holding unit SG1 is transferred to the first charge-voltage conversion unit FD1, and the charge held in the second charge holding unit SG2 is transferred to the second charge-voltage conversion unit F.
The output signals transferred to D2 and having a size corresponding to the respective charge amounts are the first output unit 23 and the second.
It is output to the output unit 24. The control circuit 130 is then subjected to the first output transistor S1 and the second.
The output transistor S2 is turned off.

According to the image pickup apparatus according to the second embodiment described above, the same effects as those of the first embodiment can be obtained.

(Third Embodiment)
The image pickup apparatus according to the third embodiment has the same configuration as the image pickup apparatus 100 according to the first embodiment, and performs an exposure auto bracketing imaging function and a high dynamic range composition (HDR composition). It has a function. Of these two functions, the exposure auto bracketing shooting function will be described in detail below.

FIG. 14 is a timing chart of the exposure auto bracketing shooting operation. Time t
It is assumed that a predetermined main shooting operation (for example, a full release operation of the release switch) is performed in 81. Further, it is assumed that the reading of the image pickup signal for live view from the first reading unit 21 currently being executed ends at the time t83 after the time t81.

The control circuit 130 resets the photoelectric conversion unit PD in the image pickup pixel 20 in the first row at the time t82, which is the time t7 before the predetermined time T7. The control circuit 130 then sets the time t
From 82 to time t83, the photoelectric conversion unit PD is sequentially reset for each line as in the second line, the third line, and so on. Here, the predetermined time T7 is the time required for the mechanical shutter 140 to travel, that is, the time required to switch the mechanical shutter 140 from an open state in which the mechanical shutter 140 is completely open to a light-shielding state in which the mechanical shutter 140 is completely closed. is there. After that, the control circuit 130 receives as soon as the reading of the live view imaging signal from the first reading unit 21 is completed.
The first charge holding unit SG1 is reset for all the imaging pixels 20. Second charge holding unit S
Similarly for G2, the second charge holding unit SG2 is reset for all the imaging pixels 20 as soon as the reading of the live view imaging signal from the second reading unit 22 is completed.

The control circuit 130 transfers the charge of the photoelectric conversion unit PD to the first charge holding unit SG1 for the image pickup pixel 20 in the first row at the time t84, which is only the predetermined exposure time T8 after the time t82. The control circuit 130 then extends from time t84 to time t85, which is a predetermined time T7 later, 2
The electric charge of the photoelectric conversion unit PD is sequentially transferred to the first charge holding unit SG1 for each line as in the rows, the third line, and so on.

The control circuit 130 transfers the charge of the photoelectric conversion unit PD to the second charge holding unit SG2 for the image pickup pixel 20 in the first row at the time t86, which is only the predetermined exposure time T9 after the time t84. The control circuit 130 then extends from time t86 to time t87, which is a predetermined time T7 later, 2
The electric charge of the photoelectric conversion unit PD is sequentially transferred to the second charge holding unit SG2 for each line as in the rows, the third line, and so on.

The control circuit 130 starts traveling of the mechanical shutter 140 at a time t88, which is a predetermined exposure time T10 after the time t86, and puts the mechanical shutter 140 in a closed state (light-shielding state). As described above, traveling of the mechanical shutter 140 requires a predetermined time T7. Therefore, at time t89, which is only a predetermined time T7 after time t88, the running of the mechanical shutter 140 is completed, and the image sensor 120 is covered by the mechanical shutter 140 (light-shielding state).

At time t89, the electric charge corresponding to the exposure time T10 is accumulated in the photoelectric conversion unit PD of the imaging pixel 20. Since the mechanical shutter 140 is closed, the amount of electric charge stored in the photoelectric conversion unit PD of the imaging pixel 20 does not change thereafter. That is, the photoelectric conversion unit P
The electric charge corresponding to the exposure time T10 is held in D.

At time t89, the control circuit 130 starts from the first reading unit 21 of the image pickup pixel 20 on the first line.
Read the imaging signal line by line. In parallel with this, the control circuit 130 has the imaging pixel 20 in the first row.
The imaging signal is read line by line in order from the second reading unit 22 of the above. As a result, the image pickup signal corresponding to the exposure time T8 and the image pickup signal corresponding to the exposure time T9 can be simultaneously obtained from the image pickup device 120.

At the time t90 when these imaging signals are read out, the control circuit 130 is connected to the photoelectric conversion unit P.
The electric charge accumulated in D (the electric charge corresponding to the exposure time T10) is transferred to the first charge holding unit SG1 and the second charge holding unit SG2. At this time, the control circuit 130 transfers the charge to the first charge holding unit SG1 for a certain half line (for example, an odd number line) among all the lines of the image sensor 120, and for the other half line (for example, an even number line). The charge is transferred to the second charge holding unit SG2.

After that, the control circuit 130 reads out the image pickup signal line by line from the first reading unit 21 of the image pickup pixel 20 in the first line for the former half line (for example, an odd number line). In parallel with this, the control circuit 130 transfers the second image pixel 20 in the first row for the latter half row (for example, even row).
The image pickup signal is read out line by line in order from the reading unit 22.

As a result, an image pickup signal corresponding to the exposure time T10 can be obtained from the image pickup device 120.
Further, since half of the lines are read from the first reading unit 21 and the other half of the lines are read from the second reading unit 22, the time required to read all the lines is the first reading unit. It is about half as compared with the case where only 21 (or only the second reading unit 22) is used.

It is not necessary to distribute the reading to the first reading unit 21 and the second reading unit 22 in this way. That is, for example, the image pickup signal corresponding to the exposure time T10 may be read out only from the first reading unit 21.

After reading out all the imaging signals, the control circuit 130 creates three types of recorded images based on the three types of captured imaging signals and records them on the recording medium 160. These three types of recorded images are recorded images based on different exposure times T8, T9, and T10, respectively. For example
The exposure time T8 corresponds to the time corresponding to the proper exposure, the exposure time T9 corresponds to the time corresponding to one step above the proper exposure (that is, slightly overexposed), and the exposure time T10 corresponds to the time corresponding to one step below the proper exposure (that is, underexposure). In terms of time (slightly), the three types of recorded images are images obtained by so-called exposure auto-bracketing photography.

In the above description, the exposure auto-bracketing based on the exposure time has been described, but the exposure auto-bracketing may be performed by changing, for example, the aperture and the ISO sensitivity for each shooting. Further, auto bracketing imaging may be performed in which images having different imaging settings (for example, focus position, etc.) other than exposure are obtained at once.

The control circuit 130 further creates three types of recorded images based on the read three types of imaging signals, and then HDR-synthesizes the three types of recorded images by a well-known method to create one recorded image and creates a recording medium. It has a function of recording in 160. For example, a portion of a short-accumulated recorded image that has been blacked out is restored with the same portion of a medium-accumulated recorded image or a long-accumulated recorded image. Similarly, the overexposed portion of the long-accumulated recorded image is restored with the same portion of the short-accumulated recorded image and the medium-accumulated recorded image. Since such a synthesis method is well known, the description thereof will be omitted.

FIG. 15 is a diagram schematically showing the exposure time of high dynamic range composition. FIG. 15 (
The exposure time of high dynamic range composition using the conventional image sensor is shown in a). With a conventional image sensor, the first imaging (short accumulation, that is, imaging based on an underexposed exposure setting) is performed, the first imaging signal is read, and then the second imaging (medium storage, that is, appropriate) is performed. The imaging signal was read out in sequence (shooting with the exposure setting), the third shooting (long-exposure, that is, shooting based on the exposure setting with a slight overexposure), and the reading of the image pickup signal. Therefore, the 1st to 3rd exposure times were spaced apart from each other.

In this way, if there is an interval between each shooting, for example, when a moving subject is shot, the subject positions are not aligned in the three imaging signals, and high dynamic range composition cannot be performed well.

FIG. 15B shows the exposure time of high dynamic range composition using the image pickup apparatus according to the present embodiment. Imaging signals corresponding to three types of exposure settings are obtained with continuous exposure times, as if one shot was taken. In particular, since the three times of accumulation (exposure) by these three types of exposure settings are within the pixel output timing for one time in total, it is practically within the time required for one shooting (that is, within one frame period). ) Means that the shooting is completed. Therefore,
It is possible to obtain an image for high dynamic range composition without shifting the subject position as in the case of using a conventional image sensor.

According to the image pickup apparatus according to the third embodiment described above, the following effects can be obtained.
(1) The mechanical shutter 140 is a so-called shutter device, and the image sensor 120
It is possible to switch between an open state that does not block the light incident on the image sensor 120 and a light-shielded state that blocks the light incident on the image sensor 120. The first reading unit 21 is the first in which the mechanical shutter 140 is in the open state.
The electric charge generated by the photoelectric conversion unit PD during the period is transferred to the first charge holding unit SG1, and then the output signal corresponding to the electric charge is output from the first output unit 23. In the second reading unit 22, the photoelectric conversion unit PD is in the second period in which the mechanical shutter 140 is open and does not overlap with the first period.
The electric charge generated in the above is transferred to the second charge holding unit SG2, and then the output signal corresponding to the electric charge is output from the second output unit 24. In the mechanical shutter 140, the photoelectric conversion unit PD is in the first period and the third period after the second period when the mechanical shutter 140 is open.
The electric charge generated in the above is switched to a light-shielding state and held in the photoelectric conversion unit PD. 1st reading unit 21
Finishes outputting the electric charge held in the photoelectric conversion unit PD when the mechanical shutter 140 is in a light-shielding state, and the output signal based on the electric charge generated by the photoelectric conversion unit PD in the first period from the first output unit 23. After that, it is transferred to the first charge holding unit SG1. Since this is done, up to three continuous shots can be taken at one time. In the continuous shooting of up to three images, it is not necessary to read out the signal between shootings, so that the interval between shootings is substantially zero.

(2) A first output signal based on the electric charge generated by the photoelectric conversion unit PD in the first period, a second output signal based on the electric charge generated by the photoelectric conversion unit PD in the second period, and photoelectric in the third period. The third output signal based on the electric charge generated by the conversion unit PD is an output signal based on different exposure settings. Since this is done, so-called exposure auto-bracketing photography can be performed up to three images at a time. In the exposure auto-bracketing shooting performed here, the interval between each shooting is extremely short (substantially zero) as compared with the normal exposure auto-bracketing shooting. Therefore, the misalignment of the subject for each shooting can be extremely reduced.

(3) The control circuit 130 has a higher dynamic range than an image based on any one output signal based on three imaging signals (first output signal, second output signal, and third output signal) having different exposure settings. It functions as a high dynamic range synthesizer that creates images. Since this is done, it is possible to obtain a high dynamic range image in which the positional deviation of the subject for each shooting is extremely small.

(Fourth Embodiment)
FIG. 16 is a schematic view showing the configuration of the image pickup apparatus according to the fourth embodiment of the present invention. In FIG. 16, the same parts as those in the first embodiment are designated by the same reference numerals as those in the first embodiment, and the description thereof will be omitted.

The image pickup device 200 includes an imaging optical system 110, an image pickup element 120, a control circuit 130, a mechanical shutter 140, a display device 150, a recording medium 160, and a projection unit 170. The projection unit 170 projects pulsed light (modulated light) onto a subject within a predetermined angle of view.

The control circuit 130 has a depth map creation function. The depth map is data in which the depth of each part of the subject (distance from the image pickup apparatus 200 to the subject part) is two-dimensionally mapped. The control circuit 130 is a so-called optical flight time measurement method (ToF; Time of Fl).
ight) is used to measure the depth (distance) of each part of the subject. In this embodiment
The subject portion is a subject portion corresponding to one imaging pixel 20 or a block composed of a plurality of imaging pixels 20. That is, the control circuit 130 can obtain the depth of the corresponding subject portion for each image pickup pixel 20 or for each block composed of a plurality of image pickup pixels 20. Hereinafter, the distance measuring operation at the time of creating the depth map will be described in detail.

FIG. 17 is a timing chart of the distance measuring operation. First, at time t100, the projection unit 170 projects pulsed light for time T0. Further, at time t100, the control circuit 1
30 simultaneously resets the photoelectric conversion unit PD, the first charge holding unit SG1 and the second charge holding unit SG2 for all the imaging pixels 20.

After that, at the time t102 when the projection of the pulsed light ends, the control circuit 130 transfers the electric charges from the photoelectric conversion unit PD to the first charge holding unit SG1 for all the imaging pixels 20.

Further, at time t104, which is only time T0 after time t102, the control circuit 130 transfers charges from the photoelectric conversion unit PD to the second charge holding unit SG2 for all the imaging pixels 20.

The pulsed light (projected light) whose projection is started at time t100 is reflected on the surface of the subject and returns to the image sensor 120 at time t101, which is a delay time Td after time t100. Here, the delay time Td is a time corresponding to the distance L to the subject portion corresponding to the image pickup pixel 20. The longer the distance L, the longer the delay time Td. The reflected light is interrupted in the vicinity of time t103, which is only time T0 after time t101.

After the time t104, the control circuit 130 reads out the imaging signal row by row in order from the first reading unit 21 of the imaging pixel 20 in the first row. In parallel with this, the control circuit 130 reads out the image pickup signal line by line in order from the second reading unit 22 of the image pickup pixel 20 in the first line. As a result, the image sensor 12
From 0, the imaging signal corresponding to the exposure period from time t100 to time t102 and time t
An imaging signal corresponding to the exposure period from 102 to time t104 can be obtained at the same time.

Here, the ratio of the image pickup signal (output signal) read from the first reading unit 21 to the image pickup signal (output signal) read from the second reading unit 22 is the reflected light received by the time t102. Is the ratio of the amount of light of the above to the amount of reflected light received after time t102. Therefore, the first reading unit 21
The delay time Td can be determined from the image pickup signal read from and the image pickup signal read from the second reading unit 22.

For example, for a certain imaging pixel 20, when the size of the imaging signal read from the first reading unit 21 and the imaging signal read from the second reading unit 22 are substantially equal, the imaging pixel 2
The delay time Td in the subject portion corresponding to 0 is half of the time T0. Further, when the magnitude of the image pickup signal read from the first reading unit 21 and the image pickup signal read from the second reading unit 22 is a ratio of 1: 2, the delay time Td is the time T0. It is two-thirds of.

Since the speed of light c is known, it is possible to calculate the distance L to the subject if the delay time Td is known. Since such a method for calculating the distance L is well known, the description thereof will be omitted.

According to the image pickup apparatus according to the fourth embodiment described above, the following effects can be obtained.
(1) The projection unit 170 projects pulsed light, which is a predetermined modulated light, on the subject. 1st reading unit 2
In 1, the photoelectric conversion unit PD transfers a part of the reflected light of the projected light by photoelectric conversion to the first charge holding unit SG1, and outputs an output signal based on the charge from the first output unit 23. .. In the second reading unit 22, the photoelectric conversion unit PD transfers the charge generated by photoelectric conversion of the rest of the reflected light of the projected light to the second charge holding unit SG2, and the second output unit transfers an output signal based on the charge. Output from 24. The control circuit 130 functions as a distance calculation unit that calculates the distance to the subject based on the output signal output from the first output unit 23 and the output signal output from the second output unit 24. Since this is done, the depth map of the subject can be created with high accuracy.

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.

(Modification example 1)
The first charge holding unit SG1 and the second charge holding unit SG2 may have a configuration different from that of the above-described embodiment. For example, a MOSFET for charge transfer and a capacitance component (for example, a diode) for accumulating charges are used to obtain a first charge holding unit SG1 and a second charge holding unit SG2.
May be configured. In addition, the first reading unit 21 and the second reading unit 22 are the first charge holding unit SG.
It is also possible to have a configuration that does not have the first and second charge holding units SG2.

The present invention is not limited to the above-described embodiment as long as the features of the present invention are not impaired.
Other forms conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention.

20 ... Imaging pixel, 21 ... First reading unit, 22 ... Second reading unit, 23 ... First output unit, 24 ... Second
Output unit, 100, 200 ... image pickup device, 110 ... imaging optical system, 120 ... image sensor, 130 ...
Control circuit, 140 ... mechanical shutter, 150 ... display device, 160 ... recording medium, 17
0 ... Projection unit, PD ... Photoelectric conversion unit, SG1 ... First charge holding unit, SG2 ... Second charge holding unit, F
D1 ... 1st charge-voltage conversion unit, FD2 ... 2nd charge-voltage conversion unit, TX1 ... 1st transfer transistor, TX2 ... 2nd transfer transistor, S1 ... 1st output transistor, S2 ... 2nd output transistor, SF1 ... 1st Amplification transistor, SF2 ... Second amplification transistor

Claims (12)

A photoelectric conversion unit that photoelectrically converts incident light to generate an electric charge, and a first photoelectric conversion unit connected to the photoelectric conversion unit.
A plurality of pixels having a second reading unit and a second reading unit are provided.
The first reading unit is a first charge-voltage conversion unit that converts the charge transferred from the photoelectric conversion unit into a voltage, and a first output unit that outputs an output signal corresponding to the voltage of the first charge-voltage conversion unit. And have
The second reading unit includes a second charge-voltage conversion unit that converts the charge transferred from the photoelectric conversion unit into a voltage, and a second output unit that outputs an output signal corresponding to the voltage of the second charge-voltage conversion unit. An image sensor having and. In the image pickup device according to claim 1,
The first reading unit further includes a first charge holding unit that holds the charge transferred from the photoelectric conversion unit.
The first charge-voltage conversion unit converts the charge transferred from the first charge holding unit into a voltage.
The second reading unit further includes a second charge holding unit that holds the charge transferred from the photoelectric conversion unit.
The second charge-voltage conversion unit is an image pickup device that converts the charge transferred from the second charge holding unit into a voltage. In the image pickup device according to claim 2,
The capacitance value of the first charge-voltage conversion unit is larger than the capacitance value of the first charge holding unit.
The capacitance value of the second charge-voltage conversion unit is larger than the capacitance value of the second charge-holding unit. In the image pickup device according to claim 2 or 3.
The first charge holding unit holds the charge generated by the photoelectric conversion unit in the first period, and
The second charge holding unit is an image pickup element that holds the charges generated by the photoelectric conversion unit in the second period that does not overlap with the first period. In the image pickup device according to claim 4,
An image sensor in which the length of the first period is different from the length of the second period. In the image pickup device according to claim 4 or 5.
The first output unit is an image pickup device that outputs an output signal in a third period that does not overlap with the first period but at least partially overlaps with the second period. The image sensor according to any one of claims 1 to 6,
An image creation unit that creates a first display image based on the output signal output from the first output unit and creates a second display image based on the output signal output from the second output unit.
An image display unit that alternately displays the first display image and the second display image,
An imaging device comprising. The image sensor according to any one of claims 2 to 6,
A shutter device capable of switching between an open state that does not block the incident light on the image sensor and a light-shielding state that blocks the incident light on the image sensor is provided.
The first reading unit transfers the electric charge generated by the photoelectric conversion unit during the first period when the shutter device is in the open state to the first electric charge holding unit, and then an output signal corresponding to the electric charge. Is output from the first output unit,
The second reading unit transfers the electric charge generated by the photoelectric conversion unit to the second charge holding unit during the second period when the shutter device is in the open state and does not overlap with the first period, and then the second reading unit transfers the electric charge to the second charge holding unit.
An output signal corresponding to the electric charge is output from the second output unit,
In the shutter device, the electric charge generated by the photoelectric conversion unit during the first period and the third period after the second period when the shutter device is in the open state is switched to the light-shielding state to perform the photoelectric conversion. Hold in the department,
The first reading unit outputs an output signal based on the electric charge held by the photoelectric conversion unit when the shutter device is in the light-shielding state and the electric charge generated by the photoelectric conversion unit during the first period. An imaging device that transfers data from one output unit to the first charge holding unit after the output is completed. In the imaging device according to claim 8,
The first output signal based on the electric charge generated by the photoelectric conversion unit in the first period, the second output signal based on the electric charge generated by the photoelectric conversion unit in the second period, and the said in the third period. An image pickup device in which a third output signal based on the electric charge generated by the photoelectric conversion unit and an output signal based on different exposure settings are used. In the imaging device according to claim 9,
An image pickup further comprising a high dynamic range compositing unit that creates an image having a higher dynamic range than an image based on any one output signal based on the first output signal, the second output signal, and the third output signal. apparatus. The image sensor according to any one of claims 2 to 6,
Equipped with a projection unit that projects a predetermined modulated light on the subject
The first reading unit transfers a charge generated by the photoelectric conversion unit by photoelectric conversion of a part of the reflected light of the modulated light to the first charge holding unit, and transfers an output signal based on the charge to the first charge holding unit. Output from the output section
The second reading unit transfers the electric charge generated by the photoelectric conversion unit by photoelectric conversion of the rest of the reflected light of the modulated light to the second charge holding unit, and outputs an output based on the charge. The signal is output from the second output unit,
An imaging device including a distance calculation unit that calculates a distance to the subject based on an output signal output from the first output unit and an output signal output from the second output unit. A photoelectric conversion unit that photoelectrically converts incident light to generate an electric charge, and a first photoelectric conversion unit connected to the photoelectric conversion unit.
A plurality of pixels having a second reading unit and a second reading unit are provided.
The first reading unit includes a first charge holding unit that holds the charge transferred from the photoelectric conversion unit, and a first charge holding unit.
It has a first output unit that outputs an output signal corresponding to the electric charge held in the first charge holding unit.
The second reading unit includes a second charge holding unit that holds the charge transferred from the photoelectric conversion unit, and a second charge holding unit.
An image sensor having a second output unit that outputs an output signal corresponding to the charge held in the second charge holding unit.

Images