JP2020136810A - Imaging apparatus, imaging system, and control method for imaging apparatus - Google Patents

Abstract

To provide a technology capable of shortening an auto-focus time lag.SOLUTION: There is provided an imaging apparatus including a plurality of pixels, each of which has a first and second photoelectric conversion units for performing photoelectric conversion. The plurality of pixels each store electric charge obtained via photoelectric conversion in a first storage period by the first and second photoelectric conversion units and, in a first output period after the first storage period, sequentially outputs image signals based on electric charge in the first storage period to a first signal line. At least one or some of the plurality of pixels each store electric charge obtained via photoelectric conversion in a second storage period after the first storage period by the first and second photoelectric conversion units in parallel with output in the first output period and, in a second output period after the second storage period, sequentially outputs image signals based on electric charge in the second storage period to a second signal line in parallel with output in the first output period.SELECTED DRAWING: Figure 5

Description

The present invention relates to an imaging device, an imaging system, and a control method for the imaging device.

The image pickup system has an image pickup element such as a CMOS (complementary metal oxide semiconductor) sensor, and is becoming more multifunctional. The image pickup system can not only generate a still image signal or a moving image signal, but also adjust the focus based on the image information obtained from the image sensor.

Patent Document 1 discloses an imaging system that performs pupil division type focus detection using an imaging signal obtained from an image sensor. The image sensor has one microlens and two photodiodes for each pixel. Each photodiode receives light that has passed through different pupil regions of the photographing lens. The imaging system performs focus detection by comparing the output signals of the two photodiodes, and generates an image signal of an image for imaging by adding the output signals of the two photodiodes.

Japanese Unexamined Patent Publication No. 2001-124984

However, the imaging system drives the lens to focus using the image information one frame before. Therefore, the image pickup system uses the image information delayed by one frame, and the subject to be originally focused may move during that time. In this case, the imaging system has a problem that the autofocus time lag becomes long and an out-of-focus image is generated.

An object of the present invention is to provide a technique capable of shortening the autofocus time lag.

According to one aspect of the present invention, a plurality of pixels each having first and second photoelectric conversion units that perform photoelectric conversion, a first signal line connectable to the plurality of pixels, and the plurality of pixels. The plurality of pixels have a second signal line that can be connected to, and in the first storage period, the first and second photoelectric conversion units accumulate the photoelectrically converted charge, and the first In the first output period after the accumulation period of, the image signal based on the charge of the first accumulation period is sequentially output to the first signal line, and at least a part of the plurality of pixels is In the second storage period after the first storage period, the first and second photoelectric conversion units accumulate the photoelectrically converted charges in parallel with the output in the first output period, and the above-mentioned In the second output period after the second storage period, the image signal based on the charge in the second storage period is sequentially output to the second signal line in parallel with the output in the first output period. An imaging device is provided.

According to the present invention, the autofocus time lag can be shortened.

It is a figure which shows the configuration example of an imaging system. It is a figure which shows the structural example of the image sensor. It is a figure for demonstrating the image sensor of the pupil division type. It is a figure which shows the structural example of 1 pixel of an image sensor. It is a timing chart which shows the operation of an imaging system. It is a timing chart which shows the control signal of an image sensor. It is a flowchart which shows the operation of an image pickup system.

FIG. 1 is a block diagram showing a configuration example of an imaging system 120 according to an embodiment of the present invention. The image pickup system 120 includes an image pickup element 100, a processing circuit 101, a control calculation unit 102, a control circuit 104, a memory unit 105, a recording medium control unit 106, a display unit 107, a recording medium 108, and an external interface. It has a unit 109 and an operation unit 110. Further, the imaging system 120 includes a shutter 111, a lens drive circuit 112, an aperture drive circuit 113, a lens actuator 114, an aperture actuator 115, a photographing lens 116, an aperture 117, and a focus lens 118.

The photographing lens 116 is a first lens group arranged at the tip of the photographing optical system (common optical system), and moves forward and backward in the optical axis direction to realize a scaling action (zoom function). The diaphragm 117 adjusts the amount of light at the time of photographing by adjusting the aperture diameter of the diaphragm 117. The focus lens 118 is a second lens group, and adjusts the focus of the photographing optical system by advancing and retreating in the optical axis direction.

The shutter 111 is a focal plane shutter for adjusting the exposure seconds when shooting a still image. The image pickup system 120 can adjust the exposure seconds of the image pickup device 100 by the shutter 111, but is not limited thereto. The image sensor 100 has an electronic shutter function, and the exposure seconds of the image sensor 100 may be adjusted by a control pulse.

The lens actuator 114 drives the focus lens 118 forward and backward in the optical axis direction to adjust the focal position of the focus lens 118. The lens drive circuit 112 controls the drive of the lens actuator 114 under the control of the control calculation unit 102.

The diaphragm actuator 115 adjusts the aperture diameter of the diaphragm 117 by driving. The diaphragm drive circuit 113 controls the drive of the diaphragm actuator 115 under the control of the control calculation unit 102.

The image pickup device 100 is an image pickup device, and generates an analog image signal by converting an optical image of a subject formed by the photographing lens 116 and the focus lens 118 into an electric signal. The image sensor 100 generates a still image signal or a moving image signal under the control of the control circuit 104.

The processing circuit 101 performs image processing on the image signal generated by the image sensor 100. For example, image processing includes processing for converting an image signal from analog to digital, low-pass filter processing for reducing noise, shading processing, white balance processing, scratch correction, dark shading correction, blackening processing, compression, and the like.

The control calculation unit 102 controls the entire image pickup system 120 and performs various calculations. The control circuit 104 generates a drive pulse for driving the image sensor 100 based on the control signal from the control calculation unit 102. The memory unit 105 temporarily stores the image signal. The recording medium control unit 106 records or reads an image signal from the recording medium 108. The display unit 107 displays the image signal. The recording medium 108 is a removable recording medium such as a semiconductor memory, and stores an image signal.

The external interface unit 109 is an interface for communicating with an external computer or the like. The operation unit 110 outputs information regarding the driving conditions of the imaging system 120 to the control calculation unit 102 in response to the user's operation. The imaging system 120 controls the entire imaging system 120 based on the information input from the operation unit 110.

FIG. 2 is a diagram showing a configuration example of the image pickup device 100 of FIG. The image pickup device 100 includes a pixel region 201, a vertical scanning circuit 202, a column amplifier circuit 203, a horizontal scanning circuit 204, an output circuit 205, and a control circuit 206.

The pixel region 201 has a plurality of pixels 207 arranged in a matrix. The vertical scanning circuit 202 supplies a control signal for controlling a plurality of transistors included in the pixel 207 on (conducting state) or off (non-conducting state) on a row-by-row basis. Each of the plurality of pixels 207 converts light into an electric charge to generate a pixel signal. Pixels 207 in each row can be connected to the row signal lines 208a and 208b in each row, respectively. The plurality of pixels 207 output pixel signals to the column signal lines 208a or 208b in units of rows.

The column amplifier circuit 203 amplifies the pixel signal of the column signal line 208a or 208b, and performs processing such as correlation double sampling processing based on the signal at the time of resetting the pixel 207 and the signal at the time of photoelectric conversion of the pixel 207. The horizontal scanning circuit 204 sequentially transfers the pixel signals of each column of the column amplifier circuit 203 to the output circuit 205. The output circuit 205 includes a buffer amplifier, a differential amplifier, and the like, and outputs a pixel signal from the column amplifier circuit 203 to the processing circuit 101 of FIG.

The vertical scanning circuit 202 may include a function of a signal processing circuit that performs signal processing such as correction of false signal components. Further, the column amplifier circuit 203 may convert an image signal from analog to digital.

FIG. 3A is a diagram showing a layout example of the pixel area 201. The pixel region 201 has a plurality of pixels 207 arranged in a matrix. The plurality of pixels 207 are arranged h in the horizontal direction and v in the vertical direction. Each of the plurality of pixels 207 has a microlens 301, a photodiode 302A, and a photodiode 302B. Each of the photodiodes 302A and 302B is a photoelectric conversion unit that performs photoelectric conversion. The photodiodes 302A and 302B are separated by a partition element separation region, respectively, and convert light into electric charges. In each pixel 207, one microlens 301 is arranged at the top for the two photodiodes 302A and 302B. That is, each of the plurality of pixels 207 has one microlens 301 corresponding to the plurality of photodiodes 302A and 302B. The photodiodes 302A and 302B share the microlens 301.

The photodiodes 302A and 302B receive the light images of the A image and the B image having a phase difference from each other by receiving the light passing through the different pupil regions of the photographing lens 116. The photodiode 302A having a plurality of pixels 207 can generate an image signal of an A image. The photodiode 302B having a plurality of pixels 207 can generate an image signal of a B image. The AB image is an image based on the mixed charge of the photodiodes 302A and 302B. As shown in FIG. 3B, the image sensor 100 obtains an image signal of the A image based only on the charge of the photodiode 302A and an image signal of the AB image based on the charge obtained by mixing the charges of the photodiodes 302A and 302B. Output.

The processing circuit 101 can obtain the image signal of the B image by inputting the image signal of the A image and the image signal of the AB image and subtracting the image signal of the A image from the image signal of the AB image. The control calculation unit 102 detects the phase difference between the A image and the B image by comparing the image signal of the A image and the image signal of the B image, and sets the lens drive circuit 112 and the lens actuator 114 according to the phase difference. Through, the focus of the focus lens 118 is adjusted. As a result, autofocus (AF) is realized. Further, the image signal of the AB image is used as an image signal of the image for imaging.

FIG. 4 is a diagram showing a configuration example of one pixel 207. The pixel 207 has a photodiode 302A, a holding portion 401A, a holding portion 402A, and a floating diffusion (hereinafter referred to as FD) 403. Further, the pixel 207 has a transfer transistor 405A, a transfer transistor 406A, a transfer transistor 407A, a transfer transistor 408A, and an overflow transistor 415A.

Further, the pixel 207 has a photodiode 302B, a holding portion 401B, a holding portion 402B, and an FD 404. Further, the pixel 207 has a transfer transistor 405B, a transfer transistor 406B, a transfer transistor 407B, a transfer transistor 408B, and an overflow transistor 415B.

Further, the pixel 207 has an amplification transistor 409, an amplification transistor 410, a selection transistor 411, a selection transistor 412, a reset transistor 413, and a reset transistor 414.

Each of the photodiode 302A and the photodiode 302B is a photoelectric conversion unit, which converts incident light into light and stores the converted electric charge. The transfer transistor (transfer switch) 405A is a transfer switch, and in the on state, transfers the electric charge converted by the photodiode 302A to the holding unit 401A. The transfer transistor 406A is a transfer switch, and in the on state, transfers the electric charge converted by the photodiode 302A to the holding unit 402A. The holding unit 401A and the holding unit 402A each hold an electric charge.

The transfer transistor 405B is a transfer switch, and in the on state, transfers the electric charge converted by the photodiode 302B to the holding unit 401B. The transfer transistor 406B is a transfer switch, and in the on state, transfers the electric charge converted by the photodiode 302B to the holding unit 402B. The holding unit 401B and the holding unit 402B each hold an electric charge.

The transfer transistor 407A is a transfer switch, and in the on state, transfers the electric charge of the holding unit 401A to the FD403. The transfer transistor 407B is a transfer switch, and in the on state, transfers the electric charge of the holding unit 401B to the FD403. The FD 403 is a holding portion and holds an electric charge.

The transfer transistor 408A is a transfer switch, and in the on state, transfers the electric charge of the holding unit 402A to the FD404. The transfer transistor 408B is a transfer switch, and in the on state, transfers the electric charge of the holding unit 402B to the FD404. The FD 404 is a holding portion and holds an electric charge.

In the amplification transistor 409, the drain is connected to the power supply voltage node 416 and the source is connected to the column signal line 208a via the selection transistor 411. In the amplification transistor 410, the drain is connected to the power supply voltage node 416 and the source is connected to the column signal line 208b via the selection transistor 412. The row signal line 208a and the row signal line 208b of each row are connected to the row amplifier circuit 203 of FIG.

The reset transistor 413 is a reset switch, and when it is on, it resets the charge (voltage) of the FD 403. The reset transistor 414 is a reset switch, and when it is on, it resets the electric charge (voltage) of the FD404.

The overflow transistor 415A is a discharge switch, the source is connected to the photodiode 302A, and the drain is connected to the power supply voltage node. When the overflow transistor 415A is on, the charge of the photodiode 302A is discharged to an overflow drain such as a power supply voltage node.

The overflow transistor 415B is a discharge switch, the source is connected to the photodiode 302B, and the drain is connected to the power supply voltage node. When the overflow transistor 415B is on, the charge of the photodiode 302B is discharged to an overflow drain such as a power supply voltage node.

When the overflow transistor 415A changes from on to off, the photodiode 302A begins accumulating photoelectrically converted charges. Further, when the overflow transistor 415B changes from on to off, the photodiode 302B starts accumulating the photoelectrically converted charge. As a result, the length of the charge accumulation period of the photodiodes 302A and 302B is set.

It is not essential to provide the overflow transistors 415A and 415B, and the overflow transistors 415A and 415B may be deleted. When the overflow transistor 415A is deleted, for example, when the transfer transistor 405A changes from on to off, the photodiode 302A starts accumulating the photoelectrically converted charge, and the length of the charge accumulation period is set. Further, when the transfer transistor 405B changes from on to off, the photodiode 302B starts accumulating the photoelectrically converted charge, and the length of the charge accumulation period is set. In this configuration, the operation method of each transistor for setting the length of the charge accumulation period is restricted, but the number of elements is reduced, so that the degree of freedom in layout is improved.

All the above transistors are turned on when the control signal is high level and turned off when the control signal is low level. The vertical scanning circuit 202 of FIG. 2 can control the control signals of each row to a high level or a low level at the same time so that the charge accumulation periods in the plurality of pixels 207 are substantially the same. With such a configuration, the image sensor 100 can perform a global electronic shutter operation in which the periods of photoelectric conversion in the plurality of pixels 207 are substantially matched.

The plurality of pixels 207 may have most of the effective pixels and some light-shielding pixels or dummy pixels. The effective pixel has the configuration of pixel 207 of FIG. 4, and the photodiodes 302A and 302B are not shielded from light. The light-shielding pixel has the configuration of pixel 207 in FIG. 4, and the photodiodes 302A and 302B are light-shielded. The dummy pixel has the same configuration as the pixel 207 of FIG. 4, but does not have the photodiodes 302A and 302B.

FIG. 5 is a timing chart showing a control method of the imaging system 120, and shows an imaging operation. FIG. 5 shows the frame period and the charge accumulation period of the photodiodes 302A and 302. Further, FIG. 5 shows a holding period of the holding portion 401A, a holding period of the holding portion 401B, a holding period of the holding portion 402A, and a holding period of the holding portion 402B. Further, FIG. 5 shows the holding period of the FD 403, the holding period of the FD 404, the output period TR1 of the amplification transistor 409, and the output period TR2 of the amplification transistor 410. Here, the output periods TR1 and TR2 are periods in which the output operations of the amplification transistors 409 and 410 are sequentially performed in the pixels 207 of each row, respectively. The output periods TR1 and TR2 include the output periods of the signals of the amplification transistors 409 and 410 and the selection transistors 411 and 412. The vertical scanning circuit 202 controls the operation of the above-mentioned transistor by a control signal.

Times TA0 to TB0 are periods of the first frame. At time TA0, the overflow transistors 415A and 415B change from on to off. Then, the photodiode 302A and the photodiode 302B start the first charge accumulation period, and start accumulating the photoelectrically converted charge. Times TA0 to TA1 are the first charge accumulation periods. At times TA0 to TA1, the photodiode 302A and the photodiode 302B of all pixels 207 accumulate the photoelectrically converted charges.

At time TA1, the transfer transistors 405A and 405B transfer the charges stored in the photodiode 302A and the photodiode 302B to the holding units 401A and 401B, respectively. The holding unit 401A accumulates the electric charge of the photodiode 302A. The holding unit 401B accumulates the electric charge of the photodiode 302B.

After that, at times TA2 to TA7, the transfer transistors 407A and 407B of the plurality of pixels 207 transfer the charges accumulated in the holding units 401A and 401B to the FD 403 in order, row by row, respectively. The FD403 mixes the charges of the photodiodes 302A and 302B and retains the mixed charges. In the output period of the times TA2 to TA7, the amplification transistors 409 of the plurality of pixels 207 sequentially transmit the image signal (voltage) of the AB image based on the amount of electric charge held in the FD 403 via the selection transistor 411. And output to the column signal line 208a. The image sensor 100 repeats the above operation in order from the first line to the N1 line. The image signal of the AB image is used as an image signal of the image for imaging.

Also, at time TA2, the overflow transistors 415A and 415B change from on to off. Then, the photodiode 302A and the photodiode 302B start the second charge accumulation period, and start the accumulation of the photoelectrically converted charge. Times TA2 to TA3 are the second charge accumulation periods. At time TA2 to TA3, the photodiode 302A and the photodiode 302B of all pixels 207 accumulate photoelectrically converted charges in parallel with the output of time TA2 to TA7.

At time TA3, the transfer transistors 406A and 406B transfer the charges stored in the photodiode 302A and the photodiode 302B to the holding units 402A and 402B, respectively. The holding portion 402A accumulates the electric charge of the photodiode 302A. The holding portion 402B accumulates the electric charge of the photodiode 302B.

At time TA4, in the pixel 207 of the first row in the autofocus frame, the transfer transistor 408A transfers the electric charge accumulated in the holding unit 402A to the FD404. The autofocus frame is a focusing position detection frame, and is a frame including a part of pixels 207 that is the target of autofocus. The FD 404 retains the charge of the photodiode 302A. The amplification transistor 410 outputs the image signal (voltage) of the first row of the A image based on the amount of electric charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

Next, in the pixel 207 of the first row in the autofocus frame, the transfer transistor 408B transfers the electric charge accumulated in the holding unit 402B to the FD404. The FD404 mixes the charges of the photodiodes 302A and 302B and retains the mixed charges. The amplification transistor 410 outputs the image signal (voltage) of the first row of the AB image based on the amount of electric charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

Next, in the pixel 207 of the second row in the autofocus frame, the transfer transistor 408A transfers the electric charge accumulated in the holding unit 402A to the FD404. The FD 404 retains the charge of the photodiode 302A. The amplification transistor 410 outputs the image signal of the second row of the A image based on the amount of charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

Next, in the pixel 207 of the second row in the autofocus frame, the transfer transistor 408B transfers the electric charge accumulated in the holding unit 402B to the FD404. The FD404 mixes the charges of the photodiodes 302A and 302B and retains the mixed charges. The amplification transistor 410 outputs the image signal of the second row of the AB image based on the amount of charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

At times TA4 to TA5, the image sensor 100 repeats the output operation of the image signal of the A image and the image signal of the AB image in order from the first line to the N2 line in the autofocus frame. In the output period of time TA4 to TA5, the pixels 207 from the first line to the N2 line in the autofocus frame in the autofocus frame are in parallel with the output of the output period of time TA2 to TA7 of the A image. The image signal and the image signal of the AB image are sequentially output to the column signal line 208b. The image signal of the A image and the image signal of the AB image are used as the image signal of the autofocus image. The output period of times TA4 to TA5 is shorter than the output period of times TA2 to TA7.

The number of lines from the first line to the N2 line of the times TA4 to TA5 is less than the number of lines from the first line to the N1 line of the times TA2 to TA7. Therefore, the period of time TA4 to TA5 is shorter than the period of time TA2 to TA7.

At times TA4 to TA5, the processing circuit 101 inputs the image signal of the A image and the image signal of the AB image in the autofocus frame from the image sensor 100, and subtracts the image signal of the A image from the image signal of the AB image. By doing so, the image signal of the B image can be obtained. The control calculation unit 102 detects the phase difference between the A image and the B image by comparing the image signal of the A image and the image signal of the B image, calculates the autofocus information according to the phase difference, and autofocuses. The information is output to the lens drive circuit 112. The lens drive circuit 112 is a control unit, and adjusts the focus of the focus lens 118 by controlling the position of the focus lens 118 via the lens actuator 114 based on the autofocus information. As a result, autofocus is realized.

The time TB0 is the start time of the second frame. At time TB0, the overflow transistors 415A and 415B change from on to off. Then, the photodiode 302A and the photodiode 302B start the first charge accumulation period, and start accumulating the photoelectrically converted charge. The times TB0 to TB7 in the second frame are the same as the times TA0 to TA7 in the first frame.

As described above, the output of the amplification transistors 410 at times TA4 to TA5 can be performed in parallel with the output of the amplification transistors 409 at times TA2 to TA7. As a result, the control calculation unit 102 can start the processing for autofocus at the time TA5 before the output end time TA7 of the image signal of the image to be captured, and can perform the autofocus control at high speed. .. The autofocus time lag is shortened by shortening the interval between the charge accumulation period of the AF image at the time TA2 to TA3 of the first frame and the charge accumulation period of the image for imaging at the time TB0 to TB1 of the second frame. can do.

Further, the control calculation unit 102 performs autofocus control based on the image signal of the A image and the image signal of the AB image based on the charges in the charge accumulation period of the times TA2 to TA3 instead of the charge accumulation period of the times TA0 to TA1. Do. As a result, the imaging system 120 can improve the accuracy of focus adjustment when the subject moves.

FIG. 6 is a timing chart showing a control method of the imaging system 120, and shows a control signal output by the vertical scanning circuit 202. FIG. 6 shows control signals OFD, TXA11, TXA12, TXA21, TXA22, TXB11, TXB12, TXB21, TXB22, SEL1 and SEL2 that control the pixel 207.

The control signal OFD is applied to the gates of the overflow transistor 415A and the B image overflow transistor 415B. The control signal TXA11 is applied to the gate of the transfer transistor 405A. The control signal TXA12 is applied to the gate of the transfer transistor 407A. The control signal TXA21 is applied to the gate of the transfer transistor 406A. The control signal TXA22 is applied to the gate of the transfer transistor 408A.

The control signal TXB11 is applied to the gate of the transfer transistor 405B. The control signal TXB12 is applied to the gate of the transfer transistor 407B. The control signal TXB21 is applied to the gate of the transfer transistor 406B. The control signal TXB22 is applied to the gate of the transfer transistor 408B.

The control signal SEL1 is applied to the gate of the selection transistor 411. The control signal SEL2 is applied to the gate of the selection transistor 412. When the vertical scanning circuit 202 raises the control signal to a high level, the transistor corresponding to the control signal is turned on. When the vertical scanning circuit 202 lowers the control signal, the transistor corresponding to the control signal is turned off.

Before time TA0, the control signal OFD goes high and the overflow transistors 415A and 415B are turned on. Then, the overflow transistors 415A and 415B discharge the charges stored in the photodiodes 302A and 302B to the power supply voltage node.

At time TA0, the control signal OFD goes low and the overflow transistors 415A and 415B are turned off. Then, the photodiode 302A and the photodiode 302B start the first charge accumulation period, and start accumulating the photoelectrically converted charge. At times TA0 to TA1, the photodiode 302A and the photodiode 302B of all pixels 207 accumulate the photoelectrically converted charges.

At time TA1, the control signals TXA11 and TXB11 become high-level pulses, and the transfer transistors 405A and 405B turn on during the high-level period. Then, the transfer transistors 405A and 405B transfer the electric charges stored in the photodiode 302A and the photodiode 302B to the holding units 401A and 401B, respectively. When the control signals TXA11 and TXB11 go low, the transfer transistors 405A and 405B are turned off, ending the first charge accumulation period. Times TA0 to TA1 are the first charge accumulation periods.

At times TA2 to TA7, the control signals TXA12 and TXB12 of each line are sequentially turned into high level pulses, and the transfer transistors 407A and 407B of each line are turned on during the high level period. Then, the transfer transistors 407A and 407B transfer the electric charges accumulated in the holding units 401A and 401B to the FD 403 in order, row by row, respectively. The FD403 mixes the charges of the photodiodes 302A and 302B and retains the mixed charges.

Further, at times TA2 to TA7, the control signal SEL1 of each line becomes a high level pulse in order, and the selection transistor 411 of each line turns on during the high level period. Then, the amplification transistor 409 outputs an image signal of the AB image based on the amount of electric charge held in the FD 403 in order in row units to the column signal line 208a via the selection transistor 411. The image sensor 100 repeats the above operation in order from the first line to the N1 line. The image signal of the AB image is used as an image signal of the image for imaging.

Further, before the time TA2, the control signal OFD becomes high level, and the overflow transistors 415A and 415B are turned on. Then, the overflow transistors 415A and 415B discharge the charges stored in the photodiodes 302A and 302B to the power supply voltage node.

At time TA2, the control signal OFD goes low and the overflow transistors 415A and 415B are turned off. Then, the photodiode 302A and the photodiode 302B start the second charge accumulation period, and start the accumulation of the photoelectrically converted charge. At times TA2 to TA3, the photodiode 302A and the photodiode 302B of all pixels 207 accumulate the photoelectrically converted charges.

At time TA3, the control signals TXA21 and TXB21 become high level pulses, and the transfer transistors 406A and 406B are turned on during the high level period. Then, the transfer transistors 406A and 406B transfer the electric charges stored in the photodiode 302A and the photodiode 302B to the holding units 402A and 402B, respectively. When the control signals TXA21 and TXB21 go low level, the transfer transistors 406A and 406B are turned off, ending the second charge accumulation period. Times TA2 to TA3 are the second charge accumulation periods.

At time TA4, the control signal TXA22 becomes a high-level pulse in the pixel 207 of the first row in the autofocus frame, and the transfer transistor 408A turns on in the high-level period. Then, the transfer transistor 408A transfers the electric charge stored in the holding unit 402A to the FD404. The FD 404 retains the charge of the photodiode 302A. Further, the control signal SEL2 becomes a high level pulse, and the selection transistor 412 turns on during the high level period. Then, the amplification transistor 410 outputs the signal of the first row of the A image based on the amount of charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

Next, in the pixel 207 of the first row in the autofocus frame, the control signal TXB22 becomes a high level pulse, and the transfer transistor 408B turns on in the high level period. Then, the transfer transistor 408B transfers the electric charge stored in the holding unit 402B to the FD404. The FD404 mixes the charges of the photodiodes 302A and 302B and retains the mixed charges. Further, the control signal SEL2 becomes a high level pulse, and the selection transistor 412 turns on during the high level period. Then, the amplification transistor 410 outputs the signal of the first row of the AB image based on the amount of charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

Next, in the pixel 207 of the second row in the autofocus frame, the control signal TXA22 becomes a high level pulse, and the transfer transistor 408A turns on in the high level period. Then, the transfer transistor 408A transfers the electric charge stored in the holding unit 402A to the FD404. The FD 404 retains the charge of the photodiode 302A. Further, the control signal SEL2 becomes a high level pulse, and the selection transistor 412 turns on during the high level period. Then, the amplification transistor 410 outputs the signal of the second row of the A image based on the amount of charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

Next, in the pixel 207 of the second row in the autofocus frame, the control signal TXB22 becomes a high level pulse, and the transfer transistor 408B turns on in the high level period. Then, the transfer transistor 408B transfers the electric charge stored in the holding unit 402B to the FD404. The FD404 mixes the charges of the photodiodes 302A and 302B and retains the mixed charges. Further, the control signal SEL2 becomes a high level pulse, and the selection transistor 412 turns on during the high level period. Then, the amplification transistor 410 outputs the signal of the second row of the AB image based on the amount of charge held in the FD 404 to the column signal line 208b via the selection transistor 412.

At times TA4 to TA5, the image sensor 100 repeats the output operation of the image signal of the A image and the image signal of the AB image in order from the first line to the N2 line in the autofocus frame. The image signal of the A image and the image signal of the AB image are used as the image signal of the autofocus image.

FIG. 7 is a flowchart showing a control method of the imaging system 120. In step S701, when the control calculation unit 102 detects that the power button included in the operation unit 110 is turned on by the user and then the still image recording button included in the operation unit 110 is turned on by the user, the step S701 is taken. Proceed to S703. The control calculation unit 102 continues the still image recording (continuous shooting) while the still image recording button is turned on.

In step S703, the control calculation unit 102 sets the conditions for the image to be captured. For example, the condition settings include ISO setting for the image sensor 100, aperture value setting for the aperture drive circuit 113, recording image area setting, and setting whether or not to acquire phase difference information. The control calculation unit 102 determines the output period TR1 of the amplification transistor 409 according to the set recorded image area and the presence / absence of phase difference information acquisition.

Next, in step S704, the control calculation unit 102 sets the conditions for the autofocus image. For example, the condition setting includes the setting of the ISO for the image sensor 100, the setting of the aperture value for the aperture drive circuit 113, the setting of the autofocus frame, and the setting of the reading method such as addition and thinning. The autofocus frame is an area for reading out an autofocus image signal. The control calculation unit 102 determines the output period TR2 of the amplification transistor 410 according to the set autofocus frame and the reading method.

Here, the frame speed at the time of continuous shooting of still images is determined, and the time taken for acquiring one image is TR. The charge accumulation period EX1 of the photographed image is the period of time TA0 to TA1 in FIG. The charge accumulation period EX2 of the autofocus image is the period of time TA2 to TA3 in FIG. The drive time TF is the drive time of the lens drive circuit 112. The charge accumulation period EX1 of the image for photographing, the charge accumulation period EX2 of the image for autofocus, the output period TR1 of the amplification transistor 409, and the output period TR2 of the amplification transistor 410 have the relationship of the equations (1) and (2). Set to have.

TR ≧ EX1 + TR1 ・ ・ ・ (1)
TR-EX1 ≧ EX2 + TR2 + TF ・ ・ ・ (2)

Next, in step S705, the control calculation unit 102 sets the start timing of the charge accumulation period EX2 of the autofocus image. The imaging system 120 can shorten the autofocus time lag by capturing an AF image immediately before the next still image (imaging image) to be captured under the condition satisfying the equation (2). Therefore, the control calculation unit 102 sets the start timing of the charge accumulation period EX2 of the autofocus image based on the values of TR-TR2-TF.

Next, in step S706, the control calculation unit 102 controls the photodiodes 302A and 302B so as to start charge accumulation of the image for imaging. The control calculation unit 102 discharges the charges of the photodiodes 302A and 302B by the control signal OFD, and controls the charge accumulation period EX1 and the photodiodes 302A and 302B to accumulate charges according to the aperture value and ISO. This process corresponds to the process of times TA0 to TA1 in FIG.

Next, in step S707, the pixel 207 outputs an imaging image signal to the column signal line 208a. This process corresponds to the process at times TA2 to TA7 in FIG.

Next, in step S708, the control calculation unit 102 determines whether or not the current timing has reached the start timing of the charge accumulation period EX2 of the AF image determined in step S705. The control calculation unit 102 waits when the current timing is not the start timing of the charge accumulation period EX2 of the AF image, and when the current timing is the start timing of the charge accumulation period EX2 of the AF image. , Step S709.

In step S709, the control calculation unit 102 controls the photodiodes 302A and 302B to start charge accumulation of the autofocus image. The control calculation unit 102 discharges the charges of the photodiodes 302A and 302B by the control signal OFD, and controls the charge accumulation period EX2 and the photodiodes 302A and 302B to accumulate the electric charges. This process corresponds to the process at times TA2 to TA3 in FIG.

Next, in step S710, the pixel 207 outputs the AF image signal to the column signal line 208b. This process corresponds to the process of times TA4 to TA5 in FIG.

Next, in step S711, the processing circuit 101 obtains an image signal of the B image of the AF image by subtracting the image signal of the A image of the AF image from the image signal of the AB image of the AF image. The control calculation unit 102 detects the phase difference between the A image and the B image by comparing the image signal of the A image and the image signal of the B image, calculates the autofocus information according to the phase difference, and autofocuses. The information is output to the lens drive circuit 112.

Next, in step S712, the lens drive circuit 112 adjusts the focus of the focus lens 118 via the lens actuator 114 based on the autofocus information. As a result, autofocus is realized.

Next, in step S713, the control calculation unit 102 determines whether or not the process of outputting the image pickup image signal to the column signal line 208a by the pixel 207 has been completed. This process corresponds to the process of time TA7 in FIG. The control calculation unit 102 waits when the pixel 207 has not completed the process of outputting the image pickup image signal to the column signal line 208a, and the pixel 207 outputs the image pickup image signal to the column signal line 208a. When finished, the process proceeds to step S714.

In step S714, the processing circuit 101 performs low-pass filter processing, shading processing, white balance processing, correction processing, etc. to reduce noise as still image development processing on the image pickup image signal, and produces the still image signal. Output.

Next, in step S715, the control calculation unit 102 records the still image signal on the recording medium 108, and displays the still image signal in live view on the display unit 107.

Next, in step S716, the control calculation unit 102 determines whether or not the still image recording button has been turned off. If the still image recording button is not turned off, the control calculation unit 102 returns to step S706 and repeats the still image shooting process. Further, when the still image recording button is turned off, the control calculation unit 102 proceeds to step S717 and ends the process of FIG. 7.

As described above, the image sensor 100 uses the FD403 as the image for imaging and outputs the image signal of the AB image. The image sensor 100 may output the image signal of the A image and the image signal of the B image as the image for imaging.

Further, the image sensor 100 uses the FD404 as an autofocus image to output an image signal of the A image and an image signal of the AB image. The image sensor 100 may output the image signal of the A image and the image signal of the B image as the autofocus image. The image signal of the B image is an image signal based on the charge of the photodiode 302B.

Further, the start time of the charge accumulation period of the autofocus image is not limited to the time TA2 immediately after the transfer of the transfer transistors 405A and 405B, and may be later than the time TA2.

Further, the autofocus frame is an area of all pixels 207 in the pixel area 201, or an area of a part of pixels 207 among all pixels 207 in the pixel area 201. That is, the autofocus frame is an area of at least a part of the pixels 207 among all the pixels 207 in the pixel area 201.

Further, the image sensor 100 may read out the image signals of the pixels 207 in a plurality of rows by thinning them out as an autofocus image. That is, during the output period from time TA4 to TA5, the pixels of the plurality of pixels 207 at predetermined intervals output the image signal of the A image and the image signal of the AB image to the column signal line 208b in order.

Further, the image sensor 100 may read out a signal in which the charges of the pixels 207 of a plurality of rows are mixed as an autofocus image. That is, during the output period from time TA4 to TA5, some of the plurality of pixels 207 are mixed with the charges of their own pixels and the charges of other pixels, and then the image signal of the A image and the AB image are displayed. The image signals are sequentially output to the column signal line 208b.

Further, the autofocus frame may be the frame of the main subject. The image sensor 100 can shorten the reading time of the autofocus image signal by reducing the number of lines to be read as the autofocus image. As a result, the image sensor 100 can delay the start of the charge accumulation period of the autofocus image at times TA2 to TA3, and shorten the autofocus time lag with respect to the charge accumulation period of the image for imaging from time TB0.

The image sensor 100 can be applied to smartphones, tablets, industrial cameras, medical cameras, in-vehicle cameras, and the like, in addition to digital cameras and video cameras.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments, and various modifications and changes can be made within the scope of the gist thereof.

100 image sensor, 208a, 208b row signal line, 302A, 302B photodiode

Claims (11)

A plurality of pixels each having first and second photoelectric conversion units that perform photoelectric conversion, and
A first signal line that can be connected to the plurality of pixels,
It has a second signal line that can be connected to the plurality of pixels.
The plurality of pixels
In the first storage period, the first and second photoelectric conversion units accumulate the photoelectrically converted charges.
In the first output period after the first storage period, the image signals based on the charges in the first storage period are sequentially output to the first signal line.
At least a part of the plurality of pixels
In the second storage period after the first storage period, the first and second photoelectric conversion units accumulate the photoelectrically converted charges in parallel with the output in the first output period.
In the second output period after the second storage period, an image signal based on the charge of the second storage period is sequentially output to the second signal line in parallel with the output of the first output period. An imaging device characterized by In the first output period, the plurality of pixels sequentially output a first image signal based on the charges of the first and second photoelectric conversion units in the first storage period to the first signal line. And
At least a part of the plurality of pixels has a second image signal based on the electric charge of the first photoelectric conversion unit in the second storage period and the second image signal in the second output period. 1. A third image signal based on the charges of the first and second photoelectric conversion units or the charges of the second photoelectric conversion unit during the accumulation period of the above is output to the second signal line in order. The imaging apparatus according to. The imaging device according to claim 1 or 2, wherein the second output period is shorter than the first output period. A claim, wherein some of the plurality of pixels sequentially output an image signal based on the electric charge in the second storage period to the second signal line in the second output period. Item 3. The imaging device according to any one of Items 1 to 3. The pixel in the autofocus frame among the plurality of pixels is characterized in that, in the second output period, the image signal based on the charge in the second storage period is sequentially output to the second signal line. The imaging device according to any one of claims 1 to 4. The pixels at predetermined intervals among the plurality of pixels are characterized in that, in the second output period, image signals based on the charges in the second accumulation period are sequentially output to the second signal line. The imaging device according to any one of claims 1 to 4. In the second output period, some of the plurality of pixels are mixed with the charges of their own pixels and the charges of other pixels, and then an image signal based on the charges in the second storage period is used. The image pickup apparatus according to any one of claims 1 to 4, wherein the images are sequentially output to the second signal line. The invention according to any one of claims 1 to 7, wherein each of the plurality of pixels has one microlens corresponding to the first photoelectric conversion unit and the second photoelectric conversion unit. Imaging device. Each of the plurality of pixels
A first transfer switch that transfers the electric charge of the first photoelectric conversion unit to the first holding unit, and
A second transfer switch that transfers the charge of the first holding unit to the second holding unit, and
A third transfer switch that transfers the electric charge of the second photoelectric conversion unit to the third holding unit, and
A fourth transfer switch that transfers the charge of the third holding unit to the second holding unit, and
A fifth transfer switch that transfers the electric charge of the first photoelectric conversion unit to the fourth holding unit, and
A sixth transfer switch that transfers the charge of the fourth holding unit to the fifth holding unit, and
A seventh transfer switch that transfers the electric charge of the second photoelectric conversion unit to the sixth holding unit, and
It has an eighth transfer switch that transfers the charge of the sixth holding portion to the fifth holding portion.
In the first output period, the plurality of pixels sequentially output an image signal based on the charge of the second holding portion to the first signal line, and in the second output period, the fifth The imaging apparatus according to any one of claims 1 to 8, wherein image signals based on the electric charge of the holding portion are sequentially output to the second signal line. The imaging device according to claim 2 and
An imaging system characterized by having a control unit that controls the position of a focus lens based on the second image signal and the third image signal. A plurality of pixels each having first and second photoelectric conversion units that perform photoelectric conversion, and
A first signal line that can be connected to the plurality of pixels,
A control method for an image pickup apparatus having a second signal line that can be connected to the plurality of pixels.
In the first storage period, the plurality of pixels are subjected to a step of accumulating charge charged by the first and second photoelectric conversion units.
In the first output period after the first storage period, the plurality of pixels sequentially output an image signal based on the charge of the first storage period to the first signal line.
At least a part of the plurality of pixels has the first and second photoelectrics in parallel with the output of the first output period in the second storage period after the first storage period. The step that the conversion unit accumulates the photoelectrically converted charge,
At least a part of the plurality of pixels is charged in the second storage period in parallel with the output in the first output period in the second output period after the second storage period. A method for controlling an image pickup apparatus, which comprises a step of sequentially outputting an image signal based on the above to the second signal line.

Images