JP2023057134A - Image pickup device and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device that can reduce as much as possible the time required for outputting a signal necessary for the drive control of an imaging apparatus from the image pickup device.

SOLUTION: An image pickup device comprises: a pixel unit in which a plurality of unit pixels each having one microlens and a plurality of photoelectric conversion units are arranged in a matrix shape; a signal holding unit that holds a signal corresponding to electric charges accumulated in one photoelectric conversion unit of the plurality of photoelectric conversion units in each of the unit pixels, and a signal corresponding to electric charges obtained by mixing the electric charges accumulated in all of the plurality of photoelectric conversion units; and an output unit that outputs a signal based on the signal corresponding to electric charges accumulated in one photoelectric conversion unit of the plurality of photoelectric conversion units in each of the unit pixels and the signal corresponding to electric charges obtained by mixing the electric charges accumulated in all of the plurality of photoelectric conversion units, which are held by the signal holding unit. The pixel unit is formed on a first semiconductor substrate, and the signal holding unit is formed on a second semiconductor substrate different from the first semiconductor substrate. The first semiconductor substrate and the second semiconductor substrate are laminated with each other.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to an imaging device and an imaging device.

2. Description of the Related Art In recent years, imaging apparatuses using CMOS imaging elements and the like are becoming more sophisticated and multi-functional in order to meet various needs. CMOS imaging devices are becoming more pixel-rich and faster, and there is an increasing demand for a system that can read out pixel signals at higher speeds.

For example, as a method for performing high-speed readout, as described in Patent Document 1, a method of arranging an analog/digital conversion circuit (hereinafter referred to as a column ADC) for each column and performing digital output has become widespread in recent years. . By introducing a column ADC, it becomes possible to perform digital transmission of pixel signals to the outside of the image pickup device, and high-speed readout becomes possible as technology for digital signal transmission improves.

On the other hand, as an example of multifunctionality, an imaging device capable of acquiring not only the light intensity distribution but also the light incident direction and distance information has been proposed. Japanese Patent Application Laid-Open No. 2002-200000 discloses an image pickup device capable of focus detection using a signal obtained from the image pickup device. By dividing a photodiode (PD) corresponding to one microlens into two, each PD is configured to receive light from different pupil planes of the photographing lens. Focus detection is performed by comparing the outputs of the two PDs. Also, by adding the output signals from two PDs forming a unit pixel, a normal captured image can be obtained.

Further, the imaging apparatus described in Patent Document 3 has a mode in which an imaging signal for live view display, a focus detection signal, and an exposure control signal are read out from the solid-state imaging device in one vertical scan of the same frame. ing. The imaging apparatus described in Japanese Patent Laid-Open No. 2002-200001 is capable of performing live view display while speeding up both focus detection control and exposure control.

JP-A-2005-278135 Japanese Patent No. 3774597 JP 2009-89105 A

However, when performing focus detection and exposure control in an image sensor as disclosed in Patent Documents 2 and 3, it is necessary to read out all PD signals. becomes longer and the frame rate drops. Even if the signal readout time is increased by a readout method using a column ADC as in Patent Document 1, it is expected that the number of pixels and the frame rate will be further increased in the future, and it is desirable to further shorten the signal readout time. be

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and an object of the present invention is to significantly reduce the time required to output a signal, such as a focus detection signal, necessary for drive control of an image pickup device from an image pickup device. It is to provide an imaging device.

An imaging device according to the present invention includes a pixel section in which a plurality of unit pixels each having one microlens and a plurality of photoelectric conversion sections are arranged in a matrix, and the plurality of photoelectric conversion sections in each of the unit pixels. Signal holding for holding a signal corresponding to charges accumulated in some of the photoelectric conversion units and a signal corresponding to charges obtained by mixing the charges accumulated in all of the plurality of photoelectric conversion units a signal corresponding to charges accumulated in some of the plurality of photoelectric conversion units in each of the plurality of unit pixels held in the signal holding unit and the plurality of photoelectric conversion units; and an output unit for outputting a signal based on a signal corresponding to the charge obtained by mixing the charges accumulated in all of the above, wherein the pixel unit is formed on the first semiconductor substrate, and the signal holding A part is formed on a second semiconductor substrate different from the first semiconductor substrate, and the first semiconductor substrate and the second semiconductor substrate are stacked on each other.

According to the present invention, it is possible to provide an imaging device capable of greatly shortening the time required to output a signal, such as a focus detection signal, necessary for drive control of an imaging device from the imaging device.

1 is a block diagram showing the configuration of an imaging device according to a first embodiment of the present invention; FIG. 4A and 4B are diagrams for explaining the configuration of a unit pixel of the image sensor according to the first embodiment; FIG. FIG. 4 is a conceptual diagram of a luminous flux emitted from an exit pupil of a photographing lens and incident on a unit pixel; FIG. 2 is a block diagram showing the configuration of the imaging element of the first embodiment; FIG. 4A and 4B are diagrams for explaining a pixel circuit and a readout circuit of the image sensor according to the first embodiment; FIG. 4 is a timing chart showing a signal readout operation of the imaging element of the first embodiment; FIG. 4 is a diagram for explaining the configuration of a signal processing unit in the image sensor according to the first embodiment; FIG. 10 is a diagram showing a focus detection signal output area in a pixel area according to the second embodiment; FIG. 11 is a block diagram showing the configuration of an imaging device according to a third embodiment of the present invention; 10 is a flow chart showing the flow of imaging plane phase difference AF processing in the third embodiment. FIG. 11 is a view showing a selection screen for focus detection areas according to the third embodiment; FIG. 11 is a view showing a subject detection result screen according to the third embodiment; FIG. The figure explaining the structure of the image pick-up element of 4th Embodiment. FIG. 10 is a diagram for explaining a pixel circuit and a readout circuit of an image sensor according to a fourth embodiment; The whole block diagram of the image pick-up element of 5th Embodiment. The whole block diagram of the image pick-up element of 5th Embodiment. The whole block diagram of the image pick-up element of 6th Embodiment. FIG. 11 is a block diagram showing the configuration of an imaging device according to the seventh embodiment; FIG. 11 is a block diagram showing the configuration of an imaging device according to an eighth embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of an image pickup apparatus having an image pickup device according to the first embodiment of the present invention. The imaging device 100 includes a light receiving section 102 , a reading section 103 , a control section 104 , a signal processing section 105 and an output section 106 . The light receiving unit 102 has a plurality of unit pixels arranged in a matrix and receives an optical image formed by the imaging lens 101 . A configuration of the light receiving unit 102 will be described later. A reading unit (AD conversion unit) 103 receives a drive control signal from the control unit 104 , A/D converts the image signal output from the light receiving unit 102 , and sends the signal to the signal processing unit 105 .

The signal processing unit 105 performs arithmetic processing such as signal addition, subtraction, and multiplication on the A/D-converted image signal, selection processing of signals to be output from the image sensor 100 to the outside via the output unit 106, and the like. I do. The signal processing unit 105 also performs various corrections such as reference level adjustment and data rearrangement. These processes are executed upon receiving control signals from the control unit 104 . Although the details of the signal processing unit 105 will be described later, the signal processing unit 105 performs signal processing for captured image and signal processing for focus detection on the image signal obtained from the light receiving unit 102, and outputs them. Send to section 106 . The output unit 106 outputs the image signal processed by the signal processing unit 105 to the outside of the imaging device 100 .

The image processing unit 107 receives a captured image signal from the output unit 106 of the image sensor 100, and performs image processing such as defective pixel correction, noise reduction, color conversion, white balance correction, image correction, resolution conversion processing, image processing, and so on. Performs compression processing, etc., and generates still images and moving images. A phase difference detection unit 108 receives a focus detection signal from the output unit 106 and calculates a phase difference evaluation value for focus detection.

An overall control/calculation unit 109 performs overall driving and control of the imaging element 100 and the imaging apparatus as a whole. The display unit 110 displays an image after shooting, a live view image, various setting screens, and the like. A recording unit 111 and a memory unit 112 are recording media such as a non-volatile memory or a memory card for recording and holding image signals and the like output from the overall control/calculation unit 109 . An operation unit 113 receives a user's command from an operation member provided in the imaging apparatus, and inputs the command to the overall control/arithmetic circuit 106 . A lens control unit 114 calculates optical system drive information based on the phase difference evaluation value calculated by the phase difference detection unit 108 and controls the focus lens position of the imaging lens 101 .

Next, the relationship between the imaging lens 101 and the light receiving unit 102 of the imaging element 100 in the imaging apparatus of this embodiment, the definition of pixels, and the principle of focus detection by the pupil division method will be described.

FIG. 2 is a schematic diagram showing the configuration of a unit pixel 200 of the imaging device 100. As shown in FIG. In FIG. 2, the microlens 202 further collects the light imaged on the image sensor 100 by the imaging lens 101 for each pixel. The photoelectric conversion units 201A and 201B, which are photodiodes (PD), receive light incident on the unit pixel 200, and generate and accumulate signal charges corresponding to the amount of received light. Since the unit pixel 200 has two photoelectric conversion units under one microlens 202, the two photoelectric conversion units 201A and 201B can respectively receive the light of the exit pupil area divided into two. . A signal obtained by mixing the two signals of the photoelectric conversion units 201A and 201B for each pixel is an output signal of one pixel for image generation. Further, by comparing signals obtained from two photoelectric conversion units for each pixel, focus detection of the imaging lens 101 can be performed. That is, in a certain area in the unit pixel 200, the signal obtained from the photoelectric conversion unit 201A and the signal obtained from the photoelectric conversion unit 201B are subjected to correlation calculation, thereby performing focus detection of the phase difference detection method with pupil division in the horizontal direction. is possible.

FIG. 3 shows how light that has passed through the imaging lens 101 passes through one microlens 202 and is received by the unit pixel 200 of the light receiving unit 102 of the image sensor 100 in the direction perpendicular to the optical axis (Z-axis). It is the figure observed from (Y-axis direction). Light passing through the exit pupils 302 and 303 of the photographing lens enters the unit pixel 200 centering on the optical axis. At that time, the amount of incident light is adjusted by the lens diaphragm 301 . As shown in FIG. 3, the luminous flux passing through the pupil region 302 passes through the microlens 202 and is received by the photoelectric conversion unit 201A, and the luminous flux passing through the pupil region 303 passes through the microlens 202 and is received by the photoelectric conversion unit 201B. Therefore, the photoelectric conversion units 201A and 201B receive light in different regions of the exit pupil of the photographing lens.

Signals of a photoelectric conversion unit 201A that pupil-divides the light from the photographing lens 101 are acquired from a plurality of unit pixels 200 arranged in the X-axis direction, and an object image composed of these output signal groups is assumed to be an A image. Similarly, the signals of the photoelectric conversion unit 201B for pupil division are obtained from a plurality of unit pixels 200 arranged in the X-axis direction, and the subject image composed of these output signal groups is referred to as a B image.

A correlation calculation is performed on the A image and the B image to detect an image shift amount (pupil division phase difference). Further, by multiplying the image shift amount by a conversion coefficient determined by the focal position of the photographing lens 101 and the optical system, the focal position corresponding to an arbitrary object position within the screen can be calculated. By controlling the focus position of the photographing lens 101 based on the focus position information calculated here, imaging plane phase difference AF (autofocus) becomes possible. Further, by using the A+B image signal as a signal obtained by adding the A image signal and the B image signal, the A+B image signal can be used for a normal photographed image.

Next, the configurations of the light receiving section 102 and the readout section 103 of the image sensor 100 will be described with reference to FIGS. 4 and 5. FIG. FIG. 4 is a block diagram showing a configuration example of the light receiving section 102 and the readout section 103 of the image sensor 100. As shown in FIG.

The light receiving portion 102 has a pixel portion 401 and a driver circuit portion 402 . In the pixel portion 401, a plurality of unit pixels 200 are arranged in a matrix in the horizontal direction (row direction) and the vertical direction (column direction). In FIG. 4, only six unit pixels 200 of 2 rows and 3 columns are shown, but actually, millions or tens of millions of unit pixels 200 are arranged. The driver circuit portion 402 includes a power supply circuit for driving the pixel portion 401, a timing generator (TG), a scanning circuit, and the like. By driving the pixel portion 401 by the drive circuit portion 402 , pixel signals of the entire imaging region of the pixel portion 401 are output from the pixel portion 401 to the reading portion 103 . The drive circuit section 402 is driven under the control of the control section 104 in FIG. A pixel signal from the pixel unit 401 input to the reading unit 103 is subjected to analog-to-digital conversion (A/D conversion). The readout unit 103 is configured by including a plurality of readout circuits, for example, one readout circuit per column.

By the way, regarding the driving method of the pixel unit 401, when adjacent rows are driven under different conditions (frame rate, accumulation time, etc.), noise such as crosstalk and blooming is likely to occur. However, in this embodiment, the drive circuit portion 402 drives the pixel portion 401 uniformly under the same conditions over the entire region, so such a problem does not occur.

FIG. 5 is a diagram showing an example of the unit pixel 200 of the image sensor 100 and a readout circuit 509 that constitutes the readout unit 103. As shown in FIG. In the unit pixel 200, a transfer switch 502A is connected to a photoelectric conversion unit 201A made up of a photodiode (PD), and a transfer switch 502B is connected to the photoelectric conversion unit 201B. The charges generated in the photoelectric conversion units 201A and 201B are transferred to a common floating diffusion (FD) 504 via transfer switches 502A and 502B, respectively, and temporarily stored. The charge transferred to the FD 504 is output to the column output line 507 as a voltage corresponding to the charge through an amplifying MOS transistor (SF) 505 forming a source follower amplifier when the selection switch 506 is turned on. A current source 508 is connected to the column output line 507 .

The reset switch 503 resets the potential of the FD 504 and the potential of the photoelectric conversion units 201A and 201B to VDD via the transfer switches 502A and 502B. The transfer switches 502A and 502B, the reset switch 503, and the selection switch 506 are controlled by control signals PTXA, PTXB, PRES, and PSEL via signal lines connected to the peripheral drive circuit section 402, respectively.

Next, the circuit configuration of the reading circuit 509 will be described. An amplifier 510 amplifies the signal output to the column output line 507, and a capacitor 512 is used to hold the signal voltage. Writing to the capacitor 512 is controlled by a switch 511 that is turned on and off by a control signal PSH. A reference voltage Vslope supplied from a slope voltage generation circuit (not shown) is input to one input of the comparator 513 , and the output of the amplifier 510 written to the capacitor 512 is input to the other input. A comparator 513 compares the output of the amplifier 510 with the reference voltage Vslope, and outputs either a low level or high level binary value depending on the magnitude relationship. Specifically, when the reference voltage Vslope is smaller than the output of the amplifier 510, it outputs a low level, and when it is greater than that, it outputs a high level. The clock CLK starts running at the same time as the transition of the reference voltage Vslope starts, and the counter 514 counts up in response to the clock CLK when the output of the comparator 513 is at high level, and counts at the same time when the output of the comparator 513 is inverted to low level. stop signal. The count value at this time is held in either memory 516 or memory 517 as a digital signal.

A memory 516 holds a digital signal obtained by AD-converting the reset level signal of the FD 504 (hereinafter referred to as “N signal”), and a memory 517 stores the signal of the photoelectric conversion unit 201A and the photoelectric conversion unit 201B as the N signal of the FD 504. A digital signal obtained by AD-converting the signal superimposed on (hereinafter referred to as "S signal") is held. A switch 515 determines to which of the memories 516 and 517 the count value of the counter 514 is written. The signals held in the memories 516 and 517 are subtracted by the CDS circuit 518 from the S signal to calculate the difference. Then, it is output to the signal processing unit 105 through the digital signal output line 519 under the control of the driving circuit unit 402 .

Note that one readout circuit 509 is arranged for each column of pixels, and pixel signals are read out in units of rows. In this case, the selection switches 506 are controlled row by row, and the pixel signals of the selected row are collectively output to the column output line 507 . As the number of readout circuits 509 increases, the pixel signals of the pixel portion 401 can be read out to the signal processing portion 105 at high speed.

FIG. 6 is a timing chart showing an example of the charge reading operation from the unit pixel 200 of the imaging device 100 having the circuit configuration shown in FIG. The timing of each drive pulse, reference voltage Vslope, clock CLK, and horizontal scanning signal are shown schematically. The potential Vl of the column output line at each timing is also shown.

Prior to reading the signal from the photoelectric conversion unit 201A, the signal line PRES of the reset switch 503 becomes Hi (t600). This resets the gate of SF (source follower amplifier) 505 to the reset power supply voltage. At time t601, the control signal PSEL is set to Hi, and the SF 505 is brought into operation. At t602, the reset of the FD 504 is released by setting the control signal PRES to Lo. The potential of the FD 504 at this time is output to the column output line 507 as a reset signal level (N signal) and input to the readout circuit 509 .

By turning on and off the switch 511 with the control signal PSH set to Hi and Lo at times t603 and t604, the N signal output to the column output line 507 is amplified with a desired gain by the amplifier 510 and then held in the capacitor 512. be. The potential of the N signal held in capacitor 512 is input to one side of comparator 513 . After the switch 511 is turned off at time t604, from time t605 to t607, a slope voltage generation circuit (not shown) decreases the reference voltage Vslope from the initial value with time. The clock CLK is supplied to the counter 514 with the start of transition of the reference voltage Vslope. The value of the counter 514 increases according to the number of CLKs. Then, when the reference voltage Vslope input to the comparator 513 becomes the same level as the N signal, the output COMP of the comparator 513 becomes low level, and the operation of the counter 514 also stops at the same time (time t606). The value when the operation of the counter 514 stops becomes the AD-converted value of the N signal. be done.

Next, at times t607 and t608 after the digitized N signal is held in the N signal memory 516, the control signal PTXA is sequentially set to Hi and Lo to transfer the photoelectric charges accumulated in the photoelectric conversion unit 201A to the FD504. Then, the potential fluctuation of the FD 504 corresponding to the amount of charge is output to the column output line 507 as a signal level (light component+reset noise component (N signal)) and input to the readout circuit 509 . The input signal (S(A)+N) is amplified with a desired gain by the amplifier 510, and then the control signal PSH is sequentially set to Hi and Lo at times t609 and t610 to turn the switch 511 on and off. retained. The potential held in capacitor 512 is input to one side of comparator 513 . After the switch 511 is turned off at time t610, the slope voltage generation circuit decreases the reference voltage Vslope from the initial value with time from time t611 to t613. CLK is supplied to the counter 514 with the start of transition of the reference voltage Vslope. The value of the counter 514 increases according to the number of CLKs. Then, when the reference voltage Vslope input to the comparator 513 becomes the same level as the S signal, the output COMP of the comparator 513 becomes low level, and the operation of the counter 514 also stops at the same time (time t612). The value when the operation of the counter 514 stops is the value obtained by AD converting the S(A)+N signal. A switch 515 connects the counter 514 and the memory 517 , and the digital value of the S(A)+N signal is held in the S signal memory 517 . A CDS circuit 518 calculates the differential signal level (light component) from the signals held in the memory 516 and the memory 517, and obtains the S(A) signal from which the reset noise component has been removed. The S(A) signal is sequentially sent to the signal processing section 105 under the control of the control section 104 .

The above is the operation of reading the signal from the photoelectric conversion unit 201A of the unit pixel 200. FIG. When reading out a signal from the other photoelectric conversion unit 201B of the unit pixel 200, it is similarly driven according to the timing chart of FIG. However, in this case, instead of the control signal PTXA, the control signal PTXB is sequentially set to Hi and Lo at times t607 and t608. That is, in the driving from time t600 to time t613 in FIG. By outputting (B), the output of pixel signals for one row is completed. By repeating this for all rows, the output of pixel signals S(A) and S(B) of all pixels is completed.

FIG. 7 is a diagram showing a configuration example of the signal processing unit 105 and the output unit 106 of the image sensor 100. As shown in FIG. The signal processing unit 105 has a captured image signal processing unit 701 and a focus detection signal processing unit 702 . A pixel signal output from the reading unit 103 is input to the signal processing unit 105 via a digital signal output line 519 . The input signal is processed according to control from the control section 104 . Note that the captured image signal processing unit 701 and the focus detection signal processing unit 702 are each provided with a memory (not shown).

The captured image signal processing unit 701 calculates a captured image signal from the signal output from the reading unit 103 . That is, the pixel signals S(A) and S(B) of the photoelectric conversion unit 201A and the photoelectric conversion unit 201B of the unit pixel 200 are received and mixed to calculate the S(A+B) signal. Then, the pixel signal S(A+B) is sent to the output unit 106 via the captured image signal output line 703 . In the captured image signal processing unit 701 , the pixel signals S(A) and S(B) of the two photoelectric conversion units are mixed and output from the output unit 106 to the outside of the image sensor 100 . It is possible to reduce the amount of signal transmission to the outside. Note that the calculation of the pixel signal S(A) and the pixel signal S(B) becomes possible at the stage when both the signals of the unit pixels are ready. The previously read pixel signal S(A) is held in a memory, and when the pixel signal S(B) is read and input to the captured image signal processing unit 701, S(A)+S The calculation of (B) is performed and output from the output unit 106 .

Note that the captured image signal processing unit 701 may further mix the signals of the unit pixels 200 or perform an averaging process. For example, in a pixel unit having a general configuration provided with Bayer array color filters of red (R), green (G), and blue (B), the signals of adjacent pixels of the same color are mixed and averaged and output. If it is sent to the unit 106, the amount of signal transmission can be further reduced. In addition, instead of outputting signals of all the pixel portions to the output portion 106, signals of only necessary regions may be output. These processes are controlled by the control unit 104 .

Next, processing of the focus detection signal processing unit 702 will be described. The focus detection signal processing unit 702 calculates and outputs a focus detection signal from the signal output from the reading unit 103 . In order to detect the phase difference, as described above, the pixel signal S(A) and the pixel signal S(B) are required. However, if the pixel signals S(A) and pixel signals S(B) of all the pixels of the pixel unit 401 are output from the output unit 106 to the outside of the image sensor 100, the amount of signal transmission becomes enormous, hindering high-speed readout. Become.

Therefore, arithmetic processing is performed in the focus detection signal processing unit 702 to reduce the signal amount and output from the output unit 106 . For example, the pixel signals S(A) and S(B) are Bayer-added to calculate the luminance value Y, and the luminance signals Y(A) and Y(B) are output. Calculations for focus detection may be performed after converting the signals into Y values. By converting the signals into Y values before output from the image sensor 100, the amount of signal transmission can be reduced to 1/4. . Since a signal in Bayer units is required for calculating the Y value, the pixel signals input to the focus detection signal processing unit 702 are held in the memory until all the signals required for the calculation are obtained. That is, after the signals for the R and G rows are output, the signals for the G and B rows are output. Then, when the signals of the G and B rows are output, the luminance signals Y(A) and Y(B) are sequentially calculated and output from the output unit 106 via the signal line 704 .

Further, correlation calculation may be performed in the focus detection signal processing unit 702 and the calculated value may be output from the signal line 704 to the output unit 106 . Phase difference detection using correlation calculation can be performed by a known technique. If only the correlation calculation value is output, the amount of signal to be output can be greatly reduced, although it depends on the number of regions divided at the time of calculation.

As described above, the image sensor 100 including the captured image signal processing unit 701 and the focus detection signal processing unit 702 performs signal processing to output only necessary signals to the outside of the image sensor 100 . As a result, the amount of signal transmission can be reduced, and both captured image data and focus detection information can be obtained at high speed.

Note that in this embodiment, the captured image signal processing unit 701 and the focus detection signal processing unit 702 are each provided with a memory. However, even if a memory is provided in the front stage of these components and signals are sent to the captured image signal processing unit 701 and the focus detection signal processing unit 702 at the stage when the signals necessary for the calculation in each processing unit are ready. good.

Further, in the description of the present embodiment, the luminance signals Y(A) and Y(B) are output as focus detection signals, but the output unit 106 may output only the luminance signal Y(A). . Specifically, the captured image signal, that is, the pixel signal S(A+B) is output from the captured image signal processing unit 701 . Therefore, after being output to the outside of the image sensor 100, the phase difference detection unit 108 or the like calculates the luminance signal Y(A+B) from the pixel signal S(A+B), performs subtraction processing of the luminance signal Y(A), and obtains the luminance signal. A focus detection signal may be obtained by calculating the signal Y(B). By outputting only the luminance signal Y(A) from the focus detection signal processing unit 702 in this manner, the amount of signal transmission can be further reduced.

For example, if the number of pixels is 20 million, it is necessary to output pixel signals S(A) and S(B) for all pixels, that is, 40 million pieces of data if the signal processing section is not provided. On the other hand, when the Y value is calculated and output as the focus detection signal by the image sensor having the signal processing section of the present embodiment, 20 million pieces of data for the captured image and 20 million/4=5 million pieces of data for focus detection. data is output, and the amount of signal transmission is reduced. As a result, high-speed reading becomes possible. Further, when the focus detection signal is a correlation calculation value, it is clear that the amount of signal transmission is further reduced.

In the timing chart of FIG. 6, it is also possible to simultaneously control the control signals PTXA and PTXB at times t607 and t608 to obtain a signal of the unit pixel 200 in which charges of the photoelectric conversion units 201A and 201B are mixed. be. Specifically, according to the timing chart of FIG. 6, after the signal of the photoelectric conversion unit 201A is read out, the control signals PTXA and PTXB are controlled to be Hi and Lo at the same time, and the signal is read out to obtain the pixel signal S (A+B). ) can be obtained. In this case, reading of the reset signal is reduced by one, so that reading can be performed at a higher speed.

When the pixel signals S(A) and S(A+B) are read out from the pixels, the focus detection signal processing unit 702 performs processing for subtracting the pixel signal S(A) from the pixel signal S(A+B). For example, a pixel signal S(B) can be obtained. Alternatively, the focus detection processing unit 702 may process and output only the pixel signal S(A), and the phase difference detection unit 108 may calculate the pixel signal S(B) or the luminance signal Y(B).

(Second embodiment)
Next, a second embodiment of the invention will be described. In the first embodiment, the focus detection signal is output in the entire area of the pixel unit. However, if the focus detection signal is selected and output only in the necessary area, the speed can be further increased. can do.

FIG. 8 is a diagram showing an example of a focus detection signal output area in a pixel area. The hatched areas output the focus detection signal and the captured image signal, and the other areas output only the captured image signal. For example, as in the example shown in FIG. 8A, focus detection signals are discretely (selectively) output only for a target area in a wide range of pixels. This makes it possible to obtain the focus detection information of the entire pixel area, while suppressing the amount of signal output to the outside of the image sensor 100 . Further, in the case of the example shown in FIG. 8B, it is possible to obtain detailed focus detection information for a partial area, and it is possible to suppress the amount of signal output to the outside of the imaging device 100. . Selection of these output areas is controlled by the control unit 104 . In the focus detection signal processing unit 702, only the signal of the output target area is read from the memory and calculated.

Note that the output of the focus detection signal in a partial area as shown in FIG. 8 may be the Y value signal or the correlation calculation result, but may also be the pixel signal S(A). Although the amount of signal to be transmitted is larger than that of the Y value signal and the result of correlation calculation, since only the required area is output, the amount of signal transmission can be suppressed. Also, it can be realized with a relatively small-scale signal processing circuit.

The focus detection signal processing unit 702 may also perform mixing processing and averaging processing on the focus detection signal. In this case, pixel signals S(A) are mixed and pixel signals S(B) are mixed and averaged.

As described above, the image sensor 100 including the captured image signal processing unit 701 and the focus detection signal processing unit 702 performs signal processing to output only necessary signals to the outside of the image sensor 100 . As a result, the transmission amount of the focus detection signal can be reduced, and both captured image data and focus detection information can be obtained quickly and efficiently.

There is a method of shortening the readout time for one frame by limiting the pixels used for focus detection in this way. Normally, only the row used for focus detection processing outputs the signals of the two photoelectric converters in the unit pixels, and the row not used for focus detection processing mixes the signals of the two photoelectric converters for image generation. An increase in read time is suppressed by outputting only the signal. In this case, individual output signals of the two photoelectric conversion units output for focus detection can be mixed and used as pixel signals for a captured image. However, since the signal readout method and the method of mixing the output signals of the two photoelectric conversion units are different between the row used for focus detection processing and the row not used for focus detection processing, a difference occurs in the noise level, etc., resulting in an image pickup. A problem arises that the image deteriorates. However, by providing the focus detection signal processing unit as in this embodiment, all the signals from the pixel units can be read out at the same readout timing, and the pixels to be output by the focus detection signal processing unit 702 can be selected. Therefore, the noise amount of the pixel signal S(A+B) used for the captured image does not vary from region to region, and a high-quality captured image can be obtained.

(Third embodiment)
Next, a third embodiment of the invention will be described. In the above-described second embodiment, an example has been described in which the focus detection signal processing unit 702 selects and outputs only the signals of the necessary output target area from the focus detection signals. In the present embodiment, this is further advanced, and an example will be described in which the necessary area of the focus detection signal is set based on the user's input or the subject area detected by the subject detection unit.

As shown in FIG. 9, the configuration of the imaging device of this embodiment is obtained by adding a subject detection unit 105a to the configuration of the imaging devices of the first and second embodiments shown in FIG. Since the rest of the configuration is the same as the configuration of FIG. 1, the description of the same parts will be omitted and only the different parts will be described.

In FIG. 9, an object detection unit 105a receives a digital signal output for image generation from the reading unit 103, detects an object using a known pattern recognition processing circuit, and detects a focus detection area for performing focus detection processing. to decide. Examples of subjects to be detected here include the faces and eyes of people and animals. Further, the subject detection unit 105a may include therein a memory for temporarily storing signals for image generation in order to perform subject detection processing.

The signal processing unit 105 has a captured image signal processing unit 701 and a focus detection signal processing unit 702, as already described in the first embodiment with reference to FIG. The operation of the captured image signal processing unit 701 is the same as in the first embodiment.

On the other hand, the focus detection signal processing unit 702 selects a necessary region from the digital signal output for focus detection and outputs it to the output unit 106, as in the second embodiment. However, when the focus detection area is set manually, the focus detection signal processing unit 702 selectively outputs the focus detection signal for the focus detection area arbitrarily selected by the user. Alternatively, when the focus detection area is automatically set, the focus detection signal processing unit 702 receives the object detection result of the object detection unit 105a and selectively outputs the focus detection signal of the area where the object is detected. do. The output unit 106 outputs the image generation digital signal received from the captured image signal processing unit 701 and the focus detection digital signal received from the focus detection signal processing unit 702 to the outside of the image sensor 100 .

The phase difference detection unit 108 receives the focus detection digital signal from the output unit 106 and calculates a phase difference evaluation value for performing phase difference detection type focus detection. In the present embodiment, the focus detection signal input to the phase difference detection unit 108 is a signal output by selecting a region from the focus detection signal processing unit 702 in the signal processing unit 105 inside the image sensor 100. be. Therefore, the focus detection signal transmitted from the output unit 106 of the image sensor 100 to the phase difference detection unit 108 is only the signal necessary for focus detection control, so the transmission band is efficiently used. Also, in the internal processing of the phase difference detection unit 108, there is no need for arithmetic processing for calculating phase difference evaluation values in areas not required for focus detection control and processing for extracting signals ultimately required for focus detection control. Therefore, the processing speed for calculating the phase difference evaluation value by the phase difference detection unit 108 can be increased. Also, the scale of the processing circuit of the phase difference detection unit 108 can be reduced.

Note that the display unit 110 in FIG. 9 is used not only to display the image signal received from the overall control/calculation unit 109, but also to display a focus detection area that can be arbitrarily selected by the user of the imaging apparatus. be. Alternatively, it is also used to display a subject area where the subject detected by the subject detection unit 105a exists. The operation unit 113 is used for various inputs, and is also used for the user of the imaging apparatus to set an arbitrary focus detection area. However, if the display unit 110 is a touch panel, the input operation on the operation unit 113 may be replaced by a touch operation on the display unit 110 .

Next, FIG. 10 is a flow chart showing the processing flow of imaging plane phase difference AF in this embodiment. When the imaging plane phase difference AF process is started, first, in step S401, the pixel unit 401 of the image sensor 100 is driven, and a plurality of PD signals (focus detection signals) are read out respectively. At this time, the drive circuit unit 402 scans the pixel unit 401 by a readout scanning method such as row thinning readout, row addition readout, row partial readout, column thinning readout, column addition readout, and column partial readout according to a required frame rate. You can drive. However, as described above, the driving circuit portion 402 uniformly drives the pixel portion 401 under the same conditions, so that crosstalk and blooming are not likely to occur.

After that, the signal of each PD is AD-converted by the reading unit 103 to obtain a digital signal for focus detection. Furthermore, the reading unit 103 can also generate a signal for image generation by mixing digital signals of a plurality of PDs for each unit pixel 200 .

The focus detection signal and the image generation signal for the entire area of the pixel unit 401 are output from the reading unit 103 to the signal processing unit 105 . A signal for image generation is output from the output unit 106 to the outside of the image sensor 100 via the signal processing unit 105 and processed by the image processing unit 107 . After that, the generated image is displayed on the display unit 110 by the overall control/calculation unit 109 . The moving image is continuously displayed on the display unit 110 by continuously outputting the image generation signal from the imaging device 100 at a predetermined frame rate.

Next, in step S402, the selection mode of the focus detection area is confirmed. Here, if the user has manually set the focus detection area in advance in order to focus the imaging lens 101 on an arbitrary area, the process proceeds to step S403. In this case, FIG. 11 shows an example of an operation screen displayed on the display unit 110 when the user selects the focus detection area. As shown in FIG. 11, a plurality of selectable focus detection areas 1101 are displayed on the operation screen. Here, the detection frame is divided into 7 horizontally and 5 vertically, but the detection frame may be divided more finely or coarsely. In addition, the user may designate an arbitrary position from the entire photographing screen instead of selecting from among predetermined detection frames. The user selects, for example, an area 1102 containing a person's face as the area to be focused on. Positional information of the focus detection area 1102 selected by the user is input from the operation unit 113 to the signal processing unit 105 via the overall control/calculation unit 109 .

Next, in step S403, out of the focus detection signals of the entire area of the pixel unit 401 output from the reading unit 103 to the signal processing unit 105 in step S401, the signal for focus detection of the area specified by the user is processed by the signal processing unit. The focus detection signal processing unit 702 in 105 selects. Then, the focus detection signal selected by the focus detection signal processing unit 702 is input to the phase difference detection unit 108 via the output unit 106 . At this time, in the present embodiment, the focus detection signal transmitted from the output unit 106 to the phase difference detection unit 108 is only the signal necessary for focus detection control, that is, only the focus detection signal for the area specified by the user. Therefore, high-speed transmission is possible.

Next, in step S404, the phase difference detection unit 108 receives the digital signal for focus detection from the output unit 106 and calculates a phase difference evaluation value for performing phase difference detection type focus detection. At this time, the phase difference detection unit 108 does not need arithmetic processing for calculating a phase difference evaluation value in an area unnecessary for focus detection control or processing for extracting a signal finally required for focus detection control. Therefore, focus detection control can be performed at high speed.

Next, in step S<b>405 , the lens control unit 114 calculates optical system drive information based on the phase difference evaluation value calculated by the phase difference detection unit 108 and controls the focus lens position of the imaging lens 101 .

Next, in step S406, it is checked whether the imaging device should end the imaging. When the user inputs an operation to end shooting from the operation unit 113, the shooting ends as it is. If the user does not input an operation to end imaging from the operation unit 113, the process proceeds to step S401 to continue imaging and imaging plane phase difference AF processing.

On the other hand, if it is determined in step S402 that the image capturing apparatus automatically determines the focus area of the photographing lens 101, the process proceeds to step S407. In step S407, the subject detection unit 105a receives the image generation signal from the reading unit 103 and performs subject detection processing for automatically determining the area on which the photographing lens 101 is focused. A subject to be detected is, for example, the face or eyes of a person or an animal. Various known pattern recognition processes can be applied to the subject detection process method. Typical pattern recognition methods include, for example, methods called template matching and deep learning.

FIG. 12A shows an example of a screen displayed on the display unit 110 for indicating the subject area detected by the subject detection unit 105a. As shown in FIG. 12A, a subject area 1201 detected by the subject detection unit 105a as an area containing a person's face is displayed. The subject detection unit 105 a outputs horizontal/vertical address information in the image of the detected subject region 1201 to the signal processing unit 105 . The signal processing unit 105 outputs the horizontal/vertical address information in the image of the object area 1201 to the overall control/calculation unit 109 via the output unit 106 in order to display it as the object area on the display unit 110 . Then, the overall control/calculation unit 109 synthesizes the generated image processed by the image processing unit 107 with the subject region information, and displays the synthesized image on the display unit 110 . Further, although the details will be described later, the signal processing unit 105 uses horizontal and vertical address information in the image of the subject area 1201 to select a focus detection signal.

As another example of subject detection results, FIG. 12B shows a case where a person's face is close-up and framed. In the example shown in FIG. 12B, since the area including the human face is wide, if the focus detection signal in the wide area including the entire face is output to the subsequent phase difference detection unit 108, the communication time is becomes long, which hinders high-speed focus detection control. Therefore, as shown in FIG. 12(b), when a person's face is detected in close-up, the subject detection unit 105a further extracts a characteristic part of the face (such as the eyes), and extracts a subject detection area 1202. preferably. This is realized, for example, by detecting a case where the number of pixels in an area where the subject detection unit 105a detects a face exceeds a predetermined number of pixels, and performing control so as to switch the subject detection target.

Next, in step S408, the detection result of the subject detection unit 105a in step S407 is confirmed. If the subject can be detected in step S407, the process proceeds to step S409.

In step S409, the signal processing unit 105 selects the focus detection signal for the area detected by the subject detection unit 105a from among the focus detection signals read out from the entire area of the pixel unit 401. FIG. A focus detection signal selectively output from the signal processing unit 105 is input to the phase difference detection unit 108 via the output unit 106 . In this case, in the present embodiment, the signal for focus detection transmitted from the output unit 106 to the phase difference detection unit 108 is only the signal in the area necessary for focus detection control, so that high-speed transmission is possible.

Next, in step S404, the phase difference detection unit 108 receives the digital signal for focus detection from the output unit 106 and calculates a phase difference evaluation value for performing phase difference detection type focus detection. In this case, the phase difference detection unit 108 does not need arithmetic processing for calculating a phase difference evaluation value for an area unnecessary for focus detection control or processing for extracting a signal finally required for focus detection control. Therefore, focus detection control can be performed at high speed. After that, the processes already described are executed in steps S405 and S406.

On the other hand, if it is confirmed in step S408 that the subject could not be detected in step S407, the process proceeds to step S410.

In step S410, the lens control unit 114 performs control so that the focus lens of the photographing lens 101 is search-driven by a predetermined amount. Further, in step S411, the focus lens position of the photographing lens 101 is confirmed to determine whether or not the search drive has ended. If search driving is in progress, the process proceeds to step S401. Therefore, the focus detection area is automatically determined in step S402, and if the state in which the subject detection unit 105a cannot detect the subject continues in step S407, search driving of the focus lens is continued. However, when the focus lens position finishes the search drive from the infinite end to the close end in step S411, the process proceeds to step S412.

In step S412, processing is performed when the subject detection unit 105a cannot detect the subject in step S407 even if the lens control unit 114 repeats the search driving operation of the focus lens in step S410. Here, in order to provisionally determine the focus lens position of the imaging lens 101 , the signal processing unit 105 selects the focus detection signal for the provisional area among the focus detection signals read from the entire area of the pixel unit 401 . A focus detection signal selectively output from the signal processing unit 105 is input to the phase difference detection unit 108 via the output unit 106 .

Next, in step S404, the phase difference detection unit 108 receives the digital signal for focus detection from the output unit 106 and calculates a phase difference evaluation value for performing phase difference detection type focus detection. After that, the processes already described are executed in steps S405 and S406.

As described above, in the present embodiment, the signals for focus detection transmitted from the output unit 106 of the image sensor 100 to the phase difference detection unit 108 are only signals necessary for focus detection control. It is possible. Then, the phase difference detection unit 108 does not need arithmetic processing for calculating phase difference evaluation values in areas not required for focus detection control and processing for extracting signals finally required for focus detection control. Therefore, the processing speed for calculating the phase difference evaluation value by the phase difference detection unit 108 can be increased. Therefore, focus detection control by imaging plane phase difference AF can be performed at high speed.

(Fourth embodiment)
Next, a fourth embodiment of the invention will be described. The configuration of the unit pixel 200 of the pixel section 402 is different in the fourth embodiment. FIG. 13 is a view of the light receiving section 102 and the microlens array of the imaging element 100 observed from the optical axis direction (Z direction). Four photoelectric conversion units 901A, 901B, 901C, and 901D are arranged for one microlens 202 . Thus, by having a total of four photoelectric conversion units, two each in the X-axis direction and the Y-axis direction, it is possible to receive the light of the exit pupil areas divided into four. A signal readout method and processing of the signal processing unit 105 in the image sensor 100 including the pixel unit 401 in which each unit pixel includes four photoelectric conversion units will be described.

FIG. 14 is a schematic diagram showing an example of the configuration of the unit pixel 900 and the readout section 103. As shown in FIG. The configuration of FIG. 14 includes a readout circuit corresponding to each photoelectric conversion unit. That is, the pixel signal of the photoelectric conversion unit 901A is output to the readout circuit 1001A. Similarly, the photoelectric conversion unit 901B is output to the readout circuit 1001B, the photoelectric conversion unit 901C is output to the readout circuit 1001C, and the photoelectric conversion unit 901D is output to the readout circuit 1001D. The signal readout operation from the photoelectric conversion unit can be performed in substantially the same manner as the driving method described with reference to FIGS. 5 and 6, so description thereof will be omitted.

Processing of the signal processing unit 105 for signals read from each photoelectric conversion unit will be described. The captured image signal processing unit 701 calculates a captured image signal from the read signal. That is, the pixel signals S(A), S(B), S(C), and S(D) of the plurality of photoelectric conversion units 901A, 901B, 901C, and 901D of the unit pixel 900 are received, mixed processing is performed, and the pixel Calculate the signal S(A+B+C+D). Then, the pixel signal S (A+B+C+D) is sent to the output unit 106 via the captured image signal output line 703 . In the captured image signal processing unit 701, signals from a plurality of photoelectric conversion units are mixed and output from the output unit 106 to the outside of the image sensor 100, thereby reducing the amount of signal transmission to the outside of the image sensor 100. can. The effect is even greater in an imaging device that includes four photoelectric conversion units, such as the unit pixel 900 .

Next, processing of the focus detection signal processing unit 702 will be described. In the case of an image pickup device having four photoelectric conversion units per unit pixel 900 as shown in FIG. , which is not desirable for high-speed reading. As described in the first embodiment, it is preferable to calculate and output the Y value or to output only the correlation calculation result. Signal transmission amount can be further reduced by outputting a signal only for a required area.

Further, in the case of a configuration in which the unit pixel 900 includes 2×2 photoelectric conversion portions as shown in FIG. 13, phase difference detection can be performed not only in the horizontal direction but also in the vertical direction. For example, the pixel signals S(A) and S(C) are mixed, and the pixel signals S(B) and S(D) are mixed and output. In this case, the obtained signal for focus detection enables phase-difference focus detection in which the pupil is divided in the horizontal direction. Further, when the pixel signals S(A) and S(B) are mixed and the pixel signals S(C) and S(D) are mixed and output, the obtained focus detection signals are divided into upper and lower It is possible to perform phase-difference focus detection in which the pupil is split in different directions. By switching and outputting these according to the object, it is possible to accurately perform focus detection for objects with vertical stripes and horizontal stripes. Also, the output pattern may be changed depending on the pixel area.

Furthermore, in the focus detection signal processing unit 702, correlation calculation may be performed using the obtained mixed signals (for example, S(A+C) and S(B+D)), and only the result thereof may be output. If the focus detection signal processing unit 702 performs correlation calculation processing, the amount of signal transmission output from the image sensor 100 can be reduced. In addition, the amount of signal transmission can be suppressed even if correlation calculations are performed in both the horizontal direction and the vertical direction in the same area and output.

In this way, in an image sensor having multi-segmented pixels, by providing the focus detection signal processing unit 702 in the image sensor, an increase in the amount of signal output from the image sensor is suppressed, and captured image data and focus can be detected at high speed. Detection information can be obtained. Furthermore, since phase difference information in the horizontal direction and the vertical direction can be obtained, focus detection can be performed with high accuracy. In this embodiment, an image pickup device having four photoelectric conversion units per unit pixel has been described as an example, but a configuration having more photoelectric conversion units may be used. In order to provide more parallax, only necessary PD signals may be output, or signals may be mixed and output in oblique directions.

(Fifth embodiment)
Since the signal processing unit 105 of the image sensor 100 as described in the first to third embodiments is a large-scale circuit, the image sensor 100 as a whole is likely to have a large area. Therefore, in the present embodiment, the configuration of the imaging device 100 that suppresses the increase in area will be described.

15 and 16 are configuration diagrams of the imaging device 100 according to the fifth embodiment. The imaging device of this embodiment has a structure (multilayer structure) in which a pixel region chip 1501 and a signal processing chip 1502 are stacked. Wirings between the semiconductor chips are electrically connected using microbumps or the like by a known substrate lamination technique.

The pixel region chip 1501 includes a pixel section 401 in which unit pixels 200 each having a plurality of photoelectric conversion sections are arranged in a matrix, a driving circuit section 402 and a reading section 103 . The driver circuit portion 402 sends driving signals to the pixels of the pixel portion 401 . Although the unit pixel 200 has two photoelectric conversion units in FIG. 15, the number of photoelectric conversion units is not limited to this.

The readout unit 103 includes a large number of readout circuits 509 , for example, one readout circuit 509 per pixel column, and reads out pixel signals from the pixel unit 401 . The read pixel signals are selected vertically or horizontally under the control of the driving circuit unit 402 and are sequentially transferred to the signal processing unit 105 .

The signal processing chip 1502 has a control section 104 , a signal processing section 105 and an output section 106 . The signal processing unit 105 has a captured image signal processing unit 701 and a focus detection signal processing unit 702 , processes the pixel signals read from the reading unit 103 , and outputs them to the image sensor 100 via the output unit 106 . Output externally. Since the signal processing in the signal processing unit 105 is the same as the processing described in the first to third embodiments, description thereof is omitted. As shown in FIG. 16, the image pickup device 100 has a configuration in which a pixel region chip 1501 and a signal processing chip 1502 are stacked together and integrated.

As described above, since the imaging device has a laminated structure, a sufficient area can be secured for the signal processing unit 105, and a large-scale circuit can be mounted. The signal processing unit 105 performs signal processing to output only necessary signals to the outside of the image pickup device 100, thereby reducing the amount of signal transmission and enabling both captured image data and focus detection information to be obtained at high speed. Become.

(Sixth embodiment)
In an image sensor configured from a pixel portion having a plurality of photoelectric conversion units per unit pixel as described in the first to third embodiments, it is preferable that the number of readout circuits 509 is large. For example, in a configuration in which one readout circuit is provided for a unit pixel, or in a configuration in which one readout circuit is provided for one photoelectric conversion unit, pixel signals are output simultaneously for all pixels, and A Since /D conversion can be performed, higher speed reading becomes possible. In this case, an area is required for arranging the readout circuit, and it is desirable that the image sensor has a laminated structure.

FIG. 17 is a diagram showing the configuration of an imaging device 100 according to the sixth embodiment. The imaging device of this embodiment has a structure in which a pixel region chip 1301, a readout circuit chip 1302, and a signal processing chip 1303 are stacked. Wirings between the semiconductor chips are electrically connected using microbumps or the like by a known substrate lamination technique.

A pixel region chip 1301 includes a pixel section 401 in which unit pixels 200 each having a plurality of photoelectric conversion sections are arranged in a matrix, and a driver circuit section 402 . The driver circuit portion 402 sends driving signals to the pixels of the pixel portion 401 . Although the unit pixel 200 includes two photoelectric conversion units in FIG. 17, the number of photoelectric conversion units is not limited to two.

The readout circuit chip 1302 includes a readout section 103 , a vertical selection circuit 1304 and a horizontal selection circuit 1305 . The readout unit 103 has a large number of readout circuits 509 corresponding to each unit pixel or photoelectric conversion unit, and outputs pixel signals of the pixel unit 401 . The pixel signals output to the readout circuit 509 are sequentially transferred to the signal processing unit 105 under the control of the vertical selection circuit 1304 and the horizontal selection circuit 1305 .

The signal processing chip 1303 has a control section 104 , a signal processing section 105 and an output section 106 . The signal processing unit 105 has a captured image signal processing unit 701 and a focus detection signal processing unit 702. The signal processing unit 105 processes the pixel signals read by the reading unit 103, and outputs them via the output unit 106. and output to the outside of the image sensor 100 . Since the signal processing in the signal processing unit 105 is the same as the processing described in the first to third embodiments, description thereof will be omitted.

The imaging device 100 has a configuration in which a pixel region chip 1301, a readout circuit chip 1302, and a signal processing chip 1303 are laminated and integrated. By the way, as in the configuration of this embodiment, for example, in the case of a configuration in which one readout circuit is provided for one photoelectric conversion unit, the time in which pixel signals are output to the signal processing unit 105 is dramatically shortened. For example, if the time required to output a signal from one photoelectric conversion unit is α, in the case of an imaging device having one readout circuit per column, it takes αx number of rows to read out pixel signals for one frame. It takes. On the other hand, when one readout circuit is provided for one photoelectric conversion unit, pixel signals for one frame can be read out in the time α. However, in this case, there is concern that the reception of pixel signals and signal processing in the signal processing unit 105 may be rate-determining. However, since the signal processing unit 105 of the image pickup device 100 having a laminated structure can be provided with a large area as in this embodiment, a large number of signal transmission lines from the readout circuit chip 1302 can be provided, and high-speed processing can be performed. The pixel signal can be sent to the signal processing unit 105 at the same time. In addition, since a large number of signal processing circuits can be mounted in the signal processing unit 105, parallel processing becomes possible and the signal processing time is increased.

As described above, since the image sensor has a laminated structure, a sufficient area can be secured for the readout section and the signal processing section. The image sensor of this embodiment can read out signals from the pixel unit at high speed, and the signal processing unit 105 performs signal processing to output only necessary signals to the outside of the image sensor, thereby reducing the amount of signal transmission. Both captured image data and focus detection information can be obtained at high speed.

(Seventh embodiment)
FIG. 18 is a block diagram showing the configuration of an imaging device according to the seventh embodiment of the present invention. In FIG. 18, the same reference numerals are given to the same components as in the third embodiment, and detailed description thereof will be omitted.

The seventh embodiment is composed of the same components as the third embodiment when viewed as an imaging apparatus as a whole, but the components included inside the image sensor 800 are different from the third embodiment. The image sensor 800 of the third embodiment includes a light receiving unit 801, a readout unit 802, a signal processing unit 803, an image processing unit 804, a subject detection unit 805, a phase difference detection unit 806, an output unit 807, and a control unit 104. It is configured.

A light receiving unit 801 receives an optical image formed by the imaging lens 101 . In the light-receiving unit 801, focus detection pixels with a plurality of PDs are arranged under one microlens so as to receive the light beams that have passed through the divided exit pupil areas of the photographing lens 101. A reading unit 802 performs analog-digital signal processing by an A/D conversion circuit and adjustment of a reference level (clamp processing).

The signal processing unit 803 receives the digital signal output from the reading unit 103 and outputs the signal to the phase difference detection unit 806 and the image processing unit 804 which will be described later. At this time, the signal processing unit 803 receives the subject detection result of the subject detection unit 805 and selectively outputs a signal for focus detection of the subject detection area.

The image processing unit 804 performs image processing such as correction of defective pixels, noise reduction, color conversion, white balance correction, gamma correction, resolution conversion processing, and image compression on the signal for image generation received from the signal processing unit 803 . processing, etc. A subject detection unit 805 receives a digital signal output from the image processing unit 804 and determines a signal region for focus detection processing.

A phase difference detection unit 806 calculates a phase difference evaluation value for performing phase difference detection type focus detection on the signal for focus detection received from the signal processing unit 803 . The output unit 807 outputs the phase difference evaluation value and the digital signal for image generation received from the phase difference detection unit 806 and the image processing unit 804 to the outside of the image sensor 800 . In this manner, the image pickup device of the seventh embodiment of the present invention also provides the same effects as the third embodiment.

By the way, in the imaging plane phase difference AF, one phase difference evaluation value is calculated by performing a correlation operation on signals for focus detection obtained from a plurality of pixels×a plurality of PDs. In the third embodiment, the output unit 106 outputs a plurality of focus detection digital signals necessary for calculating the phase difference evaluation value to the outside of the image sensor 100 . On the other hand, in the case of the seventh embodiment, the output unit 807 outputs the phase difference evaluation value calculated by the phase difference detection unit 806 to the outside of the image sensor 800 . In other words, the amount of signal output to the outside of the imaging device 800 is even smaller in the seventh embodiment than in the third embodiment. Therefore, the output unit 807 of the image sensor 800 can transmit the signal necessary for focus detection control in a shorter time than in the third embodiment, and the focus detection control by the imaging plane phase difference AF can be performed at high speed. be able to.

Further, in the image pickup device 800 of the seventh embodiment, the subject detection unit 805 uses the digital signal processed by the image processing unit 804 to perform subject detection. Therefore, it is possible to perform subject detection processing using a signal subjected to defective pixel correction and noise reduction, and further using color information. Therefore, subject detection in the seventh embodiment can be performed with higher precision than in the third embodiment.

It should be noted that the imaging device 800 of the seventh embodiment includes more components than the imaging device 100 of the third embodiment. Therefore, it is preferable to use a stacked imaging device as shown in FIG.

(Eighth embodiment)
In the first to third embodiments, it has been explained that high-speed imaging plane phase difference AF is possible by selectively outputting/transmitting signals for focus detection by the image sensor. However, the scope of application of the present invention is not limited to imaging plane phase difference AF, and can also be applied to automatic exposure control (imaging plane AE) using a signal from an imaging device. Specifically, in the image pickup plane AE, the image pickup apparatus automatically determines and controls the aperture of the photographing lens, the accumulation time of the image pickup device, the sensitivity (gain) of the image pickup device, and the like. The third embodiment of the present invention aims at speeding up the imaging plane AE by selectively outputting/transmitting signals used for the imaging plane AE by the imaging device.

FIG. 19 is a block diagram showing the configuration of an imaging device according to the eighth embodiment of the present invention. In FIG. 19, the same symbols are attached to the same components as in the third embodiment, and detailed description thereof is omitted. This embodiment differs from the third embodiment in that the imaging device includes a photometry unit 820 .

The signal processing unit 105 receives the digital signal output for image generation from the reading unit 103 and outputs a signal to the output unit 106 . The signal processing unit 105 selects a digital signal for image generation to be used for the imaging plane AE according to the photometric method set in the imaging apparatus. At this time, for example, in the case of the so-called "spot metering method" or "partial metering method", the signal processing unit 105 selects a signal of a partial area in the center of the screen. In the case of the so-called “evaluative photometry method”, the signal processing unit 105 selects the signal of the entire screen. However, instead of selecting signals from all pixels in the entire screen, it is possible to select signals by thinning out rows or columns within a range in which the amount of signal necessary for calculating the photometry evaluation value is obtained. is preferred for speeding up Further, as in the third embodiment, the image generation signal of the focus detection area arbitrarily selected by the user or the subject area detected by the subject detection unit 105a may be selected.

A photometry unit 820 receives a digital signal for image generation from the output unit 106 and calculates a photometry evaluation value for performing imaging plane AE. A digital signal for image generation input to the photometry unit 820 is a signal of an area selected by the signal processing unit 105 inside the image sensor 100 . Therefore, the image generation signal for the imaging surface AE that is transmitted from the output unit 106 of the image sensor 100 to the photometry unit 820 is only the signal necessary for exposure control, so the communication band is efficiently used. . Also, in the internal processing of the photometry unit 820, there is no need for calculation processing for photometric evaluation value calculation of finally unnecessary areas on the imaging plane AE and processing for extracting finally necessary signals. Therefore, the processing speed for calculating the photometric evaluation value by the photometry unit 820 can be increased.

The lens control unit 114 receives the output of the photometry evaluation value from the photometry unit 820, and drives the aperture of the photographing lens 101 based on the photometry evaluation value. Furthermore, the overall control/calculation unit 109 drives the image sensor 100 based on the photometric evaluation value, and controls the accumulation time and sensitivity (gain) of the image sensor 100 .

As described above, according to the eighth embodiment of the present invention, the signals for image generation transmitted from the output unit 106 of the image sensor 100 to the photometry unit 820 are only the signals necessary for the imaging surface AE. , high-speed transmission is possible. Then, the photometry unit 820 does not need arithmetic processing for calculating photometric evaluation values that are ultimately unnecessary for the imaging plane AE, and processing for extracting signals that are ultimately required. Therefore, the imaging surface AE can be speeded up.

In addition, in the pixels of the imaging device 100 of the eighth embodiment, it is not always necessary to arrange a plurality of PDs for each pixel as described with reference to FIG. In order to calculate the photometry evaluation value, it is sufficient to have an image generation signal (that is, a signal obtained by mixing signals of a plurality of PDs under one microlens for each pixel). A configuration with one PD may also be used.

Further, in the eighth embodiment, the configuration in which the imaging surface AE is applied based on the configuration of the third embodiment has been described, but in the same way, the imaging surface AE is applied based on the configuration of the seventh embodiment. can also In this case, the image sensor 800 is configured by replacing the phase difference detection unit 806 in FIG. 18 with a photometry unit.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

100: image sensor, 101: photographing lens, 102: light receiving unit, 103: readout unit, 104: control unit, 105: signal processing unit, 106: output unit, 107: image processing unit, 108: phase difference detection unit, 109 : Overall control/calculation part

Claims (14)

a pixel portion in which a plurality of unit pixels each having one microlens and a plurality of photoelectric conversion portions are arranged in a matrix;
obtained by mixing a signal corresponding to charges accumulated in some of the plurality of photoelectric conversion units in each of the unit pixels and charges accumulated in all of the plurality of photoelectric conversion units; a signal holding unit that holds a signal corresponding to the charged charge;
A signal corresponding to charges accumulated in some of the plurality of photoelectric conversion units in each of the plurality of unit pixels held in the signal holding unit and all of the plurality of photoelectric conversion units an output unit that outputs a signal based on a signal corresponding to the charge obtained by mixing the accumulated charges;
with
The pixel portion is formed on a first semiconductor substrate, and the signal holding portion is formed on a second semiconductor substrate different from the first semiconductor substrate,
An imaging device, wherein the first semiconductor substrate and the second semiconductor substrate are stacked on each other. The output unit selects a part of the signals of the plurality of photoelectric conversion units in each of the plurality of unit pixels arranged in a part of the pixel region from the signal of the pixel unit held by the signal holding unit. 2. The imaging device according to claim 1, wherein a signal based on a signal corresponding to charges accumulated in the photoelectric conversion portion is output. The output section mixes charges accumulated in all of the plurality of photoelectric conversion sections in each of the plurality of unit pixels arranged over the entire pixel region from the signal of the pixel section held by the signal holding section. 3. The imaging device according to claim 2, wherein a signal based on a signal corresponding to the charge obtained by the above is outputted. 4. The imaging device according to claim 2, wherein a part of said pixel area is a subject area. 4. The imaging device according to claim 2, wherein the part of the pixel area is an area selected by a user. 6. The imaging device according to any one of claims 1 to 5, wherein the signal holding section has a plurality of holding circuits that hold the signals read out from the pixel section for each of the unit pixels. 6. The imaging device according to claim 1, wherein the signal holding section has a plurality of holding circuits that hold the signals read out from the pixel section for each column. An imaging device, image processing means for processing a signal output from the imaging device to generate an image, and focus detection means for processing the output signal to perform focus detection,
The imaging element is
a pixel portion in which a plurality of unit pixels each having one microlens and a plurality of photoelectric conversion portions are arranged in a matrix;
obtained by mixing a signal corresponding to charges accumulated in some of the plurality of photoelectric conversion units in each of the unit pixels and charges accumulated in all of the plurality of photoelectric conversion units; a signal holding unit that holds a signal corresponding to the charged charge;
A signal corresponding to charges accumulated in some of the plurality of photoelectric conversion units in each of the plurality of unit pixels held in the signal holding unit and all of the plurality of photoelectric conversion units an output unit that outputs a signal based on a signal corresponding to the charge obtained by mixing the accumulated charges;
has
The pixel portion is formed on a first semiconductor substrate, and the signal holding portion is formed on a second semiconductor substrate different from the first semiconductor substrate,
An image pickup device, wherein the first semiconductor substrate and the second semiconductor substrate are stacked on each other. The output unit selects a part of the signals of the plurality of photoelectric conversion units in each of the plurality of unit pixels arranged in a part of the pixel region from the signal of the pixel unit held by the signal holding unit. 9. The imaging device according to claim 8, wherein a signal based on a signal corresponding to charges accumulated in the photoelectric conversion unit is output. The output section mixes charges accumulated in all of the plurality of photoelectric conversion sections in each of the plurality of unit pixels arranged over the entire pixel region from the signal of the pixel section held by the signal holding section. 10. The image pickup apparatus according to claim 9, wherein a signal based on a signal corresponding to the charge obtained by the above is outputted. 11. The imaging apparatus according to claim 9, wherein a part of said pixel area is a subject area. 11. The imaging device according to claim 9, wherein the part of the pixel area is an area selected by a user. 13. The imaging apparatus according to claim 8, wherein the signal holding section has a plurality of holding circuits that hold the signals read out from the pixel section for each of the unit pixels. 13. The imaging apparatus according to claim 8, wherein the signal holding section has a plurality of holding circuits that hold the signals read out from the pixel section for each column.

Images