JP6789707B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an image pickup apparatus and a control method thereof.

Recently, an image pickup device having an image pickup device including a pixel array including a plurality of unit pixels including a plurality of photoelectric conversion units and a plurality of microlenses provided for each of the plurality of unit pixels has been proposed. There is. In such an imaging device, it is possible to obtain image signals corresponding to the luminous fluxes that have passed through different pupil regions of the exit pupils of the imaging optical system, and focus detection can be performed using these image signals (Patent Documents). 1).

Japanese Unexamined Patent Publication No. 2001-124984

However, in the conventional technique, if a defect is present in a part of the pixel array, the focus detection may not be performed well.

An object of the present invention is to provide an image pickup apparatus capable of performing focus detection satisfactorily even when a part of the pixel array is defective, and a control method thereof.

According to one aspect of the embodiment, the image sensor includes a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix. An image sensor that generates a first signal corresponding to the charge generated in the photoelectric conversion unit and a second signal corresponding to at least the charge generated in the second photoelectric conversion unit, and positions in a plurality of rows, respectively. A first image signal is generated by synthesizing a plurality of the first signals from the unit pixels, and a plurality of the second signals from the unit pixels located in the plurality of rows are combined. When the driving means for driving the image sensor and the defective row, which is the row containing the defect, is not included in the plurality of rows so that the second image signal is generated. The drive of the first aspect of driving the image sensor so that the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of rows. When the defective row is included in the plurality of rows, the first signal from the unit pixel located in a row other than the defective row among the plurality of rows is used. When the driving means for driving the second aspect of driving the image sensor so that the first image signal is generated and the first image signal are generated by the driving of the second aspect. may have a normalized means for performing normalization processing on at least one of said first image signal and the second image signal, the first row of the plurality of rows The first signal and the second signal obtained by the unit pixel located are output from the pixel array via the first signal line, and the first row among the plurality of rows is The first signal and the second signal obtained by the unit pixels located in different second rows are output from the pixel array via a second signal line different from the first signal line. From the first signal output from the pixel array via the first signal line and the first signal output from the pixel array via the second signal line, the first signal. The second signal output from the pixel array via the first signal line and the second signal output from the pixel array via the second signal line are combined. the synthesizing means for synthesizing the second image signal from the signal, the image pickup device is provided an imaging apparatus characterized by further closed.

According to another aspect of the embodiment, the image sensor includes a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix. An image sensor that generates a first signal corresponding to the electric charge generated in the photoelectric conversion unit 1 and a second signal corresponding to at least the electric charge generated in the second photoelectric conversion unit, and a plurality of rows, respectively. A first image signal is generated by synthesizing a plurality of the first signals from the located unit pixels in a signal line, and a plurality of the second signals from the unit pixels located in the plurality of rows are generated. A defective line, which is a driving means for driving the image pickup device and is a line containing a defect, is included in the plurality of lines so that a second image signal is generated by synthesizing the signals on the signal line. When not included, the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of rows in the signal line. When the defect row is included in the plurality of rows by driving the first aspect of driving the image sensor, the said row located in a row other than the defect row among the plurality of rows. Provided is an image pickup apparatus comprising: a driving means for driving a second aspect of driving the image pickup element so that the first image signal is generated by the first signal from a unit pixel. Will be done.

According to the present invention, it is possible to provide an image pickup apparatus capable of performing focus detection satisfactorily even when a part of the pixel array is defective.

It is a block diagram which shows the structure of the image pickup apparatus according to 1st Embodiment. It is a flowchart which shows the operation of the image pickup apparatus by 1st Embodiment. It is a block diagram which shows the structure of the image sensor according to 1st Embodiment. It is a figure which shows the photographing optical system and a unit pixel. It is a circuit diagram which shows a part of the image pickup element by 1st Embodiment. It is a time chart which shows the operation of the image pickup apparatus by 1st Embodiment. It is a time chart which shows the operation of the image pickup apparatus by 1st Embodiment. It is a time chart which shows the operation of the image pickup apparatus by 1st Embodiment. It is a time chart which shows the operation of the image pickup apparatus by 1st Embodiment. It is a block diagram which shows the structure of the image pickup apparatus according to 2nd Embodiment. It is a flowchart which shows the operation of the image pickup apparatus by 2nd Embodiment. It is a time chart which shows the operation of the image pickup apparatus by 2nd Embodiment. It is a block diagram which shows the structure of the image pickup apparatus according to 3rd Embodiment. It is a flowchart which shows the operation of the image pickup apparatus according to 3rd Embodiment. It is a circuit diagram which shows a part of the image pickup element by 3rd Embodiment. It is a time chart which shows the operation of the image pickup apparatus according to 3rd Embodiment. It is a time chart which shows the operation of the image pickup apparatus according to 3rd Embodiment. It is a block diagram which shows the structure of the image sensor according to 3rd Embodiment.

Embodiments of the present invention will be described in detail below with reference to the drawings. The present invention is not limited to the following embodiments, and can be appropriately modified without departing from the gist thereof.

[First Embodiment]
The image pickup apparatus according to the first embodiment and the control method thereof will be described with reference to FIGS. 1 to 9. FIG. 1 is a block diagram showing the configuration of the image pickup apparatus 100 according to the first embodiment.

The photographing lens (lens unit) 111 is for forming an optical image of a subject, that is, a subject image, on an image pickup device (imaging means, imaging unit) 101. As will be described later, a pixel array 302 (see FIG. 3) in which a plurality of unit pixels 403 (see FIG. 5) are formed in a matrix is provided in a light receiving portion (imaging surface, light receiving region) of the image sensor 101 (not shown). Has been done. A microlens 404 (see FIG. 4B) is provided on each unit pixel 403. The image pickup optical system composed of the photographing lens 111 and the micro lens 404 forms a subject image on the light receiving portion of the image pickup element 101. The image sensor 101 converts a subject image formed by the image pickup optical system into an electric signal by photoelectric conversion, and outputs an image signal. The lens driving means 112 performs zoom control, focus control, aperture control, and the like on the photographing lens 111 based on the control signal from the control means 110.

The drive means (timing generator) 102 outputs a drive signal for driving the image sensor 101 to the image sensor 101. The defect information storage means (storage means) 103 records information about defects occurring in the image sensor 101, that is, defect information. More specifically, information indicating a defective row, which is a defective row, is stored in the defective information storage means 103. Here, the A image signal described later cannot be normally acquired from the unit pixel 403 located in the defective row, but the A + B image signal described later cannot be normally acquired from the unit pixel 403 located in the defective row. The case where it is possible will be described as an example. The defective line will be described in detail later. As the defect information storage means 103, for example, a ROM (Read Only Memory) or the like is used. The defect information storage means 103 provides such defect information to the driving means 102.

The separation means (image signal separation means) 104 uses the image signal output from the image sensor 101 as an image signal for recording, which is an image signal used for recording, and an image signal for distance measurement, which is an image signal used for distance measurement. Separate into and. The normalization means 105 normalizes the distance measuring image signal output from the separation means 104, if necessary. Similar to the defect information storage means 103, the defect information storage means 106 records information about defects occurring in the image sensor 101, that is, defect information. As the defect information storage means 106, for example, a ROM or the like is used. The defect information storage means 106 provides defect information to the control means 110. The generation means (image signal generation means) 107 generates an image signal based on the image signal output from the normalization means 105. Specifically, the generation means 107 corresponds to the luminous flux passing through the different pupil regions 408 and 409 (see FIG. 4 (b)) of the exit pupil 405 of the imaging optical system (see FIG. 4 (b)), respectively. A plurality of image signals, that is, an A image signal and a B image signal, which will be described later, are output. The image signal storage means 108 temporarily stores the image signal output from the normalization means 105. As the image signal storage means 108, for example, a RAM (Random Access Memory) or the like is used. When the generation means 107 generates an image signal corresponding to the light flux passing through the different pupil regions 408 and 409 of the exit pupil 405, the image signal from the normalization means 105 and the image signal storage means 108 are temporarily generated. The image signal stored in is used. The distance measuring means 109 performs a distance measuring calculation for calculating the distance to the subject based on a plurality of image signals generated by the generating means 107, and generates distance information for focus control.

The control means 110 controls the entire image pickup apparatus 100 according to the present embodiment. The control means 110 drives the image sensor 101 via the drive means 102. The control means 110 outputs a control signal for separating the image signal output from the image sensor 101 into a recording image signal and a distance measuring image signal to the separation means 104. Further, the control means 110 outputs a control signal for executing the normalization process to the normalization means 105 based on the defect information provided by the image signal storage means 108. The control means 110 drives the photographing lens 111 via the lens driving means 112 based on the distance information acquired by the distance measuring means 109. In this way, focus control is performed.

FIG. 3 is a block diagram showing the configuration of the image sensor 101 according to the present embodiment. The light receiving portion of the image sensor 101 is provided with a pixel array 302 in which a plurality of unit pixels 403 (see FIG. 4) are arranged two-dimensionally, that is, in a matrix. The vertical scanning unit 301 scans the pixel array 302 row by row. The vertical scanning unit 301 outputs a control signal for reading an image signal from the unit pixel 403 based on the drive signal from the drive means 102. An image signal is read from the unit pixel 403 based on the control signal output from the vertical scanning unit 301. The AD conversion unit (AD converter) 303 is provided in each row. The AD conversion unit 303 converts analog signals sequentially output from each unit pixel 403 via output signal lines 508 and 509 (see FIG. 5) into digital signals. The AD conversion unit 303 includes a comparator (voltage comparator) and a counter. An analog image signal from the unit pixel 403 is input to one input terminal of the comparator, and a lamp signal is input to the other input terminal of the comparator. A digital image signal (pixel signal) is obtained based on the counter value when the output of the comparator is inverted.

The line memory 304 temporarily stores the digital image signal output from the AD conversion unit 303. The addition unit 305 performs addition processing (composite processing) on the image signal stored in the line memory 304. The horizontal scanning unit 306 sequentially reads out the image signals obtained by performing the addition processing in the addition unit 305. In this way, the image signal is read from the pixel array 302 line by line by the horizontal scanning unit 306. The horizontal scanning unit 306 sequentially outputs an image signal read from the pixel array 302 in units of rows. The output unit 307 outputs the image signal output from the horizontal scanning unit 306 to the outside of the image sensor 101 by using, for example, a low voltage differential signaling (LVDS) technique or the like. By using the low voltage differential transmission technology, the power consumption of the image pickup apparatus 100 can be suppressed.

FIG. 4 is a diagram showing an imaging optical system and a unit pixel. FIG. 4A is a plan view showing the unit pixel 403. FIG. 4A shows an extracted unit pixel 403 of a plurality of unit pixels 403 provided in the pixel array 302. FIG. 4B is a side view showing the relationship between the exit pupil 405 of the photographing optical system and the unit pixel 403. As shown in FIG. 4B, a color filter 406 and a microlens 404 are provided on each unit pixel 403. A photoelectric conversion unit 401 and a photoelectric conversion unit 402 are formed in each unit pixel 403. That is, each unit pixel 403 has a divided pixel 410 having a photoelectric conversion unit 401 and a divided pixel 411 having a photoelectric conversion unit 402. The photoelectric conversion units 401 and 402 convert incident light into electric charges, and are each composed of, for example, a photodiode. Here, a case where two photoelectric conversion units 401 and 402 are provided in each unit pixel 403 will be described as an example, but more than two photoelectric conversion units are provided in each unit pixel 403, respectively. It may have been.

The center of the luminous flux passing through the exit pupil 405 of the imaging optical system coincides with the optical axis 407. The luminous flux that has passed through the exit pupil 405 is incident on the unit pixel 403 about the optical axis 407. The exit pupils 405 of the imaging optical system have different pupil regions 408 and 409. The luminous flux passing through the pupil region 408 is received by the photoelectric conversion unit 401 via the microlens 404. On the other hand, the luminous flux passing through the pupil region 409 is received by the photoelectric conversion unit 402 via the microlens 404. In this way, the photoelectric conversion units 401 and 402 receive light from the separate pupil regions 408 and 409 of the exit pupil 405 of the imaging optical system, respectively. Therefore, by comparing the image signals obtained by the photoelectric conversion units 401 and 402, it is possible to perform the focus detection by the phase difference method.

FIG. 5 is a circuit diagram showing a part of the image pickup device 101 according to the present embodiment. In FIG. 5, four unit pixels 403 out of a plurality of unit pixels 403 provided in the pixel array 302 are extracted and shown. Specifically, the unit pixels 403 located in the 0th row Line0, the 1st row Line1, the 2nd row Line2, and the 3rd row Line3 are shown in FIG. These four unit pixels 403 extracted and shown are located in the same row in the pixel array 302. Since all of these four unit pixels 403 have the same structure, the same reference numerals are appropriately used.

The photoelectric conversion units 401 and 402 photoelectrically convert the incident light (optical image) and generate an electric charge according to the exposure amount. The floating diffusion region (FD unit) 503 is an storage region for temporarily storing the electric charges generated in the photoelectric conversion units 401 and 402. By appropriately applying the transfer pulses pTX0_A, pTX1_A, pTX2_A, and pTX3_A to the gates of the respective transfer switches 501, the charges generated in the photoelectric conversion unit 401 are appropriately transferred to the FD unit 503. By appropriately applying the transfer pulses pTX0_B, pTX1_B, pTX2_B, and pTX3_B to the gates of the respective transfer switches 502, the charges generated in the photoelectric conversion unit 402 are appropriately transferred to the FD unit 503. In the present specification, the optical image incident on the photoelectric conversion unit 401 is referred to as an A image, and the image signal corresponding to the electric charge generated in the photoelectric conversion unit 401 is referred to as an A image signal. Further, in the present specification, the optical image incident on the photoelectric conversion unit 402 is referred to as a B image, and the image signal corresponding to the electric charge generated in the photoelectric conversion unit 402 is referred to as a B image signal.

The drain of the reset switch 504 is connected to the power supply line 506, and the source of the reset switch 504 is connected to the FD unit 503. By appropriately applying the reset pulses pRES0, pRES1, pRES2, and pRES3 to the gates of the respective reset switches 504, the electric charge accumulated in the FD unit 503 is appropriately removed. When resetting the charges of the photoelectric conversion units 401 and 402, the photoelectric conversion units 401 and 402 are reset via the FD unit 503 by turning on both the transfer switches 501 and 502 and the reset switch 504. To do. The FD unit 503 is connected to the gate of the amplification transistor 505, and the potential of the gate of the amplification transistor 505 is a potential corresponding to the amount of electric charge transferred from the photoelectric conversion units 401 and 402. The drain of the amplification transistor 505 is connected to the power supply line 506. The source of the amplification transistor 505 is connected to a current source (not shown) via the selection switch 507 and the output signal lines 508 and 509. The source follower circuit is composed of the amplification transistor 505 and the current source. By appropriately applying the selection pulses pSEL0, pSEL1, pSEL2, and pSEL3 to the gate of the selection switch 507, image signals corresponding to the gate potential of the amplification transistor 505 are output via the output signal lines 508 and 509, respectively.

The gate of the transfer switch 501 is connected line by line to the signal lines that supply the transfer pulses pTX0_A, pTX1_A, pTX2_A, and pTX3_A, respectively. The gate of the transfer switch 502 is connected line by line to the signal lines that supply the transfer pulses pTX0_B, pTX1_B, pTX2_B, and pTX3_B, respectively. The gate of the reset switch 504 is connected line by line to the signal lines that supply the reset pulses pRES0, pRES1, pRES2, and pRES3, respectively. The gate of the selection switch 507 is connected line by line to the signal lines that supply the selection pulses pSEL0, pSEL1, pSEL2, and pSEL3, respectively. Each of these signal lines is connected to the vertical scanning unit 301, and these pulse signals are supplied from the vertical scanning unit 301 in units of lines. Therefore, the transfer switches 501 and 502, the reset switch 504, and the selection switch 507 are selectively scanned row by line by the vertical scanning unit 301. In this way, the image signal acquired by each unit pixel 403 is output via the output signal lines 508 and 509. The image signal output from the unit pixel 403 located in the 0th line Line 0 and the 1st line Line 1 is output via the output signal line 508. On the other hand, the image signal output from the unit pixel 403 located in the second line Line 2 and the third line Line 3 is output via the output signal line 509.

When reading an image signal from the pixel array 302, scanning is performed by the vertical scanning unit 301, and output signals from the plurality of unit pixels 403 are sequentially read line by line. The electric signal output from each unit pixel 403, that is, the analog image signal is output from the pixel array 302 via the output signal lines 508 and 509. The image signals output from the pixel array 302 via the output signal lines 508 and 509 are added by the addition unit 305. For example, the image signal output from the unit pixel 403 located in the 0th row Line0 is read out via the output signal line 508, and the image output from the unit pixel 403 located in the 2nd row Line2. The signal is read through the output signal line 509. Then, these image signals are added in the addition unit 305. That is, the addition process is performed every other pixel. For example, when the color filters 406 are arranged in a Bayer array, it is possible to add (combine) the image signals from the unit pixels 403 in which the color filters 406 of the same color are arranged by doing so. Become. The addition unit 305 synthesizes the A image signal output from the pixel array 302 via the output signal line 508 and the A image signal output from the pixel array 302 via the output signal line 509 to form an A image. Can function as a synthesis means to generate a synthesis signal for. Further, the addition unit 305 generates a composite signal for the A + B image by synthesizing the A + B image signal output via the output signal line 508 and the A + B image signal output via the output signal line 509. Can function as a synthetic means.

Next, the operation of the image pickup apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the image pickup apparatus 100 according to the present embodiment.

In step S201, the image sensor 101 is driven based on the drive signal from the drive means 102, whereby the image sensor 101 acquires the image signal.

In step S202, the driving means 102 determines whether or not a defective row is included in the plurality of rows to be additionally read (composite read) based on the defect information supplied from the defect information storage means 103. To do. If no defective row is included in the plurality of rows to be added and read, that is, if all of the plurality of rows to be added and read are normal rows (NO in step S202), step S203. Move to. On the other hand, if a defective row is included in the plurality of rows to be additionally read (YES in step S202), the process proceeds to step S204.

In step S203, a predetermined read process, that is, a normal read process is performed. FIG. 6 is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 6 shows a time chart when reading an image signal from the pixel array 302, and corresponds to a case where a defective line is not included in a plurality of lines to be added and read. Here, it is assumed that the plurality of lines to be added and read are the 0th line Line0 and the 2nd line Line2, and all of these lines Line0 and Line2 are normal lines. FIG. 6 shows an operation when reading an image signal from the unit pixels 403 located in these lines Line 0 and Line 2.

First, the selection pulses pSEL0 and pSEL2 applied to the gates of the selection switches 507 located in the 0th row Line0 and the 2nd row Line2, respectively, are set to the High level. This makes it possible to read the image signals from these lines Line0 and Line2. Next, high-level reset pulses pRES0 and pRES2 are applied to the gates of the reset switches 504 located in the 0th row Line0 and the 2nd row Line2, respectively. As a result, the FD unit 503 is reset in these lines Line0 and Line2.

Next, high-level transfer pulses pTX0_A and pTX2_A are applied to the gates of the transfer switches 501 located in the 0th row Line0 and the 2nd row Line2, respectively. As a result, in these lines Line0 and Line2, the electric charge is transferred from the photoelectric conversion unit 401 to the FD unit 503. The FD unit 503 located at the 0th row Line 0 is connected to the gate of the amplification transistor 505 located at the 0th row Line 0. Therefore, the potential of the FD unit 503 located in the 0th row Line 0, that is, the output signal corresponding to the potential of the gate of the amplification transistor 505 located in the 0th row Line 0 is transmitted via the output signal line 508. It is output. The FD unit 503 located in the second row Line2 is connected to the gate of the amplification transistor 505 located in the second row Line2. Therefore, the potential of the FD unit 503 located in the second row Line 2, that is, the output signal corresponding to the potential of the gate of the amplification transistor 505 located in the second row Line 2 is transmitted via the output signal line 509. It is output. The output signals output via the output signal lines 508 and 509 are added by the addition unit 305. That is, the A image signal (first signal) of the 0th line Line 0 and the A image signal (1st signal) of the 2nd line Line 2 are added in the addition unit 305, and the composite signal for the A image is added. (First image signal) is obtained. In this way, the driving means 102 generates a composite signal for the A image by synthesizing a plurality of A image signals from the unit pixels 403 located in the plurality of rows to be added and read. It drives the image sensor 101. As described above, when the defective row, which is the row containing the defect, is not included in the plurality of rows to be added, the driving means 102 has the image sensor 101 in the first aspect (first mode). To drive. That is, in such a case, the driving means 102 generates a composite signal for the A image by synthesizing a plurality of A image signals from the unit pixels 403 located in the plurality of rows to be added and read. The driving of the first aspect for driving the image pickup device 101 is performed as described above. In this way, a plurality of A image signals generated by the unit pixels 403 to which the color filters 406 of the same color are arranged are combined, and a combined signal (addition signal) for the A image is obtained.

Next, high-level transfer pulses pTX0_B and pTX2_B are applied to the gates of the transfer switches 502 located at the 0th row Line0 and the 2nd row Line2, respectively. As a result, in these lines Line0 and Line2, the electric charge is transferred from the photoelectric conversion unit 402 to the FD unit 503. Since the electric charge corresponding to the A image already transferred from the photoelectric conversion unit 401 to the FD unit 503 is held in the FD unit 503, the electric charge corresponding to the A image and the electric charge corresponding to the B image are added in the FD unit 503. Will be done. In the present specification, the image signal corresponding to the combined charge obtained by adding the charge corresponding to the A image and the charge corresponding to the B image is referred to as an A + B image signal. Further, in the present specification, the optical image incident on the unit pixel 403 including the photoelectric conversion unit 401 and the photoelectric conversion unit 402 is referred to as an A + B image. The A + B image signal can be used as a recording image signal. Further, it is possible to obtain a B image signal by subtracting the A image signal from the A + B image signal. The FD unit 503 is connected to the gate of the amplification transistor 505, and output signals corresponding to the potential of the gate of the amplification transistor 505 are output via the output signal lines 508 and 509, respectively. That is, the A + B image signal is output via the output signal lines 508 and 509, respectively, like the A image signal. The A + B image signal output via the output signal lines 508 and 509 is added by the addition unit 305 in the same manner as the A image signal. That is, the A + B image signal (second signal) of the 0th row Line 0 and the A + B image signal (second signal) of the second row Line 2 are added in the addition unit 305, and the composite signal for the A + B image is added. (Second image signal) is obtained. In this way, the driving means 102 generates a composite signal for the A + B image by synthesizing a plurality of A + B image signals from the unit pixels 403 located in the plurality of rows to be added and read. It drives the image sensor 101. In this way, a plurality of A + B image signals generated by the unit pixels 403 to which the color filters 406 of the same color are arranged are combined, and a combined signal for the A + B image is obtained. After that, the selection pulses pSEL0 and pSEL2 applied to the gate of the selection switch 507 are set to the Low level. In this way, the composite signal for the A image and the composite signal for the A + B image are obtained, respectively.

In step S204, the reading process is performed without reading the A image signal from the defective row. FIG. 7 is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 7 shows a time chart when reading an image signal from the pixel array 302, and corresponds to a case where a defective line is included in a plurality of lines to be added and read. Here, the plurality of lines to be additionally read are the first line Line1 and the third line Line3, the first line Line1 is a defective line, and the third line Line3 is a normal line. It will be described as being. For example, when the signal line for supplying the transfer pulse pTX1_A and the signal line for supplying the transfer pulse pTX1_B are short-circuited, when the transfer switch 501 is turned on by applying the transfer pulse pTX1_A, the transfer switch 502 is also turned on. Become. In this case, the charge corresponding to the A + B image is transferred to the FD unit 503 simply by applying the transfer pulse pTX1_A, and the A image signal cannot be normally obtained. In the present specification, a line in which such an abnormality occurs is referred to as a defective line. A line that is not a defective line is referred to as a normal line in the present specification. Defect lines can occur, for example, in the manufacturing process of the image sensor 101. As described above, here, in the unit pixel 403 located in the defective row, the case where the A + B image signal can be acquired but the A image signal cannot be acquired normally will be described as an example. The first row Line1 is a defective row, the third row Line3 is a normal row, and the operation when reading an image signal from the unit pixels 403 located in these rows Line1 and Line3 is shown in FIG. ing.

First, the selection pulses pSEL1 and pSEL3 applied to the gates of the selection switches 507 located in the first row Line1 and the third row Line3, respectively, are set to the High level. This makes it possible to read image signals from these lines Line1 and Line3. Next, high-level reset pulses pRES1 and pRES3 are applied to the gates of the reset switches 504 located in the first row Line 1 and the third row Line 3, respectively. As a result, the FD unit 503 located in the first row Line 1 and the third row Line 3 is reset.

Next, a high level transfer pulse pTX3_A is applied to the gate of the transfer switch 501 located in the third row Line3. As a result, in the third row Line 3, the electric charge is transferred from the photoelectric conversion unit 401 to the FD unit 503. At this time, the high level transfer pulse pTX1_A is not applied to the gate of the transfer switch 501 located in the first row Line1. Therefore, in the first row Line 1, the electric charge is not transferred from the photoelectric conversion unit 401 to the FD unit 503. The reason why the high level transfer pulse pTX1_A is not applied to the gate of the transfer switch 501 located in the first row Line 1 is as follows. That is, since the signal line for supplying the transfer pulse pTX1_A and the signal line for supplying the transfer pulse pTX1_B are short-circuited, when the transfer switch 501 is turned on by applying the transfer pulse pTX1_A, the transfer switch 502 is also turned on. I will end up. In this case, the electric charge corresponding to the A + B image is transferred to the FD unit 503, and the A image signal cannot be normally obtained. For this reason, the high level transfer pulse pTX1_A is not applied to the gate of the transfer switch 501 located in the first row Line1. The FD unit 503 is connected to the gate of the amplification transistor 505 as described above, and an output signal corresponding to the potential of the gate of the amplification transistor 505 is output via the output signal lines 508 and 509. The output signals output via the output signal lines 508 and 509 are added by the addition unit 305. As described above, since the first row Line1 is a defective row, the electric charge is not transferred from the photoelectric conversion unit 401 to the FD unit 503 in the first row Line1. Therefore, the A image signal output from the unit pixel 403 located in the third row Line 3 is acquired by the addition unit 305. As described above, the driving means 102 drives the image sensor 101 in the following second mode (second mode) when the defective row is included in the plurality of rows to be added. To do. That is, in such a case, the driving means 102 refers to the A image based on the A image signal (first signal) from the unit pixel 403 located in the row other than the defective row among the plurality of rows to be added. The second aspect of driving the image sensor 101 is performed so that the image signal of the above is generated.

Next, high-level transfer pulses pTX1_B and pTX3_B are applied to the gates of the transfer switches 502 located in the first row Line1 and the third row Line3, respectively. At this time, the high level transfer pulse pTX1_A is also applied to the gate of the transfer switch 501 located in the first row Line1. As a result, in the first row Line 1, the electric charge corresponding to the A image is transferred from the photoelectric conversion unit 401 to the FD unit 503 via the transfer switch 501. Further, in the unit pixel 403 located in the first row Line 1, the electric charge corresponding to the B image is transferred from the photoelectric conversion unit 402 to the FD unit 503 via the transfer switch 502. Therefore, in the first row Line 1, the electric charge corresponding to the A image and the electric charge corresponding to the B image are added in the FD unit 503. In this way, in the first row Line 1, a signal corresponding to the charge obtained by adding the charge generated in the photoelectric conversion unit 401 and the charge generated in the photoelectric conversion unit 402, that is, an A + B image signal is obtained. Can be done. In the first line Line 1, the signal line for supplying the transfer pulse pTX1_A and the signal line for supplying the transfer pulse pTX1_B are short-circuited, but these are simultaneously set to the High level and the A + B image. It is possible to get a signal. In the third row Line 3, since the electric charge corresponding to the A image already transferred from the photoelectric conversion unit 401 to the FD unit 503 is held in the FD unit 503, the electric charge corresponding to the A image and the B image are obtained. The corresponding charges are added in the FD section 503. The FD unit 503 is connected to the gate of the amplification transistor 505 as described above, and output signals corresponding to the potential of the gate of the amplification transistor 505 are output via the output signal lines 508 and 509, respectively. That is, the A + B image signals obtained in the unit pixel 403 located in the first row Line 1 and the unit pixel 403 located in the third row Line 3 are output via the output signal lines 508 and 509, respectively. Will be done. A plurality of A + B image signals output via the output signal lines 508 and 509 are added by the addition unit 305. That is, the A + B image signal obtained by the unit pixel 403 located in the first row Line 1 and the A + B image signal obtained by the unit pixel 403 located in the third row Line 3 are added in the addition unit 305. .. As a result, a composite signal for the A + B image is obtained. After that, the selection pulses pSEL1 and pSEL3 applied to the gate of the selection switch 507 are set to the Low level. In this way, the signal for the A image and the composite signal for the A + B image are obtained, respectively.

In step S205, the separation means 104 separates the image signal obtained by the image sensor 101 into a recording image signal and a distance measuring image signal. FIG. 8 is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 8 shows how the image signal obtained by the image sensor 101 is separated into a recording image signal and a distance measuring image signal based on the recording image selection signal. “A (L0 + L2)” is A obtained by adding the A image signal obtained by the unit pixel 403 of the 0th row Line0 and the A image signal obtained by the unit pixel 403 of the second row Line2. Shows a composite signal for the image. “A + B (L0 + L2)” is A + B obtained by adding the A + B image signal obtained by the unit pixel 403 of the 0th row Line0 and the A + B image signal obtained by the unit pixel 403 of the second row Line2. Shows a composite signal for the image. “A (L3)” indicates an A image signal obtained by the unit pixel 403 of the third row Line3. “A + B (L1 + L3)” is A + B obtained by adding the A + B image signal obtained by the unit pixel 403 of the first row Line 1 and the A + B image signal obtained by the unit pixel 403 of the third row Line 3. The composite signal for the image is shown. The input image signal 801 is input from the image sensor 101 to the separation means 104. The recording image selection signal 802 is a signal for selecting a recording image signal from the input image signals 801 and is supplied from the control means 110 to the separation means 104. The recording image signal 803 is an image signal of an A + B image for recording on a storage medium (not shown), and is selected from the input image signals 801 based on the recording image selection signal 802. Examples of such a storage medium include a removable memory card and the like. The ranging image signal 804 includes an A image signal and an A + B image signal, and is output to the normalization means 105.

In step S206, the normalization means 105 normalizes the A image signal acquired by the driving of the second aspect, that is, the A image signal acquired in step S204. The signal level of the A image signal A (L3) acquired by the driving of the second aspect in step S204 is lower than the original signal level. Therefore, normalization for adjusting the level of the A image signal A (L3) is performed by the normalization means 105. FIG. 9 is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 9 shows a state of the normalization process performed by the normalization means 105. The input image signal 901 is an image signal input from the separation means 104 to the normalization means 105, and is the distance measurement image signal 804 described above with reference to FIG. The normalization control signal 902 is a signal for selecting an image signal to be normalized by the normalization means 105 from the input image signals 901. The normalization control signal 902 is transferred from the control means 110 to the normalization means 105 based on the defect information stored in the defect information storage means 106, specifically, information indicating which line is the defect line. Will be supplied. The normalization means 105 normalizes the input image signal 901 based on the normalization control signal 902 supplied from the control means 110. Here, the number of a plurality of rows Line1 and Line2 to be additionally read is, for example, two, the number of defective rows is, for example, and the number of normal rows is, for example, one. In such a case, the magnitude of the A image signal acquired by the driving of the second aspect becomes 1/2 the magnitude of the A image signal that should be originally obtained. Therefore, in such a case, normalization can be performed by multiplying the A image signal A (L3) acquired by the driving of the second aspect in step S204 by 2. In this way, the normalization means 105 normalizes the A image signal by multiplying the A image signal by a predetermined coefficient. As described above, the predetermined coefficient is set based on the number of rows of a plurality of rows to be read by addition and the number of defective rows included in the plurality of rows to be read by addition. Will be done. The output image signal 903 is an image signal output from the normalization means 105. “A (L3) × 2” indicates an A image signal normalized by multiplying the A image signal acquired by the unit pixel 403 of the third row Line 3 by 2. The normalization means 105 outputs the input image signal 901, which does not need to be normalized, as it is without performing the normalization process. As described above, when the A image signal is generated by the driving of the second aspect, the normalization means 105 performs the normalization process on the A image signal.

In step S207, the generation means 107 generates the B image signal by using the A image signal and the A + B image signal output from the normalization means 105. The B image signal is generated as follows. That is, the A image signal output from the normalization means 105 is temporarily stored in the image signal storage means 108 by the generation means 107. Then, when the A + B image signal is input from the normalization means 105, the generation means 107 reads the A image signal from the image signal storage means 108 and subtracts the A image signal from the A + B image signal so that the generation means 107 B. Generate an image signal. The generation means 107 outputs an A image signal and a B image signal. In this way, the A image signal corresponding to the light flux passing through the pupil region 408 and the B image signal corresponding to the light flux passing through the pupil region 409 are output from the generation means 107 to the distance measuring means 109.

In step S208, the distance measuring means 109 performs the distance measuring calculation based on the A image signal and the B image signal output from the generating means 107. The distance measuring means 109 acquires the correlation between the A image signal and the B image signal. That is, the distance measuring means 109 acquires the correlation value by quantifying the correlation by calculating the sum of the absolute values of the differences. The distance measuring means 109 calculates the amount of deviation between the A image and the B image based on the correlation value, and acquires distance information regarding the distance to the subject based on the amount of deviation. In step S209, the control means 110 drives the photographing lens 111 via the lens driving means 112 based on the distance information. In this way, focus control is performed.

As described above, in the present embodiment, when a defective row is included in the plurality of rows to be additionally read, the A image is read by reading from a row other than the defective row among the plurality of rows. The signal is acquired, and the normalization process is performed on the A image signal. Therefore, according to the present embodiment, there is provided an image pickup apparatus capable of obtaining a good A image signal even when a part of the pixel array 302 is defective, and by extension, performing good focus detection. can do.

[Second Embodiment]
The image pickup apparatus according to the second embodiment and the control method thereof will be described with reference to FIGS. 10 to 12. FIG. 10 is a block diagram showing an imaging device according to the present embodiment. The same components as those of the image pickup apparatus according to the first embodiment shown in FIGS. 1 to 9 and the control method thereof are designated by the same reference numerals, and the description thereof will be omitted or simplified.

As shown in FIG. 10, the image pickup element 101, the drive means 102, the defect information storage means 103, and the normalization means 1002 have an image pickup chip (imaging unit, image acquisition unit) 1001 in one chip. That is, in the present embodiment, the image sensor 101, the driving means 102, the defect information storage means 103, and the normalizing means 1002 are incorporated in the same package. In other words, in the present embodiment, the image pickup device 101 is configured as one chip including the defect information storage means 103, the drive means 102, and the normalization means 1002. The normalization means 1002 performs normalization processing on the image signal acquired by the image pickup device 101 based on the defect information supplied from the defect information storage means 103. As described above in the first embodiment, the defect information storage means 103 records information about defects occurring in the image sensor 101, that is, defect information.

Next, the operation of the image pickup apparatus according to the present embodiment will be described with reference to FIG. FIG. 11 is a flowchart showing the operation of the image pickup apparatus according to the present embodiment. In step S1101, the image sensor 101 is driven based on the drive signal from the drive means 102, and the image signal is acquired by the image sensor 101, as in step S201 described above in the first embodiment.

In step S1102, similarly to step S202 described above in the first embodiment, the defect information storage means 103 supplies whether or not the defect row is included in the plurality of rows to be additionally read. The driving means 102 determines based on the information. When no defective row is included in the plurality of rows to be added and read, that is, when all of the plurality of rows to be added and read are normal rows (NO in step S1102), step S1103 Move to. On the other hand, if a defective row is included in the plurality of rows to be additionally read (YES in step S1102), the process proceeds to step S1104.

In step S1103, a predetermined read process, that is, a normal read process is performed in the same manner as in step S203 described above in the first embodiment. That is, as in the first embodiment, the image sensor 101 is driven by the time chart as shown in FIG. As a result, for example, the A image signal read from the 0th row Line 0 and the A image signal read from the 2nd row Line 2 are added in the addition unit 305, and a composite signal for the A image is obtained. Further, for example, the A + B image signal read from the 0th row Line0 and the A + B image signal read from the 2nd row Line2 are added by the addition unit 305 to obtain a composite signal for the A + B image.

In step S1104, as in step S204 described above in the first embodiment, the reading process is performed without reading the A image signal from the defective row. That is, as in the first embodiment, the image sensor 101 is driven by the driving of the second aspect in the time chart as shown in FIG. 7. As a result, the signal for the A image and the composite signal for the A + B image can be obtained as in the case of the first embodiment. When the first row Line1 is a defective row and the third row Line3 is a normal row, the A image signal is based on the image signal from the unit pixel 403 located in the third row Line3. Is obtained. The composite signal for the A + B image is generated by adding the A + B image signal read from the first row Line 1 and the A + B image signal read from the third row Line 3.

In step S1105, the normalization means 1002 normalizes the A image signal acquired by the driving of the second aspect, that is, the A image signal acquired in step S1104. The signal level of the A image signal A (L3) acquired by driving the second aspect in step S1104 is lower than the original signal level as in the case of the first embodiment. Therefore, normalization for adjusting the level of the A image signal A (L3) is performed by the normalization means 1002. FIG. 12A is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 12A shows a state of the normalization process performed by the normalization means 1002. The input image signal 1201 is an image signal input from the image pickup device 101 to the normalization means 1002. The normalization determination signal 1202 is a signal for selecting an image signal to be normalized by the normalization means 1002 from the input image signals 1201. The normalization determination signal 1202 is generated in the normalization means 1002 based on the defect information stored in the defect information storage means 103, specifically, the information indicating which line is the defect line. The normalization means 1002 normalizes the input image signal 1201 based on the normalization determination signal 1202. Similar to the first embodiment, the number of a plurality of rows Line1 and Line2 to be additionally read is, for example, two, the number of defective rows is, for example, and the number of normal rows is, for example, one. It shall be. In such a case, as described above in the first embodiment, the magnitude of the A image signal acquired by the driving of the second embodiment is 1/2 of the magnitude of the A image signal that should be originally obtained. Become. Therefore, in such a case, normalization is performed in the same manner as in the first embodiment by multiplying the A image signal A (L3) acquired by driving the second aspect in step S1104 by 2. be able to. The output image signal 1203 is an image signal output from the normalization means 1002. “A (L3) × 2” indicates an A image signal normalized by multiplying the A image signal obtained by the unit pixel 403 of the third row Line 3 by 2. The normalization means 1002 outputs the input image signal that does not need to be normalized without performing the normalization process as it is. The image signal output from the normalization means 1002, that is, the image signal output from the image pickup chip 1001 is input to the separation means 104.

In step S1106, the separation means 104 separates the image signal obtained by the image pickup chip 1001 into a recording image signal and a distance measuring image signal. FIG. 12B is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 12B shows how the image signal obtained by the image pickup chip 1001 is separated into a recording image signal and a distance measuring image signal based on the recording image selection signal. The input image signal 1204 is output from the image pickup chip 1001 and input to the separation means 104. The recording image selection signal 1205 is a signal for selecting a recording image signal from the input image signals 1204, and is supplied from the control means 110 to the separation means 104. The recording image signal 1206 is an image signal of an A + B image for recording on a storage medium (not shown), and is selected from the input image signals 1204 based on the recording image selection signal 1205. The distance measuring image signal 1207 includes the image signal of the A image and the image signal of the A + B image, and is output to the generation means 107.

In step S1107, the generation means 107 generates the B image signal by using the A image signal and the A + B image signal output from the separation means 104. The B image signal is generated as follows. That is, the A image signal output from the separation means 104 is temporarily stored in the image signal storage means 108 by the generation means 107. Then, when the A + B image signal is input from the separation means 104, the generation means 107 reads the A image signal from the image signal storage means 108 and subtracts the A image signal from the A + B image signal so that the generation means 107 B. Generate an image signal. The generation means 107 outputs an A image signal and a B image signal. In this way, the A image signal corresponding to the light flux passing through the pupil region 408 and the B image signal corresponding to the light flux passing through the pupil region 409 are output from the generation means 107 to the distance measuring means 109.

In step S1108, similarly to step S208 in the first embodiment, the distance measuring means 109 performs the distance measuring calculation based on the A image signal and the B image signal output from the generating means 107. As a result, distance information regarding the distance to the subject is acquired. In step S1109, the control means 110 drives the photographing lens 111 via the lens driving means 112 based on the distance information. In this way, focus control is performed.

As described above, in the present embodiment, the image pickup chip 1001 in which the image pickup element 101, the drive means 102, the defect information storage means 103, and the normalization means 1002 are integrated into one chip is provided. The defect information used when driving the image sensor 101 or normalizing the image signal is supplied from the defect information storage means 103 provided in the image pickup chip 1001. Therefore, it is not necessary to provide the defect information storage means 106 (see FIG. 1) separate from the defect information storage means 103 outside the image pickup chip 1001. Therefore, in the present embodiment, it is not necessary to set the defect information outside the image pickup chip 1001. Therefore, according to the present embodiment, it is possible to eliminate the need to manage defect information when assembling the image pickup apparatus 100. Moreover, according to the present embodiment, it is not necessary to provide the defect information storage means 106 separately from the defect information storage means 103, which can contribute to cost reduction and the like.

[Third Embodiment]
The image pickup apparatus according to the third embodiment will be described with reference to FIGS. 13 to 18. FIG. 13 is a block diagram showing an imaging device according to the present embodiment. The same components as those of the image pickup apparatus according to the first or second embodiment shown in FIGS. 1 to 12 are designated by the same reference numerals, and the description thereof will be omitted or simplified.

The image pickup apparatus according to the present embodiment synthesizes a plurality of image signals acquired in each of a plurality of lines to be added and read in a common output signal line.

As shown in FIG. 13, the image pickup apparatus according to the present embodiment has an image pickup chip 1302 in which the image pickup element 1301, the drive means 102, and the defect information storage means 103 are integrated into one chip. In other words, the image pickup device 1301 is configured as one chip including the defect information storage means 103 and the drive means 102. The image sensor 1301 converts the subject image formed by the image pickup optical system into an electric signal by photoelectric conversion, and outputs the image signal as an image signal. FIG. 18 is a block diagram showing the configuration of the image sensor 1301 according to the present embodiment. The image sensor 1301 according to the present embodiment is not provided with the addition unit 305 (see FIG. 3).

FIG. 15 is a circuit diagram showing a part of the image pickup device 1301 according to the present embodiment. The image sensor 1301 according to the present embodiment is different from the image sensor 101 according to the first embodiment shown in FIG. 5 in that the unit pixels 403 of each line Line 0 to Line 3 are connected to a common output signal line 1501. There is. In the present embodiment, since the unit pixels 403 of each line Line 0 to Line 3 are connected to the common output signal line 1501, the operation is as follows. That is, in the present embodiment, when the selection switch 507 is turned on in the plurality of lines, the image signals output from the plurality of lines are added and averaged and output to the output signal line 1501. For example, when it is desired to output a composite signal of the image signal output from the 0th line Line0 and the image signal output from the 2nd line Line2, the following is performed. That is, the selection pulse pSEL0 applied to the gate of the selection switch 507 in the 0th row and the selection pulse pSEL2 applied to the gate of the selection switch 507 in the second row are set to the High level. As a result, the image signal from the 0th line Line 0 and the image signal from the 2nd line Line 2 are added and averaged on the output signal line 1501 and output to the outside of the image sensor 1301 via the output signal line 1501. To. Further, when it is desired to output only the image signal acquired by the unit pixel 403 located in the third row Line 3, only the selection pulse pSEL3 applied to the gate of the selection switch 507 of the third row Line 3 is High. Make it a level. As a result, the image signal acquired by the unit pixel 403 located in the third row Line 3 is output to the outside of the image sensor 1301 via the output signal line 1501.

The drive means 102 generates a composite signal for the A image by synthesizing a plurality of A image signals from the unit pixels 403 located in the plurality of rows to be added and read on the output signal line 1501. , Drives the image sensor 1301. Further, the driving means 102 generates a composite signal for the A + B image by synthesizing a plurality of A + B image signals from the unit pixels 403 located in the plurality of rows to be added and read on the output signal line 1501. As described above, the image sensor 1301 is driven. The driving means 102 drives the image sensor 1301 in the following first mode when the defective row, which is a row containing the defect, is not included in the plurality of rows to be added and read. That is, in such a case, the driving means 102 synthesizes the A image by synthesizing a plurality of A image signals from the unit pixels 403 located in the plurality of rows to be added and read on the output signal line 1501. The driving of the first aspect is performed so that a signal is generated. Further, the driving means 102 drives the image pickup device 1301 in the following second mode when a defective row is included in a plurality of rows to be added and read. That is, in such a case, the driving means 102 generates an image signal for the A image by the A image signal from the unit pixel 403 located in a row other than the defective row among the plurality of rows to be added and read. The driving of the second aspect is performed.

FIG. 14 is a flowchart showing the operation of the image pickup apparatus according to the present embodiment. In step S1401, the image sensor 1301 is driven based on the drive signal from the drive means 102, and the image signal is acquired by the image sensor 1301, as in step S201 described above in the first embodiment.

In step S1402, similarly to step S202 described above in the first embodiment, a defect is output from the defect information storage means 103 as to whether or not a defective row is included in the plurality of rows to be added and read. The driving means 102 determines based on the information. When no defective row is included in the plurality of rows to be added and read, that is, when all of the plurality of rows to be added and read are normal rows (NO in step S1402), step S1403 Move to. On the other hand, if a defective row is included in the plurality of rows to be additionally read (YES in step S1402), the process proceeds to step S1404.

In step S1403, a predetermined read process, that is, a normal read process is performed in the same manner as in step S203 described above in the first embodiment. That is, as in the first embodiment, the image sensor 1301 is driven by the time chart as shown in FIG. As a result, for example, the A image signal read from the 0th row Line 0 and the A image signal read from the 2nd row Line 2 are combined on the output signal line 1501 to obtain a combined signal for the A image. Further, the A + B image signal read from the 0th row Line 0 and the A + B image signal read from the 2nd row Line 2 are combined on the output signal line 1501 to obtain a combined signal for the A + B image.

In step S1404, the reading process is performed without reading the A image signal from the defective row. That is, the image sensor 1301 is driven by the time chart as shown in FIG. FIG. 16 is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 16 shows a time chart when reading an image signal from the pixel array 302, and corresponds to a case where a defective line is included in a plurality of lines to be added and read. Here, it is assumed that the first line Line1 is a defective line and the third line Line3 is not a defective line.

First, the selection pulse pSEL3 applied to the gate of the selection switch 507 located in the third row Line3 is set to the High level. This makes it possible to read the image signal from the unit pixel 403 located in the third row Line3. On the other hand, in order not to read the image signal from the unit pixel 403 located in the first row Line 1, the selection pulse pSEL1 applied to the gate of the selection switch 507 located in the first row Line 1 remains at the Low level. And.

Next, the FD unit 503 is reset by applying high-level reset pulses pRES1 and pRES3 to the gates of the reset switches 504 located in the first row Line 1 and the third row Line 3, respectively.

Next, high-level transfer pulses pTX1_A and pTX3_A are applied to the gates of the transfer switches 501 provided in the unit pixels 403 located in the first row Line 1 and the third row Line 3, respectively. In the first line Line 1, the signal line for supplying the transfer pulse pTX1_A and the signal line for supplying the transfer pulse pTX1_B are short-circuited, so that the result is as follows. That is, in the first row Line 1, when the transfer switch 501 is turned on by applying the transfer pulse pTX1_A, the transfer switch 502 to which the transfer pulse pTX1_B is supplied to the gate is also turned on. Therefore, in the unit pixel 403 located in the first row Line 1, the electric charge is transferred to the FD unit 503 from both the photoelectric conversion unit 401 and the photoelectric conversion unit 402. However, the selection pulse pSEL1 applied to the gate of the selection switch 507 located in the first row Line1 remains at the Low level. Therefore, the signal corresponding to the potential of the FD portion 503 of the unit pixel 403 located in the first row Line 1, that is, the signal corresponding to the potential of the gate of the amplification transistor 505 located in the first row Line 1 is output. It is not output to the signal line 1501. On the other hand, in the third line Line3, the signal line for supplying the transfer pulse pTX3_A and the signal line for supplying the transfer pulse pTX3_B are not short-circuited. Therefore, in the unit pixel 403 located in the third row Line 3, the electric charge generated in the photoelectric conversion unit 402 is transferred to the FD unit 503 without transferring the electric charge generated in the photoelectric conversion unit 402 to the FD unit 503. To. The selection pulse pSEL3 applied to the gate of the selection switch 507 located in the third row Line3 remains at the High level. Therefore, a signal corresponding to the potential of the FD portion 503 of the unit pixel 403 located in the third row Line 3, that is, a signal corresponding to the potential of the gate of the amplification transistor 505 located in the third row Line 3 is output. It is output to the signal line 1501. As described above, in the present embodiment, the electric charge is transferred to the FD unit 503 with the selection switch 507 of the first row Line 1 turned off. Therefore, in the present embodiment, the A image signal from the unit pixel 403 located in the third row Line 3 is read without reading the image signal from the unit pixel 403 located in the first row Line 1. Then, the A image signal read from the unit pixel 403 located in the third row Line 3 is output from the image sensor 1301.

Next, the selection pulse pSEL1 applied to the gate of the selection switch 507 located in the first row Line1 is set to the High level. As a result, the selection switch 507 is turned on in both the first row Line 1 and the third row Line 3.

Next, charges are transferred from the photoelectric conversion unit 402 to the FD unit 503 by applying high-level transfer pulses pTX1_B and pTX3_B to the gates of the transfer switches 502 of the first row Line 1 and the third row Line 3. .. In the first line Line 1, the signal line for supplying the transfer pulse pTX1_A and the signal line for supplying the transfer pulse pTX1_B are short-circuited, but these are simultaneously set to the High level, which is good. It is possible to obtain an A + B image signal. In the third row Line 3, since the electric charge corresponding to the A image already transferred from the photoelectric conversion unit 401 to the FD unit 503 is held in the FD unit 503, the electric charge corresponding to the A image and the B image are obtained. The corresponding charges are added in the FD section 503. The FD unit 503 is connected to the gate of the amplification transistor 505 as described above, and an output signal corresponding to the potential of the gate of the amplification transistor 505 is output to the output signal line 1501. In this way, the electric charge is transferred to the FD unit 503 with both the selection switch 507 of the first row Line 1 and the selection switch 507 of the third row Line 3 turned on. Therefore, the A + B image signal from the first row Line 1 and the A + B image signal from the third row Line 3 are combined on the output signal line 1501, and the combined signal for the A + B image is passed through the output signal line 1501. It is output. After that, the selection pulses pSEL1 and pSEL3 applied to the gate of the selection switch 507 located in each of the first row Line1 and the third row Line3 are set to the Low level.

In step S1405, the separation means 104 separates the image signal obtained by the image sensor 1301 into a recording image signal and a distance measuring image signal, as in the first embodiment. FIG. 17 is a time chart showing the operation of the image pickup apparatus according to the present embodiment. FIG. 17 shows how the image signal obtained by the image sensor 1301 is separated into a recording image signal and a distance measuring image signal based on the recording image selection signal. The input image signal 1701 is output from the image pickup device 1301 and input to the separation means 104. “{A (L0) + A (L2)} / 2” is an A image signal obtained by the unit pixel 403 of the 0th row Line 0 and an A image signal obtained by the unit pixel 403 of the second row Line 2. The A image signal obtained by the summing and averaging of is shown. “{A + B (L0) + A + B (L2)} / 2” is an A + B image signal obtained by the unit pixel 403 of the 0th row Line0 and an A + B image signal obtained by the unit pixel 403 of the second row Line2. The A + B image signal obtained by the summing and averaging of is shown. “A (L3)” indicates an A image signal obtained by the unit pixel 403 of the third row Line3. “{A + B (L1) + A + B (L3)} / 2” is an A + B image signal obtained by the unit pixel 403 of the first row Line 1 and an A + B image signal obtained by the unit pixel 403 of the third row Line 3. The A + B image signal obtained by the averaging of is shown. The recording image selection signal 1702 is a signal for selecting the recording image signal from the input image signals 1701, and is supplied from the control means 110 to the separation means 104. The recording image signal 1703 is an image signal of an A + B image for recording on a storage medium (not shown), and is selected from the input image signals 1701 based on the recording image selection signal 1702. The ranging image signal 1704 includes an A image signal and an A + B image signal, and is output to the generation means 107. Since steps S1406 to S1408 are the same as steps S1107 to S1109 described above in the second embodiment, the description thereof will be omitted.

As described above, in the present embodiment, the plurality of image signals acquired in each of the plurality of lines to be added and read are combined on the common output signal line 1501. When a defective line is included in a plurality of lines to be additionally read, reading is performed with the selection switch 507 located in the defective line turned off. Therefore, according to the present embodiment, a good A image signal can be obtained even when a defective line is included in a plurality of lines to be added and read. Therefore, also in the present embodiment, there is provided an image pickup apparatus capable of obtaining a good A image signal even when a part of the pixel array 302 is defective, and by extension, performing good focus detection. be able to.

[Modification Embodiment]
Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof.

For example, in the above embodiment, the case where the image signals from the two rows are combined at the time of additive reading has been described as an example, but the present invention is not limited to this. For example, image signals from three or more rows may be combined. Then, normalization may be appropriately performed based on the number of rows of the plurality of rows to be added and read and the number of defective rows included in the plurality of rows to be added and read.

Further, the reliability of the first distance information obtained by using the A image signal obtained by driving in the second aspect is obtained by using the A image signal obtained by driving in the first aspect. It is lower than the reliability of the second distance information. Therefore, the control means 110 may perform focus control based on the distance information obtained by considering the reliability of the first distance information and the reliability of the second distance information, respectively. For example, the focus control may be performed based on the distance information obtained by the weighted average in consideration of the reliability.

Further, in the above embodiment, the case where the pixel array 302 has a defect that cannot be normally acquired for the A image signal but can be normally acquired for the A + B image has been described as an example, but the present invention is limited to this. It is not something that is done. For example, the present invention can be applied even when the pixel array 302 has a defect in which not only the A image signal but also the A + B image signal cannot be normally acquired. In such a case, the driving in the second mode is performed not only when the image signal for the A image is generated but also when the image signal for the A + B image is generated. That is, an image signal for the A + B image is generated based on the A + B image signal from the unit pixel 403 located in the row other than the defective row among the plurality of rows to be added and read. Then, normalization is performed as appropriate.

Further, in the above embodiment, the image sensor 101 that acquires the A + B image signal after acquiring the A image signal has been described as an example, but the image sensor 101 is not limited to such an image sensor. For example, the present invention can be applied to the case where an image pickup device is used in which the FD unit 503 is reset after acquiring the A image signal and then the B image signal is acquired. When a defective line that cannot be normally acquired for the A image signal but can be normally acquired for the B image signal is generated in such an image sensor, among a plurality of lines to be additionally read out. When such a defective line is included, the following driving is performed. When generating the image signal for the A image, the driving in the second aspect is performed. That is, an image signal for the A image is generated based on the A image signal from the unit pixel 403 located in a row other than the defective row among the plurality of rows to be added and read. On the other hand, when the image signal for the B image is generated, the driving in the first aspect is performed. That is, an image signal for the B image is generated by synthesizing a plurality of B image signals from the unit pixels 403 located in the plurality of rows to be added and read. Then, normalization is performed as appropriate. When a defective line in which not only the A image signal but also the B image signal cannot be normally acquired occurs in such an image sensor, the defective line among a plurality of lines to be added and read is applied. When is included, the following drive is performed. When generating the image signal for the A image, the driving in the second aspect is performed. That is, an image signal for the A image is generated based on the A image signal from the unit pixel 403 located in a row other than the defective row among the plurality of rows to be added and read. The driving in the second aspect is also performed when the image signal for the B image is generated. That is, an image signal for the B image is generated based on the B image signal from the unit pixel 403 located in a line other than the defective line among the plurality of lines to be added and read. Then, normalization is performed as appropriate.

Further, in the above embodiment, when the A image signal is generated by the driving of the second aspect, the case where the normalization means 105 and 1002 perform the normalization process on the A image signal will be described as an example. , Not limited to this. For example, when the A image signal is generated by the driving of the second aspect, the normalization means 105 and 1002 may perform the normalization process on the signals other than the A image signal. For example, the A image signal A (L3) obtained by driving in the second aspect is not multiplied by a coefficient, and signals other than the A image signal A (L3) are multiplied by, for example, 0.5. , Normalization may be performed. That is, the composite signal A (L0 + L2) for the A image, the composite signals A + B (L0 + L2), and A + B (L1 + L3) for the A + B image may be normalized.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101 ... Image sensor 102 ... Driving means 103 ... Defect information storage means 104 ... Separation means 105 ... Normalization means 106 ... Defect information storage means 107 ... Generation means 108 ... Image signal storage means 109 ... Distance measuring means 110 ... Control means

Claims (17)

An image sensor including a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix, and the electric charge generated in the first photoelectric conversion unit is used. An image pickup device that generates a first signal corresponding to the response and a second signal corresponding to at least the electric charge generated in the second photoelectric conversion unit.
A first image signal is generated by synthesizing a plurality of the first signals from the unit pixels located in the plurality of rows, respectively, and the plurality of the first signals from the unit pixels located in the plurality of rows are generated. A defect line, which is a driving means for driving the image pickup device and is a line containing a defect, is included in the plurality of lines so that a second image signal is generated by synthesizing the two signals. If not, the image sensor is driven so that the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of rows. When the driving of the first aspect is performed and the defective row is included in the plurality of rows, the first from the unit pixel located in a row other than the defective row among the plurality of rows. A driving means for driving the second aspect of driving the image pickup device so that the first image signal is generated based on the signal of 1.
When the first image signal is generated by driving the second aspect, normalization processing is performed on at least one of the first image signal and the second image signal. have a normalized means,
The first signal and the second signal obtained by the unit pixel located in the first row of the plurality of rows are output from the pixel array via the first signal line.
The first signal and the second signal obtained by the unit pixel located in the second row different from the first row among the plurality of rows are different from the first signal line. Output from the pixel array via the second signal line
The first signal from the first signal output from the pixel array via the first signal line and the first signal output from the pixel array via the second signal line. The second signal that synthesizes an image signal and is output from the pixel array via the first signal line and the second signal that is output from the pixel array via the second signal line. combining means for combining the second image signal from the a, the imaging apparatus wherein said imaging device is further closed. The normalization means normalizes the first image signal or the second image signal by multiplying the first image signal or the second image signal by a predetermined coefficient. The imaging device according to claim 1. The predetermined coefficient according to claim 2, wherein the predetermined coefficient is set based on the number of rows of the plurality of rows and the number of rows of the defective row included in the plurality of rows. Imaging device. Further having a storage means in which information about the defect is stored,
Any one of claims 1 to 3 , wherein the image pickup device is configured as one chip including at least one of the storage means, the driving means, and the normalization means. The image sensor according to. An image sensor including a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix, and the electric charge generated in the first photoelectric conversion unit is used. An image pickup device that generates a first signal corresponding to the response and a second signal corresponding to at least the electric charge generated in the second photoelectric conversion unit.
A first image signal is generated by synthesizing a plurality of the first signals from the unit pixels located in the plurality of rows in a signal line, and a plurality of the unit pixels located in the plurality of rows are generated. A plurality of defective rows, which are driving means for driving the image pickup device and include defects, so that the second image signal is generated by synthesizing the second signal in the signal line. When it is not included in the line, the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of lines in the signal line. When the defect row is included in the plurality of rows, the drive is performed in the first aspect of driving the image pickup device so as to be performed, and the defect row is not included in the plurality of rows. It is characterized by having a driving means for driving a second aspect of driving the image pickup element so that the first image signal is generated by the first signal from the unit pixel located in a row. Image sensor. Further having a storage means in which information about the defect is stored,
The image pickup device according to claim 5 , wherein the image pickup device is configured as one chip including at least one of the storage means and the drive means. Claims 1 to 6 are characterized in that the second signal can be acquired from the unit pixel located in the defect row, but the first signal cannot be acquired normally. The imaging apparatus according to any one of the above. The unit pixel includes a floating diffusion region for temporarily accumulating charges generated in at least one of the first photoelectric conversion unit and the second photoelectric conversion unit, and the first photoelectric conversion unit and the floating diffusion. It includes a first transfer switch provided between the regions and a second transfer switch provided between the second photoelectric conversion unit and the floating diffusion region.
The imaging apparatus according to any one of claims 1 to 7 , wherein in the defective row, the gate of the first transfer switch and the gate of the second transfer switch are short-circuited. The image sensor further has a plurality of color filters provided for each of the plurality of unit pixels.
The first image signal is generated by the plurality of first signals from the unit pixel to which the color filter of the same color is arranged.
The second image signal is any one of claims 1 to 8 , characterized in that the second image signal is generated by the plurality of second signals from the unit pixel to which the color filter of the same color is arranged. The imaging apparatus according to. The image pickup device further has a plurality of microlenses provided for each of the plurality of unit pixels.
The first image signal is a signal corresponding to a light flux passing through the first pupil region of the exit pupil of the imaging optical system.
The second image signal is a signal corresponding to the light flux passing through the first pupil region and the light flux passing through a second pupil region different from the first pupil region of the exit pupil. ,
A claim further comprising a generation means for generating a third image signal corresponding to the light flux passing through the second pupil region based on the first image signal and the second image signal. Item 2. The imaging apparatus according to any one of Items 1 to 9 . The imaging apparatus according to claim 10 , further comprising a distance measuring means for generating distance information based on the first image signal and the third image signal. The reliability of the distance information obtained by using the first image signal obtained by driving the second aspect and the first image signal obtained by driving the first aspect are used. The imaging device according to claim 11 , further comprising a control means for performing focus control in consideration of the reliability of the distance information obtained thereby. The second signal is a signal corresponding to the combined electric charge obtained by synthesizing the electric charges generated in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively. The imaging apparatus according to any one of 1 to 12 . An image sensor including a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix, and the electric charge generated in the first photoelectric conversion unit is used. A control method for an image pickup device having an image pickup element that generates a first signal corresponding to the response and a second signal corresponding to at least the electric charge generated in the second photoelectric conversion unit.
A first image signal is generated by synthesizing a plurality of the first signals from the unit pixels located in the plurality of rows, respectively, and the plurality of the first signals from the unit pixels located in the plurality of rows are generated. A defect line, which is a step of driving the image pickup device and is a line containing a defect, is included in the plurality of lines so that a second image signal is generated by synthesizing the two signals. If not, the first image sensor is driven so that the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of rows. When the defective row is included in the plurality of rows, the first from the unit pixel located in a row other than the defective row among the plurality of rows is performed. The step of driving the second aspect of driving the image pickup element so that the first image signal is generated based on the signal of
When the first image signal is generated by driving the second aspect, normalization processing is performed on at least one of the first image signal and the second image signal. and the step,
A step of outputting the first signal and the second signal obtained by the unit pixel located in the first row of the plurality of rows from the pixel array via the first signal line.
The first signal and the second signal obtained by the unit pixel located in the second row different from the first row among the plurality of rows are different from the first signal line. A step of outputting from the pixel array via the second signal line,
The first signal from the first signal output from the pixel array via the first signal line and the first signal output from the pixel array via the second signal line. The second signal that synthesizes an image signal and is output from the pixel array via the first signal line and the second signal that is output from the pixel array via the second signal line. And the step of synthesizing the second image signal from
A method for controlling an imaging device, which comprises. An image sensor including a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix, and the electric charge generated in the first photoelectric conversion unit is used. A control method for an image pickup device having an image pickup element that generates a first signal corresponding to the response and a second signal corresponding to at least the electric charge generated in the second photoelectric conversion unit.
A first image signal is generated by synthesizing a plurality of the first signals from the unit pixels located in the plurality of rows in a signal line, and a plurality of the unit pixels located in the plurality of rows are generated. The step of driving the image sensor so that the second image signal is generated by synthesizing the second signal in the signal line, and the defect row is a plurality of defect rows. When it is not included in the row, the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of rows in the signal line. When the defective row is included in the plurality of rows, the row other than the defective row among the plurality of rows is driven so as to drive the image pickup element. An image pickup apparatus comprising: driving a second aspect of driving the image pickup element so that the first image signal is generated by the first signal from the unit pixel located in. Control method. An image sensor including a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix, and the electric charge generated in the first photoelectric conversion unit is used. A program for controlling an image pickup device having an image pickup element that generates a first signal corresponding to the response and a second signal corresponding to at least the electric charge generated in the second photoelectric conversion unit.
On the computer
A first image signal is generated by synthesizing a plurality of the first signals from the unit pixels located in the plurality of rows, respectively, and the plurality of the first signals from the unit pixels located in the plurality of rows are generated. A defect line, which is a step of driving the image pickup device and is a line containing a defect, is included in the plurality of lines so that a second image signal is generated by synthesizing the two signals. If not, the first image sensor is driven so that the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of rows. When the defective row is included in the plurality of rows, the first from the unit pixel located in a row other than the defective row among the plurality of rows is performed. The step of driving the second aspect of driving the image pickup element so that the first image signal is generated based on the signal of
When the first image signal is generated by driving the second aspect, normalization processing is performed on at least one of the first image signal and the second image signal. and the step,
A step of outputting the first signal and the second signal obtained by the unit pixel located in the first row of the plurality of rows from the pixel array via the first signal line.
The first signal and the second signal obtained by the unit pixel located in the second row different from the first row among the plurality of rows are different from the first signal line. A step of outputting from the pixel array via the second signal line,
The first signal from the first signal output from the pixel array via the first signal line and the first signal output from the pixel array via the second signal line. The second signal that synthesizes an image signal and is output from the pixel array via the first signal line and the second signal that is output from the pixel array via the second signal line. And the step of synthesizing the second image signal from
A program to execute. An image sensor including a pixel array in which a plurality of unit pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a matrix, and the electric charge generated in the first photoelectric conversion unit is used. A program for controlling an image pickup device having an image pickup element that generates a first signal corresponding to the response and a second signal corresponding to at least the electric charge generated in the second photoelectric conversion unit.
On the computer
A first image signal is generated by synthesizing a plurality of the first signals from the unit pixels located in the plurality of rows in a signal line, and a plurality of the unit pixels located in the plurality of rows are generated. The step of driving the image sensor so that the second image signal is generated by synthesizing the second signal in the signal line, and the defect row is a plurality of defect rows. When it is not included in the row, the first image signal is generated by synthesizing the plurality of first signals from the unit pixels located in the plurality of rows in the signal line. When the defective row is included in the plurality of rows, the row other than the defective row among the plurality of rows is driven so as to drive the image pickup element. A program for executing a step of driving a second aspect of driving the image sensor so that the first image signal is generated by the first signal from the unit pixel located in.

Images