JP6259492B2 - Image processing apparatus and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method for capturing a still image or a moving image, and more particularly to a technique for detecting defective pixels when a photoelectric conversion unit of an image sensor that is a component of the image capturing apparatus has a divided structure. It is.

  2. Description of the Related Art Conventionally, there are many imaging devices such as electronic cameras that record and reproduce still images and moving images captured by a solid-state imaging device such as a CCD or CMOS.

  As an example of the technology related to the solid-state imaging device mounted on these imaging devices, among a plurality of pixels constituting the solid-state imaging device, a part or all of the photoelectric conversion units of the pixels are divided into a plurality (divided pixels) It is proposed in Patent Document 1 and the like. Applications of such an image sensor include pupil division focus detection and generation of a stereoscopic image based on output signals obtained from each of the divided photoelectric conversion units. Further, by adding the output of the divided photoelectric conversion unit for each divided pixel, it can be used as a normal imaging signal.

  By the way, many recent electronic cameras have an image pickup device having millions to tens of millions of pixels, and all the pixels are appropriately converted into electric signals corresponding to the amount of incident light. It is very difficult to manufacture an image sensor that performs the above. In reality, some “defective pixels” that do not operate normally are mixed with the “pixels” of the image sensor.

  Therefore, in the conventional imaging apparatus, interpolation processing using the image signal of the surrounding pixels is performed on the image signal corresponding to the position of such a defective pixel, and the finally generated image is corrected. ing.

  There are several methods for detecting defective pixels to be corrected. For example, in the manufacturing process of an imaging device or an imaging device, the address of a defective pixel detected based on a photographed image is recorded, and the imaging device There is a method to save in the memory.

  Another detection method is to determine whether or not the pixel is a defective pixel from the level difference between the image signal from the pixel to be detected and the image signal from the surrounding pixels for each shooting by the imaging device. There is a defective pixel detection method, which is proposed in Patent Document 2 and the like.

JP 2003-244712 A JP 61-261974 A

  However, defective pixel detection in a solid-state imaging device using an imaging element having divided pixels as described above has the following problems. That is, it is difficult to always accurately determine whether a pixel to be detected is a defective pixel or a normal pixel that correctly outputs subject information. For this reason, if a normal pixel is erroneously determined to be a defective pixel, there is a risk of erroneous correction that unnecessarily corrects an image signal based on incident light from an actual subject. Specifically, the output values obtained from the plurality of photoelectric conversion units of the divided pixels as described above are different from each other when the subject is not imaged in the vicinity of the divided pixels. For this reason, the defective pixel detection method for determining whether or not the corresponding pixel is a defective pixel by comparing with the output values of peripheral pixels has a problem that correct detection cannot be performed depending on the state of the photographing lens.

The present invention has been made in view of the above problems, and an object of the present invention is to improve the detection accuracy of a defective pixel signal with respect to an output signal output from an imaging device using an imaging device having divided pixels.

In order to achieve the above object, the present invention makes a determination on an output signal output from an imaging device having a plurality of pixels each having a microlens and a plurality of photoelectric conversion means sharing the microlens. In the image processing apparatus, the determination target included in an output signal output from the photoelectric conversion unit to be determined and a pixel around the pixel having the photoelectric conversion unit to be determined among the plurality of photoelectric conversion units. A first determination process for outputting information relating to a defect in the photoelectric conversion means to be determined by comparing the photoelectric conversion means and an output signal from the photoelectric conversion means corresponding to the positional relationship with respect to the microlens; Among the plurality of pixels, the sum of the output signals of the plurality of photoelectric conversion means included in the determination target pixel and the plurality of pixels included in the peripheral pixels of the determination target pixel. By comparing of the total output signal of the photoelectric conversion means and a defective pixel determination means for performing a second determination process for outputting information on the defective pixel of the determination target.

The present invention can increase the detection accuracy of a defective pixel signal with respect to an output signal output from an imaging apparatus using an imaging element having divided pixels.

1 is a block diagram showing a schematic configuration of an imaging apparatus according to a first embodiment of the present invention. FIG. 3 is a diagram illustrating a pixel array of an image sensor in the first embodiment. FIG. 2 is a diagram illustrating a schematic cross section and a circuit configuration of one pixel in the first embodiment. FIG. 3 is a circuit diagram illustrating the configuration of the image sensor according to the first embodiment. 4 is a timing chart showing first drive timing in the first embodiment. Explanatory drawing of the 1st defective pixel detection method in 1st Embodiment. Explanatory drawing of the 2nd defective pixel detection method in 1st Embodiment. 6 is a flowchart showing a defective pixel correction procedure in the first embodiment. The block diagram which shows schematic structure of the imaging device concerning the 2nd Embodiment of this invention. The figure explaining the pixel arrangement | sequence of the image pick-up element in 2nd Embodiment. A circuit diagram explaining composition of an image sensor in a 2nd embodiment. 9 is a timing chart showing first drive timing in the second embodiment. 9 is a timing chart showing second drive timing in the second embodiment. Explanatory drawing of the 1st defective pixel detection method in 2nd Embodiment. Explanatory drawing of the 2nd defective pixel detection method in 2nd Embodiment. Explanatory drawing of the 3rd defective pixel detection method in 2nd Embodiment. 9 is a flowchart showing a defective pixel correction procedure in the second embodiment.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings. However, the dimensions, shapes, relative arrangements, and the like of the components exemplified in the present embodiment should be changed as appropriate according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to.

(First embodiment)
First, a first embodiment of the present invention will be described.

  FIG. 1 is a block diagram illustrating a schematic configuration of the imaging apparatus according to the first embodiment.

  In FIG. 1, 101 is an optical system comprising a photographic lens and a diaphragm, 102 is a mechanical shutter, 103 is a photoelectric conversion unit that converts incident light into an electric signal, and a signal amplification circuit that amplifies the electric signal. An image sensor that converts an electrical signal.

  Reference numeral 106 denotes an analog signal processing circuit that performs analog signal processing on an image signal output from the image sensor 103. The analog signal processing circuit 106 includes a CDS circuit 107 that performs correlated double sampling, a signal amplifier 108 that amplifies the analog signal, a clamp circuit 109 that performs horizontal OB clamping, and an A / D converter 110 that converts the analog signal into a digital signal. Including.

  Reference numeral 111 denotes a timing signal generation circuit that generates signals for operating the image sensor 103 and the analog signal processing circuit 106, and 112 denotes a drive circuit for the optical system 101 and the mechanical shutter 102. Reference numeral 128 denotes a second digital signal processing circuit that generates an imaging signal and a ranging signal from the image signal converted into a digital signal by the A / D converter 110 and performs necessary digital signal processing. , A signal generation circuit 129, a correction circuit 130, and a correlation calculation circuit 131. The signal generation circuit 129 generates an imaging signal and a ranging signal from the digital image signal. The correction circuit 130 performs various correction processes on the distance measurement signal generated by the signal generation circuit 129, and the correlation calculation circuit 131 calculates the distance to the subject by performing the correlation calculation process on the distance measurement signal. Generate the ranging data to be used.

  Reference numeral 113 denotes a digital signal processing circuit that performs digital signal processing necessary for the imaging signal (image data) and distance measurement data output from the second digital signal processing circuit 128. The digital signal processing circuit 113 includes an image correction circuit 114 that performs necessary correction processing on the image data, a signal amplification circuit 115 that amplifies the digital signal corrected by the image correction circuit 114, and a necessary image for the image data. An image processing circuit 116 that performs processing is included. Note that processing performed by the image correction circuit 114 and the image processing circuit 116 will be described later.

  117 is an image memory for storing processed image data, 118 is a recording medium removable from the imaging apparatus, and 119 is a recording circuit for recording signal processed image data on the recording medium 118. Reference numeral 120 denotes an image display device that displays signal-processed image data, and 121 denotes a display circuit that displays an image on the image display device 120.

  A system control unit 122 controls the entire imaging apparatus. 123 is a non-volatile memory that stores a program describing a control method executed by the system control unit 122, control data such as parameters and tables used when the program is executed, and correction data such as a scratch address (ROM). Reference numeral 124 denotes a volatile memory (RAM) that is used to transfer and store the program, control data, and correction data stored in the nonvolatile memory 123 and used when the system control unit 122 controls the imaging apparatus.

  A temperature detection circuit 125 detects the temperature of the image sensor 103 or its peripheral circuit. Reference numeral 126 denotes an accumulation time setting circuit for setting the accumulation time of the image sensor 103, and 127 denotes a shooting mode setting circuit for setting shooting conditions such as ISO sensitivity setting and switching of shooting modes including still image shooting and moving image shooting. It is.

  Here, a photographing operation in the imaging apparatus having the above configuration will be described. Prior to the shooting operation, necessary programs, control data, and correction data are transferred from the nonvolatile memory 123 to the volatile memory 124 and stored at the start of the operation of the system control unit 122 such as when the imaging apparatus is turned on. Keep it. These programs and data are used when the system control unit 122 controls the imaging apparatus. Further, as necessary, additional programs and data are transferred from the nonvolatile memory 123 to the volatile memory 124, or the system control unit 122 directly reads and uses the data in the nonvolatile memory 123. .

  First, the optical system 101 drives a diaphragm and a photographing lens according to a control signal from the system control unit 122 to form a subject image controlled to have an appropriate brightness on the image sensor 103. Next, during still image shooting, the mechanical shutter 102 is driven by the control signal from the system control unit 122 so as to shield the image sensor 103 in accordance with the operation of the image sensor 103 so that a necessary exposure time is obtained. Is done. At this time, when the image sensor 103 has an electronic shutter function, it may be used together with the mechanical shutter 102 to secure a necessary exposure time. In addition, the mechanical shutter 102 is maintained in an open state so that the image sensor 103 is always exposed during shooting by a control signal from the system control unit 122 during moving image shooting.

  The image sensor 103 is driven with a drive pulse based on an operation pulse generated by the timing signal generation circuit 111 controlled by the system control unit 122. The subject image is converted into an electric signal by photoelectric conversion in the photoelectric conversion unit, and the gain of the amplification factor set in accordance with the amount of incident light is applied to the electric signal in the signal amplification circuit and output as an analog image signal.

  The analog image signal output from the image sensor 103 removes clock synchronization noise by the CDS circuit 107 of the analog signal processing circuit 106 by an operation pulse generated by the timing signal generation circuit 111 controlled by the system control unit 122. . Further, the gain of the amplification factor set in accordance with the amount of incident light is applied by the signal amplifier 108, the signal output in the horizontal OB region is clamped as a reference voltage by the clamp circuit 109, and converted into a digital image signal by the A / D converter 110. Is done.

  Next, the digital image signal output from the analog signal processing circuit 106 is processed in the second digital signal processing circuit 128 controlled by the system control unit 122. First, based on the digital image signal, the signal generation circuit 129 generates an imaging signal and a distance measurement signal. The imaging signal generated by the signal generation circuit 129 is output as it is. On the other hand, the distance measurement signal generated by the signal generation circuit 129 is subjected to various correction processes such as flaw detection, flaw correction, and smoothing processing according to the present invention in the correction circuit 130, and then the correlation calculation in the correlation calculation circuit 131. By performing processing, distance measurement data is generated and output.

  Next, the imaging signal output from the second digital signal processing circuit 128 is processed in the digital signal processing circuit 113 controlled by the system control unit 122. First, the image signal is subjected to various image correction processes such as flaw detection and flaw correction according to the present invention in the image correction circuit 114 and dark shading correction, and the signal is set according to the amount of incident light. Apply gain gain. Further, the image processing circuit 116 performs various image processing such as image processing such as color conversion, white balance, and gamma correction, resolution conversion processing, and image compression processing. At this time, the image memory 117 is used for temporarily storing a digital image signal during signal processing or for storing image data that is a digital image signal subjected to signal processing.

  The image data signal-processed by the digital signal processing circuit 113 and the image data stored in the image memory 117 are converted into data suitable for the recording medium 118 (for example, file system data having a hierarchical structure) by the recording circuit 119. The Then, it is recorded on the recording medium 118. Alternatively, the image data that has undergone resolution conversion processing by the digital signal processing circuit 113 is converted into a signal suitable for the image display device 120 (for example, an NTSC analog signal) by the display circuit 121 and displayed on the image display device 120. To do.

  Here, the digital signal processing circuit 113 may output the digital image signal as it is as image data to the image memory 117 or the recording circuit 119 without performing signal processing by the control signal from the system control unit 122. Further, the digital signal processing circuit 113 outputs information on digital image signals and image data generated in the signal processing process to the system control unit 122 when requested by the system control unit 122. The image data information includes, for example, information such as the spatial frequency of the image, the average value of the designated area, the data amount of the compressed image, or information extracted from them. Further, the recording circuit 119 outputs information such as the type and free capacity of the recording medium 118 to the system control unit 122 when requested by the system control unit 122.

  Next, the reproduction operation when image data is recorded on the recording medium 118 will be described. In response to a control signal from the system control unit 122, the recording circuit 119 reads image data from the recording medium 118. Similarly, in response to a control signal from the system control unit 122, the digital signal processing circuit 113 performs an image expansion process when the read image data is a compressed image, and stores it in the image memory 117. Further, the image data stored in the image memory 117 is subjected to resolution conversion processing by the digital signal processing circuit 113, converted into a signal suitable for the image display device 120 by the display circuit 121, and displayed on the image display device 120. Is done.

  The distance measurement data output from the second digital signal processing circuit 128 is sent to the system control unit 122 via the digital signal processing circuit 113. The system control unit 122 calculates the distance to the subject based on the distance measurement data, and controls the drive circuit 112 based on the result to drive the optical system 101.

  FIG. 2 is a layout diagram illustrating the pixel arrangement of the image sensor 103 in the first embodiment.

  In FIG. 2, each region indicated by (0, 0), (1, 0), (0, 1) or the like in the drawing represents one pixel in the image sensor 103, and one microlens is arranged for each pixel. ing. In each pixel, each region indicated by a and b is a photoelectric conversion unit, and shares one microlens arranged for each pixel. Hereinafter, in the first embodiment, an output obtained by adding the output signals of the photoelectric conversion units a and b constituting each pixel in the image sensor 103 is referred to as a pixel output. In the first embodiment, the outputs of the photoelectric conversion units a and b are called divided outputs. When the divided output is used as a focus detection signal, a stereoscopic image generation signal, or the like, two signals of the divided output of the photoelectric conversion unit a and the divided output of the photoelectric conversion unit b are used.

  In the first embodiment described with reference to FIG. 2, each pixel is composed of two photoelectric conversion units arranged in 2 × 1, and the photoelectric conversion units a and b having the same symbol are In the positional relationship with the microlens, the corresponding positions are indicated. The characters “R”, “G”, and “B” written for each pixel represent the hue of the color filter formed on each pixel.

FIG. 3 is a diagram showing a schematic cross section and a circuit configuration for one pixel in the first embodiment of the present invention. In FIG. 3, reference numeral 411 denotes a P-type well, and 412 denotes a gate insulating film composed of a SiO ₂ film. 414a and 414b are P + layers formed on the surface of the P-type well 411, and constitute photoelectric conversion regions 415a and 415b together with the n layers 413a and 413b. Reference numerals 403a and 403b denote transfer switches for transferring signal charges generated in the photoelectric conversion regions 415a and 415b to the floating diffusion (FD) unit 407. 441 is a color filter, 442 is a microlens (on-chip lens), and the microlens 442 is such that the pupil of the photographing lens included in the optical system 101 and the photodiodes 402a and 402b of the image sensor 103 are substantially conjugate. Formed in shape and position.

  FIG. 4 is a circuit diagram illustrating a CMOS image sensor as an example of the pixel portion of the image sensor 103 in the first embodiment. As shown in FIG. 4, each pixel includes two photoelectric conversion units 401 a and 401 b arranged for each pixel and one pixel common unit 408.

  The photoelectric conversion unit 401a includes a photodiode 402a that is a photoelectric conversion element, and a transfer switch 403a that transfers charges generated by photoelectric conversion of the photodiode 402a by a pulse PTXa. Note that the photoelectric conversion unit 401b has the same configuration as the photoelectric conversion unit 401a.

  The pixel common unit 408 includes an FD unit 407 that accumulates charges transferred by the transfer switches 403a and 403b of the photoelectric conversion units 401a and 401b, respectively. In other words, one FD unit 407 is connected to the two photoelectric conversion units 401a and 401b. The pixel common unit 408 also includes a reset switch 404 that resets the FD unit 407 connected to the gate of the MOS transistor 406 to the level of the potential SVDD by a reset pulse PRES. Further, the pixel common unit 408 includes a MOS transistor 406 that amplifies the charge accumulated in the FD unit 407 as a source follower, and a row selection controlled by a selection pulse PSEL for selecting a row to be read by a vertical scanning circuit (not shown). Switch 405.

  Note that in the configuration of the pixel portion of the imaging element 103 described with reference to FIG. 4, the charges photoelectrically converted by the photodiodes 402a and 402b are both controlled by the transfer switches 403a and 403b, so that the FD portion 407 is used. It is set as the structure transferred to.

  The charges of the pixels in the row selected by the row selection switch 405 are output to the vertical output line 422 by the source follower circuit configured by the row selection switch 405 and the load current source 421. The signal output pulse PTS1a turns on the transfer gate 425 and accumulates the signal output to the vertical output line 422 in the transfer capacitor CTS1a427. The signal output pulse PTS1ab turns on the transfer gate 433 and accumulates the signal output to the vertical output line 422 in the transfer capacitor CTS1ab434. The noise output pulse PTN1 is stored in the transfer capacitor CTN1 426 by turning on the transfer gate 424.

  The signal transfer pulse PTS2a turns on the transfer gate 436 and accumulates the signal accumulated in the transfer capacitor CTS1a427 in the transfer capacitor CTS2439. In addition, the signal transfer pulse PTS2ab turns on the transfer gate 437 and causes the signal accumulated in the transfer capacitor CTS1ab434 to be accumulated in the transfer capacitor CTS2439. The noise transfer pulse PTN2 is stored in the transfer capacitor CTN2 438 with the transfer gate 435 turned ON.

  Subsequently, the noise components are stored in the capacitor CHN 430 and the signal components are stored in the capacitor CHS 431 via the transfer gates 428 and 429 by the control signals PHS and PHN from the horizontal scanning circuit (not shown), and the difference between the two is stored by the differential amplifier 432. Output as a pixel signal.

  In FIG. 4, transfer gates 424, 425, 433, 435, 436, and 437 subsequent to the vertical output line 422 are provided for each column. Further, the configuration subsequent to the transfer gates 428 and 429 is grouped into a plurality of columns, and has the same number as the output terminals of the image sensor 103.

  FIG. 5 is a timing chart showing drive timing (reading method) in the first embodiment. The drive timing described with reference to FIG. 5 includes the divided output signal a of the photoelectric conversion unit a described with reference to FIG. 2, the divided output signal a from the photoelectric conversion unit a, and the divided output signal b from the photoelectric conversion unit b. Is a drive timing for continuously reading out the pixel output signal ab obtained by adding. Hereinafter, the drive timing will be described with reference to FIGS.

  First, the divided output signal of the photoelectric conversion unit 401a is read during the period of HBLK1a, HBLK2a, and HSRa. As the signal HD indicating the start of one horizontal scanning period falls, the vertical output line 422 is reset to a constant potential by a circuit (not shown). After that, the PRES signal is set to a low (L) level and the reset switch 404 is turned on, whereby the charge accumulated in the capacitance of the FD portion 407 provided in the gate of the MOS transistor 406 during the period T1a becomes the constant potential SVDD. It is reset to become.

  Subsequently, the PRES signal is set to the high (H) level, the MOS transistor 406 is turned off, and then the PSEL signal is set to the H level, so that the source follower circuit configured by the row selection switch 405 and the load current source 421 is brought into an operating state. Become. As a result, noise corresponding to the potential of the FD unit 407 reset on the vertical output line 422 is output via the MOS transistor 406. The transfer capacitor CTN1 426 that accumulates the noise component is connected to the vertical output line 422 by setting the PTN signal to the high level during the period in which the PSEL signal is high, and the transfer capacitor CTN1 426 holds the signal of the noise component. It becomes like this.

  Subsequently, the mixed signal of the photoelectric charge and noise component generated in the photoelectric conversion element is accumulated. First, the vertical output line 422 is reset to a constant potential by a circuit (not shown). Thereafter, the PTXa signal is set to the H level, and the photoelectric charge accumulated in the photodiode 402a is transferred to the FD unit 407 by turning on the transfer switch 403a during the period T3a. At this time, since the PSEL signal remains at the H level, the source follower circuit is activated, and an “optical signal + noise signal” corresponding to the potential of the FD unit 407 is output via the MOS transistor 406 on the vertical output line 422. Is done. During the period T4a including the period T3a, the PTS1a signal is set to the H level, so that the transfer capacitor CTS1a427 for storing “photocharge component + noise component” is connected to the vertical output line 422. As a result, the transfer capacitor CTS1a427 holds the signal of the photocharge component + noise component.

  As described above, the noise component for one row and the optical signal + noise component generated by the photodiode 402a are accumulated in the transfer capacitors CTN1 426 and CTS1a 427, respectively.

  Next, during the period HBLK2a, the two signals accumulated in the transfer capacitors CTN1 426 and CTS1a 427 are transferred to the capacitors CTN2 438 and CTS2 439 by control pulses PTN2 and PTS2a controlled by a horizontal shift register (not shown), respectively. .

  Next, during the period HSRa, the two signals accumulated in the transfer capacitors CTN2 438 and CTS2 439 are transferred to the capacitors CHN430 and CHS431 by control pulses PHN and PHS controlled by a horizontal shift register (not shown), respectively. The noise component and the optical signal + noise component accumulated in the capacitors CHN 430 and CHS 431 are processed as (optical signal + noise component) −noise component by the differential amplifier 432 to become an optical signal, which is a divided output signal. is output as a.

  The pixel output signal ab is obtained by controlling the control signals PTXa, PTXb, PRES, and PSEL during the period of HBLK1ab and adding the signals of the photoelectric conversion units 401a and 401b at the same time as the periods HBLK2a and HSRa. Is read out.

  First, a mixed signal of photoelectric charge and noise component generated in the photoelectric conversion element is accumulated. First, the vertical output line 422 is reset to a constant potential by a circuit (not shown). Thereafter, the PTXa signal and the PTXb signal are set to the H level, and the photoelectric charges accumulated in the photodiodes 402a and 402b are transferred to the FD unit 407 by turning on the transfer switches 403a and 403b during the period T3ab. At this time, since the PSEL signal remains at the H level, the source follower circuit is activated, and an “optical signal + noise signal” corresponding to the potential of the FD unit 407 is output via the MOS transistor 406 on the vertical output line 422. Is done. During the period T4ab including the period T3ab, the PTS1ab signal is set to the H level, whereby the transfer capacitor CTS1ab434 that accumulates “photocharge component + noise component” is connected to the vertical output line 422. As a result, the transfer capacitor CTS1ab434 holds the signal of the photocharge component + noise component.

  As described above, the optical signal + noise component generated in the photodiode 402a and the photodiode 402b is accumulated in the transfer capacitor CTS1ab434.

  Next, two signals accumulated in the transfer capacitors CTN1 426 and CTS1ab 434 during the period HBLK2ab are transferred to the capacitors CTN2 438 and CTS2 439 by control pulses PTN2 and PTS2ab controlled by a horizontal shift register (not shown), respectively.

  Next, during the period HSRa, the two signals accumulated in the transfer capacitors CTN2 438 and CTS2 439 are transferred to the capacitors CHN430 and CHS431 by control pulses PHN and PHS controlled by a horizontal shift register (not shown), respectively. The noise component and the optical signal + noise component accumulated in the capacitors CHN 430 and CHS 431 are processed by the differential amplifier 432 as (optical signal + noise component) −noise component to become an optical signal, which is a pixel output signal. Output as ab.

  As described above, pixel output obtained by adding the divided output signal from the photoelectric conversion unit 401a and the divided output signal from the photoelectric conversion unit 401a and the divided output signal from the photoelectric conversion unit 401b of the two photoelectric conversion units arranged in 2 × 1. Complete signal reading.

  FIG. 6 is a diagram for explaining a first defective pixel detection method according to the first embodiment. The first defective pixel detection method is a method for detecting defective pixels by comparing divided output values from photoelectric conversion units located at positions corresponding to each other in a plurality of pixels of the same color.

  Here, the photoelectric conversion unit a of the pixel (2, 3) illustrated in FIG. 2 will be described as a defect detection processing target. In this case, the divided output value of the photoelectric conversion unit a of the pixel (2, 3) is set to pixels (0, 1), (2, 1), (4) which are photoelectric conversion units on the same side of the surrounding same color pixels. , 1), (0, 3), and (4, 3) are compared with the divided output values of the respective photoelectric conversion units a. Here, for example, the comparison is made with the average value of the divided output values of the photoelectric conversion unit a of the peripheral five pixels described above, but the comparison method is not limited to this, and various methods are conceivable. For example, the divided output value of the photoelectric conversion unit a of the pixel (2, 3) may be multiplied by 5 and compared with the sum of the divided output values of the photoelectric conversion unit a of the surrounding five pixels. Further, the difference between the divided output value of the photoelectric conversion unit a of the pixel (2, 3) and the divided output value of the photoelectric conversion unit a of the five neighboring pixels is obtained and added and / or averaged. It does not matter.

  When the difference is equal to or smaller than a predetermined first scratch detection threshold value α (first threshold value), it is determined that the photoelectric conversion unit a of the detection target pixel (2, 3) is not a defective pixel. . If it is larger than the first scratch detection threshold value α, it is determined that the pixel is a defective pixel and is detected as a scratch pixel.

  Then, the above-described detection processing is repeatedly performed for all the photoelectric conversion units, with a plurality of photoelectric conversion units included in the image sensor 103 being sequentially detected. However, the detection process may not be performed for a photoelectric conversion unit that is known to have a defect in advance.

  Since the first defective pixel detection method is a detection method based on comparison between photoelectric conversion units located at the same position with respect to the microlens, detection that is not easily influenced by the defocus amount with respect to the subject in the vicinity of the detection target pixel is possible. .

  FIG. 7 is a diagram for explaining a second defective pixel detection method according to the first embodiment. This second defective pixel detection method is a method for detecting defective pixels by comparing pixel output values of peripheral pixels.

  In FIG. 7, the pixel (2, 3) shown in FIG. 2 is described as a defect detection processing target. That is, the second defective pixel detection method is a detection method in the case where the divided signals of the photoelectric conversion units a and b of each pixel are added within the pixel and read out collectively as a pixel output. For example, when the photoelectric conversion unit a of the pixel (2, 3) is a defective pixel, the pixel (2, 3) including the photoelectric conversion unit a that is the defective pixel can be detected as a defective pixel. In the second defective pixel detection method, the pixel output value of the pixel (2, 3) is set to the pixels (0, 1), (2, 1), (4, 1), (0, 3) Compare with the pixel output values of (4, 3). Here, for example, the comparison is made with the average value or the like of the five pixel output values described above, but the comparison method is not limited to this, and various methods can be considered in the same manner as the first defective pixel detection method described above.

  If the difference is equal to or smaller than a predetermined second scratch detection threshold β (second threshold), it is determined that the pixel (2, 3) to be detected is not a defective pixel, and the second scratch is detected. If it is larger than the detection threshold β, it is determined that the pixel is a defective pixel, and is detected as a scratch pixel.

  Then, the above-described detection process is repeated for all the pixels, with a plurality of pixels included in the image sensor 103 being sequentially detected. However, it is not necessary to perform the detection process for pixels that are known to be defective in advance.

  FIG. 8 is a flowchart for explaining an example of a defective pixel correction procedure in the first embodiment.

  In FIG. 8, first, mode settings such as still image shooting, moving image shooting, and stereoscopic image shooting are performed by the shooting mode setting circuit 127 or the like (S801), and shooting conditions such as sensitivity, aperture value, and exposure time are set according to the mode. Is initially set (S802). Subsequently, the shutter 102 is controlled to expose the image sensor 103 (S803).

  Next, the information of the fixed defective pixels extracted in advance in the production process of the imaging device is read from the nonvolatile memory 123 (S804).

  Next, in the second digital signal processing circuit 128, defective pixels are detected in the ranging signal. First, the signal generation circuit 129 generates a ranging signal (S805). Specifically, the difference signal between the pixel output signal ab obtained by adding the output signal from the photoelectric conversion unit 401a and the output signal from the photoelectric conversion unit 401b and the divided output signal a from the photoelectric conversion unit 401a is photoelectrically converted. Generated as a divided output signal b from the unit 401b. The divided output signal b generated in this way is combined with the divided output signal a to generate a ranging signal.

  Subsequently, the correction circuit 130 uses the divided output signal a and the generated divided output signal b to perform defective pixel detection by the first defective pixel detection method described with reference to FIG. 6 (S806). Then, correction processing is performed on each ranging signal together with the fixed defective pixels (S807). Then, the correlation calculation circuit 131 performs correlation calculation processing on the distance measurement signals subjected to various corrections, and calculates distance data to the subject (S808).

  Next, in the digital signal processing circuit 113, defective pixels are detected in the captured image. In the image correction circuit 114, defective pixels are detected by the second defective pixel detection method described with reference to FIG. 7 (S809), and correction processing is performed on each defective pixel together with the fixed defective pixels. (S810).

  Finally, the image signal is output to the image memory 117, the recording circuit 119, or the display circuit 121 (S811), and the main photographing is finished.

  As described above, according to the first embodiment, the detection accuracy of defective pixels can be increased in an imaging apparatus using an imaging element having a photoelectric conversion unit.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  FIG. 9 is a block diagram illustrating a schematic configuration of the imaging apparatus according to the second embodiment. The difference between the configuration shown in FIG. 9 and the configuration shown in FIG. 1 is that the second digital signal processing circuit 128 is not provided in the configuration shown in FIG. 9, but the rest is the same as FIG. The description is omitted.

  The photographing operation in the imaging apparatus according to the second embodiment having the above-described configuration is the same as that in the first embodiment described above until the digital image signal is output from the analog signal processing circuit 106. The processing in the later stage will be described.

  The digital signal processing circuit 113 controlled by the system control unit 122 performs processing on the digital image signal output from the analog signal processing circuit 106. First, the image signal converted into the digital signal is subjected to various image correction processes such as defect detection and defect correction according to the present invention in the image correction circuit 114 and dark shading correction. Apply the gain of the amplification factor set accordingly. Further, the image processing circuit 116 performs various image processing such as image processing such as color conversion, white balance, and gamma correction, resolution conversion processing, and image compression processing. At this time, the image memory 117 is used for temporarily storing a digital image signal during signal processing or for storing image data that is a digital image signal subjected to signal processing.

  The image data signal-processed by the digital signal processing circuit 113 and the image data stored in the image memory 117 are converted into data suitable for the recording medium 118 (for example, file system data having a hierarchical structure) by the recording circuit 119. The Then, it is recorded on the recording medium 118. Alternatively, the image data that has undergone resolution conversion processing by the digital signal processing circuit 113 is converted into a signal suitable for the image display device 120 (for example, an NTSC analog signal) by the display circuit 121 and displayed on the image display device 120. To do.

  Here, the digital signal processing circuit 113 may output the digital image signal as it is as image data to the image memory 117 or the recording circuit 119 without performing signal processing by the control signal from the system control unit 122. Further, the digital signal processing circuit 113 outputs information on digital image signals and image data generated in the signal processing process to the system control unit 122 when requested by the system control unit 122. The image data information includes, for example, information such as the spatial frequency of the image, the average value of the designated area, the data amount of the compressed image, or information extracted from them. Further, the recording circuit 119 outputs information such as the type and free capacity of the recording medium 118 to the system control unit 122 when requested by the system control unit 122.

  Next, the reproduction operation when image data is recorded on the recording medium 118 will be described. In response to a control signal from the system control unit 122, the recording circuit 119 reads image data from the recording medium 118. Similarly, in response to a control signal from the system control unit 122, the digital signal processing circuit 113 performs an image expansion process when the read image data is a compressed image, and stores it in the image memory 117. Further, the image data stored in the image memory 117 is subjected to resolution conversion processing by the digital signal processing circuit 113, converted into a signal suitable for the image display device 120 by the display circuit 121, and displayed on the image display device 120. Is done.

  FIG. 10 is a layout diagram illustrating the pixel arrangement of the image sensor 103 according to the second embodiment.

  In FIG. 10, each area indicated by (0, 0), (1, 0), (0, 1), etc. in the figure represents one pixel in the image sensor 103, and one microlens is arranged for each pixel. ing. In each pixel, each region indicated by a, b, c, and d is a photoelectric conversion unit, and shares one microlens arranged for each pixel. Hereinafter, in the second embodiment, an output obtained by adding the output signals of the photoelectric conversion units a to d constituting each pixel in the image sensor 103 is referred to as a pixel output. In the second embodiment, the outputs of the photoelectric conversion units a to d output without being added in the image sensor 103 are referred to as divided outputs. When the divided output is used as a focus detection signal, a stereoscopic image generation signal, or the like, for example, “sum of divided outputs from photoelectric conversion unit a and photoelectric conversion unit c” and “photoelectric conversion unit b and photoelectric conversion”. The two signals “sum of the divided outputs from the part d” are generated and used.

  In the second embodiment described with reference to FIG. 10, each pixel is composed of four photoelectric conversion units arranged in 2 × 2, and photoelectric conversion units a, b, “c” and “d” indicate the corresponding positions in the positional relationship with the microlens. The characters “R”, “G”, and “B” written for each pixel represent the hue of the color filter formed on each pixel.

  The schematic cross section and circuit configuration for one pixel in the second embodiment are the same as those described above with reference to FIG. However, although two photoelectric conversion areas are shown in FIG. 3, in the second embodiment, two more photoelectric conversion areas having the same configuration are configured as shown in FIG.

  FIG. 11 is a circuit diagram illustrating a CMOS image sensor as an example of a pixel portion of the image sensor 103 in the second embodiment. As shown in FIG. 11, each pixel includes four photoelectric conversion units 401 a to 401 d arranged for each pixel and one pixel common unit 408.

  The photoelectric conversion unit 401a includes a photodiode 402a that is a photoelectric conversion element, and a transfer switch 403a that transfers charges generated by photoelectric conversion of the photodiode 402a by a pulse PTXa. The other photoelectric conversion units 401b to 401d also have the same configuration as the photoelectric conversion unit 401a.

  The pixel common unit 408 includes an FD unit 407 that accumulates charges transferred by the transfer switches 403a to 403d of the photoelectric conversion units 401a to 401d. That is, one FD unit 407 is connected to the four photoelectric conversion units 401a to 401d. The pixel common unit 408 also includes a reset switch 404 that resets the FD unit 407 connected to the gate of the MOS transistor 406 to the level of the potential SVDD by a reset pulse PRES. Further, the pixel common unit 408 includes a MOS transistor 406 that amplifies the charge accumulated in the FD unit 407 as a source follower, and a row selection controlled by a selection pulse PSEL for selecting a row to be read by a vertical scanning circuit (not shown). Switch 405.

  Note that in the configuration of the pixel portion of the imaging element 103 described with reference to FIG. 11, the charges photoelectrically converted by the photodiodes 402 a to 402 d are all controlled by the transfer switches 403 a to 403 d, so that the FD portion 407 is used. It is set as the structure transferred to.

  The charges of the pixels in the row selected by the row selection switch 405 are output to the vertical output line 422 by the source follower circuit configured by the row selection switch 405 and the load current source 421. The signal output pulse PTS may turn on the transfer gate 425 and accumulate the signal output to the vertical output line 422 in the transfer capacitor CTS 427, or the noise output pulse PTN may cause the transfer gate 424 to turn on and accumulate in the transfer capacitor CTN 426. it can. Subsequently, the noise components are stored in the capacitor CHN 430 and the signal components are stored in the capacitor CHS 431 via the transfer gates 428 and 429 by the control signals PHS and PHN from the horizontal scanning circuit (not shown), and the difference between the two is stored by the differential amplifier 432. Output as a pixel signal.

  In FIG. 11, transfer gates 424 and 425 in the subsequent stage from the vertical output line 422 are provided for each column, and the configuration in the subsequent stage from the transfer gates 428 and 429 is grouped into a plurality of columns, and the image sensor The number of output terminals 103 is the same as that of 103 output terminals.

  FIG. 12 is a timing chart showing the first driving timing (first reading method) in the second embodiment of the present invention. The first drive timing described with reference to FIG. 12 is a drive timing for independently reading the divided output signals of the photoelectric conversion units a to d described in FIG. In the case of performing signal reading at the first drive timing, the divided output signal can be used after being processed by the digital signal processing circuit 113 as a focus detection signal or a stereoscopic image generation signal. In the driving described in the timing chart of FIG. 12, the photoelectric conversion units a, b, c, and d (that is, photoelectric conversion units 401a, 401b, 401c, and 401d in FIG. 11) are read in this order. Yes. Hereinafter, the first drive timing will be described with reference to FIGS. 11 and 12.

  First, the signal of the photoelectric conversion unit 401a is read during the period of HBLKa and HSRa. As the signal HD indicating the start of one horizontal scanning period falls, the vertical output line 422 is reset to a constant potential by a circuit (not shown). After that, the PRES signal is set to a low (L) level and the reset switch 404 is turned on, whereby the charge accumulated in the capacitance of the FD portion 407 provided in the gate of the MOS transistor 406 during the period T1a becomes the constant potential SVDD. It is reset to become.

  Subsequently, the PRES signal is set to the high (H) level, the MOS transistor 406 is turned off, and then the PSEL signal is set to the H level, so that the source follower circuit configured by the row selection switch 405 and the load current source 421 is brought into an operating state. Become. As a result, noise corresponding to the potential of the FD unit 407 reset on the vertical output line 422 is output via the MOS transistor 406. By setting the PTN signal to the high level during the period when the PSEL signal is at the high level, the transfer capacitor CTN 426 for storing the noise component is connected to the vertical output line 422, and the transfer capacitor CTN 426 holds the signal of the noise component. Become.

  Subsequently, the mixed signal of the photoelectric charge and noise component generated in the photoelectric conversion element is accumulated. First, the vertical output line 422 is reset to a constant potential by a circuit (not shown). Thereafter, the PTXa signal is set to the H level, and the photoelectric charge accumulated in the photodiode 402a is transferred to the FD unit 407 by turning on the transfer switch 403a during the period T3a. At this time, since the PSEL signal remains at the H level, the source follower circuit is activated, and an “optical signal + noise signal” corresponding to the potential of the FD unit 407 is output via the MOS transistor 406 on the vertical output line 422. Is done. By setting the PTS signal to H level during the period T4a including the period T3a, the transfer capacitor CTS427 that accumulates “photocharge component + noise component” is connected to the vertical output line 422, and the transfer capacitor CTS427 is optical The signal of charge component + noise component is held.

  As described above, the noise component for one row and the optical signal + noise component generated by the photodiode 402a are accumulated in the transfer capacitors CTN426 and CTS427, respectively.

  Next, during the period HSRa, the two signals accumulated in the transfer capacitors CTN 426 and CTS 427 are transferred to the capacitors CHN 430 and CHS 431 by control pulses PHN and PHS controlled by a horizontal shift register (not shown), respectively. The noise component and the optical signal + noise component accumulated in the capacitors CHN 430 and CHS 431 are processed as (optical signal + noise component) −noise component by the differential amplifier 432 to become an optical signal, which is a divided output signal. Is output as

  Subsequently, during the period of HBLKb and HSRb, the control signals PTXb, PRES, and PSEL are controlled, and the signal of the photoelectric conversion unit 401b is read. Since the signal read timing of the photoelectric conversion unit 401b is the same as the signal read timing of the photoelectric conversion unit 401a described above, description thereof is omitted.

  Similarly, the control signals PTXc, PRES, and PSEL are similarly controlled during the period of HBLKc and HSRc, and the signal of the photoelectric conversion unit 401c is read. Further, the control signals PTXd, PRES, and PSEL are similarly controlled during the period of HBLKd and HSRd, and the signal of the photoelectric conversion unit 401d is read. Thereby, the readout of the divided output signals from the four photoelectric conversion units 401a, 401b, 401c, and 401d arranged in 2 × 2 is completed.

  FIG. 13 is a timing chart showing the second driving timing (second reading method) in the second embodiment. The second drive timing described with reference to FIG. 13 is for adding together the signals of the photoelectric conversion units a to d included in each pixel described in FIG. 10 as pixel output signals for each pixel and reading them in a batch. Drive timing. In the case of performing signal readout at the second drive timing, it is possible to read out at a high speed when individual divided output signals are unnecessary, such as during normal imaging such as still image shooting and moving image shooting. Hereinafter, the second drive timing will be described with reference to FIGS. 11 and 13.

  In the drive timing in FIG. 13, the first drive timing described above until the PTN signal is set to the H level during the period in which the PSEL signal is at the H level so that the transfer capacitor CTN 426 holds the noise component signal. Is the same.

  Subsequently, the mixed signal of the photoelectric charge and noise component generated in the photoelectric conversion element is accumulated. First, the vertical output line 422 is reset to a constant potential by a circuit (not shown). Thereafter, the PTXa, PTXb, PTXc, and PTXd signals are simultaneously set to the H level. Accordingly, the transfer switches 403a to 403d are turned on, and the photocharges accumulated in the photodiodes 402a, 402b, 402c, and 402d are transferred to the FD unit 407 during the period T3. At this time, since the PSEL signal remains at the H level, the source follower circuit is activated, and an “optical signal + noise signal” corresponding to the potential of the FD unit 407 is output via the MOS transistor 406 on the vertical output line 422. Is done. By setting the PTS signal to H level during the period T4 including the period T3, the transfer capacitor CTS427 for storing “photocharge component + noise component” is connected to the vertical output line 422, and the transfer capacitor CTS427 is The signal of the photocharge component + noise component is held.

  As described above, the noise component for one row and the optical signal + noise component generated by the photodiodes 402a to 402d are accumulated in the CTN 426 and the CTS 427, respectively.

  Next, during the period HSR, the two signals accumulated in the transfer capacitors CTN 426 and CTS 427 are transferred to the capacitors CHN 430 and CHS 431 by control pulses PHN and PHX controlled by a horizontal shift register (not shown), respectively. The noise component and the optical signal + noise component accumulated in the capacitors CHN 430 and CHS 431 are processed by the differential amplifier 432 as (optical signal + noise component) −noise component to become an optical signal, which is a pixel output signal. Is output as

  FIG. 14 is a diagram for explaining a first defective pixel detection method according to the second embodiment. The first defective pixel detection method is a method for detecting defective pixels by comparing divided output values from photoelectric conversion units located at positions corresponding to each other in a plurality of pixels of the same color.

  Here, the photoelectric conversion unit c of the pixel (2, 3) illustrated in FIG. 10 will be described as a defect detection processing target. In this case, the divided output value of the photoelectric conversion unit c of the pixel (2, 3) is converted into pixels (0, 1), (2, 1), (4, Comparison is made with the divided output values of the photoelectric conversion units c of 1), (0, 3), and (4, 3). Here, for example, the comparison is made with the average value of the divided output values of the photoelectric conversion unit c of the peripheral five pixels described above, but the comparison method is not limited to this, and various methods are conceivable. For example, the divided output value of the photoelectric conversion unit c of the pixel (2, 3) may be multiplied by 5 and compared with the sum of the divided output values of the photoelectric conversion units c of the five neighboring pixels. Further, the difference between the divided output value of the photoelectric conversion unit c of the pixel (2, 3) and the divided output value of the photoelectric conversion unit c of the five neighboring pixels is obtained and added and / or averaged. It does not matter.

  When the difference is equal to or smaller than a predetermined first scratch detection threshold value α (first threshold value), it is determined that the photoelectric conversion unit c of the detection target pixel (2, 3) is not a defective pixel. . If it is larger than the first scratch detection threshold value α, it is determined that the pixel is a defective pixel and is detected as a scratch pixel.

  Then, the above-described detection processing is repeatedly performed for all the photoelectric conversion units, with a plurality of photoelectric conversion units included in the image sensor 103 being sequentially detected. However, the detection process may not be performed for a photoelectric conversion unit that is known to have a defect in advance.

  Since the first defective pixel detection method is a detection method based on comparison between photoelectric conversion units located at the same position with respect to the microlens, detection that is not easily influenced by the defocus amount with respect to the subject in the vicinity of the detection target pixel is possible. .

  FIG. 16 is a diagram for explaining a third defective pixel detection method according to the second embodiment. This third defective pixel detection method is a method for detecting a defective pixel by comparing divided output values of a plurality of photoelectric conversion units included in each pixel.

  In FIG. 16, the photoelectric conversion unit c of the pixel (2, 3) shown in FIG. 10 will be described as a defect detection processing target. In this case, the divided output value of the photoelectric conversion unit c of the pixel (2, 3) is compared with the divided output values of the other photoelectric conversion units a, b, d of the same pixel (2, 3). Here, for example, the average value of the divided output values of the three photoelectric conversion units a, b, and d described above is compared. However, the comparison method is not limited to this, and the first defective pixel detection method described above is used. Similarly, various methods are conceivable.

  When the difference is equal to or smaller than a predetermined third defect detection threshold γ (third threshold), it is determined that the photoelectric conversion unit c of the detection target pixel (2, 3) is not a defective pixel. . If it is larger than the third flaw detection threshold γ, it is determined as a defective pixel and detected as a flaw pixel.

  Then, the above-described detection processing is repeatedly performed for all the photoelectric conversion units, with a plurality of photoelectric conversion units included in the image sensor 103 being sequentially detected. However, the detection process may not be performed for a photoelectric conversion unit that is known to have a defect in advance.

  Since the third defective pixel detection method is a detection method based on comparison between photoelectric conversion units in each pixel, even if the subject around the pixel including the photoelectric conversion unit to be detected is a subject having a high-frequency component, the accuracy is high. It is possible to perform high detection. However, the third defective pixel detection method has the following problems. That is, when the focal length of the photographic lens matches the subject around the pixel including the photoelectric conversion unit to be detected, the divided output values of the photoelectric conversion units in the pixel, that is, under the same microlens, are mutually Close value. However, as the defocus amount increases, the divided output values of the photoelectric conversion units under the same microlens differ according to the luminance distribution of the subject. Therefore, the third flaw detection threshold γ is determined in advance larger than the first flaw detection threshold α when the defocus amount near the detection target pixel is detected and the defocus amount is larger than a predetermined value. Set the specified value. When the defocus amount is equal to or smaller than the predetermined value, a predetermined value smaller than the first scratch detection threshold value α is set. Further, the third flaw detection threshold γ may be decreased as the defocus amount is decreased.

  FIG. 15 is a diagram for explaining a second defective pixel detection method according to the second embodiment. This second defective pixel detection method is a method for detecting defective pixels by comparing pixel output values of peripheral pixels.

  In FIG. 15, the pixel (2, 3) shown in FIG. 10 will be described as a defect detection processing target. That is, the second defective pixel detection method is a detection method when the divided signals of the photoelectric conversion units a, b, c, and d of each pixel are added in the pixel and read out collectively as a pixel output. For example, when the photoelectric conversion unit c of the pixel (2, 3) is a defective pixel, the pixel (2, 3) including the photoelectric conversion unit c that is the defective pixel can be detected as a defective pixel. In the second defective pixel detection method, the pixel output value of the pixel (2, 3) is set to the pixels (0, 1), (2, 1), (4, 1), (0, 3) Compare with the pixel output values of (4, 3). Here, for example, the comparison is made with the average value or the like of the five pixel output values described above, but the comparison method is not limited to this, and various methods can be considered in the same manner as the first defective pixel detection method described above.

  If the difference is equal to or smaller than a predetermined second scratch detection threshold β (second threshold), it is determined that the pixel (2, 3) to be detected is not a defective pixel, and the second scratch is detected. If it is larger than the detection threshold β, it is determined that the pixel is a defective pixel, and is detected as a scratch pixel.

  Then, the above-described detection process is repeated for all the pixels, with a plurality of pixels included in the image sensor 103 being sequentially detected. However, it is not necessary to perform the detection process for pixels that are known to be defective in advance.

  The second defective pixel detection method is a method for detecting a defective pixel at the time of reading a pixel signal at a high speed for moving image shooting or at the time of still image shooting. In this second defective pixel detection method, since the photoelectric conversion unit of each pixel is read in the added state, defect detection cannot be performed for each photoelectric conversion unit. Therefore, at the time of focus detection using each divided output value or stereoscopic image generation Sometimes it is an inappropriate detection method.

  FIG. 17 is a flowchart for explaining an example of a defective pixel correction procedure in the second embodiment.

  In FIG. 17, first, mode settings such as still image shooting, moving image shooting, stereoscopic image shooting, and focus detection shooting are performed by the shooting mode setting circuit 127 or the like (S1001), and sensitivity, aperture value, exposure are set according to the mode. Shooting conditions such as time are initially set (S1002). Subsequently, the shutter 102 is controlled to expose the image sensor 103 (S1003).

  Next, information on fixed defective pixels extracted in advance in the production process of the imaging device is read from the nonvolatile memory 123 (S1004). Then, in the digital signal processing circuit 113, defective pixel detection in the captured image is performed. First, the shooting mode set in S1001 is determined. The determined mode is a mode in which the divided output signals are individually read out at the first driving timing described with reference to FIG. 12, or the pixel signal is read out at the second driving timing described with reference to FIG. It is determined whether the mode is selected (S1005).

  If the set shooting mode is a mode for reading out pixel signals such as normal still image or moving image shooting, the defective pixel detection by the second defective pixel detection method described with reference to FIG. 15 is continued. (S1006).

  On the other hand, when the set shooting mode is a mode for individually reading out the photoelectric conversion unit, such as for stereoscopic image shooting or focus detection, the first defective pixel detection method described with reference to FIG. Defective pixel detection is performed (S1007). Subsequently, defective pixel detection is performed by the third defective pixel detection method described with reference to FIG. First, a defocus amount is calculated for each predetermined area in the screen from the read image signal (S1008), and based on the result, a third flaw detection threshold γ is set for each pixel (S1009). . Then, using the threshold γ, defective pixel detection is performed by the third defective pixel detection method (S1010).

  In the second embodiment, a pixel determined to be a defective pixel in any one of the first defective pixel detection method and the third defective pixel detection method is a defect to be corrected in the next step. Let it be a pixel. However, depending on the shooting conditions and modes, a configuration may be adopted in which only the pixels for which defective pixels are determined by both the first and third defective pixel detection methods are used as defective pixels to be corrected.

  After the defective pixel detection processing by the first and third defective pixel detection methods in S1006 to S1010 or the second defective pixel detection method, the digital signal processing circuit 113 performs correction processing on each defective pixel. (S1011).

  Finally, the image signal is output to the image memory 117, the recording circuit 119, or the display circuit 121 (S1012), and the main photographing is finished.

  As described above, according to the second embodiment, the detection accuracy of defective pixels can be increased in an imaging apparatus using an imaging device having a photoelectric conversion unit.

  The embodiment of the imaging apparatus according to the second embodiment has been described above with reference to FIGS. 9 to 17, but the present invention is not limited to this and can take various forms. .

  For example, in the pixel configuration of the second embodiment described with reference to FIGS. 10 and 14 to 16, in order to explain the defective pixel detection method in an easy-to-understand manner, the pixel is configured by a 2 × 2 4-photoelectric conversion unit. did. However, the present invention is not limited to this, and it may be simply divided into left and right or upper and lower two divisions, or 3 × 3 nine divisions.

  Further, in the flowchart of the second embodiment described with reference to FIG. 17, a configuration in which defective pixels are detected by a combination of fixed defective pixels and detection results obtained by three types of real-time defective pixel detection methods. did. However, the present invention is not limited to this, and any configuration may be used as long as defective pixels are detected using at least the first defective pixel detection method.

  103: Image sensor, 106: Analog signal processing circuit, 112: Drive circuit, 113: Digital signal processing circuit, 114: Image correction circuit, 115: Signal amplification circuit, 116: Image processing circuit, 122: System control unit, 123: Nonvolatile memory (ROM), 124: volatile memory (RAM), 128: second digital signal processing circuit, 129: signal generation circuit, 130: correction circuit, 131: correlation operation circuit, 401a, 401b: photoelectric conversion unit 402a, 402b: photodiode, 408: pixel common part, 441: color filter, 442: microlens

Claims (11)

In an image processing apparatus that makes a determination on an output signal output from an imaging device that includes a plurality of pixels each having a microlens and a plurality of photoelectric conversion means that share the microlens.
Among the plurality of photoelectric conversion units, the determination target photoelectric conversion unit included in an output signal output from the determination target photoelectric conversion unit, and pixels around the pixel having the determination target photoelectric conversion unit, and the determination target photoelectric conversion unit A first determination process for outputting information on a defect of the photoelectric conversion means to be determined by comparing an output signal from the photoelectric conversion means corresponding to a positional relationship with respect to the microlens, and among the plurality of pixels, By comparing the sum of the output signals of the plurality of photoelectric conversion means included in the determination target pixel with the sum of the output signals of the plurality of photoelectric conversion means included in pixels around the determination target pixel An image processing apparatus comprising: a defective pixel determination unit that performs a second determination process for outputting information on a defect of the determination target pixel. The imaging device has a color filter composed of a plurality of different hues corresponding to any hue for each pixel,
In the first determination process, the defective pixel determination unit uses an output signal of the photoelectric conversion unit included in the peripheral pixels having a color filter of the same color as the pixel including the photoelectric conversion unit to be determined as the comparison. The image processing apparatus according to claim 1, wherein the image processing apparatus is used. The imaging device has a color filter composed of a plurality of different hues corresponding to any hue for each pixel,
In the second determination process, the defective pixel determination unit uses an output signal from a photoelectric conversion unit included in the peripheral pixel having a color filter of the same color as the determination target pixel for the comparison. The image processing apparatus according to claim 1 or 2.   In the first determination process, the defective pixel determination unit has a difference between an output signal of the photoelectric conversion unit to be determined and a value calculated from an output signal from a photoelectric conversion unit included in the peripheral pixels. 4. The image processing apparatus according to claim 1, wherein the determination target photoelectric conversion unit determines that there is a defect when larger than a predetermined first threshold value. 5.   In the second determination process, the defective pixel determination unit includes a sum of output signals of the plurality of photoelectric conversion units included in the determination target pixel and a pixel around the determination target pixel. When the difference from the value calculated from the sum of the output signals of a plurality of photoelectric conversion means is larger than a predetermined second threshold, it is determined that the pixel to be determined is defective. The image processing apparatus according to claim 1. First correction means for correcting an output signal of the photoelectric conversion means to be determined based on a result of the first determination processing ;
6. The image according to claim 1 , further comprising: a second correction unit configured to correct an output signal of the pixel to be determined based on a result of the second determination process. Processing equipment.   The image processing apparatus according to claim 1, wherein the pixel to be determined is a single pixel.   The image processing apparatus according to claim 1, wherein the determination target photoelectric conversion unit is one photoelectric conversion unit. An image processing method for making a determination on an output signal output from an imaging device including a plurality of pixels each having a microlens and a plurality of photoelectric conversion means sharing the microlens,
Among the plurality of photoelectric conversion units, the determination target photoelectric conversion unit included in an output signal output from the determination target photoelectric conversion unit, and pixels around the pixel having the determination target photoelectric conversion unit, and the determination target photoelectric conversion unit A first determination step of outputting information on defects of the photoelectric conversion means to be determined by comparing an output signal from the photoelectric conversion means corresponding to the positional relationship with respect to the microlens;
Of the plurality of pixels, the sum of the output signals of the plurality of photoelectric conversion units included in the determination target pixel and the output signals of the plurality of photoelectric conversion units included in pixels around the determination target pixel And a second determination step of outputting information on the defect of the determination target pixel by comparing the total. A program for causing a computer to execute each step of the image processing method according to claim 9 . A computer-readable storage medium storing the program according to claim 10 .

Images