JP4977541B2 - Digital camera and operation control method thereof - Google Patents

Description

  The present invention relates to a digital camera and an operation control method thereof.

In a solid-state electronic image pickup device such as a CCD, defective pixels may be generated, so that defective pixel correction may be performed. Since defective pixels are conspicuous when the exposure time is long, there is one that determines whether or not to perform defective pixel correction based on the exposure time (Patent Documents 1 and 2).
Japanese Patent No. 376188 Japanese Patent Laid-Open No. 2001-211388

  However, many photoelectric conversion elements provided in a solid-state electronic image pickup device are not all of the same sensitivity, but often have different sensitivities. If all the defective pixels are corrected when the exposure time is long, the defective pixels that do not need to be corrected are corrected, and the image quality of the obtained image may deteriorate.

  An object of the present invention is to correct a defective pixel that needs to be corrected.

  In the digital camera according to the present invention, a large number of photoelectric conversion elements having different sensitivities are arranged on a light receiving surface, and a solid-state electronic image sensor that outputs image data representing a subject image by imaging the subject differs in sensitivity. Determining means for determining whether or not to correct a defective pixel for each photoelectric conversion element, Of the image data output from the solid-state electronic image sensor, the determination means determined to correct a defective pixel Image data obtained on the basis of signal charges accumulated in the photoelectric conversion element, and correction means for correcting image data corresponding to a predetermined defective pixel, and an image output from the solid-state electronic image sensor Among the data, the image data corrected by the correcting means and the image data not corrected by the correcting means are The photoelectric conversion elements having different amplification factors for each of the photoelectric conversion elements having different sensitivities are provided, and the determination means includes the photoelectric conversion elements to be determined as to whether or not to correct defective pixels among the different amplification factors. Using the value for the amplification factor of the image data obtained based on the accumulated signal charge, it is determined that the defective pixel is corrected when the value obtained by correcting the value for the exposure time exceeds the threshold value. Is.

  The present invention also provides an operation control method for the digital camera. That is, in this method, a solid-state electronic image sensor is provided with a large number of photoelectric conversion elements having different sensitivities on the light receiving surface, and by imaging the subject, image data representing the subject image is output. Determining whether or not to correct the defective pixel for each of the photoelectric conversion elements having different sensitivities, and correcting means corrects the defective pixel by the determining means out of the image data output from the solid-state electronic image sensor. Image data obtained based on the signal charges accumulated in the photoelectric conversion element determined to be performed, the image data corresponding to a predetermined defective pixel is corrected, and an amplifier circuit Of the image data output from the image sensor, the image data corrected by the correcting means and the image data not corrected by the correcting means are processed. Signal charges accumulated in the photoelectric conversion elements to be subjected to the determination by the determination means to determine whether or not to correct a defective pixel out of the different amplification factors. Using the value for the amplification factor of the image data obtained based on the above, it is determined that the defective pixel is corrected when the value obtained by correcting the value for the exposure time exceeds the threshold value.

  According to the present invention, a large number of photoelectric conversion elements having different sensitivities are arranged on the light receiving surface of the solid-state electronic image sensor, and image data representing the subject image can be obtained by imaging the subject. Whether or not to perform defective pixel correction is determined for each photoelectric conversion element having a different sensitivity. Image data representing a subject image obtained based on the signal charge accumulated in the photoelectric conversion element determined to perform the defective pixel determination, and corresponding to the predetermined defective pixel is corrected. The corrected image data and the uncorrected image data are amplified with different amplification factors for the photoelectric conversion elements having different sensitivities. The determination of whether or not the defective pixel is corrected is based on the amplification factor obtained based on the signal charge accumulated in the photoelectric conversion element to be determined whether or not to correct the defective pixel among the different amplification factors. Using this value, it is determined that the value obtained by correcting the value for the exposure time exceeds the threshold value.

  According to the present invention, it is determined whether or not defective pixel correction is performed for each photoelectric conversion element having different sensitivities, and when it is determined to be performed, defective pixel correction is performed. Whether or not to perform defective pixel correction can be determined for each photoelectric conversion element having a different sensitivity.

  For image data obtained from photoelectric conversion elements having different sensitivities, the image data may be amplified by changing the amplification factor according to the sensitivity. When the image data is amplified by changing the amplification factor, the threshold value of the exposure time considered to be necessary to correct the defective pixel also changes for each photoelectric conversion element having a different sensitivity. According to the present invention, using the value of the amplification factor of the image data obtained based on the signal charge accumulated in the photoelectric conversion element to be determined whether or not to correct the defective pixel among the different amplification factors, Since it is determined whether or not the defective pixel is corrected by correcting the value for the exposure time, whether or not the defective pixel is performed even if the image data is amplified by changing the amplification factor according to the photoelectric conversion element having different sensitivity The determination of whether or not can be performed relatively accurately.

  You may further provide the memory which memorize | stores the amplification factor for every said photoelectric conversion element from which a sensitivity differs. In this case, the amplification circuit will correct the image data corrected by the correction means using the amplification factor stored in the memory.

  Temperature detecting means for detecting the temperature around the digital camera and changing means for changing the threshold based on the temperature detected by the temperature detecting means may be further provided.

  The solid-state electronic image sensor includes, for example, a large number of photoelectric conversion elements in which color filters of different colors are attached to the light receiving area, and a large number of photoelectric conversion elements in which color filters of different colors or transparent filters are attached to the light receiving area. A conversion element or a large number of photoelectric conversion elements having different light receiving area sizes is provided.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing an electrical configuration of a digital camera.

  The entire operation of the digital camera is controlled by the CPU 8.

  The digital camera is provided with an operation switch 9 including switches such as a power button, a shutter release button, and a mode switch. A signal output from the operation switch 9 is input to the CPU 8. The digital camera is also provided with a timer 11 for measuring the exposure time and other times and a temperature sensor 12 for measuring the ambient temperature of the digital camera.

  The driving circuit 7 drives the lens / barrier / shutter / aperture 1 and the image sensor 2. The light beam that has passed through the lens / barrier / shutter / aperture 1 enters the light receiving surface of the image sensor 2.

  FIG. 2 shows a part of the light receiving surface of the image sensor 2.

  A large number of photoelectric conversion elements 21 are arranged in the horizontal direction and the vertical direction on the light receiving surface 20 of the imaging element 2. These photoelectric conversion elements 21 correspond to pixels. On the light receiving surface (light receiving region) of the photoelectric conversion element 21, a filter that transmits red component light (indicated by R), a filter that transmits green component light (indicated by G), or a filter that transmits blue component light (Indicated by B) is formed. A signal charge corresponding to the exposure amount is accumulated in the photoelectric conversion element 21. The accumulated signal charge is output from the image sensor 2 as image data via a vertical transfer path and a horizontal transfer path (both not shown). Since the transmittance of the filter R that transmits the red component light, the filter G that transmits the green component light, or the filter B that transmits the blue component light is different, the filter R that transmits the red component light, The sensitivity of the photoelectric conversion element 21 in which the filter G that transmits light or the filter B that transmits blue component light is formed is different.

  Returning to FIG. 1, the image data of the red component, the image data of the green component, or the image data of the blue component are sequentially output from the image sensor 2. Image data output from the image sensor 2 is input to the defect correction circuit 4 via the offset correction circuit 3.

  The defect correction circuit 4 is a defective pixel of the image sensor 2 (a so-called flaw of the image sensor 2 or the like, and image data corresponding to the accumulated amount although signal charges are accumulated in the photoelectric conversion element 21). This is a circuit that corrects a pixel from which no image can be obtained. The position of the defective pixel is detected in advance, and data indicating the position is stored in the memory 10. The digital camera according to this embodiment does not perform defective pixel correction for all defective pixels detected in advance, but performs defective pixel correction for defective pixels that satisfy a predetermined determination formula.

  In particular, in this embodiment, the parameters used for the predetermined determination formula differ depending on the sensitivity of the image sensor. As described above, the photoelectric conversion element 21 of the image pickup device 2 has a filter formed on the light receiving surface that transmits a red light component, a green light component, or a blue light component. Since the sensitivity varies depending on the object, white balance adjustment is performed. The gain in this white balance adjustment differs depending on whether it is red component image data, green component image data, or blue component image data. The above-described determination formulas are calculated using different gains for each of the red component image data, the green component image data, and the blue component image data.

  The image data output from the defective pixel correction circuit 4 is input to the gain correction circuit 5 and white balance adjustment is performed. After the white balance adjustment, the defective pixel may be corrected. The image data output from the gain correction circuit 5 is subjected to predetermined signal processing such as color interpolation and data compression in the signal processing circuit 6 and input to the image display device 14. A subject image obtained by imaging is displayed on the image display device 14.

  When the shutter release button is pressed, the image data output from the signal processing circuit 6 as described above is recorded on a memory card (not shown) connected to the memory card interface 13.

  FIG. 3 is a flowchart showing the processing procedure of the digital camera.

  The subject is imaged by the image sensor 2 (step 31). An APEX value of ISO sensitivity (for example, 5 for ISO 100) Sv, an APEX value of exposure time (for example, 0 for 1 second) Tv, gain Ga, and the like are acquired (step 32). If the ISO sensitivity is fixed, it is stored in the memory 10, and if the user can set it, the set value will be read. The exposure time is counted by the timer 11. The gain Ga is a white balance gain, and is, for example, 1.2 for red component image data, 1 for green component image data, and 1.8 for blue component image data.

  Subsequently, the determination formula of Expression 1 is calculated for each of the red component (R pixel) image data, the green component (G pixel) image data, and the blue component (B pixel) image data (step 33). When Equation 1 is satisfied for the image data of the defective pixel, the defective pixel correction is performed. The threshold value Th is, for example, a fixed value and is 5. Equation 1 corrects the exposure time using a value for the amplification factor of the image data.

  Sv + log2 (Ga) -Tv> Th ... Formula 1

  It is determined whether the image data inputted from the defective pixel correction circuit 4 and obtained from the photoelectric conversion element 21 corresponding to the defective pixel is an R pixel, a G pixel or a B pixel (step 34). .

  In the case of R pixel image data, it is determined whether Formula 1 (determination formula) is satisfied when Formula 1 is calculated using the gain Ga of the R pixel. If Equation 1 is satisfied (YES in step 35), it is considered that the defective pixel has been exposed for a time necessary for correction, and therefore, the defective pixel correction is performed in the defect correction circuit 4 for the defective pixel of the R pixel (step 36). ). If Equation 1 does not hold (NO in step 35), the process of step 36 is skipped, and defective pixel correction is not performed for the defective pixel of the R pixel.

  Similarly, in the case of G pixel image data, it is determined whether Equation 1 is satisfied when Equation 1 is calculated using the gain Ga of the G pixel. If Expression 1 is satisfied (YES in step 37), defective pixel correction is performed for the defective pixel of the G pixel (step 38). If Equation 1 does not hold (NO in step 37), the process of step 38 is skipped, and defective pixel correction is not performed for the defective pixel of the G pixel.

  Further, in the case of image data of B pixels, it is determined whether Equation 1 is satisfied when Equation 1 is calculated using the gain Ga of B pixels. If Equation 1 is satisfied (YES in step 39), defective pixel correction is performed for the defective pixel of the B pixel (step 40). If Equation 1 does not hold (NO in step 39), the process of step 40 is skipped, and defective pixel correction is not performed for the defective pixel of the B pixel.

  The processing from step 34 to step 40 is repeated for the image data of all the pixels of the image for one frame (step 41). When the processing from step 34 to step 40 for the image data of one frame is completed (YES in step 41), predetermined image signal processing such as pixel interpolation and data compression described above is performed on the image data (step). 42).

  When the image sensor includes photoelectric conversion elements with different sensitivities, it is necessary to adjust the exposure time for each sensitivity when deciding whether or not to perform defective pixel correction based on the exposure time. Since the above-described determination formula 1 considers a different gain for each pixel (that is, considers a difference in sensitivity of photoelectric conversion elements), the exposure time is adjusted to correct a defective pixel. You can decide whether or not to do it.

  FIG. 4 shows the relationship between temperature and threshold value Th.

  Although the threshold value Th is fixed in the determination formula of the above-described formula 1, the threshold value Th may be changed according to the ambient temperature of the digital camera. The ambient temperature can be measured using the temperature sensor 12 as described above.

  As the temperature (° C.) increases to 0 °, 10 °, 20 °, 30 °, 40 °, and 50 °, the threshold Th decreases to 8, 7, 6, 5, 4, and 3. Since the sensitivity of the photoelectric conversion element also changes in accordance with the temperature change, it is possible to determine whether or not to perform defective pixel correction in consideration of the temperature change.

  FIG. 5 is an example of the photoelectric conversion element 21 formed on the light receiving surface 22 of the imaging element.

  As described above, a large number of photoelectric conversion elements 21 are arranged in the horizontal direction and the vertical direction of the light receiving surface 22 of the imaging element. In this figure as well, the filter formed on the light receiving surface of the photoelectric conversion element 21 is denoted by the symbol R for the filter that transmits the red light component, and the green light component is designated as shown in FIG. Those that pass through are labeled G, and those that pass the blue light component are labeled B. As described above, R pixel image data is obtained based on the signal charges accumulated in the photoelectric conversion elements in which the filter with the R symbol is formed, and the filter with the G symbol is formed. G pixel image data is obtained based on the signal charge accumulated in the photoelectric conversion element, and the B pixel is obtained based on the signal charge accumulated in the photoelectric conversion element in which a filter having a symbol B is formed. Image data is obtained.

  In this embodiment, a photoelectric conversion element 21A in which no color filter is formed is provided on the light receiving surface of the image pickup element. The photoelectric conversion element 21A in which no color filter is formed is denoted by the reference symbol Y. The sensitivity of the photoelectric conversion element 21A in which no color filter is formed is higher than the sensitivity of the photoelectric conversion element 21 in which a color filter is formed. Luminance data is obtained based on the signal charges accumulated in these photoelectric conversion elements 21A.

  FIG. 6 is a flowchart showing a processing procedure of the digital camera when the subject is imaged using the image sensor having the photoelectric conversion element array shown in FIG. In this figure, the same processes as those shown in FIG.

  As described above, a pixel corresponding to the photoelectric conversion element 21A in which no color filter is formed is referred to as a Y pixel. The determination formula of Formula 1 is calculated for each of the R pixel, G pixel, B pixel, and Y pixel (step 33A). In the case where a photoelectric conversion element in which no color filter is formed is arranged in the imaging device, for example, the gain Ga of the R pixel is 2.4, the gain Ga of the G pixel is 2, and the gain Ga of the B pixel is 3.6, The gain Ga of the Y pixel is 1. Using the respective gains Ga, the above-described determination formula of Formula 1 is calculated. When the image data of R pixel, G pixel, B pixel, and Y pixel is input, if the judgment formula of Formula 1 is satisfied (YES in each of Steps 35, 37, 39, and 43), defective pixel correction is performed. (Steps 36, 38, 40, 44).

  FIG. 7 is an example of a light receiving surface of another image sensor.

  A large number of photoelectric conversion elements 24 and 25 are arranged in the horizontal direction and the vertical direction on the light receiving surface of the imaging element. The area of the light receiving surface of one photoelectric conversion element (main photoelectric conversion element) 24 is larger than the area of the light receiving surface of the other photoelectric conversion element (sub photoelectric conversion element) 25. For this reason, the sensitivity of the sub photoelectric conversion element 25 is 20 times that of the main photoelectric conversion element 24. The main photoelectric conversion elements 24 are arranged in the odd rows and the odd columns, and the sub photoelectric conversion elements 25 are arranged in the even rows and the even columns, but the main photoelectric conversion elements 24 are arranged in the even rows and the even columns, and the odd rows and the odd columns. The sub photoelectric conversion element 25 may be disposed.

  FIG. 8 shows the sensitivity of the main photoelectric conversion element 24 and the sensitivity of the sub photoelectric conversion element 25.

  As described above, since the area of the main photoelectric conversion element 24 is larger than the area of the sub photoelectric conversion element 25, the signal stored in the main photoelectric conversion element 24 is more accumulated than the sub photoelectric conversion element 25 if the brightness is the same. Large amount of charge. The amount of signal charge accumulated in the main photoelectric conversion element 24 is saturated at the brightness L0. It can be seen that the sub photoelectric conversion element 25 has higher sensitivity than the main photoelectric conversion element 24.

  FIG. 9 is an example of a light receiving surface of still another image sensor.

  A large number of photoelectric conversion elements 27 are arranged in the horizontal direction and the vertical direction. These photoelectric conversion elements 27 are divided into a main light receiving region 28 and a sub light receiving region 29. The area of the main light receiving region 28 is larger than the area of the sub light receiving region 29. As described above, even when the light receiving area of one photoelectric conversion element 27 is divided into the main light receiving area 28 and the sub light receiving area 29, the sensitivity of the sub light receiving area 29 is similar to that of the image sensor shown in FIG. This is higher than the sensitivity of the main light receiving region 28.

  The main light receiving area 28 of the photoelectric conversion element 27 is given an R code, a G code or a B code, and the sub light receiving area 29 of the photoelectric conversion element 27 is given an r code, a g code or a b code. Has been. Signal charges having a red light component are accumulated in the photoelectric conversion elements 27 to which the symbol R or the symbol r is attached. A signal charge having a green light component is accumulated in the photoelectric conversion element 27 to which the symbol G or the symbol g is attached. A signal charge having a blue light component is accumulated in the photoelectric conversion element 27 to which the symbol B or the symbol b is attached.

  10 shows an image sensor having a main photoelectric conversion element 24 and a sub photoelectric conversion element 25 as shown in FIG. 7, or an image sensor in which a main light receiving region 28 and a sub light receiving region 29 are formed as shown in FIG. 6 is a flowchart showing a processing procedure of a digital camera provided with a digital camera. In this figure, the same processes as those shown in FIG.

  The pixel represented by the image data obtained based on the signal charge accumulated in the main photoelectric conversion element 24 or the main light receiving area 28 is determined based on the signal charge accumulated in the main pixel, the sub photoelectric conversion element 25 or the main light receiving area 29. A pixel represented by image data obtained in this way is defined as a subpixel.

  The gain Ga of the main pixel is 1, for example, and the gain Ga of the sub-pixel is 20, for example. Using these gains Ga, the above-described determination formula 1 is calculated for each main pixel and sub-pixel (step 51).

  If the determination formula 1 is satisfied for the main pixel (YES in step 52), defective pixel correction of the main pixel is performed (step 53). If the determination formula 1 is satisfied for the subpixel (YES in step 54), defective pixel correction of the subpixel is performed (step 55).

  Steps 52 to 55 are repeated for one frame of image data (step 56). When the processing of steps 52 to 55 is completed for one frame of image data (YES in step 56), image signal processing is performed (step 57).

  In the above-described embodiment, the judgment formula of Formula 1 is calculated in consideration of the gains of the main pixel and the sub-pixel, but a color filter is formed on the light receiving surface of the photoelectric conversion element 27 as shown in FIG. In such a case, the determination formula of Formula 1 may be calculated in consideration of the gain of the color filter and the gains of the main pixel and subpixel.

  FIG. 11 to FIG. 13 show a modification, and show a block diagram of a digital camera. In these drawings, the same components as those shown in FIG.

  A gain correction table memory 15 is provided in the digital camera of the block diagram shown in FIG. The gain correction table memory 15 stores a gain correction table in which data indicating the gain for each pixel is stored. Data indicating the gain correction table stored in the gain correction table memory 15 is given to the defect correction circuit 4 and the gain correction circuit 5. A determination formula of Formula 1 for determining whether or not to perform the above-described defect correction is calculated using data indicating the gain correction table. Further, white balance adjustment is performed in the gain correction circuit 5 using data indicating the gain correction table.

  The digital camera of the block diagram shown in FIG. 12 is provided with an offset / gain correction table memory 16. The offset / gain correction table memory 16 stores an offset correction table in addition to the gain correction table described above. The offset correction in the offset correction circuit 3 can be performed using the offset correction table.

  The digital camera in the block diagram shown in FIG. 13 does not perform defect correction in the defect correction circuit 4 as described above, but performs it by software. The digital camera is also provided with a strobe 65.

  The overall operation of the digital camera is controlled by the CPU 66.

  As described above, the image data output from the image sensor 2 is input to the digital signal processing circuit 61, and predetermined digital signal processing such as offset correction is performed. The image data output from the digital signal processing circuit 61 is applied to the memory 62 and temporarily stored. The image data is read from the memory 62, and the defective pixel correction determination process is performed under the control of the CPU 66 as described above. For defective pixels determined to be defective pixel correction, the correction is executed under the control of the CPU 66.

It is a block diagram which shows the electric constitution of a digital camera. It is an example of the light-receiving surface of an image pick-up element. It is a flowchart which shows the process sequence of a digital camera. The relationship between temperature and threshold is shown. It is an example of the light-receiving surface of an image pick-up element. It is a flowchart which shows the process sequence of a digital camera. It is an example of the light-receiving surface of an image pick-up element. It shows the brightness and the amount of accumulated signal charge. It is an example of the light-receiving surface of an image pick-up element. It is a flowchart which shows the process sequence of a digital camera. It is a block diagram which shows the electric constitution of a digital camera. It is a block diagram which shows the electric constitution of a digital camera. It is a block diagram which shows the electric constitution of a digital camera.

Explanation of symbols

2 Image sensor 4 Defect correction circuit 5 Gain correction circuit 8 CPU
21, 21A, 24, 25, 27 Photoelectric conversion element

Claims (5)

A solid-state electronic image sensor that outputs a plurality of photoelectric conversion elements with different sensitivities on the light-receiving surface and outputs image data representing a subject image by imaging the subject,
Determining means for determining whether or not to correct a defective pixel for each of the photoelectric conversion elements having different sensitivities;
Of the image data output from the solid-state electronic image sensor, the image data obtained based on the signal charge accumulated in the photoelectric conversion element determined to correct the defective pixel by the determination means, Correction means for correcting image data corresponding to a predetermined defective pixel, and among image data output from the solid-state electronic image sensor, image data corrected by the correction means and not corrected by the correction means Amplifying circuit for amplifying the obtained image data at a different amplification factor for each photoelectric conversion element having a different sensitivity,
The determination means is
Said different among the amplification factor, the value exposure time using for amplification ratio of the image data obtained based on correction whether the signal charge accumulated in the subject of the photoelectric conversion element is determined performs the defective pixel When the value corrected for the value exceeds the threshold value , it is determined that the defective pixel is corrected.
Digital camera. A memory for storing an amplification factor for each of the photoelectric conversion elements having different sensitivities;
The amplifying circuit corrects the image data corrected by the correcting means using the amplification factor stored in the memory.
The digital camera according to claim 1. Temperature detecting means for detecting the temperature around the digital camera, and changing means for changing the threshold based on the temperature detected by the temperature detecting means;
The digital camera according to claim 1, further comprising: The solid-state electronic imaging device includes a large number of photoelectric conversion elements having color filters of different colors attached to the light receiving region, and a large number of photoelectric conversion elements having color filters of different colors or transparent filters attached to the light receiving region. Or a large number of photoelectric conversion elements having different light receiving area sizes,
The digital camera according to any one of claims 1 to 3. The solid-state electronic image sensor has a large number of photoelectric conversion elements with different sensitivities on the light-receiving surface. By imaging the subject, image data representing the subject image is output.
The determination means determines whether to correct the defective pixel for each of the photoelectric conversion elements having different sensitivities,
Image data obtained based on the signal charge accumulated in the photoelectric conversion element determined to be corrected by the determination means among the image data output from the solid-state electronic image sensor by the correction means And correcting the image data corresponding to the predetermined defective pixel,
The amplification circuit differs among the photoelectric conversion elements having different sensitivities, among the image data output from the solid-state electronic image sensor, the image data corrected by the correction unit and the image data not corrected by the correction unit. Amplify with gain,
Among the different amplification factors, the determination means determines a value for the amplification factor of the image data obtained based on the signal charge accumulated in the photoelectric conversion element to be determined whether or not to correct a defective pixel. value obtained by correcting the value of the exposure time used is determined that the correction of the defective pixel when the threshold is exceeded,
Digital camera operation control method.

Images