JP4077956B2 - Imaging apparatus and imaging method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus and an imaging method, and is particularly suitable for application to an imaging apparatus that corrects pixel defects in a color imaging element.
[0002]
[Prior art]
In recent years, performances such as the number of pixels, sensitivity, smear suppression ratio, dynamic range, and the like of a solid-state imaging device such as a CCD have been improved year by year due to advances in semiconductor technology. However, a manufacturing process of the solid-state imaging device and a technique for completely preventing the occurrence of pixel defects after the manufacturing have not been established, which greatly affects the manufacturing yield.
[0003]
This pixel defect occurs in the manufacturing process of the solid-state imaging device. In addition to the non-uniformity of exposure caused by dust and the incident light to the pixel being blocked by dust contamination, crystal defects of the semiconductor substrate, There are reports of the effects of heavy metal ions and the like existing in the inside. In addition, pixel defects that occur after manufacture are caused by alpha rays generated from surrounding materials and the environment.
[0004]
Pixel defects generated due to these causes include a white defect having a larger output than a normal pixel and a black defect having a smaller output than a normal pixel, and appear as white or black dots on the screen. Many methods for correcting such pixel defects have been proposed. As a conventional first example, there is Japanese Patent No. 2603640. This is a method of performing correction by removing a sampling pulse corresponding to a defective pixel when sample-holding an image sensor output signal. By this operation, the signal of the pixel before the defective pixel is replaced with the signal of the defective pixel and output.
As a second conventional example, there is JP-A-05-083638. This is a technique for replacing the average value of the signals before and after the defective pixel with the signal of the defective pixel and outputting it.
[0005]
[Problems to be solved by the invention]
However, although the circuit configuration is simplified in the correction of the conventional example, there is a drawback that an error due to replacement occurs when a fine pattern exists near the defective pixel.
[0006]
In the first conventional example, since correction is performed using only the previous pixel, for example, if a picture is taken with a white screen on the left side and a black screen on the right side, and a defective pixel just in the black screen on the right side, The white color appears to bite into the black area. In particular, in the case of a color imaging element using a color mosaic filter, correction is performed using pixels of the same color and correction is performed using the signal of the pixel two pixels before, so the degree of biting increases.
[0007]
In the second conventional example, correction is performed using the left and right pixels, so there is no phenomenon that looks like it bites into the side, but when shooting a thin line vertically, the thin line is defective. A phenomenon occurs in which the pixel portion appears to be cut off by correction. In particular, in the case of a color imaging element using a color mosaic filter, correction is performed using a signal of a pixel that is two pixels away from each other, so that this phenomenon appears to be significant.
[0008]
SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an imaging apparatus and an imaging method capable of correcting defective pixels with extremely little image quality deterioration, excluding these drawbacks of the conventional example.
[0009]
[Means for Solving the Problems]
The imaging device of the present invention includes an imaging element in which pixels of different colors are repeatedly arranged, a pixel having the same color as the pixel to be corrected on the line where the pixel to be corrected of the imaging element exists, Detecting means for detecting a color carrier level indicating a difference between the same color and the different color from the pixel to be corrected and the different color pixel, and using the different color pixel on the line, Interpolation means for obtaining an interpolation value of the pixel to be corrected, and correction means for correcting the value of the pixel to be corrected using a value obtained by combining the color carrier level and the interpolation value.
[0010]
The imaging method of the present invention includes a pixel of the same color as the pixel to be corrected on a line on which a pixel to be corrected of an imaging element in which pixels of different colors are repeatedly arranged, and the correction target on the line. A detection step of detecting a color carrier level indicating a difference between the same color and the different color from the pixel and the different color pixel, and interpolation of the pixel to be corrected using the different color pixel on the line An interpolation step for obtaining a value, and a correction step for correcting the value of the pixel to be corrected using a value obtained by combining the color carrier level and the interpolation value.
[0011]
The storage medium of the present invention is a computer-readable medium characterized by storing a program for causing a computer to execute each step of the above-described imaging method of the present invention.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.
[0020]
FIG. 1 is a block diagram showing the main configuration of a color imaging apparatus according to an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a CCD which is a color image pickup element having a color mosaic filter disposed on an image pickup surface, 2 denotes a timing generation unit for generating a CCD drive pulse and a synchronization pulse Ps for driving the CCD 1, and 3 denotes an output of the CCD 1 An AD converter that converts a signal into a digital signal, 4 is a carrier detection unit that detects a color carrier having a frequency determined by the repetition period of the color mosaic filter of the CCD 1 around the defective pixel, from the output signal of the AD converter 3, An interpolation unit that interpolates defective pixels using signals of pixels around the pixel, 6 is a synthesis unit that combines the carrier detected by the carrier detection 4 and the signal interpolated by the interpolation unit 5, and 7 is a switch circuit. 8 is a defect position memory in which position information of defective pixels of the CCD 1 is written in advance, and 9 is a timing generator. The correction pulse generator 10 generates a correction pulse Pc at a timing corresponding to the defective pixel of the CCD 1 based on the output of the synchronization pulse Ps and the output of the defect position memory 8. Reference numeral 10 denotes a clamp circuit, a color separation circuit, a gamma correction circuit, A process circuit including a contour correction circuit, a white clip circuit, and a dark clip circuit, which forms a color video signal from the output signal of the CCD 1, and 11 is an output terminal for the video signal.
A correction unit is configured including the interpolation unit 5, the synthesis unit 6, and the switch circuit 7.
[0021]
The operation of FIG. 1 will be described below along the signal flow. First, a subject image (not shown) is imaged on a light receiving surface of the CCD 1 through a color mosaic filter on the CCD 1 by an imaging optical system (not shown), and is photoelectrically converted into a signal charge. The signal charges are sequentially transferred in accordance with the CCD driving pulse generated by the timing generator 2, and are converted into an imaging signal by the charge voltage converter in the output portion of the CCD 1 and output. This imaging signal is converted into a digital imaging signal by the AD converter 3. This digital imaging signal is first input to the switch circuit 7. The switch circuit 7 is controlled by the correction pulse Pc that is the output of the correction pulse generator 9, but when it is not a signal corresponding to a defective pixel, the digital imaging signal that is the output of the AD converter 3 is selected and the process circuit 10 Is input.
[0022]
In the process circuit 10, a color video signal is formed by each of the processing units described above, and is output from the output terminal 11 to a television monitor or VTR which is an external device (not shown).
[0023]
The digital image signal output from the AD converter 3 is also input to the carrier detection unit 4 and the interpolation unit 5. The carrier detection unit 4 detects the level of the color carrier having a frequency determined by the repetition period of the color mosaic filter of the CCD 1 around the defective pixel as described above. In addition, the interpolation unit 5 obtains an interpolation value for interpolating the level of the defective pixel from the signals of the pixels around the defective pixel. The color carrier level obtained from the carrier detection unit 4 and the interpolated value obtained from the interpolation unit 5 are synthesized in the synthesis unit 6 and replaced with the defective pixel signal to obtain a synthesized signal. The synthesized signal is input to the switch circuit 7 and, as described above, the synthesized signal is selected only for the signal corresponding to the defective pixel by the correction pulse Pc and replaced with the digital imaging signal of the defective pixel, thereby correcting the digital imaging. The signal is input to the process circuit 10 as described above.
[0024]
The correction pulse generator 9 reads the defective pixel data from the CCD 1 from the synchronization pulse Ps generated from the timing generator 2 and the data of the defective pixel position of the CCD 1 previously written in the defect position memory 8. To generate a correction pulse Pc. The defective pixel position data is stored by, for example, a method of detecting the position of the defective pixel by inspecting the CCD 1 alone, writing it in the defect position memory 8 in advance, and incorporating it into the apparatus. At this time, the number of defective pixels may be zero, or may be one or more, depending on the state of the CCD 1. The correction pulse generator 9 outputs a number of correction pulses Pc corresponding to the number of defective pixels within one field period.
[0025]
The generated correction pulse Pc is supplied to the selection terminal of the switch circuit 7. As described above, when it does not correspond to the position of the defective pixel, the output of the AD converter 3 is selected, and when it corresponds to the position of the defective pixel, It operates so that the output of the synthesis unit 6 is selected.
[0026]
FIG. 2 is an explanatory diagram of the operation of the imaging apparatus of the present embodiment.
FIG. 2A shows an example of the color mosaic filter of the CCD 1. This color mosaic filter is of a type called a Bayer arrangement in which RGB primary color filters are arranged by repeating RGRG for odd lines and GBGB for even lines. In this case, since the repetition period in the horizontal direction of each color filter is 2 pixels, the color carrier frequency in the horizontal direction is ½ of the horizontal transfer frequency.
[0027]
In FIG. 2B, a solid line indicates a digital image pickup signal output from the AD converter 3, and represents an odd-numbered line in the arrangement of the color mosaic filter in FIG. An example of the subject image is an image representing an upwardly convex luminance change in FIG.
[0028]
In the odd-numbered lines, signals are obtained in the order of RGRG, but R and G have a predetermined ratio depending on the ratio of the color components of the subject. In FIG. 2B, for example, when A, C, E, G, and I are R pixels and B, D, F, and H are G pixels, the ratio of the R component is smaller than that of the G component. At this time, the difference between R and G is the color carrier, and is shown as the color carrier level in the figure. The color carrier level can be obtained as an average difference between the plurality of R pixels and the plurality of G pixels.
[0029]
In the example, the pixel E is defective among the pixels A to I, and a value larger than the normal value is output. At this time, the carrier detection 4 first detects and outputs the above-described color carrier level. Further, the interpolation unit 5 obtains an interpolation value from an average of two pixels on the left and right of the defective pixel, for example. In the synthesis unit 6, for example, the color carrier level and the interpolation value are simply added. The combined value obtained by the addition becomes a digital image pickup signal selected and corrected by the switch circuit 7, and is input to the process circuit 10 as described above to form a video signal.
[0030]
Compared with the interpolated values of the same color pixels in the figure, the interpolated values from the left and right pixels are reflected, so that the signal of the original image is better interpolated. Also, compared with the simple left-right average interpolation value, the color carrier level values are combined, so that the color carrier value does not change partially due to the correction.
[0031]
FIG. 3 is a block diagram showing in detail the main part of the color imaging apparatus of FIG. 101 to 109 and 118 to 121 are D-type flip-flops (hereinafter abbreviated as DFF) each having a predetermined bit width, 110 to 112, 114, 117, and 122 are adders, and 115 and 123 are 1 / 2 is a coefficient multiplier, 113 is a quarter coefficient multiplier, and 116 is a subtractor.
[0032]
In FIG. 3, the carrier detection unit 4 in FIG. 1 includes 104 to 116, the interpolation unit 5 includes 118 to 123, and the synthesis unit 6 includes 117.
[0033]
First, an operation corresponding to the carrier detection unit 4 will be described.
The input digital imaging signal S _DCCD is delayed by one pixel in each of the DFFs 104 to 109. The signals of the same color are added to the outputs of these DFFs. First, the imaging signal S _DCCD and the output of the DFF 109 are added by the adder 111, and then the outputs of the DFF 105 and the DFF 107 are added by the adder 110. The outputs of the adders 110 and 111 are further added by an adder 112 and multiplied by a coefficient of ¼ by a coefficient unit 113 to obtain an average value of odd-numbered pixel values. Further, the output of the DFF 104 and the output of the DFF 108 are added by the adder 114 and multiplied by a coefficient of 1/2 by the coefficient unit 114 to obtain an average value of even-numbered pixels. The difference between the two average values obtained in this way is taken by the subtractor 116 to obtain the color carrier level.
[0034]
Subsequently, an operation corresponding to the interpolation unit 5 will be described.
The output of the DFF 119 and the output of the DFF 121 obtained by delaying the output of the DFF 119 by two pixels are added by the adder 122, multiplied by a factor of 1/2 by the coefficient unit 123, and the pixel adjacent to the defective pixel that is the interpolation value is added. An average value is obtained.
[0035]
The color carrier level and the interpolated value obtained in this way are added by the adder 117 which is the synthesizer 6, and the output thereof is output by the switch circuit 7 and the imaging signal S _DCCD is output by the DFFs 101 to 103 according to the interpolation pulse Pc. Switch with delayed signal. The DFFs 101 to 103 are inserted in order to match the delay time of the signals by the DFFs 104 to 106 or the DFFs 118 to 120.
[0036]
The signal switched by the switch circuit 7 is input to the process circuit 10 as described above. By this operation, the color carrier level and the interpolation value are obtained from the digital image pickup signal, and an operation of replacing the synthesized signal with the defective pixel signal according to the correction pulse is performed.
[0037]
As described above, according to the color imaging apparatus of the present embodiment, correction of defective pixels with extremely little image quality deterioration can be performed easily and reliably by switching the switch circuit 7 using the correction pulse Pc from the correction pulse generator 9. Is possible.
[0038]
Here, some modified examples of the color imaging apparatus of the present embodiment will be described. In addition, about the component etc. which are the same as that of the imaging device of this embodiment, a same sign is attached and description is abbreviate | omitted.
[0039]
(Modification 1)
The color imaging apparatus of the first modification is configured as shown in FIG. In FIG. 4, 201 to 206 are DFFs, 207 to 209 and 214 are adders, 210 and 212 are 1/4 coefficient units, 211 is a 1/2 coefficient unit, and 213 is a subtractor. is there. In this example, common parts in FIG. 3 are shared. For example, the DFFs 101, 104, and 118 in FIG. 3 are shared with the DFF 201, the DFFs 102, 105, and 119 are shared with the DFF 202, and the DFFs 103, 106, and 120 are shared with the DFF 203. Further, the combination of the adder is changed according to the normal replacement law, but the calculation results are the same. Therefore, since the operation function of FIG. 4 is exactly the same as that of FIG. 3, detailed description is omitted.
[0040]
(Modification 2)
The main part of the color imaging apparatus of the second modification is configured as shown in FIG. In FIG. 4, reference numerals 301 to 306 are delay lines (1HDL) for one horizontal period, 307 is a two-dimensional band pass filter (BPF), 308 and 314 are delay lines having a predetermined delay time, 309 and 310 are DFFs, 311 and 313 are adders, and 312 is a 1/2 coefficient unit.
[0041]
In the configuration of FIG. 5, the carrier detection unit 4 in FIG. 1 includes 301 to 307, the interpolation unit 5 includes 308 to 312, and the combining unit includes 313.
[0042]
The 1HDL 301 to 306 delay the signal in the vertical direction, and detect the level of the color carrier by the two-dimensional BPF 307 using their outputs. The characteristics of the two-dimensional bandpass filter 307 are obtained from the signal obtained from the color filter of the CCD 1 corresponding to the current pixel that is the output of the delay line 314 and the color filter corresponding to the interpolation pixel corresponding to the output of the interpolation unit 5. It is set to a characteristic for detecting an output difference from the obtained signal.
[0043]
The interpolating unit 5 composed of 308 to 312 is substantially the same as the interpolating unit of FIG. 3, but a delay line 308 is inserted in order to match the delay time in the two-dimensional BPF 307.
[0044]
The output of the interpolation unit 5 is combined with the output of the color carrier detection 4 by the adder 313 as in FIG. The output is switched to the output of the delay line 314 in accordance with the PC, whereby the defective pixel is replaced.
[0045]
Note that the program code itself for operating various devices and means for supplying the program code to a computer so as to realize the functions of the imaging apparatus described in the present embodiment and the modifications thereof, and the imaging apparatus The program code itself for realizing the imaging method using the computer and the means for supplying the program code to the computer, for example, the storage medium storing the program code belong to the category of the present invention.
[0046]
In this case, the program code stored in the storage medium is read out by a predetermined storage / reproduction device, and the EEPROM operates. As a storage medium for storing the program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
[0047]
Further, by executing the program code supplied by the computer, not only the functions of the embodiments are realized, but also the OS (operating system) or other application software running on the computer. Such a program code is also included in the present invention even when the functions of the respective embodiments are realized together.
[0048]
Further, after the supplied program code is stored in the memory provided in the function expansion board of the computer or the function expansion unit connected to the computer, the CPU provided in the function expansion board or function expansion unit based on the instruction of the program code A system in which the functions of the present embodiment are realized by performing part or all of the actual processing and the processing is also included in the present invention.
[0049]
【The invention's effect】
According to the present invention, it is possible to easily and reliably perform correction of a defective pixel with extremely little image quality deterioration by switching the switch circuit 7 with a correction pulse from the correction pulse generating means.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a main configuration of a color imaging apparatus according to an embodiment of the present invention.
FIG. 2 is a schematic diagram for explaining the operation of the color imaging apparatus according to the embodiment of the present invention.
3 is a block diagram showing in detail a main part of the color imaging apparatus of FIG. 1;
FIG. 4 is a block diagram illustrating in detail a main part of a color imaging apparatus in Modification 1 of the embodiment of the present invention.
FIG. 5 is a block diagram illustrating in detail a main part of a color imaging apparatus in a second modification of the embodiment of the present invention.
[Explanation of symbols]
1 CCD
2 Timing generator 3 AD converter 4 Carrier detector 5 Interpolator 6 Synthesizer 7 Switch circuit 8 Defect position memory 9 Correction pulse generator 10 Process circuit 11 Output terminals 101 to 109, 118 to 121, 201 to 206, 309, 310 D-type flip-flop (DFF)
110 to 112, 114, 117, 122, 207 to 209, 214, 311, 313 Adder 115, 123, 211, 312 1/2 coefficient unit 113, 210, 212 1/4 coefficient unit 116, 213 subtractor 301 to 306 Delay line for one horizontal period (1HDL)
307 Two-dimensional bandpass filter (BPF)
308,314 Delay line

Claims (7)

An image sensor in which pixels of different colors are repeatedly arranged;
From the pixel of the same color as the pixel to be corrected on the line where the pixel to be corrected of the image sensor is present, and the pixel of the color different from the pixel to be corrected on the line, the same color and the different color Detecting means for detecting a color carrier level indicating a difference between
Interpolation means for obtaining an interpolation value of the pixel to be corrected using the different color pixels on the line;
An image pickup apparatus comprising: a correction unit that corrects a value of the pixel to be corrected using a value obtained by combining the color carrier level and the interpolation value. A memory for storing a position of a defective pixel of the image sensor;
2. The correction unit according to claim 1, wherein the correction unit performs the correction on a pixel at a position stored in the memory and does not perform the correction on a pixel that is not at a position stored in the memory. Imaging device. Generating means for outputting a correction pulse according to the position stored in the memory;
The imaging apparatus according to claim 2, wherein the correction unit performs the correction when the correction pulse is supplied. The detecting device, the average of the correction that the pixel with the same color of a plurality of pixels of the signal, the difference between the average of a plurality of pixels of the signal of the pixel different from the color to be the corrected, detecting the color carrier level The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus.   5. The image pickup apparatus according to claim 1, wherein the interpolation unit obtains the interpolation value by an average of left and right pixels of the correction target pixel. 6. A pixel of the same color as the pixel to be corrected on the line where the pixel to be corrected of the imaging device in which pixels of different colors are repeatedly arranged, and a pixel of a different color from the pixel to be corrected on the line From the detection step of detecting a color carrier level indicating a difference between the same color and the different color,
An interpolation step for obtaining an interpolation value of the pixel to be corrected using the differently colored pixels on the line;
An imaging method comprising: a correction step of correcting a value of a pixel to be corrected using a value obtained by combining the color carrier level and the interpolation value.   A computer-readable storage medium storing a program for causing a computer to execute each step of the imaging method according to claim 6.

Images