JPH0441866B2 - - Google Patents

Description

[Detailed description of the invention]

INDUSTRIAL APPLICATION FIELD The present invention relates to a scratch correction device for a video camera or the like using a solid-state image sensor. BACKGROUND ART In recent years, video cameras using solid-state image sensors are entering the practical stage. However, the degree of integration is high;
Solid-state imaging devices are very expensive because they have a large chip area and handle analog signals and have a low yield. Therefore, this is a major obstacle to the widespread use of video cameras using solid-state image sensors. When a solid-state image sensor has an image defect such as a white spot or a black spot scratch, it is necessary to correct the defect in a signal processing circuit. A conventional scratch correction device is reported, for example, in the Technical Report of the Television Society Vol. 7 No. 14, pages 19 to 24. FIGS. 9 and 10 show block diagrams of this conventional scratch correction device. The color filter array of the solid-state image sensor to be corrected is
For example, it is configured as shown in FIG. In other words, the nH line is made up of repeated W (white) and G (green) color filters, and (n+1)H
The line is made up of repeated Cy (cyan) and Ye (yellow) color filters. Now, if the m-th pixel signal Sm of the nH line is a defect signal, the state of the input and output signals of the scratch correction device shown in FIGS.
This is shown in Figures 1 and 12. The operations of these conventional examples will be explained below. In FIG. 9, when the control signal 906 of the multiplier 902 is at a low level (hereinafter referred to as "L"), a clamp level is output from the multiplier 902, and when it is at a high level (hereinafter referred to as "H"), the multiplier 902 outputs a clamp level. , an input pixel signal 900 is output. The inversion of the defect correction control signal 908 is input to the multiplier 902, and the multiplier 902
Non-inversion is input to 05. Therefore, when the defect correction control signal 908 is in the "L" state, the output 907 of the adder 903 is the output signal 909 of the clamp circuit 901 as it is. Furthermore, in the "H" state, the output signal 910 of the delay circuit 904 is output. As a result, as shown in FIG. 11, the defective pixel signal Sm is replaced with the signal S _n-2 (indicated by * in the figure) two pixels before, and the defective pixel and the defective pixel are When the light intensity is the same, the defective pixel signal Sm is S _n-2
It is corrected to Also in FIG. 10, the non-inverted and inverted signals of the defect correction control signal 1003 are input to multipliers 1001 and 1002, respectively. There are two delay circuits that delay the time by two pixels, and the output signal 1005 of the adder 1004
When the defect correction control signal 1003 is in the "L" state, the output signal 1007 of the delay circuit 1006 is output as is. In addition, in the "H" state, the output signal of the delay circuit 1008 and the clamp circuit 1009
, and the output signal are added by an adder 1010 , and the added signal 1011 is halved by a multiplier 1001 and output from an adder 1004 . As a result, as shown in Fig. 12, the defective pixel signal Sm is replaced with the average value of the other two pixel signals S _n-2 and S _n+2 , so if the following equation holds true, correction can be made. I can do it. Signal entering the defective pixel = 1/2 (S _n-2 + S _n+2 ) The position of the defective pixel is known in advance, and in the configuration shown in Figure 9 it is always pre-predicted, and in the configuration shown in Figure 10 it is always averaged. It can be said that the defective pixel signal Sm is corrected by performing signal processing based on prediction. Problems to be Solved by the Invention However, in the above configuration, the replacement signal for correcting defective pixels such as scratches is a signal with the same positional relationship for all captured images. Signal errors occur due to changes in type or image, which appear on the screen and significantly degrade image quality. Furthermore, since multipliers and adders are used, the circuit configuration becomes large-scale and complicated, and power consumption is also large. In view of the above, an object of the present invention is to provide a flaw correction device that can select a replacement signal for correcting a defective pixel in response to the type of image captured and changes in the image with a simple circuit configuration. . Means for Solving the Problems The present invention converts the m-th pixel signal extracted from the solid-state image sensor into a difference signal between the i-th pixel signal and the j-th pixel signal, and a difference signal between the k-th pixel signal and the l-th pixel signal. The flaw correction device replaces the pixel signal with the p-th pixel signal, the q-th pixel signal, the average value of the p-th and q-th pixel signals, or the r-th pixel signal, depending on the value of the difference signal from the pixel signal. . Effects According to the present invention, with the above-described configuration, a defective pixel correction replacement signal most suitable for the type and change of the captured image can be easily obtained, and deterioration of image quality can be sufficiently prevented. Embodiment FIG. 1 shows a block diagram of a scratch correction device in an embodiment of the present invention. In FIG. 1, 101 is a solid-state image sensor, 102 is an A/D converter, 103 is a delay circuit, 104 and 110 are switching circuits, 105 is a drive circuit, 106 is a read-only storage device (hereinafter referred to as ROM), 107 108 is a detection correction pixel signal extraction circuit, 108 is a correction signal generation circuit,
109 is a correction signal switching judgment circuit. The operation of the scratch correction device of this embodiment configured as described above will be described below. Pixel signals taken out from the solid-state image sensor 101 controlled by the drive circuit 105 are converted into digital signals by the A/D converter 102. When this digital pixel signal includes a defect signal, the pixel signals before and after the defect signal are extracted using the ROM 106 and the detection/correction pixel signal extraction circuit 107. These pixel signals are input to the correction signal generation circuit 108 and the correction signal switching determination circuit 109, and the switching circuit 1
10, a corrected pixel signal is created. Then, the original defect signal passing through the delay circuit 103 is switched to a corrected pixel signal by the switching circuit 104, and the corrected pixel signal is obtained as an output. Since the position of the defective pixel is known in advance, if it is stored in the RAM 106, the pixel signal for detection correction can be easily extracted. For example, suppose that one pixel 201 on the nH line is a defective pixel in a solid-state imaging device having a repeating color filter arrangement as shown in FIG. 2 as well as FIG. 8. This defective pixel signal W ₃ 2
The pixel signals for correcting 01 are the two pixel signals W ₂ and W ₄ on both sides thereof, their average value signal (W ₂ +W ₄ )/2, and (n-2) on the H line, which is the defective pixel 201. There is a pixel signal _W1 located directly above. (n-1)H line is Cy, Ye
Since the pixel signal is a repetition of
It cannot be used as a correction signal for W ₃ 201. Further, a judgment signal for selecting the optimum signal among the four correction signals listed above is generated from the pixel signals G ₁ , G ₂ , G ₃ , and G ₄ shown in FIG. 2. That is,
In order to understand the signal changes before and after defective pixel signal _W3 ,
Two difference signals G ₂ -G and G ₃ -G ₄ are thus created. As described above, W ₁ , G ₁ ,
It is necessary to extract seven signals: W ₂ , G ₂ , G ₃ , W ₄ , and G ₄ . FIG. 3 is a configuration diagram of the detection correction pixel signal extraction circuit 107 that extracts these seven signals. 301
is a shift register with 1 input and 7 outputs. Also,
Output data of ROM 106, pixel signal continuous sampling clock pulse and shift register 30
The waveform of the clock pulse 303 applied to
They are shown in FIGS. 4a, b, and c, respectively. A pixel signal is input to the shift register 301, but the clock pulse 303 that controls it is
The output data of the ROM 106 and the continuous sampling pulse of the pixel signal are signals created through the AND circuit 302. Therefore, as can be seen from FIG. 4c, the signals taken into the shift register 301 are W ₁ , G ₁ ,
These are seven pixel signals: W ₂ , G ₂ , G ₃ , W ₄ , and G ₄ .
Since these seven signals are output in parallel from the shift register 301, the required pixel signals are extracted. FIG. 5 is a configuration diagram of the correction signal generation circuit 108 and the switching circuit 110. in the correction signal
Since W ₁ , W ₂ , and W ₄ are already extracted pixel signals themselves, it is sufficient to input W ₂ and W ₄ to the averaging circuit 501 to generate only the average value signal (W ₂ +W ₄ )/2. Switching signal 5 for selecting these four correction signals
02 can be obtained from FIG. FIG. 6 is a block diagram of the correction signal switching judgment circuit 109. Among the correction signals extracted by the detection correction pixel signal extraction circuit 107, four pixel signals G ₁ , G ₂ , G ₃ , and G ₄ are used. The difference signal is obtained by subtracters 601 and 602.
G ₂ −G ₁ and G ₃ −G ₄ are generated, and absolute value circuits 6, 603, and 604 calculate the absolute value of these difference signals |
Obtain G ₂ −G ₁ |, |G ₃ −G ₄ |. Then, these two values are compared with the setting data 607 by comparators 605 and 606, respectively, and when the value is larger than the setting data 607, "H" is output, and when it is not, "L" is output. Also, the difference signal G ₂
For the sign bits of −G ₁ and G ₃ −G ₄ ,
Through EXCLUSIVE-OR and NOT circuit 609,
When the two difference signals have the same sign, "H" is output, and when they have different signs, "L" is output. Output signals 610 from comparators 605, 606,
611 and EXCLUSIVE-OR and NOT circuit 6
The signal 612 from 09 is the switching signal generation circuit 608
A selection signal 613 of the optimal correction signal is obtained. The judgment table is shown in Table 1.

[Table] However, "X" indicates that either "L" or "H" may be used. FIG. 7 shows the levels of pixel signals before and after the defective pixel signal _W3 corresponding to the selected replacement signal. Figure a is an example where |G ₂ - G ₁ | is smaller than the setting data 607 and |G ₃ - G ₄ | is larger than the setting data 607, and the defective pixel is the previous pixel signal.
Replaced by W ₂ . In addition, FIG. b is an example in which |G ₂ - _{G 1} | is larger than the setting data 607 and |G ₃ - G ₄ | is smaller than the setting data 607. In this case, the defective pixel is the subsequent pixel signal W Replaced by ₄ . It is clear that the optimal replacement pixel signals in a and b of the figure are W ₂ and W ₄ , respectively. If both are replaced with the average value signal of W ₂ and W ₄ (W ₂ +W ₄ )/2, the pixel signal level will be set at the portion marked with Δ, resulting in a large error. In the same figure, c is the absolute value of the difference signal |G ₂ −G ₁ |, |G ₃ −
When G ₄ | is both smaller than the setting data 607,
Or, if both are large and the difference signals are of opposite signs, the defective pixel is the average value of W ₂ and W ₄ (W ₂ +
W ₄ )/2. Figure d shows a case where edges are formed on both sides of the defective pixel, and there is, for example, a thin bright line in the vertical scanning direction on the screen. In this case, even if the pixel signals before and after the defective pixel are replaced with their average value signal, the originally required
The error with the _W3 pixel signal is very large. Therefore,
This error will disappear if it is replaced with _W1 , which is the pixel signal closest to the vertical scanning direction. The conditions are as shown in Table 1, the absolute value of the difference signal |
G ₂ −G ₁ |, |G ₃ −G ₄ | are both setting data 60
7 and these difference signals have the same sign. However, cases like d in the same figure rarely occur with ordinary subjects, so cases a, b, and c in the same figure
Even if the replacement is performed in the three cases described above, the effect of flaw correction is sufficient and the image quality is significantly improved. In that case, there are six pixel signals for detection and correction, so the circuit scale of the flaw correction device becomes even smaller. Although this embodiment has been described assuming that the color filter arrangement of the solid-state image sensor is as shown in FIG. 2, it goes without saying that the present invention can be similarly applied to other color filter arrangements. Effects of the Invention As described above, according to the present invention, defective pixels in a solid-state image sensor can be corrected by replacing them with optimal pixel signals, thereby preventing deterioration of image quality and reducing the cost of the solid-state image sensor. I can do it. Further, the flaw correction device itself of the present invention has a simple circuit configuration and low power consumption, and its practical effects are very large.

[Brief explanation of drawings]

FIG. 1 is a block diagram of a scratch correction device according to an embodiment of the present invention, FIGS. 2 and 8 are color filter arrangement diagrams of a solid-state image sensor, and FIG. 3 is a configuration diagram of a pixel signal extraction circuit for detection correction. Fig. 4 is a timing chart of the clock pulse in Fig. 3, Fig. 5 is a block diagram of the correction signal generation circuit and switching circuit, Fig. 6 is a block diagram of the correction signal switching judgment circuit, and Fig. 7 is the pixels before and after the defective pixel. A waveform diagram showing signal levels, FIGS. 9 and 10 are block diagrams of a conventional scratch correction device, and FIGS. 11 and 12 are signal timing charts of each block in FIGS. 9 and 10, respectively. . 101...solid-state image sensor, 106...ROM,
107... pixel signal extraction circuit for detection correction, 108
... Correction signal generation circuit, 109 ... Correction signal switching judgment circuit, 104, 110 ... Switching circuit, 301
...Shift register, 501 ...Average circuit, 60
1,602...Subtractor, 603,604...Absolute value circuit, 605,606...Comparator, 607...
Setting data, 608...Switching signal generation circuit.

Claims (1)

[Claims] 1. The m-th pixel signal (m is a natural number) extracted from the solid-state image sensor and the i-th pixel signal obtained from the pixel on the same line as the m-th pixel and j
th pixel signal (i, j are natural numbers, i<j<m)
and the value of the difference signal between the k-th pixel signal and the l-th pixel signal (k, l are natural numbers, m<k<l) obtained from pixels on the same line as the above line. p-th or q-th pixel signal (p, q are natural numbers, p<m<q), the p-th and q
Replace with the average value of the pixel signal of the m-th pixel, or the r-th pixel signal obtained from the pixel located directly above the m-th pixel in the upper line where color filters of the same combination as the line are arranged. A scratch correction device characterized by: (However, i, j,
The color filters corresponding to the k-th and l-th pixels are the same color. There are equal intervals between the m-th, i-th, and k-th pixels. The mth, jth, and lth pixels are equally spaced. ) 2 The color filter array of the solid-state image sensor consists of lines made up of repeated color filters of W (white) and G (green), and lines of Cy (cyan) and Ye (yellow).
2. The scratch correction device according to claim 1, wherein lines formed by repeating color filters are arranged alternately. 3. The m-th pixel signal Sm (m is a natural number) extracted from the solid-state image sensor is converted into the i-th pixel signal Si obtained from the pixel on the same line as the m-th pixel.
and j-th pixel signal Sj (i, j are natural numbers, i<j
<m) absolute value of the difference signal |Sj-Si| is smaller than a certain constant value A, and the k-th pixel signal Sk and the l-th pixel signal Sl (k, l are natural numbers, m<k<l)
When the absolute value of the difference signal |Sk−Sl| is larger than A, the p-th pixel signal Sp obtained from the pixel on the same line as the m-th pixel (p is a natural number, p
<m), and when the above |Sj−Si| is larger than A and the above |Sk−Sl| is smaller than A, the qth pixel signal obtained from the pixel on the same line as the mth pixel. A scratch correction device characterized in that the substitution is made with Sq (q is a natural number, m<q), and in other cases, the substitution is made with average values Sp and q of the p-th and q-th pixel signals. (However, the color filters corresponding to the i, j, k, and l-th pixels are the same color. The m-th, i-th, and k-th pixels are equally spaced. The m-th, j-th, and l-th pixels are (The pixels are equally spaced.) 4 The color filter array of the solid-state image sensor consists of lines made up of repeated color filters of W (white) and G (green), as well as lines of Cy (cyan) and Ye (yellow).
4. The scratch correction device according to claim 3, wherein lines formed by repeating color filters are arranged alternately. 5 The m-th pixel signal Sm (m is a natural number) extracted from the solid-state image sensor is converted into the i-th pixel signal Si obtained from the pixel on the same line as the m-th pixel.
and j-th pixel signal Sj (i, j are natural numbers, i<j
<m) absolute value of the difference signal |Sj-Si| is smaller than a certain constant value A, and the k-th pixel signal Sk and the l-th pixel signal Sl (k, l are natural numbers, m<k<l)
When the absolute value of the difference signal |Sk−Sl| is larger than A, the p-th pixel signal Sp obtained from the pixel on the same line as the m-th pixel (p is a natural number, p
<m), and when the above |Sj−Si| is larger than A and the above |Sk−Sl| is smaller than A, the qth pixel signal obtained from the pixel on the same line as the mth pixel. Replaced with Sq (q is a natural number, m < q), and the above |Sj - Si | and the above |Sk - Sl | are both greater than A, and the value obtained by subtracting Si from the above Sj
When the product (Sj-Si)/(Sk-Sl) of Sj-Si and Sk-Sl, the value obtained by subtracting Sl from Sk, is positive, a color filter of the same combination as the m-th pixel line is arranged. The r-th pixel signal Sr obtained from the pixel located directly above the m-th pixel in the upper line where A scratch correction device characterized by replacing. (However, the color filters corresponding to the i, j, k, and l-th pixels are the same color. The m-th, i-th, and k-th pixels are equally spaced. m
The intervals between the th, jth, and lth pixels are also equal. ) 6 The color filter array of a solid-state image sensor consists of lines made up of repeated color filters of W (white) and G (green), and lines of Cy (cyan) and Ye (yellow).
6. The scratch correction device according to claim 5, wherein lines formed by repeating color filters are arranged alternately.