JP2011114473A - Pixel defect correction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pixel defect correction device capable of accurately correcting a defective pixel in exactly real time or a small number of processing times even when there is the defective pixel among eight peripheral adjacent pixels. <P>SOLUTION: The pixel defect correction device includes: a central pixel defect-determining means for determining whether video data of a predetermined pixel obtained by converting an output signal from a solid-state imaging element into a digital signal is an effective pixel or a defective pixel; an adjacent pixel defect-determining means for determining whether eight adjacent pixels in all directions of the pixel determined to be the defective pixel are effective pixels or defective pixels, when determined to be the defective pixel; an interpolation value-calculating means for calculating any one value of an average value, a median or a mode of the video data of effective pixels among the eight adjacent pixels as an interpolation value; and a correction means for correcting the defective pixel with the calculated interpolation value. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a pixel defect of a solid-state image sensor, and more particularly to a pixel defect of a solid-state image sensor used for a video camera or a television camera.

The pixel defect correction of a solid-state image sensor used in a conventional video camera or television camera will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an example of a pixel defect correction device in a conventional solid-state imaging device (see Patent Document 3). 101 is an input terminal, 102 and 103 are 1 line delay elements, 104 is a current line signal, 105 is a 1 line delay signal, 106 is a 2 line delay signal, 107 to 112 are 1 pixel delay elements, 113 is a pixel defect detection unit, 114 is a preamplifier gain signal, 115 is a CPU (Central Processing Unit), 116 is a detection threshold signal, 117 is a detection signal, 118 and 119 are detection signals of other colors, 125 is a detection result processing unit, 120 is a detection signal, 121 is an interpolation signal level, 122 is a delay element, 123 is a selector, and 124 is an output signal.
Note that a pixel defect correction circuit is required for each solid-state imaging device such as a CCD that images imaging light of R (red), G (green), and B (blue) colors. Since all are the same, only the case of R (red) will be described here.
Further, hereinafter, a video camera or a television camera is collectively referred to as a camera.

In FIG. 1, a video signal obtained from a solid-state image pickup device such as an R CCD (not shown) is digitalized by an A / D (Analog to Digital) converter (not shown) via a correlated double sampling circuit, a preamplifier circuit, etc. A video signal is input to the input terminal 101.
The digital video signal input to the input terminal 101 is delayed by one horizontal scanning period by each of the one-line delay elements 102 and 103, thereby being delayed by one horizontal scanning period from the current line signal 104 and the current line signal 104. The video signals for three lines of the one-line delay signal 105 and the two-line delay signal 106 delayed by one horizontal scanning period from the one-line delay signal 105 are one-pixel delay elements 107 to 111 and a pixel defect detection unit. It is output to 113. At the same time, the one-line delay signal 105 is output to the delay element 122. The delay element 122 delays a signal input so as to be synchronized with an interpolation signal repel 121 input to one input terminal of the selector 123 described later, and outputs the delayed signal to the other input terminal of the selector 123.
Further, the one-pixel delay elements 107 to 112 are, for each of the three lines of video signals, a video signal without pixel delay, a one-pixel delay signal delayed by one pixel data signal period from a video signal without pixel delay, In addition, the video signal for three pixels of the two-pixel delay signal delayed by one pixel data signal period from the one-pixel delay signal is output to the pixel defect detection unit 113.

  Hereinafter, an operation of the pixel defect detection unit 113 for detecting a defective pixel using the above-described video signal and correcting a signal from the defective pixel will be described. Here, a video signal from a defective pixel to be detected and corrected is described as “center” with a signal delayed by one line in the vertical direction and further delayed by one pixel in the horizontal direction. The delay signal is “upper left”, the current pixel 1 pixel delay signal is “up”, the current line 2 pixel delay signal is “upper right”, the non-delay signal after 1 line delay is “left”, and the 1 line delay 2 The pixel delay signal is “right”, the non-delay signal after 2 line delay is “lower left”, the 1 pixel delay signal after 2 line delay is “lower”, and the 2 pixel delay signal after 2 line delay is “lower right”. Describe.

A total of nine pixel data signals of the above-described central pixel and adjacent eight pixels around it are input to the pixel defect detection unit 113. The pixel defect detection unit 113 first calculates the pixel data signal obtained by averaging the “center” pixel data signal level and the “upper” and “lower” as the first determination condition for the input nine pixel data signals. The absolute value of the difference from the signal level and the absolute value of the difference between the “center” pixel data signal level and the pixel data signal level averaged from “right” and “left” are calculated. Then, it is checked whether the pixel data signal level to be detected has a higher correlation in the vertical direction or the horizontal direction.
When shooting a subject with a camera, the output of the solid-state image sensor is spread evenly up, down, left, and right, regardless of how small the object is a point light source. The video signal has a correlation of On the other hand, in the case of a video signal with a white defect, there is no correlation in the vertical direction because the video signal has a spread only on the same line, that is, in the left-right direction for low-pass filter processing before A / D conversion. Therefore, when the correlation is stronger in the vertical direction than in the horizontal direction, it can be determined that the pixel data signal level is not a flaw defect.

Further, in the second determination condition, the difference obtained by subtracting the “upper” and “lower” pixel data signal levels from the “center” pixel data signal level, and the “center” pixel data signal level, respectively. The absolute value of the difference between the level and the pixel data signal level of “right” and “left” is calculated.
The difference between the experimentally obtained white-scratch defective pixel and the level of the adjacent pixels above, below, left and right is set as a preset level, and at least one difference value is equal to or less than the set level Is determined not to be a pixel defect within the normal noise range.

Further, in the third determination condition, a pixel obtained by averaging the pixel data signal level of “center” and the four pixel data signal levels of “upper left”, “upper right”, “lower left”, and “lower right”. Calculate the difference from the data signal lepel. The detection threshold signal 16 is set by the CPU 115 based on a preamplifier amplification factor signal 114 representing the signal amplification factor of the preamplifier before A / D conversion.
When the signal amplification factor is large, the detection threshold is set high, thereby preventing erroneous detection of pixel defects due to noise. Further, by using the pixel data signal level in the oblique direction, when the signal amplification factor is large, it is possible to ignore a particularly significant spread of the pixel defect correlation in the left-right direction.

The above three determination conditions are expressed as follows.
Absolute value of first determination condition (P _C − (P _T + P _B ) / 2)> Absolute value of (P _C − (P _L + P _R ) / 2) Expression (1)
Second determination condition absolute value of _{(P C -P L)> T} C ··· formula (2)
Absolute value of (P _C −P _R )> T _C Formula (3)
(P _C −P _T )> T _C (4)
(P _C −P _B )> T _C (5)
Third determination condition (P _C − (P _TL + P _TR + P _BL + P _BR ) / 4> T _V Expression (6)
Here, P _C , P _T , P _B , P _L , P _R , P _TL , P _TR , P _BL , P _BR are respectively “center”, “up”, “down”, “left”, “ right "," upper left "," upper right "," lower left "represents a pixel data signal level of the" lower right ", T _C is the detection threshold of the fixed, T _V is the detection threshold which varied according to the signal amplification factor of the preamplifier Represents. When Pc satisfies all of the expressions (1) to (6), the pixel defect detection unit 113 determines that the center pixel of the input video signal is the video signal from the defective pixel, and outputs the detection signal 117. Output. That is, the pixel defect detection unit 113 outputs, as the detection signal 117, information indicating whether the center pixel is an effective pixel (defective pixel).
In addition, the pixel defect detection unit 113 interpolates the average values of the pixel data signal levels of “upper left”, “upper right”, “lower left”, and “lower right” obtained under the third determination condition of pixel defect detection described above. The signal level 121 is output to one input terminal of the selector 123.

Further, a fourth determination condition is introduced in order to reduce pixel defect detection errors. In this method, the detection results of the solid-state image pickup elements of R, G, and B colors are mutually used. This is based on the premise that the probability that a defective pixel is generated at the same location of each of the R, G, and B color solid-state imaging devices is small. That is, the pixel defect detection unit 113 outputs the above three determination results (detection signals 117) performed on the video signal from the solid-state image sensor of a certain color to the pixel defect detection unit of the solid-state image sensor of another color. . For example, in FIG. 1, the detection result processing unit 125 receives the detection signal 117 output from the pixel defect detection unit 113 of the own color and the detection signals 118 and 119 output from the pixel defect detection unit of other colors. Is done. That is, the detection result processing unit 125 for each color detects a pixel defect based on the detection signals 118 and 119 for other colors and the detection signal 117 for its own color.
That is, when the detection signal processing unit 125 indicates that the detection signal 117 of its own color is a pixel defect, at least one of the detection signals 118 and 119 of the other two colors is a pixel defect. It is determined whether or not it is a pixel defect only when it is determined that the detection signals 118 and 119 of the solid-state image pickup devices of other colors are not pixel defects.
The detection result processing unit 125 outputs information as to whether or not the pixel is defective to the control terminal of the selector 123 as the detection signal 120.

After detecting a defective pixel as described above, a correction process for correcting a pixel defect based on the detection result will be described. Here, the display error in the screen display is made inconspicuous by interpolating the pixel having the pixel defect with the peripheral pixel data signal level.
Therefore, the interpolation signal level is the average value of the pixel data signal levels of “upper left”, “upper right”, “lower left”, and “lower right” obtained under the third determination condition of the pixel defect detection method described above. Is used. Although the degree of correlation is slightly weaker than when interpolated at the pixel data signal level of the upper, lower, left, and right, the pixel defect signal component can be reduced because it is not affected by the spread of the pixel defect signal due to the low-pass filter.
That is, the selector 123 is controlled by the final detection signal 120 determined based on the detection results of the three colors input to its control terminal, and the interpolation signal level corresponding to the “center” pixel data signal level. The signal 121 is switched from the delay element 122 delayed by the time required for detection to the bell 121 and output as an output signal 124. An output signal 124 from the selector 123 is output from the encoder to the outside of the television camera via various digital process circuits not shown.

JP 200631296 A JP 2007-143131 A JP 2002-271806 A

As described above, conventional defective pixel correction (pixel defect correction) is performed by interpolating and correcting from predetermined surrounding pixels (vertical, horizontal, diagonal, etc.). As a result, detection of a defective pixel and correction of a video signal from the defective pixel can be performed simultaneously with imaging and in real time, so that no special setting work is required. From this, it is possible to cope with the occurrence of a sudden scratch defect during the operation of the camera. Moreover, since all can be performed by digital processing and can be realized by a digital processing circuit incorporated in a digital LSI, the circuit scale can be further reduced.
However, in such correction, if correction is continued, even if surrounding pixels become defective pixels due to aging, the pixels are also used as interpolation values, and as a result, accurate correction cannot be performed. there were. This problem will be described with reference to FIGS. 2 and 3 are diagrams for simply explaining an example of conventional pixel defect correction. FIG. 2 shows an example in which there are no defects in the 8 pixels around the defective pixel, and FIG. 3 shows an example in which there are defects in the 8 pixels around the defective pixel (adjacent 8 pixels).

In FIG. 2A, the center pixel (VFF22) has a defect (determination value such as an average of a plurality of frames of pixel values is equal to or greater than the threshold value), and the upper left (VFF11), upper (VFF12), and upper right (8FF). VFF13), left (VFF21), right (VFF23), lower left (VFF31), lower (VFF32), and lower right (VFF33) pixels are all normal (valid). FIG. 2B shows the average value of the video signal levels of the surrounding 8 pixels as the video signal level value of the central pixel (VFF 22) in FIG. 2A.
As shown in FIG. 2, when there are no defective pixels in the adjacent eight pixels, there is no problem even if the center pixel which is a defective pixel is corrected by interpolation.

FIG. 3A shows a case where there are defects in the surrounding 8 pixels as well as the central pixel (VFF 22). In FIG. 3, the upper left (VFF11), upper (VFF12), left (VFF21), right (VFF23), lower (VFF32), and lower right (VFF33) are normal pixels, and the upper right (VFF13) and lower left (VFF31) are It is a defective pixel.
FIG. 3B shows the average value of the video signal levels of the surrounding 8 pixels as the video signal level value of the central pixel (VFF 22) in FIG. 3A.
In the correction as shown in FIG. 3, even if the positions of the defective pixels and the correction pattern (from which surrounding pixels are interpolated) are accurately tabulated in advance, even if the surrounding pixels become defective pixels due to secular change, The pixel is also used as an interpolation value, and accurate correction cannot be performed.

Patent Document 1 discloses a technique for replacing a defective pixel value with an average value of only normal pixels. However, this is a color correction of the pixels in the false color generation portion between the whiteout pixels and the other (normal) pixels, and the purpose of the correction is different from that of the present application. Further, the correction process must be performed a plurality of times (for example, five times). Further, additional information regarding correction is required for each pixel, and the amount of memory used and the number of processes are large, which takes time.
Patent Document 2 discloses a technique for detecting a defective pixel in real time from a deviation from an average of neighboring pixels of a correction target pixel. Further, it is detected whether the correction target pixel is an edge image region or a flat image region, and the correction method is changed depending on whether it is an edge image region or a flat image region. Therefore, the number of processing is large.

  An object of the present invention is to provide a pixel defect correction apparatus capable of correcting in real time or with a small number of processes even when there are defective pixels in neighboring eight pixels.

In order to solve the above problems, the pixel defect correction apparatus of the present invention determines in real time whether a pixel used as interpolation data is a defective pixel or an effective pixel, and based on the effective pixel excluding the defective pixel, The defective pixel is corrected by interpolation.
That is, the pixel defect correction apparatus according to the present invention includes a pixel defect determination unit that receives a video signal from a solid-state imaging device in which pixels are arranged in a matrix and determines whether each pixel is an effective pixel or a defective pixel. Interpolation value calculation that calculates, as an interpolation value, an average value of only effective pixels among eight pixels adjacent to a pixel determined as a defective pixel in the vertical and horizontal diagonal directions of the pixel determined as the defective pixel And a correction means for correcting the defective pixel by the calculated interpolation value, wherein the pixel defect determination means calculates an average pixel value between frames calculated for fixed pattern noise correction as a predetermined threshold value. And the correction by the correction means is performed with a delay of about 2H (H is one horizontal line time) or less from the input of the video signal.

  According to the present invention, an image in which a pixel defect correction defect such as a column defect in a solid-state image sensor such as a CCD or CMOS sensor of a camera is removed by real-time operation can be obtained.

The block diagram which shows the structure of an example of the conventional pixel defect correction apparatus. The figure for demonstrating simply an example of the conventional pixel defect correction | amendment. The figure for demonstrating simply an example of the conventional pixel defect correction | amendment. The figure for demonstrating one Example of the correction | amendment by the pixel defect correction apparatus of this invention. The figure for demonstrating one Example of the correction | amendment by the pixel defect correction apparatus of this invention. The figure for demonstrating one Example of the correction | amendment by the pixel defect correction apparatus of this invention. 1 is a block diagram illustrating a configuration of a camera according to an embodiment of the present invention. FIG. 3 is a block diagram showing the internal configuration of an FPN correction circuit 15 and a white defect correction circuit 16. FIG. 10 is a diagram illustrating an OUT_V calculation formula in a selection average circuit 877.

In a camera that generates an image using a solid-state imaging device, there is a pixel that outputs a level higher than a certain level when an iris is closed (black image). This is due to pixel defects. The present invention corrects such defective pixels. That is, the present invention uses a frame memory or the like to correct in real time using only normal values even if there are defects connected to adjacent pixels.
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Constituent elements having common functions in the description of each drawing, including the drawings already described, are denoted by the same reference numerals, and overlapping description is avoided as much as possible.

  FIG. 7 is a block diagram showing the configuration of a camera according to one embodiment of the present invention. Reference numeral 1 denotes a lens that collects imaging light incident from a subject. Reference numeral 2 denotes imaging light that has passed through the lens 1 as a plurality of spectral lights, for example, red, green, and blue (hereinafter referred to as R, G, and B, respectively). The prisms 3, 3, and 5 that split the light into wavelengths of light respectively receive the spectral light from the prism 2, photoelectrically convert it according to the amount of spectral light received for each pixel, and convert the electric charges obtained by the conversion respectively. It is a solid-state image sensor provided with a plurality of light receiving elements to be accumulated. 6, 7, and 8 input a video signal generated by sequentially reading out the charges accumulated in the image pickup devices 3 to 5 from the light receiving device, and removing noise components existing in the input video signal. Thus, it is a correlated double sampling (CDS) circuit that outputs a video signal from which a noise component has been removed by sample-holding only the signal component. Reference numerals 9, 10, and 11 denote preamplifier circuits that perform video signal processing such as gain correction and dynamic range compression on the video signals output from the CDS circuits 6 to 8, and reference numerals 12, 13, and 14 denote preamplifier circuits 9. To A / D converters for converting the processed video signals output from .about.11 into digital signals. 15 is an FPN correction circuit that corrects FPN (fixed pattern noise) for each pixel by subtracting the pixel value averaged over a long period of time, and 16 is a digital video signal of each color output from the FPN correction circuit 15. The white defect correction circuit detects a pixel defect for each pixel and outputs an average value of normal pixel pixels around the detected defective pixel by replacing the defective pixel. Further, the video signal processing circuit 17 appropriately performs chromatic aberration correction, color adjustment, video signal format conversion processing, and the like, and outputs the processed video signal to a subsequent stage (not shown). Reference numeral 18 denotes a CPU for causing the imaging elements 3 to 5 and the above-described circuits to operate at a predetermined timing, and for establishing external control between the imaging device and an external device (not shown). is there. Note that the solid-state imaging device is, for example, a CCD. The camera is, for example, an HD (High Definition) television camera.

  In FIG. 7, the imaging light that has passed through the lens 1 of the camera is split by the prism 2, and spectral lights of R, G, and B colors are obtained. These spectral lights are received by the solid-state imaging devices 3, 4, and 5 for each color. In each solid-state image sensor, electric charge is obtained by photoelectric conversion according to the amount of spectral light received for each pixel, the electric charge is accumulated, and further, it is sequentially output as a video signal, and video for each color A signal is generated. These video signals are input to the FPN correction circuit 15 and the white defect correction circuit 16 through the CDS circuits 6, 7 and 8, the preamplifier circuits 9, 10 and 11, and the A / D converters 12, 13 and 14.

FIG. 8 is a block diagram showing the internal configuration of the FPN correction circuit 15 and white defect correction circuit 16 of the present invention. FIG. 8 shows only the configuration for one color. The video signal (12 bits) from the A / D converter 12 is input to the FPN correction circuit 15. In the FPN correction circuit 15, a short line obliquely intersecting with a solid line connecting each component indicates a data bus, and a number written in the vicinity thereof indicates the number of bits of the data bus.
In FIG. 8, there are paths that connect to the selection averaging circuit 877 from the pixel storage memories 854 to 862 and the comparison result storage memories 867 to 875, respectively. Not shown.

In the FPN correction circuit 15, the video signal is input to the subtraction unit 809 and the 256 subtraction circuit 801.
The 256 subtraction circuit 801 subtracts 256, which is a value corresponding to the black level, and outputs a value proportional to the accumulated charge (including the dark current) of the pixel, that is, a true pixel value. In general, inside the camera, the black level of the video signal is not set to the minimum value (0) so that the black level can be adjusted at a later stage, and clogs are put on. The 64-time adder 802 adds the pixel value input from the 256 subtraction circuit 801 and the accumulated pixel value (16 bits) of the corresponding pixel position input from the frame memory 804 (described later), and the clip circuit 803. Output to.
When the input addition value (17 bits) exceeds the maximum value that can be handled by 16 bits, the clip circuit 803 performs clip processing to replace the maximum value and outputs the result to the frame memory 804.
The frame memory 804 has a capacity of at least two frames, reads out the accumulated pixel value corresponding to the current processing pixel by address control in synchronization with the video signal input to the FPN correction circuit 15, and adds 64 times By overwriting with the pixel value added and returned in 802, the video data added 64 times is obtained and output to the rounding processing circuit 805. As an example, the memory area is alternately switched as a two-plane configuration of one frame holding video data added 64 times and one frame holding a cumulative pixel value.
The rounding processing circuit 805 performs rounding such as rounding off at a predetermined digit of the input data and outputs the result to the 1/64 circuit 806.
The 1/64 circuit 806 divides the input video data by 64 (6-bit right shift), and the video data (inter-frame average pixel value) of the quotient is sent to the temperature correction coefficient multiplier 807 and the white spot correction circuit 17. Output.

  The circuits from the 256 subtraction circuit 801 to the 1/64 circuit 806 have a configuration for obtaining an average pixel value for a predetermined plurality of frames, and may have other configurations that function similarly. Desirably, the moving average of the latest 64 frames or more is updated constantly, but the averaged frame may be spaced, and the average value is also updated at intervals of a plurality of frames (for example, 64 frames) as described above. May be.

The temperature correction coefficient multiplier 807 multiplies the input video data by the temperature correction coefficient and outputs the result to the on / off switch 808. The temperature correction coefficient is a value from 0 to 1 (or around 1), and the larger the temperature correction coefficient, the stronger the FPN correction. The dark current increases as the temperature increases, but the temperature correction coefficient is selected so that the FPN is not visually noticeable. Although not shown, a temperature sensor is attached in the vicinity of the solid-state image sensor of the camera, and temperature data detected by the temperature sensor is input to the CPU 18 at a predetermined cycle. A predetermined temperature correction coefficient is input to the temperature correction coefficient multiplier 807 from the CPU 18 based on the input temperature data.
The on / off switch 808 outputs the video data input from the temperature correction coefficient multiplier 807 to the subtraction circuit 809 in accordance with the input on / off signal. That is, the on / off switch 808 outputs the video data input from the temperature correction coefficient multiplier 807 when the on signal is input, and outputs 0 to the subtraction circuit 809 when the off signal is input. To do. The on / off signal is given by manual setting. Since the inter-frame average pixel value output from the 1/64 circuit 806 is required by the subsequent white defect correction circuit 16, the 256 subtraction circuit 801 to the 1/64 circuit 806 operate even if the on / off signal indicates OFF. Keep doing.

  The subtraction circuit 809 outputs video data obtained by subtracting the video data (data B) input from the on / off switch 808 from the video signal (data A) input to the FPN correction circuit 15 to the white defect correction circuit 850. To do. The processing for each pixel by the FPN correction circuit 15 of this example can function not only for the reduction of FPN but also for the adaptive dynamic range compression similar to the retina.

Next, the white defect correction circuit 16 will be described. The white spot correction circuit 16 receives the corrected video data and the inter-frame average pixel value from the FPN correction circuit 15. It is assumed that these two signals are synchronized (that is, the delay is always adjusted to indicate the same pixel).
A line memory (Line MEM) 851 holds one line of video data input from the FPN correction circuit 15 and sequentially outputs the data to the VFF 11 pixel storage memory 854 and the line memory 852 one by one. The line memory 852 outputs video data for one line to the VFF 21 pixel storage memory 855 and the line memory 853 pixel by pixel. The line memory 853 outputs video data for one line to the VFF 31 pixel storage memory 856 pixel by pixel. The line memories 851, 852, and 853 are FIFO (First-In-First-Out) memories. The line memory 851 may be composed of a stack (LIFO: Last-In-First-Out) memory that operates in one line cycle, or may be unnecessary. The pixel storage memories 854 to 862 are 12-bit flip-flops (registers). Further, the comparison result storage memories 867 to 875 are 1-bit flip-flops.

The VFF 11 pixel storage memory 854 holds data for one pixel input from the line memory 851 and outputs it to the VFF 12 pixel storage memory 857 and the selection averaging circuit 877 at the next clock (pixel clock). The VFF 12 pixel storage memory 857 holds data for one pixel input from the VFF 11 pixel storage memory 851 and outputs it to the VFF 13 pixel storage memory 860 and the selection averaging circuit 877 at the next clock. The VFF 13 pixel storage memory 860 holds data for one pixel input from the VFF 12 pixel storage memory 857 and outputs the data to the selection averaging circuit 877 at the next clock.
The VFF 21 pixel storage memory 855 holds data for one pixel input from the line memory 852, and outputs it to the VFF 22 pixel storage memory 858 and the selection averaging circuit 877 at the next clock. The VFF 22 pixel storage memory 858 outputs the input data for one image to the VFF 23 pixel storage memory 861 and holds the data for one image input from the VFF 21 pixel storage memory 855, and VFF 23 pixels at the next clock. The data is output to the storage memory 861 and the selector 876. The VFF 23 pixel storage memory 861 holds data for one pixel input from the VFF 22 pixel storage memory 858 and outputs it to the selection averaging circuit 877 at the next clock.
The VFF 31 pixel storage memory 856 holds data for one pixel input from the line memory 853 and outputs it to the VFF 32 pixel storage memory 859 and the selection averaging circuit 877 at the next clock. The VFF 32 pixel storage memory 859 outputs the input data for one image to the VFF 33 pixel storage memory 862 and holds the data for one image input from the VFF 31 pixel storage memory 856. The VFF 33 pixel is stored at the next clock. The data is output to the storage memory 862 and the selector 876. The VFF 33 pixel storage memory 862 holds data for one pixel input from the VFF 32 pixel storage memory 859 and outputs it to the selection averaging circuit 877 at the next clock.
The pixel held in the VFF 22 pixel storage memory 858 is a central pixel to be corrected, and the pixel storage memories 854 to 862 can use the values of the central pixel and the peripheral pixels together.

  The white defect correction circuit 16 or the white defect correction circuit 850 determines that the central pixel is a defective image according to the embodiment of FIG. 3 from the image data stored in the pixel storage memories 854 to 862 as described above. The pixel defect at the center is corrected by performing interpolation with the image data of the surrounding effective pixels.

Next, the comparator 863 compares the inter-frame average pixel value input from the 1/64 circuit 806 with a threshold value (threshold value), and when the inter-frame average pixel value is smaller, true (1) is obtained. If it is larger, false (0) is output to the line memory (Line MEM) 864. Except in the case of still photography, a pixel having a high inter-frame average pixel value is likely to have a white spot with always high luminance. Such a pixel is regarded as a defective pixel, and the other pixels are defined as normal pixels (referred to as effective pixels).
The line memory 864 holds the logical value (indicating whether it is a valid pixel or a defective pixel) input from the comparator 863 for one line and sequentially outputs it to the CFF11 comparison result storage memory 867 and the line memory 865. The line memory 865 sequentially outputs a logical value for one line to the CFF 21 comparison result storage memory 868 and the line memory 866. The line memory 866 sequentially outputs the logical values for one line to the CFF 31 comparison result storage memory 869.

The CFF11 comparison result storage memory 867 holds the logical value for one pixel input from the line memory 864 and outputs it to the CFF12 comparison result storage memory 870 and the selection averaging circuit 877 at the next clock (pixel clock). The CFF12 comparison result storage memory 870 holds the logical value for one pixel input from the CFF11 comparison result storage memory 867 and outputs it to the CFF13 comparison result storage memory 873 and the selection averaging circuit 877 at the next clock (pixel clock). . The CFF13 comparison result storage memory 873 holds the logical value for one pixel input from the CFF12 comparison result storage memory 870 and outputs it to the selection averaging circuit 877 at the next clock (pixel clock).
The CFF 21 comparison result storage memory 868 holds a logical value for one pixel input from the line memory 865 and outputs it to the CFF 22 comparison result storage memory 871 and the selection averaging circuit 877 at the next clock (pixel clock). The CFF22 comparison result storage memory 871 holds a logical value for one pixel input from the CFF21 comparison result storage memory 868, and outputs it to the CFF23 comparison result storage memory 874 and the selection averaging circuit 877 at the next clock (pixel clock). . The CFF 23 comparison result storage memory 874 holds the logical value for one pixel input from the CFF 22 comparison result storage memory 871 and outputs it to the selection averaging circuit 877 at the next clock (pixel clock).
The CFF31 comparison result storage memory 869 holds the logical value for one pixel input from the line memory 866 and outputs it to the CFF32 comparison result storage memory 872 and the selection averaging circuit 877 at the next clock (pixel clock). The CFF32 comparison result storage memory 872 holds the logical value for one pixel input from the CFF31 comparison result storage memory 869, and outputs it to the CFF33 comparison result storage memory 875 and the selection averaging circuit 877 at the next clock (pixel clock). . The CFF33 comparison result storage memory 875 holds the logical value for one pixel input from the CFF32 comparison result storage memory 872 and outputs it to the selection averaging circuit 877 at the next clock (pixel clock).
By these comparison result storage memories 867 to 875, logical values corresponding to the central pixel and the peripheral pixels can be used together.

The selection average circuit 877 stores nine video data (pixel values) from each of the VFF11 pixel storage memory 854 to VFF33 pixel storage memory 862, and the CFF11 comparison result storage memory 867 to CFF33 comparison result excluding the CFF22 comparison result storage memory 871. Eight logical values from each of the memories 875 are input (wiring is not shown in FIG. 8), and an arithmetic average value (OUT_V) using only effective pixels is output.
FIG. 9 is a diagram illustrating a calculation formula (formula (7)) for calculating the arithmetic average value (OUT_V) in the selection average circuit 877. “VFF11” in the figure indicates the output value of the VFF11 pixel storage memory 854, and is the pixel value of the upper left pixel of the correction target pixel. “VFF11 [11..0] ·” indicates that each of the 11th to 0th bits of the VFF 11 is logically ANDed with the subsequent logical value. An upper score above the logical value indicates negation.
If all of CFF11 to CFF33 are false, the denominator becomes 0. In this case (or when the denominator is equal to or smaller than a predetermined value), the calculation formula of FIG. 9 is not used and the output of the VFF22 comparison result memory 858 (VFF22) Is output as is.

Returning to FIG. 8 again, the correction output processing circuit 876 is a selection circuit (2 × 1 multiplexer) having inputs A, B, and M, and the output (VFF22) of the VFF22 comparison result memory 858 is input to the input A. The output (OUT_V) of the selection averaging circuit 877 is input to the input M, and the output (CFF22) of the CFF22 comparison result memory 871 is input to the input M.
The output of the VFF22 pixel storage memory 858 (VFF22) is input to the input terminal A, the output (OUT_V) of the selection averaging circuit 877 is input to the input terminal B, and the output (CF22) of the CFF22 comparison result memory 871 is input to the input terminal M. Are respectively input. If the CFF 22 of the input M is false, the input A is selected, and if it is true, the input B is selected and output.
In the above configuration, the delay of the real-time processing by the FPN correction circuit 15 and the white defect correction circuit 16 is at most about 2H (H is the time of one horizontal line).

FIG. 4 is a diagram for explaining the state of pixel defect correction in this example. FIG. 4A shows a case where a defect (the video signal level is equal to or higher than the threshold value) is present in the surrounding eight pixels as well as the central pixel (VFF 22), as in FIG. 3A. In FIG. 4A, the upper left (VFF11), upper (VFF12), left (VFF21), right (VFF23), lower (VFF32), and lower right (VFF33) pixel values are less than the threshold value in the comparator 863. The pixel values on the upper right (VFF13) and the lower left (VFF31) are equal to or greater than the threshold value.
FIG. 4 (b) detects whether or not the pixel value is equal to or greater than a predetermined value (threshold value) for the adjacent eight pixels in FIG. 4 (a), and excludes pixels equal to or greater than the predetermined value. The average value of the detected pixels is used as the pixel value of the center image (VFF22), which is a defective image, and the pixels in the upper right (VFF13) and lower left (VFF31) in which pixel values greater than or equal to a predetermined value are detected , Not shown, indicating that these pixels are excluded from the target of averaging. The averaging is usually an arithmetic average, but a geometric average or an average norm (an average norm in an arbitrary dimension including a square average) may be used.

  According to the embodiment of FIG. 4, a pixel defect in a solid-state imaging device such as a CCD or CMOS sensor of a camera can be corrected in real time by a relatively small circuit (memory, multiplier / divider). In particular, even if there is a pixel defect that appears continuously, such as a column defect, the pixel is not used as an interpolation value, so accurate correction can be performed. Further, when all (or most) of the central pixel and the surrounding eight pixels are regarded as defective, the pixel value is used as it is, and thus, for example, a high-luminance subject having a predetermined size (that is, not a point) that hardly moves. Therefore, unnecessary correction processing can be prevented.

FIG. 5 is a diagram for explaining another embodiment of correction by the pixel defect correction apparatus of the present invention. In FIG. 5A, as in FIG. 3A, a defect (upper right (VFF 13) and lower left (VFF 31)) is present not only in the central pixel (VFF 22) but also in the adjacent eight pixels.
FIG. 5 (b) is a pixel obtained by detecting whether or not the pixel values of the neighboring eight pixels in FIG. 5 (a) are greater than or equal to a predetermined value (threshold value), and excluding pixels greater than or equal to the predetermined value. The pixel value of the center pixel (VFF22), which is a defective pixel, is replaced with the center value of.

FIG. 6 is a diagram for explaining another embodiment of the correction by the pixel defect correcting device of the present invention. In FIG. 6A, as in FIG. 3A, there are defects (upper right (VFF 13) and lower left (VFF 31)) in the eight neighboring pixels as well as the central pixel (VFF 22).
FIG. 6 (b) is a pixel obtained by detecting whether or not the pixel value is equal to or higher than a predetermined value (threshold value) for the adjacent eight pixels in FIG. 6 (a), and excluding pixels higher than the predetermined value. The pixel value of the central pixel (VFF22), which is a defective pixel, is replaced with the mode value.
For example, a histogram is created by grouping pixel values into predetermined intervals based on the first pixel value that is less than a predetermined value, and any one pixel value belonging to the group with the highest frequency is maximized. It is a frequent value.

  In the embodiment of FIG. 5 or FIG. 6 described above, detection of whether or not a predetermined value or more may be omitted. In these embodiments, it is difficult to make the processing hardware (fixing the processing time), but the defective pixel can be replaced by the effective pixel from which the black scratched pixel is also removed.

  Furthermore, after calculating the median and mode, the average (midrange) of the maximum and minimum values of the data is calculated, and the median value and mode value closer to the midrange value is calculated in the center. You may employ | adopt as a pixel value. As a result, the image data to be interpolated becomes closer to the surrounding image. Also, if they are close to each other, either a predetermined median or mode is set.

  1: Lens, 2: Prism, 3, 4, 5: Solid-state imaging device, 6, 7, 8: CDS circuit, 9, 10, 11: Preamplifier circuit, 12, 13, 14: A / D converter, 15: FPN correction circuit, 16: white defect correction circuit, 17: video signal processing circuit, 18: CPU, 101: input terminal, 102, 103: 1 line delay element, 104: current line signal, 105: 1 line delay signal, 106 : 2-line delay signal, 107 to 112: 1 pixel delay element, 113: pixel defect detection unit, 114: preamplifier amplification factor signal, 115: CPU, 116: detection threshold signal, 117, 118, 119: detection signal, 125: Detection result processing unit, 120: detection signal, 121: interpolation signal level, 122: delay element, 123: selector, 124: output signal, 800 FPN correction circuit, 801: 256 subtraction circuit, 802: 64 times adder, 803: clip circuit, 804: frame memory, 805: rounding circuit, 806: 1/64 circuit, 807: temperature correction coefficient multiplier, 808: ON / OFF switch, 809: subtraction unit, 850: white defect correction circuit, 851, 852, 853: line memory (Line MEM), 854 to 862: pixel storage memory, 863: comparator, 864, 864, 866: line memory (Line MEM), 867 to 875: comparison result storage memory, 876: selector, 877: selected average circuit.

Claims (1)

  A pixel defect determination unit that receives a video signal from a solid-state imaging device in which pixels are arranged in a matrix and determines whether each pixel is an effective pixel or a defective pixel, and a pixel that is determined to be a defective pixel An interpolation value calculating means for calculating an average value of only effective pixels among eight pixels adjacent in the vertical and horizontal diagonal directions of the pixel determined as the defective pixel, and the calculated interpolation value Correction means for correcting defective pixels, the pixel defect determination means performs the determination by comparing an average value of pixels between frames calculated for fixed pattern noise correction with a predetermined threshold, The pixel defect correction apparatus according to claim 1, wherein the correction by the correction means is performed with a delay of about 2H (H is one horizontal line time) or less from the input of the video signal.

Images