JP4457874B2 - Video signal processing apparatus and method - Google Patents

Description

  The present invention relates to an image signal processing apparatus and a defective pixel correction method for correcting a signal value at a position corresponding to a defective pixel of an image sensor.

  In a solid-state imaging device such as a CCD (Charge Coupled Device), some pixels may not operate normally in the manufacturing or actual operation process. For example, there may be a pixel that outputs a signal level that is larger or smaller than a signal level that should be originally obtained, or a pixel that always outputs a certain level. These pixels that do not operate normally are called defective pixels. The presence or absence of defective pixels or the number thereof is largely due to the deterioration of image quality.

  Therefore, conventionally, a defect correction method by signal processing for making a defective pixel inconspicuous has been developed.

  For example, Patent Document 1 proposes a defect correction method for correcting a defective pixel using a color signal different from the color of the defective pixel. This method is very effective because interpolation can be performed using a large amount of information and interpolation is performed using a pixel at the same position as the defective pixel.

JP 2004-228931 A

  However, in the defect correction method described in Patent Document 1, in an imaging apparatus in which the chromatic aberration of magnification of the optical system cannot be ignored, the spatial phase of each color image on the imaging element does not match, so that the correction accuracy decreases. End up.

  FIG. 9 is a schematic diagram illustrating the chromatic aberration of magnification of the lens 100. As shown in FIG. 9, the imaging positions of the long wavelength light beam 103 and the short wavelength light beam 104 are shifted at a position away from the optical axis 101 on the imaging surface 102. For this reason, if interpolation processing is performed on defective pixels on the imaging surface 102 away from the optical axis 101 with reference to other color information, an error increases.

  The present invention solves such a problem, and an object of the present invention is to provide an image signal processing apparatus and a defective pixel correction method capable of correcting a signal value at a position corresponding to a defective pixel of an image sensor with high accuracy. And

  An image signal processing apparatus according to the present invention is provided with a plurality of defect correction means for correcting a value of a position corresponding to a defective pixel of an image pickup device, to which an image pickup signal output from the image pickup device is input, and on a light receiving surface of the image pickup device Selecting one of the plurality of defect correction means according to the position of the defective pixel in the image sensor, and selecting the correction of the defective pixel. The selection means is centered on the light receiving surface of the image sensor. The defect correction means is switched between the case where the defect pixel is located in the central region and the case where the defect pixel is located in the peripheral region. The position value corresponding to the defective pixel of the image sensor is corrected.

  A defective pixel correction method according to the present invention is a defective pixel correction method for correcting a value of a position corresponding to a defective pixel of an imaging element with respect to an imaging signal output from the imaging element. When the pixel is divided into two regions, the side region and the peripheral region, and the position of the defective pixel is located in the central region, refer to the value of the pixel of a color different from the defective color among the colors of the defective pixel The position value corresponding to the defective pixel is corrected with respect to the imaging signal output from the imaging device by the correction processing algorithm, and the position of the defective pixel is determined when the position of the defective pixel is located in the peripheral area. The position value corresponding to the defective pixel is corrected with respect to the image pickup signal output from the image pickup device by a correction algorithm different from the case of being located in the center side region.

  In the image signal processing apparatus and the defective pixel correction method according to the present invention, when the light receiving surface of the image sensor is divided into two regions, a central region and a peripheral region, and the position of the defective pixel is located in the central region. Corrects a defective pixel by a process referring to the value of a pixel of a color different from the defective color among the colors of the defective pixel, and when the position of the defective pixel is located in the peripheral area, other processing is performed. Thus, the defective pixel is corrected.

  In the image signal processing apparatus and the defective pixel correction method according to the present invention, the signal value at the position corresponding to the defective pixel of the image sensor can be corrected with high accuracy.

  An electronic video camera (hereinafter simply referred to as a video camera) to which the present invention is applied will be described below.

Configuration Figure 1 of a video camera, a block diagram of a video camera 1 to which the present invention is applied.

  As shown in FIG. 1, the video camera 1 has a lens optical system (hereinafter abbreviated as “lens”) 2 on which reflected light from a subject is incident and a wavelength of the light incident from the lens 2 to separate red light and green light. A color separation optical system (hereinafter referred to as a prism) 3 that separates into blue light, a red image sensor 4R provided corresponding to the imaging position of red light separated by the prism 3, and a prism 3 The green image sensor 4G provided corresponding to the green light image forming position and the blue image sensor 4B provided corresponding to the blue light image forming position separated by the prism 3 are provided. .

  Each color image sensor 4 (4R, 4G, 4B) is a solid-state image sensor such as a C-MOS image sensor or a CCD image sensor. Each of the image sensors 4 (4R, 4G, 4B) has unit pixels arranged in a matrix on the light receiving surface, and photoelectrically converts light irradiated on each unit pixel. The image sensor 4 (4R, 4G, 4B) outputs an image signal by scanning unit pixels arranged in a matrix and reading out an electric signal.

  The video camera 1 also includes a CDS (Correlated Double Sampling) circuit 5 that removes noise from the image signal output from the image sensor 4 (4R, 4G, 4B), and level adjustment of the image signal output from the CDS circuit 5. An AGC (Automatic Gain Control) circuit 6 that performs image processing, an LPF (Low Path Filter) circuit 7 that removes high-frequency noise from the image signal output from the AGC circuit 6, and an image signal output from the LPF circuit 7. On the other hand, a white balance (WB) circuit 8 for adjusting white balance and an AD conversion circuit 9 for converting an image signal output from the WB circuit 8 into a digital signal are provided.

  Furthermore, the video camera 1 outputs the signal value at the position corresponding to the defective pixel of the image sensor 4 (4R, 4G, 4B) for each color with respect to the digitized image signal output from the AD conversion circuit 9. A defective pixel correction circuit 10 for correction is provided.

  In the video camera 1, since the defective pixel correction circuit 10 is provided, for example, a signal level larger or smaller than a signal level that should be originally obtained is output, or a constant level is always output. Even if a defective pixel exists in the image sensor 4 (4R, 4G, 4B), it is possible to make the noise of the pixel portion inconspicuous and generate an image signal with good image quality.

  Hereinafter, the defective pixel correction circuit 10 will be described in more detail.

Defective Pixel Correction Circuit FIG. 2 shows a block configuration diagram of the defective pixel correction circuit 10.

  As shown in FIG. 2, the defective pixel correction circuit 10 includes an address counter 11, a defective address storage unit 12, a first correction unit 13, a second correction unit 14, a selector 15, and a control unit 16. It has.

  The address counter 11 receives a video signal synchronization signal. The address counter 11 generates a pixel address in the screen of the input video signal by counting the synchronization signal.

  The defect address storage unit 12 stores address information indicating the position of the defective pixel of each image sensor 4 (4R, 4G, 4B). The address information indicating the position of the defective pixel is stored, for example, by inspecting the defective pixel when the video camera 1 is shipped from the factory. The defect address storage unit 12 receives pixel address information from the address counter 11 when the video camera 1 is in operation. The defective address storage unit 12 determines whether or not the pixel address generated from the address counter 11 matches the address of the defective pixel stored therein, and indicates that it is the position of the defective pixel at the matching timing. Is generated (defective pixel identification flag).

  Both the first correction unit 13 and the second correction unit 14 receive the video signal output from the image sensor 4, and at a position corresponding to the defective pixel of the image sensor 4 with respect to the input video signal. It is a circuit that performs processing for correcting a value. Both the first correction unit 13 and the second correction unit 14 are input with a video signal and a defective pixel specifying flag indicating the position of the defective pixel. Both the first correction unit 13 and the second correction unit 14 perform the defect correction process on the video signal at the generation timing of the defective pixel specifying flag.

  The correction algorithm of the defective pixel of the first correction unit 13 is as follows.

  First, the 1st correction | amendment part 13 hold | maintains the value of each color of R, G, B of a defective pixel and several adjacent pixels adjacent to a defective pixel. Subsequently, the first correction unit 13 calculates the difference between the value of the defect-free color among the color values of the defective pixel and the value of the color corresponding to the defect-free color of each adjacent pixel. For example, if R is defective, the difference between the G (or B) value of the defective pixel and the G (or B) value of each adjacent pixel is calculated. Then, the first correction unit 13 specifies one adjacent pixel that is most similar to the defective pixel as a reference pixel based on the difference from each adjacent pixel, and determines the defect pixel defect based on the specified reference pixel value. Correct a certain defect color.

  As described above, the first correction unit 13 performs defect correction processing with reference to the value of a pixel having a color different from the defective color among the colors of the defective pixel.

  In contrast, the second correction unit 14 performs correction by an algorithm different from the defective pixel correction algorithm by the first correction unit 13. For example, the second correction unit 14 employs a method (sampling hold method) that retains the value immediately before (to the left of) the defective pixel and employs it as the value of the defective pixel (sampling hold method), or an average value of the pixels before and after the defect. Method (average before and after), edge detection by taking the difference between the color information of defective pixels and the color information of adjacent pixels adjacent to the defective pixels, and correcting from the pixels that seem to be less affected (edge detection) The defective pixel is corrected using the method.

  The selector 15 has three video signals: a video signal output from the first correction unit 13, a video signal output from the second correction unit 14, and an intact video signal input to the defective pixel correction circuit 10. A signal is input. The selector 15 selects and outputs one of the three video signals according to the control of the control unit 16.

  The control unit 16 performs switching control of the selector 15.

  The control unit 16 selects a video signal that has not been subjected to defect correction at the timing at which a flag (defective pixel specifying flag) indicating the position of the defective pixel is generated from the defective address storage unit 12. On the contrary, the control unit 16 at the timing when the flag (defective pixel specifying flag) indicating the position of the defective pixel is generated from the defective address storage unit 12, the first correction unit 13 or the second correction unit 14. The video signal output from either one of them is selected.

  Further, at the timing when the defective pixel specifying flag is generated, the control unit 16 refers to the pixel address of the address counter 11 to determine the position of the defective pixel in the screen, and determines the position of the defective pixel according to the position in the screen. One of the first correction unit 13 and the second correction unit 14 is selected.

  Specifically, as shown in FIG. 3, the screen is divided into a region (center region) 31 at the center of the screen and a region (peripheral region) 32 at the other peripheral portion of the screen. The first correction unit 13 is selected when the pixel is positioned in the position, and the second correction unit 14 is selected when the defective pixel is positioned in the peripheral region 32.

  The above processing contents are shown in the flowchart in FIG. That is, as shown in FIG. 4, the control unit 16 determines whether or not the input video signal is a defective pixel (step S1). If it is not a defective pixel, it is input as it is without correction. A video signal is output (step S11). If it is a defective pixel, it is subsequently determined whether or not the defective pixel is located in the central region 31 (step S2). If the defective pixel is located in the central region 31, the video signal corrected by the first correction unit 13 is output (step S12). If the defective pixel is not located in the center region 31, that is, if the defective pixel is located in the peripheral region 32, the video signal corrected by the second correction unit 14 is output.

  Here, the central area 31 of the screen means an area on the light receiving surface of the image sensor 4 that corresponds to a portion close to the optical axis center of the imaging light and has very little chromatic aberration. On the contrary, the peripheral area 32 of the screen means that the area on the light receiving surface of the image sensor 4 corresponds to a portion far from the center of the optical axis of the imaging light and has a large chromatic aberration.

  That is, in the correction algorithm of the first correction unit 13, correction processing is performed by referring to the value of a pixel having a color different from the defective color among the colors of the defective pixel. Becomes lower. For this reason, in the defective pixel correction circuit 10, the defective pixels are corrected in principle by the first correction unit 13 that can correct with high accuracy. Exceptionally, the correction is performed by the second correction unit 14 to improve the correction accuracy of defective pixels comprehensively.

  The center area 31 of the screen may have a rectangular shape as shown in FIG. 3 or a circular shape as shown in FIG.

  Further, the two correction processes of the first correction unit 13 and the second correction unit 14 are switched. However, in the present invention, not only two algorithms but also three or four correction units are switched. A plurality of correction algorithms may be switched.

First Correction Unit Next, the details of processing performed by the first correction unit 13 will be described in more detail.

  FIG. 6 is a functional block diagram of the first correction circuit 13.

  The first correction unit 13 includes a color information holding unit 21, a difference calculation unit 22, a reference pixel determination unit 23, and a correction calculation unit 24.

  The color information holding unit 21 refers to defect information such as the position of the defective pixel held in the defect address storage unit 12 and the color where the defect exists, and the defect pixel and the adjacent pixel having no defect adjacent thereto are Holds multiple types of color information. Specifically, as shown in FIG. 6, the values of the video signals from the image sensors 4R, 4G, and 4B provided for each of a plurality of colors such as R (Red), G (Green), and B (Blue), Each pixel is stored as a plurality of types of color information.

  FIG. 7 is a diagram illustrating an example of retained color information.

  Here, for the sake of simplicity, only the adjacent pixels adjacent to the left and right (front and rear) and above the defective pixel 0 having a defect in the color G0 are shown. RGB values that are color information of the defective pixel 0 and the pixels 1, 2, 3, 4, and 5 adjacent to the defective pixel are held. For example, for the defective pixel 0, the value of the color R0 is “59”, the color G0 is “250”, and the color B0 is “48” for each color. Similarly, the values of the colors R1 to R5, the colors G1 to G5, and the colors B1 to B5 are held as color information for the other pixels 1 to 5 as well.

  The difference calculation unit 22 calculates a difference between color information having no defect among color information of defective pixels and color information of adjacent pixels. For example, in the case of FIG. 7, the difference between the values of the defective pixels 0 that have no defects R0 and B0 and the values of the colors R1 to 5 and B1 to 5 of the adjacent pixels is calculated for each color.

  The reference pixel determination unit 23 determines an adjacent pixel having a minimum sum of absolute values of differences calculated by the difference calculation unit 22 among adjacent pixels as a pixel (reference pixel) to be referenced for correction.

  The correction calculation unit 24 calculates the average value of the difference between the reference pixels and corrects the defect color information of the defective pixel based on the average value.

  Hereinafter, the operation of the first correction unit 13 will be described with reference to FIGS. 6 and 7.

  When color information is input through the external image sensors 4R, 4G, and 4B, the color information holding unit 21 refers to the defective pixel specifying flag generated from the defective address storage unit 12 as shown in FIG. Thus, RGB values are held for each of the defective pixel 0 and the pixels 1 to 5 that are adjacent to the defective pixel 0 and have no defect.

  Next, the difference calculation unit 22 calculates the difference between the values of the non-defective colors R0 and B0 among the defective pixels 0 and the values of the colors R1 to R5 and B1 to B5 of the pixels 1 to 5. Further, the reference pixel determination unit 23 selects a pixel having the minimum sum of absolute values of differences.

  FIG. 8 is a diagram illustrating a state in which the sum of absolute values of differences is calculated.

  Here, dr is the difference between the values of the colors R1 to R5 of the adjacent pixels 1 to 5 and the value of the color R0 of the defective pixel 0, and db is the value of the colors B1 to B5 of the adjacent pixels 1 to 5 This is the difference from the value of the color B0 of the defective pixel 0. The sum of the absolute values of the differences is represented by | dr | + | db |. As can be seen from FIG. 8, the sum of the absolute values of the differences of the pixel 1 is “34”, which is the smallest among the other pixels 2 to 5. From this, it is determined that the pixel 1 is a pixel having color information most similar to the defective pixel 0, and the pixel 1 is determined as a reference pixel for correction.

  Next, in the correction unit 24, for the reference pixel and the defective pixel 0, the color R, B is calculated from the value of the color G 1 of the pixel 1 so that the average value of the differences for the colors R and B matches the difference of the color G. Subtract the average of the differences for B. Specifically, as shown in FIG. 8, since the sum of the absolute values of the differences of the pixel 1 is “34”, the average value of the differences is “17”. By subtracting this from “63” which is the value of the color G1 of the pixel 1, a correction value of the defective color G0 of the defective pixel 0 of “46” is obtained.

  As described above, the color information (color R0, B0 or any one of the colors R0 and B0) of the defective pixel 0 that does not have a defect is used to determine the defective pixel 0 based on the sum of the absolute values of the differences from the color information of the adjacent pixels. By determining the most similar reference pixel and correcting the defect color (color G0) so as to match the average value of the differences, the defective pixel 0 is more effective than when correcting by focusing on one color. Can be corrected automatically.

  Although omitted in the above description, a weighting unit that applies a coefficient to the difference according to the distance between the defective pixel 0 and the adjacent pixel of interest is provided between the difference calculation unit 22 and the reference pixel determination unit 23. May be.

It is a block diagram of a video camera to which the present invention is applied. It is a block diagram of a defective pixel correction circuit to which the present invention is applied. It is a figure which shows the example which divided | segmented the inside of a screen into the area | region of the center side, and the area | region of the peripheral side. It is a flowchart which shows the processing content of a defective pixel correction circuit. It is a figure which shows the other example which divided | segmented the inside of a screen into the area | region of the center side, and the area | region of the peripheral side. It is a block diagram of the 1st correction circuit. It is a figure which shows the example of the hold | maintained color information. It is a figure which shows a mode that the sum of the absolute value of a difference is calculated. It is a figure for demonstrating chromatic aberration.

Explanation of symbols

1 video camera, 10 defective pixel correction circuit, 11 address counter, 12 defective address storage unit, 13 first correction unit, 14 second correction unit, 15 selector, 16 control unit

Claims (4)

A plurality of defect correction means for inputting an imaging signal output from the imaging element and correcting a value of a position corresponding to the defective pixel of the imaging element;
Selecting one of the plurality of defect correction means according to the position of the defective pixel on the light receiving surface of the image sensor, and a selection means for correcting the defective pixel,
The selection unit divides the light receiving surface of the image sensor into two regions, a center region and a peripheral region, and the defective pixel is located in the central region and the defective pixel is located in the peripheral region. The image signal processing apparatus is characterized in that the defect correction means is switched and the position value corresponding to the defective pixel of the image sensor is corrected by different processing. The plurality of defect correction means includes a first defect correction means that performs correction by referring to a pixel value of a color different from a defective color among the colors of the defective pixels, and the first defect correction means. Second defect correction means for correcting the value of the position corresponding to the defective pixel of the image sensor by a process different from the correction process,
The selection unit selects the first defect correction unit when the defective pixel is located in the central region, and selects the second defect correction unit when the defective pixel is located in the peripheral region. The image signal processing apparatus according to claim 1. The first defect correction means includes
Holding a plurality of types of color information of the defective pixel and adjacent pixels adjacent to the defective pixel;
Calculating a difference between the defect-free color information having no defect in the color information of the defective pixel and the color information of the adjacent pixel corresponding to the defect-free color information;
Identifying the color information most similar to the defect-free color information based on the difference, determining a reference pixel as a pixel having the specified color information among the adjacent pixels,
The image signal processing apparatus according to claim 2, wherein defective color information having a defect of a defective pixel is corrected based on the determined reference pixel value. In the defective pixel correction method for correcting the value of the position corresponding to the defective pixel of the image sensor with respect to the image signal output from the image sensor
The light receiving surface of the image sensor is divided into two regions, a central region and a peripheral region,
When the position of the defective pixel is located in the central region, the image signal output from the image sensor is processed by a correction processing algorithm that refers to the value of a pixel having a color different from the defective color among the colors of the defective pixel. For the position value corresponding to the defective pixel,
When the position of the defective pixel is located in the peripheral area, the defective pixel corresponds to the imaging signal output from the image sensor by a correction algorithm different from the case where the position of the defective pixel is located in the central area. A defective pixel correction method comprising correcting a position value.

Images