JP4397869B2 - Smear correction method and smear correction circuit - Google Patents

Description

The present invention relates to an image processing apparatus mounted on an imaging apparatus using a solid-state imaging element such as a CCD (Charge Coupled Devices) or a CiS (CMOS image sensor).
In an imaging apparatus using a solid-state imaging device, image quality deterioration due to a smear phenomenon occurs when a high-luminance subject is imaged. Therefore, an image processing apparatus that corrects image quality deterioration due to a smear phenomenon is required.

  FIG. 11 shows an image in which image quality deterioration has occurred due to the smear phenomenon. In the smear phenomenon, for example, when a high-brightness subject such as a light source is imaged, a white vertical streak 2 extending vertically from the high-brightness portion 1 corresponding to the subject is generated. This phenomenon is due to the charge that was not trapped in the original charge accumulation region in the high luminance portion 1 getting over the potential region and leaking into the vertical transfer path.

  As a method for correcting this phenomenon, since the smear 2 is generated at substantially the same level in the vertical direction, the following processing is performed. In other words, in the light-shielded OB (optical black) region 3 outside the effective pixel region, the integrated value of the pixel output where the smear 2 is generated is calculated as a smear component. Then, the smear 2 is corrected by subtracting the smear component from the pixel output of each vertical line in which the smear 2 is generated.

  However, there is a limit in the output level of the image sensor used in the image pickup device, and when strong light exceeding the output level is received, the smear correction as described above may cause image quality degradation. There is.

That is, when the imaging device is saturated in a plurality of vertical lines in the high luminance portion 1 and the smear component of the smear 2 in the central portion of the high luminance portion 1 is larger than the smear component in the vicinity of the outer peripheral portion of the high luminance portion 1, The subtraction value of the smear component in the central portion of the luminance portion 1 becomes larger than the subtraction value of the smear component in the vicinity of the outer peripheral portion of the high luminance portion 1, and black vertical stripes are generated in the central portion of the high luminance portion 1.
JP 2001-14493 A JP 2003-174642 A

  As described above, black streaks are generated in the high luminance portion 1 due to the smear correction. The image sensor is saturated in a plurality of vertical lines included in the high luminance portion 1 and is compared with the smear component of each line. This is a case where there is a significant difference.

  In the imaging apparatus described in Patent Document 1, a configuration is disclosed in which the occurrence of black stripes is prevented by not performing correction when the image signal level is saturated. However, since the smear correction is not performed even when the output level of the adjacent vertical line is not saturated, the smear cannot be reliably removed.

  In the image processing apparatus described in Patent Document 2, a configuration is disclosed in which a smear occurrence region is detected based on a luminance difference between a pixel in a monitoring region and its surrounding pixels without using an OB region.

  An object of the present invention is to reliably remove smear and to prevent black streaks from occurring due to smear correction.

The above object calculates a smear correction value for each horizontal address from the image data of the OB region, in the smear correction method is subtracted from the pixel value of each pixel in the effective pixel region the smear correction value, the smear correction value At least one of the pixel value of the pixel of interest that is subject to smear correction or the pixel value of the pixel adjacent to the pixel of interest at the same horizontal address without exceeding the preset first threshold value is the pixel value of each pixel. When a second threshold value set in advance to determine whether or not the saturation level is exceeded, a smear correction value is subtracted from the pixel of interest, and the smear correction value is the first threshold value. the not exceed, when the pixel values of the pixels pixel value and the adjacent the pixel of interest is greater than both the second threshold value, the smear correction method does not perform subtraction of smear correction value for the target pixel More is achieved.

In addition, the purpose is to calculate a smear correction value for each horizontal address from the imaging data in the OB area and whether the pixel value of each pixel exceeds the saturation level from the imaging data in the effective pixel area. Based on the result of determination by the saturation determination unit and the saturation determination unit, both the pixel value of the target pixel subjected to smear correction and the pixel value of the pixel adjacent to the target pixel at the same horizontal address have threshold values. than when not I row subtraction smear correction value for the target pixel, at least one of pixel values of pixels adjacent the target pixel and the target pixel and the same horizontal address to be smear correction exceeds the threshold If not, the smear correction circuit includes a correction processing unit that subtracts the smear correction value for the target pixel.

  According to the present invention, it is possible to surely remove smear and prevent black streaks from occurring due to smear correction.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. An imaging signal (imaging data) S is input from the imaging device to the smear detection correction unit 12 in the image processing unit 11 illustrated in FIG. The smear detection correction unit 12 performs a smear correction process on the imaging signal S and outputs a processed signal to the pixel interpolation unit 13.

The pixel interpolation unit 13 performs pixel interpolation on the Bayer array data output from the smear detection correction unit 12 and outputs the data to the display device as RGB data.
A specific configuration of the smear detection correction unit 12 is shown in FIG. The imaging signal S is input to the smear subtraction correction circuit 14, the saturation determination unit 15, the correction table generation unit 16, and the shift register 17.

  As the imaging signal S, a signal obtained by scanning the captured image in the horizontal direction is input for each pixel according to the clock signal, and the scanning is sequentially performed from the OB area. Each circuit constituting the smear detection correction unit 12 operates in synchronization with the clock signal.

  A threshold value T1 for determining whether or not the imaging signal S exceeds a saturation level is input to the saturation determination unit 15. Then, the saturation determination unit 15 determines whether or not each imaging signal S exceeds the threshold value T1, and sequentially outputs the determination result to the shift register 18 as a determination signal “1” or “0”.

  The correction table generation unit 16 generates a correction table for smear correction from the input image signal S of the OB area. That is, a difference between the minimum value of the output level of the imaging signal S in the OB area and the output level of each line in the vertical direction in the OB area is detected as a smear component, and the smear component is detected for each line in the vertical direction (horizontal This is stored in the smear correction table 19 as a smear correction value for each direction address).

  When the imaging signal S of the effective pixel region is sequentially input, the smear correction value corresponding to the horizontal address of the imaging signal is output to the threshold value comparison unit 20 from the smear correction values stored in the smear correction table 19. Is done. A threshold value T2 having a preset correction level is input to the threshold value comparison unit 20. Then, the threshold comparison unit 20 determines whether or not the smear correction value exceeds the threshold T2, and outputs the determination result to the shift register 21 as a determination signal “1” or “0”.

  The smear correction values for each line stored in the smear correction table 19 are sequentially output to the shift register 22 in the order of horizontal addresses based on the clock signal. The shift register 22 always holds smear correction values for three addresses and outputs each correction value to an LPF (low-pass filter) 23.

The LPF 23 smoothes the difference between the smear correction values for three addresses, and outputs it to the smear subtraction correction circuit 14 as a smear correction value for an intermediate address among the three addresses.
The shift register 18 always stores the determination results output from the saturation determination unit 15 for three addresses, and the determination results for the three addresses are output to the correction determination unit 25.

  The shift register 21 sequentially stores the determination results determined by the threshold value comparison unit 20 for two addresses, and the previously stored determination results are output to the correction determination unit 25. Therefore, the determination result output to the correction determination unit 25 corresponds to the determination result of the intermediate address among the determination results for three addresses output from the shift register 18 to the correction determination unit 25.

The correction determination unit 25 determines whether or not to perform smear correction based on the determination results output from the shift registers 18 and 21, and outputs the determination result to the smear subtraction correction circuit 14.
The shift register 17 always stores pixel values for five addresses and outputs the pixel values to the smear subtraction correction circuit 14 in parallel.

  If the determination result of the correction determination unit 25 is correctable, the smear subtraction correction circuit 14 performs a process of subtracting the smear correction value from the pixel value and outputs a processed signal. On the other hand, if the determination result of the correction determination unit 25 is uncorrectable, the processed signal is output without performing the process of subtracting the smear correction value.

  Further, when the pixel value exceeds the threshold value T1 and the smear correction value of the horizontal address corresponding to the pixel value exceeds the threshold value T2, the pixel value is replaced with an average of adjacent pixels of the same color. And output as a processed signal.

  FIG. 3 shows a specific configuration of the correction table generator 16. The image pickup signal S is input to the offset processing unit 26, and the minimum smear component for each color type of the image pickup device in the OB region is input as the offset value OF. Then, the offset processing unit 26 generates a smear correction value by subtracting the offset value OF from the input image signal of the OB area.

Next, operations of the correction determination unit 25 and the smear subtraction correction circuit 14 of the smear detection correction unit 12 configured as described above will be described with reference to FIGS.
When the imaging signal S in the effective pixel area is sequentially input, the correction determination unit 25 uses the central address among the determination results for three addresses input from the shift register 18 as the determination result of the target pixel, and the target pixel. Is determined based on the determination result output from the shift register 21 (step 1).

  If the smear correction value does not exceed the threshold value T2 in step 1, it is determined whether or not the pixel value (output level) of the pixel of interest exceeds the threshold value T1 (step 2). If the threshold value T1 is not exceeded, the smear subtraction correction circuit 14 performs processing for subtracting the smear correction value output from the LPF 23 from the output level of the target pixel (steps 2 and 3).

  FIG. 5 shows a case where the processing of steps 1 to 3 is performed. Three address pixel values A1 to A3 indicate pixel values at the same horizontal address in three adjacent vertical lines. Here, the attention address is A2. The smear correction values B1 to B3 are smear correction values corresponding to the pixel values A1 to A3, that is, the smear correction values of the corresponding vertical lines.

  As described above, the smear correction value B2 corresponding to the target pixel A2 is less than the threshold value T2, and the pixel value of the target pixel A2 is less than the threshold value T1. Therefore, the smear correction value B2 is subtracted from the pixel value of the target pixel A2, and is output as the processed signal a2. In FIG. 5, the smear correction values B1 and B3 are subtracted based on the same determination result for the pixel values A1 and A3, and processed signals a1 and a3 are output.

  If the pixel value of the target pixel exceeds the threshold value T1 in step 2, it is determined whether or not the pixel value of one of the pixels adjacent to the target pixel exceeds the threshold value T1 (step S1). 4). If the pixel value of the adjacent pixel does not exceed the threshold value T1, the smear correction value is subtracted from the pixel value of the target pixel (step 5).

  FIG. 6 shows the processing of steps 4 and 5. In the figure, the pixel value of the target pixel A2 exceeds the threshold value T1, but the pixel values A1 and A3 of the adjacent pixels do not exceed the threshold value T1. Therefore, the corresponding smear correction value B2 is subtracted from the pixel value A2 of the target pixel, and the processed signal a2 is output. In FIG. 5, smear correction values B1 and B3 are subtracted even with pixel values A1 and A3, and processed signals a1 and a3 are output.

  FIG. 7 shows a case where the adjacent pixels A1 and A3 exceed the threshold value T1, but the target pixel A2 does not exceed the threshold value T1. In this case, since the target pixel A2 does not exceed the threshold value T1, the processing in steps 2 and 3 is performed, and the corresponding smear correction value B2 is subtracted from the pixel value A2 of the target pixel.

  In FIG. 7, the pixel values A1 and A3 exceed the threshold value T1, but the pixel values of the adjacent pixels do not exceed the threshold value T1, and the smear correction values B1, Processed signals a1 and a3 obtained by subtracting B3 are output.

If the pixel value of one of the pixels adjacent to the target pixel exceeds the threshold value T1 in step 4, the smear correction value is not subtracted from the target pixel (step 6).
FIG. 8 shows the processing of steps 5 and 6. In the figure, the pixel value of the target pixel A2 exceeds the threshold value T1, and the pixel value A3 of one adjacent pixel also exceeds the threshold value T1. In this case, the smear correction value B2 is not subtracted from the target pixel A2, and is output as the processed signal a2.

  Since the pixel value A1 does not exceed the threshold value T1, the processed signal a1 obtained by subtracting the smear correction value B1 is output, and the pixel value A3 is obtained because the pixel value A2 of the adjacent pixel exceeds the threshold value. The subtraction of the smear correction value B3 is not performed.

FIG. 9 shows a case where the pixel value A1 of the target pixel A2 and the other adjacent pixel exceeds the threshold value T1. Also in this case, the smear correction value B2 is not subtracted from the target pixel A2.
Since the pixel value A3 does not exceed the threshold value T1, the processed signal a3 obtained by subtracting the smear correction value B3 is output, and the pixel value A1 is obtained because the pixel value A2 of the adjacent pixel exceeds the threshold value. The smear correction value B1 is not subtracted.

  If the smear correction value corresponding to the target pixel exceeds the threshold value T2 in step 1, the pixel value of the target pixel is replaced with the average of adjacent pixel values of the same color (step 7). In other words, if the smear correction value exceeds the threshold value T2, the smear component is excessive, and therefore subtracting the smear correction value will cause the image quality to deteriorate, such as black streaks.

  FIG. 10 shows the processing of step 7. In the figure, pixel values A1 to A5 are pixel values of five adjacent addresses output from the shift register 17. Here, when the smear correction value B3 corresponding to the target pixel A3 exceeds the threshold value T2, the target pixel A3 also exceeds the threshold value T1.

  At this time, the pixel values A1 to A5 are the same color in A1, A3 and A5, and A2 and A4 are different colors. Accordingly, the target pixel A3 is replaced with an adjacent pixel of the same color, that is, the average value of the pixel values A1 and A5, and is output as the processed signal a3.

In FIG. 10, since the pixel values A2 and A4 do not exceed the threshold value T1, they are output as processed signals a2 and a4 obtained by subtracting the corresponding smear correction values B2 and B4.
In the image processing unit 11 including the smear detection correction unit 12 as described above, the following operational effects can be obtained.
(1) A smear correction process for subtracting a smear component from each pixel value in the effective pixel area can be performed.
(2) When the pixel value of the pixel adjacent to the target pixel on which smear correction is to be performed exceeds the threshold value T1, smear correction is not performed on the target pixel. Accordingly, it is possible to prevent the occurrence of black streaks due to smear correction in the high luminance portion 1 shown in FIG. 11 and to prevent image quality deterioration due to smear correction.
(3) When the smear correction value of the pixel of interest exceeds the threshold value, it is replaced with the average of adjacent pixel values of the same color, so that image quality deterioration can be prevented.

It is a block diagram which shows an image process part. It is a block diagram which shows a smear detection correction | amendment part. It is a block diagram which shows a correction table production | generation part. It is explanatory drawing which shows a smear correction process. It is explanatory drawing which shows a smear correction process. It is explanatory drawing which shows a smear correction process. It is explanatory drawing which shows a smear correction process. It is explanatory drawing which shows a smear correction process. It is explanatory drawing which shows a smear correction process. It is explanatory drawing which shows a smear correction process. It is explanatory drawing which shows the image which the smear generate | occur | produced.

Explanation of symbols

12 Smear detection correction unit 14 Correction processing unit (Smear subtraction correction circuit)
15 Saturation Judgment Unit 16 Correction Table Generation Unit 25 Correction Processing Unit (Correction Judgment Unit)
A1-A5 Pixel value B1-B3 Smear correction value T1, T2 Threshold value

Claims (5)

A smear correction method for calculating a smear correction value for each horizontal address from imaging data in an OB area, and subtracting the smear correction value from a pixel value of each pixel in an effective pixel area,
The smear correction value does not exceed a preset first threshold value, and at least one of a pixel value of a target pixel to be subjected to smear correction or a pixel value of a pixel adjacent to the target pixel at the same horizontal address is When the pixel value of the pixel does not exceed a second threshold value set in advance to determine whether the pixel value exceeds a saturation level, a smear correction value is subtracted from the target pixel, and the smear correction value is When the pixel value of the target pixel and the pixel value of the adjacent pixel both exceed the second threshold value without exceeding the first threshold value, the smear correction value is not subtracted for the target pixel. And smear correction method.   The pixel value of the target pixel is replaced with an average of pixel values of adjacent pixels of the same color when the smear correction value of the target pixel to be subjected to the smear correction exceeds the first threshold value. Item 2. The smear correction method according to Item 1.   A correction table generation unit that calculates a smear correction value for each horizontal address from the imaging data in the OB area;
  A saturation determination unit that determines whether or not the pixel value of each pixel exceeds the saturation level from the imaging data in the effective pixel region;
  Based on the determination result of the saturation determination unit, when the pixel values of the target pixel subject to smear correction and the pixel adjacent to the target pixel at the same horizontal address exceed the threshold value, the smear correction value for the target pixel When the pixel value of the target pixel subject to smear correction and at least one of the pixels adjacent to the target pixel at the same horizontal address does not exceed the threshold value, the smear correction value for the target pixel A correction processing unit that performs subtraction, and
A smear correction circuit comprising:   The correction processing unit replaces the pixel value of the target pixel with the average of the pixel values of adjacent pixels of the same color when the smear correction value of the target pixel to be subjected to smear correction exceeds a preset threshold value. The smear correction circuit according to claim 3, wherein:   The correction processing unit
  A threshold value comparison unit for determining whether a smear correction value of an address corresponding to the target pixel exceeds a threshold value;
  A correction determination unit that determines whether or not to subtract smear correction values based on the determination results for three addresses sequentially output from the saturation determination unit;
  A smear subtraction correction circuit that subtracts the smear correction value from a pixel value of the target pixel based on a determination result of the correction determination unit;
The smear correction circuit according to claim 3, wherein the smear correction circuit is provided.

Images