JP5228929B2 - Image processing apparatus, imaging apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing device, an imaging device, and an image processing program.

  Conventionally, various image processing techniques for correcting pixel values at positions of defective pixels and focus detection pixels in an image sensor have been proposed. Some of the proposed image processing techniques are devised to reproduce the fine structure of an image at a pixel position that needs to be corrected.

  For example, in Patent Document 1, the pixel value of an imaging pixel with a low arrangement density is corrected based on the pixel value of an imaging pixel with a high arrangement density among the imaging pixels around the focus detection pixel, and the corrected arrangement density is low. A technique for calculating the pixel value at the position of the focus detection pixel based on the pixel value of the imaging pixel is disclosed.

  However, the conventional image processing method has a problem in that, for example, if a complicated arithmetic processing as in Patent Document 1 is mounted on an electronic camera or the like as it is, the circuit scale increases.

  In view of the above-described problems of the conventional technique, an object of the present invention is to provide a technique that can perform pixel value correction processing in image data without increasing the circuit scale.

  In order to solve the above problems, an image processing apparatus according to the present invention calculates first image data having a bit accuracy lower than the bit accuracy of original image data, and pixels around a processing target pixel of the first image data. The first pixel value calculation unit that calculates the first pixel value of the processing target pixel using the pixel value of the pixel, and the pixel values of the pixels around the processing target pixel of the original image data, A second pixel value calculation unit that calculates a second pixel value; a bit error suppression unit that adds a correction value obtained from the first pixel value and the second pixel value to the first pixel value; Is provided.

  In the present invention, the correction value is a correction value within a predetermined range.

  In the present invention, the predetermined range is set based on the bit accuracy of the first image data.

  In the present invention, the bit error suppression unit uses a value obtained by clipping the difference between the second pixel value and the first pixel value in the processing target pixel within a predetermined range as a correction value.

  In the present invention, the bit error suppression unit obtains a difference between the second pixel value and the first pixel value in the processing target pixel, and when the difference is not within a predetermined range, the bit error suppression unit calculates the correction value. When the difference is within a predetermined range, the difference is set as the correction value.

  In the present invention, the first pixel value calculation unit calculates the first image data by performing gamma conversion on the original image data.

  In the present invention, the image processing apparatus further includes an image direction determination unit that determines a direction related to a pixel value distribution around the pixel to be processed based on the first image data, and includes a first pixel value calculation unit and a second pixel value. The calculation unit calculates the first pixel value and the second pixel value in the processing target pixel with reference to the direction by the image direction determination unit.

  In the present invention, the processing target pixel is an imaging pixel that outputs an abnormal pixel value or a focus detection pixel.

  The present invention further includes an input unit that receives original image data captured by the image sensor and a position information storage unit that stores position information of the processing target pixel.

  The imaging device of the present invention includes an imaging element and the image processing device of the present invention.

  The image processing program of the present invention realizes the processing of the image processing apparatus of the present invention by a computer.

  According to the present invention, pixel value correction processing in image data can be performed without increasing the circuit scale.

1 is a block diagram showing a configuration of an electronic camera 10 according to an embodiment of the present invention. The figure which shows the arrangement | sequence of the one part cell of the image pick-up element 2 The figure which shows the flow of the image data between the memory | storage part 9 and the image process part 12 of the electronic camera 10. Flow chart of image processing in electronic camera 10

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<< One Embodiment >>
FIG. 1 is a configuration diagram of an electronic camera 10 according to an embodiment of the present invention.

  The electronic camera 10 includes an imaging lens 1, an imaging device 2, an A / D conversion unit 3, a buffer memory 4, a CPU 5, a card interface (card I / F) 6, an operation member 8, a storage unit 9, a display unit 11, and image processing. The unit 12 is configured. The buffer memory 4, the CPU 5, the card I / F 6, the operation member 8, the storage unit 9, the display unit 11, and the image processing unit 12 are connected via a bus 13 so that information can be transmitted. FIG. 1 shows only the main part of the electronic camera 10. For example, in FIG. 1, a timing generator that emits timing pulses for shooting instructions to the image sensor 2 and the A / D converter 3 in accordance with a command from the CPU 5 is omitted.

  The imaging lens 1 is composed of a plurality of optical lenses, and forms a subject image on the light receiving surface of the imaging element 2.

  The image pickup device 2 is a CCD in which any one of a plurality of image pickup pixels arranged on the light receiving surface is provided with a color filter of R (red), G (green), or B (blue) in a Bayer array type. Or a semiconductor image sensor of CMOS, etc., which can be appropriately selected and used. Furthermore, the imaging device 2 of the present embodiment has a plurality of focus detection pixels (AF pixels) (processing target pixels) arranged one-dimensionally in the horizontal scanning direction in a partial region on the light receiving surface. These AF pixels are provided with a white filter and receive a light beam passing through the left or right side of the pupil of the optical system of the imaging lens 1. That is, each AF pixel outputs a detection signal obtained by dividing the left or right pupil according to the luminance of white light. FIG. 2 shows a part of image data centered on an area where AF pixels are arranged, among image data captured by the image sensor 2. Each cell represents one pixel. Symbols R, G, and B at the head of each cell indicate an imaging pixel having each color filter. On the other hand, symbols X and Y indicate AF pixels that are sensitive to light beams from the left side or the right side, and they are alternately arranged one-dimensionally in the horizontal scanning direction. The two-digit number following these symbols indicates the pixel position.

  The image sensor 2 operates based on a timing pulse generated by a timing generator (not shown) in response to a command from the CPU 5 and acquires a subject image formed by the imaging lens 1 provided in front. The image signal output from the image sensor 2 is converted into a digital signal by the A / D converter 3. The digital image signal is temporarily recorded in a frame memory (not shown) and then recorded in the buffer memory 4. Note that the A / D converter 3 in the present embodiment converts the pixel value of each pixel of the image sensor 2 into a 14-bit digital signal and outputs the digital signal. As the buffer memory 4, any non-volatile memory among semiconductor memories can be appropriately selected and used.

  When the electronic camera 10 is turned on by the user pressing the power button on the operation member 8, the CPU 5 reads the control program and the image processing program stored in the storage unit 9 and initializes the electronic camera 10. To do. The CPU 5 receives an instruction signal from the user via the operation member 8, outputs a subject image imaging command to a timing generator (not shown) based on a control program, or the image captured by the image processing unit 12. Control is performed such as processing, recording in the card memory 7, and display on the display unit 11. The CPU 5 can be a general computer CPU.

  A card memory 7 is detachably attached to the card I / F 6. The image recorded in the buffer memory 4 is subjected to image processing by the image processing unit 12 based on an instruction from the CPU 5 and then recorded in the card memory 7 as a file in JPEG format or YUV format.

  The operation member 8 outputs an operation signal corresponding to the content of the member operation by the user to the CPU 5. The operation member 8 includes operation members such as a power button, a mode setting button such as a shooting mode, and a release button. The operation member 8 may be a touch panel type button displayed on the screen of the display unit 11 to be described later.

  The storage unit 9 stores image data captured by the electronic camera 10 and various programs such as a control program for the CPU 5 to control the electronic camera 10. Further, the storage unit 9 stores data such as position information of AF pixels of the image sensor 2 and correction coefficients used in image processing of the image processing unit 12. Programs and data stored in the storage unit 9 can be appropriately referred to from the CPU 5 via the bus 13. As the storage unit 9, a general hard disk device, a magneto-optical disk device, or a storage device such as a semiconductor memory can be appropriately selected and used.

  The display unit 11 displays a through image, a captured image, a mode setting screen, or the like. A liquid crystal monitor or the like can be appropriately selected and used for the display unit 11.

  The image processing unit 12 performs interpolation calculation using the pixel values at the pixel positions of the AF pixels using the pixel values of the surrounding imaging pixels, along with image processing such as contour emphasis processing and white balance correction, based on an instruction for image processing of the CPU 5. Is a digital front-end circuit for performing

  Next, regarding the electronic camera 10 according to the present embodiment, referring to the flowchart of the image processing in FIG. 4 together with a diagram showing the flow of image data in the storage unit 9 and the image processing unit 12 based on the CPU 5 in FIG. explain.

  When the user presses the power button of the operation member 8, the CPU 5 reads the control program stored in the storage unit 9 of the electronic camera 10 and initializes the electronic camera 10. The CPU 5 stands by until an instruction to capture a subject image is issued from the user. When the release button of the operation member 8 is pressed by the user, the CPU 5 determines that an imaging instruction has been issued, and performs the processes of steps S10 to S21. In this embodiment, since the color filter installed in each of the imaging pixels is a Bayer array pattern, a green (G) pixel value is interpolated at the position of the AF pixel of the symbol X, and the symbol Y A blue (B) pixel value is interpolated at the pixel position of the AF pixel.

  Step S10: The CPU 5 issues an imaging command to a timing generator (not shown). A timing generator (not shown) emits a timing pulse to the image sensor 2, and the image sensor 2 captures a subject image formed by the imaging lens 1. The captured image data is converted into digital image data (original image data) with 14-bit precision by the A / D conversion unit 3 and recorded in the buffer memory 4. The CPU 5 issues a command for transferring the image data to the image processing unit 12 to the buffer memory 4. The buffer memory 4 transfers the image data to the image processing unit 12 via the bus 13.

  Step S11: The low bit image generation unit 101 of the image processing unit 12 divides each pixel value of the transferred 14-bit precision image data by 64 values and truncates the decimal point, thereby reducing the 8-bit precision. Image data with reduced bit accuracy is generated. The generated 8-bit precision image data is transferred to the image direction determination unit 104 and the first pixel value calculation unit 105. In the present embodiment, when the value is divided by 64, the decimal part is rounded down, but may be rounded off or rounded up. Further, in the following description, 14 subscripts are attached to parameters representing pixel values of 14-bit precision image data, and 8 subscripts are attached to parameters representing pixel values of 8-bit precision image data. To do.

  Step S12: The AF pixel position setting unit 102 of the image processing unit 12 reads AF pixel position information from the position information storage unit 103 of the storage unit 9 simultaneously with the processing by the low bit image generation unit 101 of step S10. Here, the position information storage unit 102 stores the coordinates YV in the vertical scanning direction and the coordinates XH1 and XH2 of the start and end points in the horizontal scanning direction of the AF pixel row.

Step S13: The image direction determination unit 104 of the image processing unit 12 uses the pixel values of the imaging pixels around each AF pixel in order to interpolate the imaging pixel values at each AF pixel, and performs three directions (this embodiment). Then, the values of the direction fluctuations V1 to V3, which are the change rates of the pixel values in the vertical scanning direction and the 45 ° oblique direction with respect to the horizontal scanning direction), are used as the absolute value of the difference between the same color pixels around the AF pixel. Is calculated. Specifically, the image direction determination unit 104 uses the following equations (1) to (3) when the AF pixel is an AF pixel X that detects light divided into the left pupil, and the right pupil is divided. In the case of the AF pixel Y that detects light, the determination is made using the following equations (4) to (6). This is because the AF pixel X and the AF pixel Y have slightly different R, G, and B arrangement patterns around them. The coordinates of each AF pixel are (x, y), and the R, G, and B components at that position are R ₈ (x, y), G ₈ (x, y), and B ₈ (x, y). ).
The direction change V1 in the vertical scanning direction in the case of the AF pixel X =
2 × (| G ₈ (x−1, y−1) −G ₈ (x−1, y + 1) | + | G ₈ (x + 1, y−1) −G ₈ (x + 1, y + 1) |) + | R ₈ (x−2, y−1) −R ₈ (x−2, y + 1) | + | R ₈ (x, y−1) −R ₈ (x, y + 1) | + | B ₈ (x−1, y−2) −B ₈ (x−1, y + 2) | + | B ₈ (x + 1, y−2) −B ₈ (x + 1, y + 2) | (1)
The direction variation V2 = 45 degrees oblique to the horizontal scanning direction in the case of the AF pixel X =
2 × (| G ₈ (x + 2, y−2) −G ₈ (x + 1, y−1) | + | G ₈ (x−1, y + 1) −G ₈ (x−2, y + 2) |) + | R _{8 (x, y-1)} -R 8 (x-2, y + 1) | + | R 8 (x + 2, y-1) -R 8 (x, y + 1) | + | B 8 (x + 1, y-2) −B ₈ (x−3, y + 2) | + | B ₈ (x + 3, y−2) −B ₈ (x−1, y + 2) | (2)
A direction variation V3 = 135 degrees oblique to the horizontal scanning direction in the case of the AF pixel X =
2 × (| G ₈ (x−2, y−2) −G ₈ (x−1, y−1) | + | G ₈ (x + 1, y + 1) −G ₈ (x + 2, y + 2) |) + | R ₈ (x−2, y−1) −R ₈ (x, y + 1) | + | R ₈ (x, y−1) −R ₈ (x + 2, y + 1) | + | B ₈ (x−3, y− 2) −B ₈ (x + 1, y + 2) | + | B ₈ (x−1, y−2) −B ₈ (x + 3, y + 2) | (3)
Direction change V1 in the vertical scanning direction in the case of AF pixel Y =
2 × (| G ₈ (x, y−1) −G ₈ (x, y + 1) | + | G ₈ (x−2, y−1) −G ₈ (x−2, y + 1) |) + | R _{8 (x-1, y-} 1) -R 8 (x-1, y + 1) | + | R 8 (x + 1, y-1) -R 8 (x + 1, y + 1) | + | B 8 (x-2, _{y-2) -B 8 (x} -2, y + 2) | + | B 8 (x, y-2) -B 8 (x, y + 2) | ... (4)
The direction variation V2 = 45 degrees oblique to the horizontal scanning direction in the case of the AF pixel Y =
2 × (| G ₈ (x + 1, y−2) −G ₈ (x, y−1) | + | G ₈ (x, y + 1) −G ₈ (x−1, y + 2) |) + | R ₈ ( x−1, y−1) −R ₈ (x−3, y + 1) | + | R ₈ (x + 1, y−1) −R ₈ (x−1, y + 1) | + | B ₈ (x + 2, y−) 2) -B ₈ (x−2, y + 2) | + | B ₈ (x + 4, y−2) −B ₈ (x, y + 2) | (5)
A direction change V3 = 135 degrees oblique to the horizontal scanning direction in the case of the AF pixel Y =
2 × (| G ₈ (x−1, y−2) −G ₈ (x, y−1) | + | G ₈ (x, y + 1) −G ₈ (x + 1, y + 2) |) + | R ₈ ( x−1, y−1) −R ₈ (x + 1, y + 1) | + | R ₈ (x + 1, y−1) −R ₈ (x + 3, y + 1) | + | B ₈ (x−2, y−2) −B ₈ (x + 2, y + 2) | + | B ₈ (x, y−2) −B ₈ (x + 4, y + 2) | (6)
The image direction determination unit 104 outputs the direction of the smallest value among the direction variations V1 to V3 in each AF pixel as image direction information to the first pixel value calculation unit 105 and the second pixel value calculation unit 107. S14: The first pixel value calculation unit 105 of the image processing unit 12 performs each AF based on the image direction information at each AF pixel X by the image direction determination unit 104 obtained in step S12 together with the 8-bit precision image data. A first pixel value G1 having an 8-bit accuracy with the G component in the pixel X is obtained. Therefore, the first pixel value calculation unit 105 calculates the average values of R, G, and B in the vicinity of each AF pixel X based on the image direction information of the image direction determination unit 104 by the following formulas (7) to (9). ) To calculate.
When the minimum direction change is the vertical scanning direction, R _ave (x, y) = (R ₈ (x, y−1) + R ₈ (x, y + 1)) / 2 (7)
G _ave (x, y) = (G ₈ (x−1, y−1) + G ₈ (x−1, y + 1) + G ₈ (x + 1, y−1) + G ₈ (x + 1, y + 1)) / 4
B _ave (x, y) = (B ₈ (x−1, y−2) + B ₈ (x−1, y + 2) + B ₈ (x + 1, y−2) + B ₈ (x + 1, y + 2)) / 4
When the minimum direction change is 45 degrees diagonally, R _ave (x, y) = (R ₈ (x, y−1) + R ₈ (x, y + 1) + R ₈ (x−2, y−1) + R ₈ ( x + 2, y + 1)) / 4 (8)
G _ave (x, y) = (G ₈ (x−1, y−1) + G ₈ (x + 1, y + 1)) / 2
B _ave (x, y) = (B ₈ (x−1, y−2) + B ₈ (x−3, y−2) + B ₈ (x + 1, y + 2) + B ₈ (x + 3, y + 2)) / 4
When the minimum direction change is 135 degrees diagonally, R _ave (x, y) = (R ₈ (x, y−1) + R ₈ (x, y + 1) + R ₈ (x + 2, y−1) + R ₈ (x−) 2, y + 1)) / 4 (9)
G _ave (x, y) = (G ₈ (x + 1, y−1) + G ₈ (x−1, y + 1)) / 2
B _ave (x, y) = (B ₈ (x + 1, y−2) + B ₈ (x + 3, y−2) + B ₈ (x−1, y + 2) + B ₈ (x−3, y + 2)) / 4
The first pixel value calculation unit 105 uses the calculated R _ave , G _ave, and B _ave and reads the values of the correction coefficients KR, KG, and KB from the correction coefficient storage unit 106 of the storage unit 9, and An average white component W _ave (x, y) in the pixel X is calculated using the following equation (10).
W _ave (x, y) = KR × R _ave (x, y) + KG × G _ave (x, y) + KB × B _ave (x, y) (10)
Here, the values of the correction coefficients KR, KG, and KB stored in the correction coefficient storage unit 106 are the sum of the spectral sensitivities of the left and right pupil-divided AF pixels according to the brightness of the white light (for example, The sum of the pixel values of the AF pixel X45 and the AF pixel Y46) best satisfies the approximate expression expressed by the spectral sensitivity of the KR × R pixel + the spectral sensitivity of the KG × G pixel + the spectral sensitivity of the KB × B pixel. This value is determined at the development stage. In the present embodiment, the value is around 1 of 0.5 to 1.2.

On the other hand, the first pixel value calculation unit 105 simultaneously uses the pixel values of each AF pixel X and each AF pixel Y to calculate the white component W ₈ (x, y) at the position of each AF pixel X as follows: Obtained using (11).
W ₈ (x, y) = X ₈ (x, y) + (Y ₈ (x−1, y) + Y ₈ (x + 1, y)) / 2 (11)
From the values obtained above, the first pixel value calculation unit 105 calculates the first pixel value G1 (x, y) with 8-bit accuracy with the G component in each AF pixel X,
G1 (x, y) = G _ave (x, y) + W ₈ (x, y) / W _ave (x, y) (12)
It calculates using the formula (12).

Step S15: The first pixel value calculation unit 105 obtains a first pixel value B1 having an 8-bit accuracy with the B component in each AF pixel Y in the same manner as in step S14. For this purpose, the first pixel value calculation unit 105 calculates the average values of R, G, and B in the vicinity of each AF pixel Y based on the image direction information of the image direction determination unit 104 by the following formulas (13) to (15). ) To calculate.
When the minimum direction change is the vertical scanning direction, R _ave (x, y) = (R ₈ (x−1, y−1) + R ₈ (x−1, y + 1) + R ₈ (x + 1, y−1) + R ₈ (x + 1, y + 1)) / 4 (13)
G _ave (x, y) = (G ₈ (x, y−1) + G ₈ (x, y + 1)) / 2
B _ave (x, y) = (B ₈ (x, y−2) + B ₈ (x, y + 2)) / 2
When the minimum direction change is 45 degrees diagonally, R _ave (x, y) = (R ₈ (x−1, y−1) + R ₈ (x + 1, y + 1)) / 2 (14)
G _ave (x, y) = (G ₈ (x, y−1) + G ₈ (x−2, y−1) + G ₈ (x, y + 1) + G ₈ (x + 2, y + 1)) / 4
B _ave (x, y) = (B ₈ (x−2, y−2) + B ₈ (x + 2, y + 2)) / 2
When the minimum direction change is 135 degrees diagonally R _ave (x, y) = (R ₈ (x + 1, y−1) + R ₈ (x−1, y + 1)) / 2 (15)
G _ave (x, y) = (G ₈ (x, y−1) + G ₈ (x + 2, y−1) + G ₈ (x, y + 1) + G ₈ (x−2, y + 1)) / 4
B _ave (x, y) = (B ₈ (x + 2, y−2) + B ₈ (x−2, y + 2)) / 2
The first pixel value calculation unit 105 uses the calculated R _ave , G _ave, and B _ave to calculate the average white component W _ave (x, y) using Equation (7) as in step S14. calculate.

On the other hand, the first pixel value calculation unit 105 simultaneously calculates the white component W ₈ (x, y) in each AF pixel Y using the pixel values of each AF pixel X and each AF pixel Y by the following formula (16 ).
W ₈ (x, y) = Y ₈ (x, y) + (X ₈ (x−1, y) + X ₈ (x + 1, y)) / 2 (16)
From the values obtained above, the first pixel value calculation unit 105 calculates the first pixel value B1 (x, y) with 8-bit accuracy with the B component in each AF pixel Y,
B1 (x, y) = B _ave (x, y) + W ₈ (x, y) / W _ave (x, y) (17)
This is calculated using equation (17). The first pixel value calculation unit 105 transfers the first pixel values G1 and B1 with 8-bit accuracy to the bit error suppression unit 108 together with the result of step S14.

Step S16: The second pixel value calculation unit 107 of the image processing unit 12 sets each AF pixel X based on the image direction information of the image direction determination unit 104 obtained in step S13 together with the original 14-bit precision image data. A second pixel value G2 having a 14-bit accuracy with the G component at is calculated using one of the following equations (18) to (20).
When the minimum direction change is the vertical scanning direction, G2 (x, y) = (G ₁₄ (x-1, y-1) + G ₁₄ (x-1, y + 1) + G ₁₄ (x + 1, y-1) + G ₁₄ (X + 1, y + 1)) / 4 (18)
When the minimum direction change is 45 degrees, G2 (x, y) = (G ₁₄ (x−1, y−1) + G ₁₄ (x + 1, y + 1)) / 2 (19)
When the minimum direction change is oblique 135 degrees G2 (x, y) = (G ₁₄ (x + 1, y−1) + G ₁₄ (x−1, y + 1)) / 2 (20)
Step S17: Similar to step S16, the second pixel value calculation unit 107 sets each AF pixel based on the image direction information of the image direction determination unit 104 obtained in step S13 together with the original 14-bit precision image data. A second pixel value B2 of 14-bit accuracy with a B component in Y is calculated using one of the following equations (21) to (23). The second pixel value calculation unit 107 transfers the second pixel values G2 and B2 with 14-bit accuracy to the bit error suppression unit 108 together with the result of step S16.
When the minimum direction change is the vertical scanning direction B2 (x, y) = (B ₁₄ (x, y−2) + B ₁₄ (x, y + 2)) / 2 (21)
When the minimum direction change is 45 degrees diagonally B2 (x, y) = (B ₁₄ (x−2, y−2) + B ₁₄ (x + 2, y + 2)) / 2 (22)
When the minimum direction change is oblique 135 degrees B2 (x, y) = (B ₁₄ (x + 2, y−2) + B ₁₄ (x−2, y + 2)) / 2 (23)
Step S18: First, the bit error suppression unit 108 of the image processing unit 12 obtains the pixel value of the G component in each AF pixel X using the first pixel value G1 and the second pixel value G2. Specifically, the first pixel value G1 (x, y) of each AF pixel X is multiplied by 64 to calculate a 14-bit precision pixel value G1 ′ (x, y). The bit error suppression unit 108
ΔG ₁₄ (x, y) = G 2 (x, y) −G 1 ′ (x, y) (24)
Using the equation (24), the difference between the second pixel value G2 and the first pixel value G1 ′ obtained by multiplying the second pixel value G2 by 64 and obtaining a pixel value with 14-bit accuracy is obtained. The bit error suppression unit 108 clips ΔG ₁₄ (x, y) in the range of 0 to 63. That is, if ΔG ₁₄ (x, y) is 0 or less, it is 0, if it is 63 or more, it is 63, and if it is within the range of 0 to 63, it is left as it is. The bit error suppression unit 108 obtains a 14-bit accurate pixel value G _X (x, y) as a G component at the pixel position of each AF pixel X as in the following equation (25).
G _X (x, y) = G 1 ′ (x, y) + ΔG ₁₄ (x, y) (25)
Step S19: Next, the bit error suppression unit 108 obtains the pixel value in each AF pixel Y by using the first pixel value B1 and the second pixel value B2. Similarly to step S18, the first pixel value B1 (x, y) of each AF pixel Y is multiplied by 64 to calculate a 14-bit precision pixel value B1 ′ (x, y). The bit error suppression unit 108
ΔB ₁₄ (x, y) = B 2 (x, y) −B 1 ′ (x, y) (26)
Using the equation (26), the difference between the second pixel value B2 and the first pixel value B1 ′ obtained by multiplying the second pixel value B2 by 64 and obtaining a pixel value with 14-bit accuracy is obtained. The bit error suppression unit 108 clips ΔB ₁₄ (x, y) in the range of 0 to 63, and uses the corrected B component of each AF pixel Y to obtain a pixel value B _Y (x, y) with 14-bit accuracy. The following equation (27) is obtained.
B _Y (x, y) = B1 ′ (x, y) + ΔB ₁₄ (x, y) (27)
Bit error suppression unit 108, together with the results of step S18, and transfers the pixel values _{G X} and _{B Y} of 14 bit precision of each AF pixel to cash join the club 109.

Step S20: The cash join the club 109 of the image processing unit 12, a pixel value _{G X} and _{B Y} of 14 bit precision of each AF pixel determined in step S18 and step S19, the image data of the original 14-bit precision, the AF By substituting as a pixel value at the pixel position, image data with 14-bit accuracy corrected to a complete Bayer array image is generated.

  Step S21: The other image processing unit 110 of the image processing unit 12 performs other interpolation processing, contour enhancement processing, white balance correction, and other image processing on the 14-bit precision image data corrected in step S20. At the same time, a file in JPEG format, YUV format or the like is recorded in the card memory 7 or the storage unit 9 via the bus 13 and the card I / F 6 to complete a series of operations.

  As described above, in the present embodiment, the low-bit image generation unit 101 generates 8-bit precision image data from the image data captured by the electronic camera 10 and corrects the image data based on the 8-bit precision image data. In addition to calculating the 8-bit precision pixel value in the power AF pixel, the bit error suppression unit 108 calculates the original pixel value from the 8-bit precision pixel value based on the difference from the pixel value calculated from the original 14-bit precision image data. The circuit scale of the electronic camera 10 can be suppressed by suppressing the bit error that occurs when the 14-bit accuracy is restored.

  Further, the first pixel value calculation unit 105 first uses the image data of 8-bit accuracy to determine the pixel value of the imaging pixel around the AF pixel, and the pixel value of the G component or B component in each AF pixel. However, when the optical image of a subject having a thin line structure is formed at the position of the AF pixel, the G component and B component pixel values obtained from the surrounding pixels cannot accurately resolve the line structure. In order to solve the problem, the surrounding W component obtained using the pixel value of the surrounding imaging pixel and the W component obtained using the pixel value of the AF pixel are used to determine the AF pixel position with respect to the surrounding light amount. By calculating the ratio of the amount of light and multiplying it by the average value of the pixel values of the G and B components obtained from the surrounding imaging pixels (Equation (12) and Equation (17)), the amount of light at the AF pixel position is Even if they are different, it is possible to accurately calculate the light amounts of the G component and B component at that position.

  In addition, the second pixel value calculation unit 107 processes the image data with 14-bit accuracy and calculates the pixel value of the G component or the B component at the position of each AF pixel with 14-bit accuracy. If it is to be performed, the processing in the first pixel value calculation unit 105 must also be performed with 14-bit accuracy. However, it is costly to produce a circuit that performs all such complex operations with 14-bit accuracy. Therefore, in this embodiment, the cost can be significantly reduced by introducing the processing by the first pixel value calculation unit 105 performed with 8-bit accuracy.

  However, due to the bit error caused by dividing the decimal by dividing by 64 when the bit is reduced, the pixel value obtained by multiplying the first pixel value B1 by 64 and returning to the 14-bit accuracy is less than the appropriate value. It becomes small about 0-63. In a flat image area, even a bit error of this level may be noticeable. Therefore, in the present invention, a second pixel value calculation unit 107 and a bit error suppression unit 108 are added to suppress the problem. That is, the second pixel value calculation unit 107 performs the calculation with 14-bit accuracy, but the circuit scale is small because it is a simple calculation that simply obtains the average of two surrounding pixels. When the thin line structure is just at the AF pixel position, the second pixel values G2 and B2 are not correct, but are correct in a region where the image is flat. Further, the bit error suppression unit 108 corrects the first pixel values G1 ′ and B1 ′ with 14-bit accuracy so as to approach the second pixel values G2 and B2 in the range of 0 to 63, for each AF pixel. Are output as pixel values. According to this processing, in a region where the image is flat, a value close to the second pixel value is output as the pixel value, so that a correct result can be obtained. On the other hand, when the thin line structure is at the position of the AF pixel, a value that is lost when the contrast of the structure is within 63 is calculated as the pixel value. 1/256) is hardly a problem.

Note that, by calculating the pixel value according to the image direction information obtained by the image direction determination unit 104, the pixel value can be calculated with high accuracy as is well known, but in the present embodiment, the second pixel value calculation unit 107 In the processing, the image direction information referred to by the first pixel value calculation unit 105 is referred to in common, thereby effectively utilizing the image direction information.
<< Other Embodiments >>
An electronic camera according to another embodiment of the present invention is the same as the electronic camera 10 according to one embodiment. Therefore, the same electronic camera 10 as that of the first embodiment shown in FIG. 1 is used as the electronic camera in the present embodiment, and a detailed description of each component is omitted.

  Also for the image processing procedure of the electronic camera 10 according to the present embodiment, the flow of image data in the storage unit 9 and the image processing unit 12 based on the CPU 5 in FIG. 3 and steps S10 to S21 shown in FIG. This is basically the same as the flowchart of FIG. However, the electronic camera 10 in the present embodiment is different from the case of the first embodiment as follows.

1) When the low-bit image generation unit 101 of the image processing unit 12 generates 8-bit precision image data from the 14-bit precision image data in step S11 of this embodiment, the 8-bit precision pixel value = 2 × (14-bit precision pixel value) Gamma conversion is performed as in ^1/2 .

2) Along with the gamma conversion processing in step S11, in steps S15 and S16, the formula (12) for obtaining first pixel values G1 and B1 with 8-bit accuracy with the G component and B component in each AF pixel and Expression (17) is transformed into the following expressions (12) ′ and (17) ′, respectively.
G1 (x, y) = G _ave (x, y) + W ₈ (x, y) −W _ave (x, y) (12) ′
B1 (x, y) = B _ave (x, y) + W ₈ (x, y) −W _ave (x, y) (17) ′
That is, in the present embodiment, the processing performed by multiplication and division is addition and subtraction in this embodiment. This is because, as is well known, addition and subtraction of pixel values having gamma characteristics correspond to multiplication and division of pixel values having linear characteristics. In this embodiment as well, when it is divided by 64, the decimal part is rounded down, but it may be rounded off or rounded up.

3) Similarly, in step S18 and step S19 following the gamma conversion processing in step S11, the bit error suppression unit 108 determines the pixel value in each AF pixel as the first pixel value G1 or B1 and the second pixel value G2. Alternatively, when obtaining from B2, the first pixel value G1 and the B1 pixel values G1 ′ (x, y) and B1 ′ (x, y) of each AF pixel are obtained from the following equation (28) and It calculates using Formula (29).
G1 ′ (x, y) = G1 (x, y) × G1 (x, y) / 4 (28)
B1 ′ (x, y) = B1 (x, y) × B1 (x, y) / 4 (29)
At the same time, the clipping range of ΔG ₁₄ calculated by Expression (24) is 0 to G1 (x, y) / 2, and the clipping range of ΔB ₁₄ calculated by Expression (26) is the same. , 0 to B1 (x, y) / 2, respectively.

  As described above, in the present embodiment, the low-bit image generation unit 101 generates 8-bit precision image data from the image data captured by the electronic camera 10 and corrects the image data based on the 8-bit precision image data. In addition to calculating the 8-bit precision pixel value in the power AF pixel, the bit error suppression unit 108 calculates the original pixel value from the 8-bit precision pixel value based on the difference from the pixel value calculated from the original 14-bit precision image data. The circuit scale of the electronic camera 10 can be suppressed by suppressing the bit error that occurs when the 14-bit accuracy is restored.

In addition, when converting a 14-bit precision image into an 8-bit precision image in the low bit image generation unit 101, the bit precision is lowered after performing gamma conversion. By doing so, the bit error is small in a dark (small pixel value) image region, and the bit error is large in a bright (large pixel value) image region. Correspondingly, the correction amount of bit error (ΔG and ΔB after clipping) is small in the dark image region, and the loss of contrast is also small in the dark image region. Human vision senses a minute change in the amount of light in a dark image region, so that it is difficult to detect the loss of contrast if the loss of contrast in the dark image region is kept small.
≪Supplementary items for the embodiment≫
In one embodiment and another embodiment, the difference between the second pixel value G2 or B2 and the first pixel value G1 ′ or B1 ′ having a 14-bit precision is predetermined in the process of obtaining the bit error correction value. In the case of out of range (out of bit error range), the correction amount is set to 0 when 0 or less, and when 63, G (x, y) / 2 or B (x, y) / 2 or more. The correction amount is set to 63, G (x, y) / 2 or B (x, y) / 2, but the present invention is not limited to this. For example, when the difference is a value outside the predetermined range Alternatively, all the correction amounts may be set to 0, and if the difference is within a predetermined range, the difference may be used as a correction value. By doing so, when the contrast of the correction portion is sufficiently high, the loss of contrast can be avoided by the difference being outside the predetermined range.

  In one embodiment and other embodiments, the values of the correction coefficients KR, KG, and KB stored in the correction coefficient storage unit 106 are about 1 to 0.5 to 1.2. The present invention is not limited to this, and any correction coefficient value can be appropriately used depending on the situation.

  In one embodiment and other embodiments, the case has been described in which captured image data has 14-bit precision and image data generated by the low-bit image generation unit 101 has 8-bit precision. The present invention is not limited to this. The bit accuracy of the process of the first pixel value calculation unit 105 is made lower than the bit accuracy of the captured image, and the calculation accuracy of the second pixel value calculation unit 107 is set to be a bit of the process of the first pixel value calculation unit 105. By making it higher than the accuracy, the effect of the present invention can be obtained.

  In the embodiment and the other embodiments, the procedure for calculating the pixel value at the position of the AF pixel has been described, but the present invention is not limited thereto. The present invention can be applied to all processes for locally correcting an image, such as correction of pixel defects and correction for each frame image of a moving image by thinning readout. In such a case, it is preferable to store the position information of the imaging pixel to be corrected in the position information storage unit 103 of the storage unit 9.

  In one embodiment and other embodiments, each AF pixel is a focus detection pixel that divides a light flux from the left side or the right side into pupils. However, the present invention is not limited to this, and one AF pixel is used. Focus detection pixels that divide the light flux from the left side and the right side into pupils may be used.

  In one embodiment and another embodiment, the case of the electronic camera 10 has been described. However, the present invention is not limited to this, and in FIG. 1, the imaging lens 1, the imaging element 2, and the A / D conversion unit 3. The image processing apparatus can be applied to an image processing apparatus provided with an input unit for reading an image captured by an electronic camera or the like.

  Note that the present invention can also be applied to a program for realizing the processing in the image processing apparatus according to the present invention by a computer.

  The present invention can be implemented in various other forms without departing from the spirit or main features thereof. For this reason, the above-described embodiment is merely an example in all respects and should not be construed in a limited manner. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Imaging lens, 2 Imaging device, 3 A / D conversion part, 4 Buffer memory, 5 CPU, 6 Card I / F, 7 Card memory, 8 Operation member, 9 Storage part, 10 Electronic camera, 11 Display part, 12 Image Processing unit, 13 bus, 101 low bit image generation unit, 102 AF pixel position setting unit, 103 position information storage unit, 104 image direction determination unit, 105 first pixel value calculation unit, 106 correction coefficient storage unit, 107 second Pixel value calculation unit, 108-bit error suppression unit, 109 substitution unit, 110 other image processing unit

JP 2007-282109 A

Claims (11)

First image data having a bit accuracy lower than the bit accuracy of the original image data is calculated, and the pixel values of the pixels around the processing target pixel of the first image data are used to calculate the first image data of the processing target pixel. A first pixel value calculation unit for calculating a pixel value;
A second pixel value calculation unit that calculates a second pixel value of the processing target pixel using pixel values of pixels around the processing target pixel of the original image data;
A bit error suppression unit that adds a correction value obtained from the first pixel value and the second pixel value to the first pixel value;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The image processing apparatus according to claim 1, wherein the correction value is a correction value within a predetermined range. The image processing apparatus according to claim 2,
The image processing apparatus according to claim 1, wherein the predetermined range is set based on bit accuracy of the first image data. The image processing apparatus according to any one of claims 2 to 3,
The bit error suppression unit is
An image processing apparatus, wherein a value obtained by clipping a difference between the second pixel value and the first pixel value in the processing target pixel within the predetermined range is used as the correction value. The image processing apparatus according to any one of claims 2 to 3,
The bit error suppression unit is
A difference between the second pixel value and the first pixel value in the processing target pixel is obtained, and when the difference is not within a predetermined range, the correction value is set to 0, and the difference is within the predetermined range. In the case of the image processing apparatus, the difference is set as the correction value. The image processing apparatus according to any one of claims 1 to 5,
The image processing apparatus according to claim 1, wherein the first pixel value calculation unit calculates the first image data by performing gamma conversion on the original image data. The image processing apparatus according to any one of claims 1 to 6,
An image direction determination unit that determines a direction related to a pixel value distribution around the processing target pixel based on the first image data;
The first pixel value calculation unit and the second pixel value calculation unit refer to the direction by the image direction determination unit, and calculate the first pixel value and the second pixel value in the processing target pixel. An image processing apparatus characterized by calculating. The image processing apparatus according to any one of claims 1 to 6,
The image processing apparatus, wherein the processing target pixel is an imaging pixel that outputs an abnormal pixel value or a focus detection pixel. The image processing apparatus according to any one of claims 1 to 6,
An input unit that receives the original image data captured by the image sensor;
A position information storage unit for storing position information of the processing target pixel;
An image processing apparatus further comprising: An image sensor;
An image processing device according to claim 9;
An imaging apparatus comprising:   The image processing program for implement | achieving the process of the image processing apparatus of any one of Claim 1 to 9 with a computer.

Images