WO2013125198A1

WO2013125198A1 - Image processor, imaging device, and image processing program

Info

Publication number: WO2013125198A1
Application number: PCT/JP2013/000860
Authority: WO
Inventors: 慎哉海老原
Original assignee: 株式会社ニコン
Priority date: 2012-02-22
Filing date: 2013-02-18
Publication date: 2013-08-29
Also published as: JP2013172402A

Abstract

An image smoothing means generates a plurality of smooth images that are the smoothed images of a target image for different degrees of smoothing. A calculation means calculates the color difference between the color components of the specified color component of the target image and color components that differ from the specified color components of the smoothed image, and calculates the color difference variance. An adjusting means determines the color component having the highest sharpness in the pixels of the target image based on the color difference variance, and calculates the adjustment amount for adjusting the sharpness between the color components in the pixels of the target image. A correction means determines the image structure of the pixels of the target image based on the color difference and the adjustment amount, and corrects the sharpness between the color components by the adjustment amount.

Description

Image processing apparatus, imaging apparatus, and image processing program

The present invention relates to an image processing device, an imaging device, and an image processing program.

Conventionally, an image obtained by imaging a subject imaged by an optical system such as an imaging lens is affected by chromatic aberration, particularly axial chromatic aberration, by the optical system.

In order to solve this, for example, the MTF characteristic mismatch between the color components is adjusted so that the color plane of the reference color component is smoothed and the color difference from the color plane of the other color components is minimized. Thus, a technique for correcting axial chromatic aberration has been developed (see Patent Document 1, etc.).

JP 2007-28041 A

However, in the prior art, when smoothing the color plane of the reference color component to minimize the color difference from the color planes of other color components, the behavior in which the color difference cannot be predicted if the degree of smoothing increases. May be.

In addition, there is a problem in that the color structure portion of the image other than the influence of the longitudinal chromatic aberration reduces the saturation and loses color.

In view of the above-described problems of the prior art, an object of the present invention is to provide a technique capable of correcting axial chromatic aberration with high accuracy while considering the color structure of an image.

An aspect of the image processing apparatus illustrating the present invention includes an image smoothing unit that smoothes a target image having pixel values of a plurality of color components to generate a plurality of smooth images having different degrees of smoothing, and pixels of the target image In each of the above, in the pixel of the target image based on the variance of the color difference, arithmetic means for calculating the color difference and the variance of the color difference, which is the difference between the predetermined color component of the target image and the color component different from the predetermined color component of the smooth image A color component having the highest sharpness is determined, an adjustment means for calculating an adjustment amount for adjusting the sharpness between the color components in the pixels of the target image based on the determined color component having the highest sharpness, and a color difference and an adjustment amount. Correction means for determining the image structure of the pixels of the target image based on the result and correcting the sharpness between the color components of the pixels of the target image with the adjustment amount according to the determination result.

Further, the adjustment means may determine the color component having the highest sharpness in the pixel of the target image based on the minimum value of the color difference variance.

Further, the calculation means calculates a texture amount indicating the number of image structures in the target image by calculating the difference between the target image and the one smoothed image for each same color component in each pixel of the target image, and the correction means The image structure at the pixel of the target image may be determined based on the color difference, the adjustment amount, and the texture amount.

Further, the adjustment means may extract a blurred image area where the texture amount is equal to or less than a predetermined value, and smooth the minimum value of the color difference variance in the pixels of the blurred image area.

Further, the correcting means determines whether the image structure in the pixel of the target image is a color boundary based on the change rate of the color difference in the pixel of the target image and the size of the adjustment amount. You may exclude the pixel of an image from correction object.

Further, the correction means compares the adjustment amount of the pixel of the target image with the adjustment amount of the adjacent pixel to determine whether or not the image structure of the pixel of the target image is an isolated structure. If it is determined, the pixel of the target image may be excluded from the correction target.

Further, the correction means determines whether the image structure of the pixel of the target image is a pseudo-axial chromatic aberration region similar to the axial chromatic aberration based on the texture amount and the correction amount of the pixel of the target image. When it is determined as the axial chromatic aberration region, the pixel of the target image may be excluded from the correction target.

Further, the correcting means determines whether or not the image structure of the pixel of the target image determined to be a color boundary is a color blur due to a density difference around the saturation region based on the distribution of each color component, and determines that the color blur The pixels of the target image may be the correction target.

Further, color difference correction means for correcting the pixel value of the pixel of the target image whose sharpness has been adjusted to be the same as the direction of the color difference of the pixel value before the sharpness adjustment in the color difference space may be provided.

The image smoothing means smoothes the target image with at least two different smoothing levels to generate a set of smoothed images, and mixes the set of smoothed images at a predetermined ratio to generate a plurality of smoothed images. May be.

Another aspect of the image processing apparatus illustrating the present invention includes a determination unit that determines an image structure in a pixel of a target image having pixel values of a plurality of color components, and a color component having the highest sharpness in the pixel of the target image. Determine the amount of adjustment to adjust the sharpness of the color component in the target image pixel based on the determined color component with the highest sharpness, and adjust the sharpness of the color component in the target image pixel according to the determination result Correction means for adjusting the amount.

An aspect of an imaging apparatus that exemplifies the present invention includes imaging means for capturing a subject and generating a target image having pixel values of a plurality of color components, and the image processing apparatus of the present invention.

One aspect of an image processing program that exemplifies the present invention is an input procedure for reading a target image having pixel values of a plurality of color components, and image smoothing that smoothes the target image and generates a plurality of smooth images having different degrees of smoothing. Based on the procedure, the calculation procedure for calculating the color difference and the variance of the color difference, which is the difference between the predetermined color component of the target image and the color component different from the predetermined color component of the smooth image, for each pixel of the target image, based on the variance of the color difference An adjustment procedure for determining a color component having the highest sharpness in the pixel of the target image, and calculating an adjustment amount for adjusting the sharpness between the color components of the pixel of the target image based on the determined color component having the highest sharpness, a color difference And a correction procedure for determining the image structure of the pixel of the target image based on the adjustment amount and correcting the sharpness between the color components of the pixel of the target image with the adjustment amount according to the determination result. It is executed by the over data.

Another aspect of the image processing program illustrating the present invention is a determination procedure for determining an image structure in a pixel of a target image having pixel values of a plurality of color components, and determining a color component having the highest sharpness in the pixel of the target image The amount of adjustment for adjusting the sharpness of the color component in the pixel of the target image is calculated based on the determined color component having the highest sharpness, and the amount of sharpness of the color component in the pixel of the target image is adjusted according to the determination result. Makes the computer execute the correction procedure to be adjusted with.

According to the present invention, it is possible to correct axial chromatic aberration with high accuracy while considering the color structure of an image.

1 is a block diagram showing the configuration of a computer that operates as an image processing apparatus according to an embodiment of the present invention. Diagram explaining color structure 7 is a flowchart showing image processing operations performed by a computer according to an embodiment. The figure which shows the relationship between a pixel (i, j) and a reference area. The figure which shows distribution of the standard deviation DEVr [k '] in a pixel (i, j). The figure which shows an example of a structure of the digital camera which concerns on this invention The figure which shows the correction output rate according to the magnitude | size of the color difference component after correction | amendment

FIG. 1 is a block diagram showing a configuration of a computer 10 that operates as an image processing apparatus according to an embodiment of the present invention.

A computer 10 shown in FIG. 1A includes a CPU 1, a storage unit 2, an input / output interface (input / output I / F) 3, and a bus 4. The CPU 1, the storage unit 2, and the input / output I / F 3 are connected via the bus 4 so that information can be transmitted. The computer 10 is connected to an output device 30 for displaying the progress of image processing and processing results, and an input device 40 for receiving input from the user via the input / output I / F 3. A general liquid crystal monitor, a printer, or the like can be used for the output device 30, and a keyboard, a mouse, or the like can be appropriately selected and used for the input device 40.

Note that it is assumed that the target image processed by the computer 10 has pixel values of red (R), green (G), and blue (B) color components in each pixel. That is, it is assumed that the target image of the present embodiment is an image captured by a three-plate color digital camera or an image captured by a single-plate color digital camera and subjected to color interpolation processing. The target image is assumed to have an influence of axial chromatic aberration due to the imaging lens when picked up by a digital camera or the like, and the sharpness between the color components is different.

The CPU 1 is a processor that comprehensively controls each unit of the computer 10. For example, the CPU 1 reads an image processing program stored in the storage unit 2 based on an instruction from the user received by the input device 40. The CPU 1 operates as an image smoothing unit 20, a calculation unit 21, an adjustment unit 22, a correction unit 23, and a color difference correction unit 24 by executing the image processing program (FIG. 1 (b)), and is on the axis with respect to the target image. Chromatic aberration correction processing is performed. The CPU 1 displays the result of image processing for the image on the output device 30.

The image smoothing unit 20 uses, for example, N known Gaussian filters with different degrees of smoothing (blur index), and smoothes the target image according to the blur index of each Gaussian filter to generate N smooth images. (N is a natural number of 2 or more). Note that the blur index in the present embodiment refers to the size of the blur radius, for example.

As will be described later, the calculation unit 21 uses the target image and N smooth images, and the difference between the color component of the target image and the color component different from the color component of the target image of the smooth image at the pixel position of each pixel. From the absolute value of, the value of the color difference surface (color difference) and its standard deviation (variance) corresponding to the blurring index are calculated. In addition, the calculation unit 21 of the present embodiment makes a difference between the target image and the smooth image having the smallest blurring index (that is, the smallest smoothing degree) for each identical color component, and the image structure of the target image is large. The amount of texture of each color component indicating is calculated.

The adjustment unit 22 applies a known interpolation method to the standard deviation distribution with respect to the blur index at the pixel position of each pixel of the target image, and obtains the blur index that gives the minimum standard deviation at the pixel position. The adjustment unit 22 determines a color component having the highest sharpness based on a blurring index (hereinafter referred to as a minimum blurring index) that gives the minimum standard deviation of each color difference plane obtained in each pixel. The adjustment unit 22 calculates a correction value (adjustment amount) for adjusting the sharpness between the color components at the pixel position of each pixel of the target pixel based on the determined color component having the highest sharpness.

Based on the color difference calculated by the calculation unit 21 and the texture amount of each color component and the correction value calculated by the adjustment unit 22, the correction unit 23 determines whether the image structure at the pixel position of each pixel of the target image is on-axis. It is determined whether or not the structure can correct chromatic aberration. In the case of a correctable image structure, the correction unit 23 corrects the sharpness between color components in each pixel using the correction value calculated by the adjustment unit 22.

Here, FIG. 2 shows a typical color structure considered in this embodiment. FIG. 2A is a target image obtained by imaging a subject in which a single black line is arranged on a white background, for example, an R component (dotted line), a G component (solid line) in the scanning direction orthogonal to the black line, The distribution of pixel values of the B component (broken line) is shown. FIG. 2B shows, as an example, a target image obtained by imaging a subject having a color boundary composed of red and white, and an R component (dotted line), a G component (solid line), and a B component in a scanning direction orthogonal to the color boundary. The distribution of pixel values (broken line) is shown. FIG. 2C shows, for example, an image of a subject with a bright light source such as a streetlight, and a target image affected by so-called purple fringe. The R component (dotted line) and G component (in the scanning direction passing through the center of the light source) A distribution of pixel values of a solid line) and a B component (broken line) is shown. Here, purple fringing refers to purple color bleeding that occurs around high-luminance areas (saturated areas) where the pixel values of each color component are saturated due to a large amount of light, such as around a light source such as a streetlight or the reflection of water. It is. Note that the imaging lens of the camera that captured each target image in FIG. 2 has axial chromatic aberration and is focused on the G component.

As shown in FIG. 2 (a), the G component reproduces the color structure well, whereas the R and B components are blurred in color structure due to axial chromatic aberration. As a result, the black line portion is blurred in green or magenta. However, since the distributions of the respective color components essentially match each other, as will be described later, the sharpness adjustment (that is, correction of axial chromatic aberration) with respect to the R component and the B component by the calculation unit 21, the adjustment unit 22, and the correction unit 23 is performed. As a result, the distributions of the respective color components shown in FIG. In other words, the axial chromatic aberration correction processing according to the present embodiment is based on the premise that the distributions of the respective color components of the original target pixel match each other. In the vicinity of such a color structure, the texture amount calculated by the calculation unit 21 shows a distribution that differs greatly for each color component, whereas the minimum blurring index of each color difference calculated by the adjustment unit 22 is similar to each other. It shows such a gentle and uniform distribution.

Next, at the color boundary shown in FIG. 2B, the color structure of each color component is greatly different. That is, in the vicinity of the color boundary, the distribution of the texture amount is greatly different for each color component, and the distribution of the minimum blurring index for each color difference also shows an abrupt change. For this reason, at the color boundary as shown in FIG. 2B, the above-mentioned premise is not satisfied, and the correction value calculated by the adjustment unit 22 becomes a large value, and the axial chromatic aberration cannot be corrected appropriately. . Therefore, the correction unit 23 of the present embodiment determines whether the color structure is a color boundary as shown in FIG. 2B based on the distribution of the minimum blurring index of each color difference and the size of the correction value. When the correction unit 23 determines that the color structure is a color boundary, in this embodiment, the correction of the axial chromatic aberration with respect to the pixel at the color boundary is not performed. Thereby, the discoloration which arises by performing the correction process of the axial chromatic aberration with respect to a color boundary can be suppressed.

On the other hand, in the purple fringe shown in FIG. 2 (c), the saturation region is different for each color component, the G component first decreases as the distance from the light source increases, and the R component is distributed over the widest range. Due to such a distribution, a purple color appears. Such distribution of each color component is different from the color structure shown in FIG. However, as described above, the black line portion shown in FIG. 2A is similar to the purple fringe in that the black line portion blurs in green or magenta due to axial chromatic aberration. Therefore, in the present embodiment, the same correction process as that for the longitudinal chromatic aberration is performed on the purple fringe image area. For this purpose, as shown in FIG. 2C, the correction unit 23 obtains a distribution of pixel values of each color component in the peripheral area centering on the pixel of the target image or the entire target image, and from the distribution, A saturation region of pixel values is obtained. The correction unit 23 extracts the saturation region of the color component that is most widely distributed (the R component in the case of FIG. 2C) and the region that is widened by β from the end of the saturation region as a purple fringe region. The correcting unit 23 determines whether the color structure is a purple fringe based on whether the pixel position is included in the extracted purple fringe region. The correction unit 23 performs axial chromatic aberration correction processing based on the correction value calculated by the adjustment unit 22 for each color component of the pixel determined to be purple fringe.

Note that, in the vicinity of the purple fringe, the texture amount distribution varies greatly for each color component, and the distribution of the minimum blurring index for each color difference also shows an abrupt change, similar to the color boundary shown in FIG.

In the present embodiment, in addition to the color structure shown in FIG. 2, an isolated structure that shows an abnormal correction value by the pixel value of one pixel due to the influence of shot noise or the like, or a color similar to the image area of axial chromatic aberration In spite of showing the structure, when the axial chromatic aberration correction process is performed, the area that is excessively corrected and reduces the image quality of the target image (hereinafter referred to as the pseudo-axial chromatic aberration area) is also considered. To do. In this embodiment, axial chromatic aberration correction processing is not performed on these image structures. Note that these image structure determination processes will be described later.

The color difference correction unit 24 corrects the pixel value of each color component of the pixel whose sharpness has been adjusted to be the same as the direction of the color difference before the adjustment in the color difference space, and changes the color change caused by the correction process of the longitudinal chromatic aberration. Suppress.

The storage unit 2 records an image processing program and the like for correcting axial chromatic aberration in the target image along with the target image. Images, programs, and the like stored in the storage unit 2 can be appropriately referred to from the CPU 1 via the bus 4. A storage device such as a general hard disk device or a magneto-optical disk device can be selected and used for the storage unit 2. Although the storage unit 2 is incorporated in the computer 10, it may be an external storage device. In this case, the storage unit 2 is connected to the computer 10 via the input / output I / F 3.

Next, the operation of image processing for correcting axial chromatic aberration by the computer 10 of this embodiment will be described with reference to the flowchart shown in FIG.

The user uses the input device 40 to input an image processing program command or double-click the icon of the program displayed on the output device 30 to instruct the CPU 1 to start the image processing program. The CPU 1 receives the instruction via the input / output I / F 3 and reads and executes the image processing program stored in the storage unit 2. CPU1 starts the process from step S10 of FIG.

Step S10: The CPU 1 reads the target image to be corrected for the longitudinal chromatic aberration designated by the user via the input device 40.

Step S11: The image smoothing unit 20 of the CPU 1 smoothes the read target image according to the blur index of each Gaussian filter, and generates N smooth images. In this embodiment, the target image itself is also one of the smooth images, and the total number of smooth images in this embodiment is (N + 1).

Step S12: The calculation unit 21 of the CPU 1 uses the color difference surface Cr of the R component and the G component, the color difference surface Cb of the B component and the G component, and the color difference surface Crb of the R component and the B component as the target image and each smoothed image. And using

For example, the calculation unit 21 calculates the pixel value G0 (i, j) of the G component that is a predetermined color component of the target image and the pixel value Rk of the R component that is a color component different from the predetermined color component of the smooth image. The absolute value of the difference from (i, j) is obtained at each pixel position, and the color difference plane Cr [−k] (i, j) shown in the following equation (1) is calculated.
Cr [−k] (i, j) = | Rk (i, j) −G0 (i, j) | (1)
Here, (i, j) indicates the coordinates of the pixel position of each pixel of the target image. k is a blurring index of a smooth image and is an integer of 0 ≦ k ≦ N. In Expression (1), the negative blur index k indicates that the color difference surface Cr is obtained by sequentially blurring the R surface on the negative side. The blur index k = 0 represents the target image itself, that is, an image that has not been smoothed.

Similarly, the calculation unit 21 calculates the pixel value R0 (i, j) of the R component that is a predetermined color component of the target image and the pixel value of the G component that is a color component different from the predetermined color component of the smooth image. The absolute value of the difference from Gk (i, j) is obtained at each pixel (i, j), and the color difference plane Cr [k] (i, j) of the following equation (2) is calculated.
Cr [k] (i, j) = | R0 (i, j) −Gk (i, j) | (2)
In Expression (2), the blurring index k being positive indicates that the color difference surface Cr is obtained by sequentially blurring the G surface on the positive side.

Similarly, based on the equations (3) to (6), the calculation unit 21 determines the color difference plane Cb of the B component and the G component and the color difference plane Crb of the R component and the B component, respectively, for each pixel (i, j). In the calculation.
Cb [−k] (i, j) = | Bk (i, j) −G0 (i, j) | (3)
Cb [k] (i, j) = | B0 (i, j) −Gk (i, j) | (4)
Crb [−k] (i, j) = | Rk (i, j) −B0 (i, j) | (5)
Crb [k] (i, j) = | R0 (i, j) −Bk (i, j) | (6)
Step S13: The calculation unit 21 calculates the standard deviations DEVr, DEVb, and DEVrb of each color difference plane in the pixel (i, j) for each blur index using the color difference planes Cr, Cb, and Crb calculated in step S12. In other words, as shown in FIG. 4, the calculation unit 21 displays each color difference plane Cr of a pixel in the reference area AR <b> 1 having a size of 15 pixels × 15 pixels centered on the calculation target pixel (i, j) indicated by diagonal lines. , Cb and Crb are used to calculate the standard deviation. In the present embodiment, the size of the reference area AR1 is 15 pixels × 15 pixels. However, it is preferable that the size of the reference area AR1 is appropriately determined according to the processing capability of the CPU 1 and the accuracy of correction of the axial chromatic aberration. It is preferably set in the range of 30 pixels.

The calculation unit 21 calculates standard deviations DEVr, DEVb, and DEVrb of each color difference plane using the following equations (7) to (9).

Here, k ′ is an integer blur index of −N to N. r indicates the number of pixels on one side of the reference area AR1, and in this embodiment, r = 15 pixels. Further, (l, m) and (x, y) represent pixel positions in the reference area AR1, respectively.

Step S14: The calculation unit 21 calculates a difference for each color component using the target image, the smooth image with the blurring index k = 1, and the following equation (10) in order to calculate the texture amount at the pixel (i, j). To do.
TR (i, j) = R0 (i, j) −R1 (i, j)
TG (i, j) = G0 (i, j) −G1 (i, j) (10)
TB (i, j) = B0 (i, j) −B1 (i, j)
The calculation unit 21 adds, for each color component, a value obtained by adding the differences obtained by Expression (10) of each pixel in the range of 5 pixels × 5 pixels centered on the pixel (i, j) for each color component. A value normalized in units of one pixel is calculated as the texture amount of the pixel (i, j). In addition, the calculating part 21 is good also considering each difference of Formula (10) as the texture amount of each color component.

Step S15: The calculation unit 21 uses the standard deviations DEVr, DEVb, and DEVrb of the color difference surfaces calculated in Step S13, and uses the standard deviation DEVr, DEVb, and DEVrb for each pixel (i, j) to provide the minimum standard deviation index k ′ For each color difference plane. For example, as illustrated in FIG. 5, the calculation unit 21 calculates the minimum blur index k ′ = α in the pixel (i, j) based on the distribution of the standard deviation DEVr [k ′] of the color difference plane Cr corresponding to the blur index. _{Find r} .

The calculation unit 21 performs the same processing for the color difference plane Cb and the color difference plane Crb, and calculates the minimum blurring indices k ′ = α _b and k ′ = α _rb that give the minimum standard deviations DEVb and DEVrb to the target image. It calculates | requires about all the pixels (i, j).

Step S16: The adjusting unit 22 of the CPU 1 extracts a blurred image region (hereinafter referred to as a background region) other than the focused region from the target image based on the texture amount obtained in Step S14. For this purpose, the adjustment unit 22 determines whether the texture amount of the pixel (i, j) is equal to or less than a predetermined value d. The adjustment unit 22 extracts the pixel (i, j) as the background area when the texture amount is equal to or less than the predetermined value d. The adjustment unit 22 performs such extraction processing for all the pixels of the target image.

Step S17: The adjustment unit 22 _{selects a} color component having the highest sharpness in the pixel (i, j) of the target image based on the minimum blurring indexes α _r , α _b , and α _rb obtained in steps S15 and S16. A correction value for adjusting the sharpness between the color components is calculated based on the determined color component having the highest sharpness.

For example, when the minimum blurring index α _r is positive, the adjustment unit 22 determines that it is the G component that has a higher sharpness in the pixel (i, j). On the other hand, when the minimum blurring index α _r is negative, the adjustment unit 22 determines that it is the R component that has a higher sharpness in the pixel (i, j). The adjustment unit 22 also determines color components having higher sharpness based on the signs of the minimum blurring indexes α _b and α _rb .

Based on the above result, the adjustment unit 22 determines whether or not the color component having the highest sharpness has been determined for each pixel. That is, when the same color component is determined in two of the results on the three color difference planes, the adjustment unit 22 determines the color component as the color component having the highest sharpness in the pixel. On the other hand, for example, when the R component, the G component, and the B component are respectively determined based on the blur coefficients α _r , α _b , and α _{rb of} each color difference surface, the adjusting unit 22 determines whether the pixel (i, j) The color component with the highest sharpness cannot be determined as one. In such a case, it is preferable that the adjustment unit 22 determines that the pixel is indefinite and does not perform the process of correcting axial chromatic aberration for the pixel. The color component having the highest sharpness may be determined by comparing the sharpness of each color component determined for each color difference plane.

Next, the adjustment unit 22 calculates a correction value for adjusting the sharpness between the color components based on the determined color component having the highest sharpness. For example, the adjustment unit 22 obtains the blurring index s that gives the true standard deviation value in the pixel (i, j) based on the distribution of the standard deviation DEVr of the color difference plane Cr shown in FIG. That is, the minimum blurring index α _r obtained by the calculation unit 21 in step S15 is not necessarily a blurring index that truly gives the minimum standard deviation DEVr, as indicated by the dotted line in FIG. Therefore, adjustment unit 22, the minimum and the blur index alpha _r, by applying the interpolation to three points with minimum blur index alpha _r -1 and alpha _r +1 adjacent ends, more accurate minimum blur indicator (internal (Insert point) s is obtained. An example in which the color component having the highest sharpness in the pixel (i, j) is the G component is shown below.

Here, in the distribution of the standard deviation DEVr [k ′] (i, j), when DEVr [α _r −1] (i, j)> DEVr [α _r +1] (i, j), the minimum blur index s Is expressed by the following equation (11).
s = ((α _r +1) + α _r ) / 2 + (DEVr [α _r +1] (i, j) −DEVr [α _r ] (i, j)) / 2 / a (11)
Here, the coefficient a is a slope and is (DEVr [α _r −1] (i, j) −DEVr [α _r ] (i, j)) / ((α _r −1) −α _r ).

On the other hand, in the case of DEVr [α _r −1] (i, j) <DEVr [α _r +1] (i, j), the minimum blur index s is expressed as the following equation (12).
s = ((α _r −1) + α _r ) / 2 + (DEVr [α _r −1] (i, j) −DEVr [α _r ] (i, j)) / 2 / a (12)
The slope a is (DEVr [α _r +1] (i, j) −DEVr [α _r ] (i, j)) / ((α _r +1) −α _r ).

Thereafter, the adjustment unit 22 smoothes the distribution of the interpolation points s that are the minimum blurring indexes of the color difference surfaces in the background region extracted in step S16. That is, since the background area is a blurred image area, it is considered that the minimum blur index of each color difference plane is gently and uniformly distributed. However, when there is a pixel in which the minimum blur index greatly changes in the background area, the axial chromatic aberration correction process cannot be appropriately performed on the pixel, and the image quality of the target image is reduced as noise. In order to avoid this, the adjustment unit 22 of the present embodiment, for example, sets the interpolation point s of the pixel (i, j) in the background region to 5 pixels × 5 pixels centered on the pixel (i, j). Is averaged with the interpolation point s of the adjacent pixels in the range of, and the average value is overwritten as a new interpolation point s of the pixel (i, j). Similar processing is performed for the interpolation point s of the other color difference planes. As a result, the minimum blurring index of each color difference surface is gently and uniformly distributed in the background region.

Then, the adjusting unit 22 performs known weighted addition using Gα _r (i, j) and G (α _r +1) (i, j) of the blurring indexes α _r and α _r +1 together with the interpolation point s, A correction value G ′ (i, j) is calculated.

Step S18: The correction unit 23 of the CPU 1 determines whether or not the color structure of the pixel (i, j) is a color boundary based on the minimum blurring indexes α _r , α _b , α _rb and the correction value. First, the correction unit 23 obtains the rate of change of each of the minimum blurring indexes α _r , α _b , α _{rb in} the pixel (i, j). For this purpose, the correcting unit 23 uses, for example, a known differential coefficient formula for each of the minimum blur indices α _r , α _b , α _rb of adjacent pixels at a width of about 5 pixels centered on the pixel (i, j). The inclination is obtained by applying to the above, and the inclination is set as the change rate of the minimum blurring index of each color difference of the pixel (i, j). Correcting unit 23, their rate of change of the threshold epsilon ₁ or more, and determines whether or not the correction value is the threshold epsilon ₂ or more. In the present embodiment, for example, the threshold value ε ₁ = 4 and the threshold value ε ₂ = 40 (when the target image is an image having 255 gradations) are set. Note that the values of the threshold ε ₁ and the threshold ε ₂ are preferably determined according to the accuracy required for the axial chromatic aberration correction processing, the processing capability of the CPU 1, and the like.

Then, when the determination result is true, the correction unit 23 determines that the color structure of the pixel (i, j) is a color boundary, and proceeds to step S19 (YES side). On the other hand, when the determination result is false, the correction unit 23 proceeds to step S20 (NO side).

Step S19: The correcting unit 23 determines whether or not the color structure of the pixel (i, j) determined as the color boundary in Step S18 is a purple fringe. For this purpose, the correction unit 23 uses the pixel value of each color component of the pixel (i, j) and its surrounding pixels or the entire target image, and calculates the pixel value for each color component as shown in FIG. A saturated region in which the pixel value is saturated (in the case of an image with 255 gradations, the pixel value is 255) is obtained. The correction unit 23 has a widest area among the saturation areas of the respective color components. For example, an area obtained by combining a saturation area of the R component and an area widened by β from the end of the saturation area is referred to as a purple fringe area. To do. Note that the value of the width β in this embodiment is, for example, about 10 pixels. However, the size of the width β is preferably determined according to the processing capability of the CPU 1, the accuracy of the axial chromatic aberration correction processing, and the degree of decrease from the saturated state in each color component.

The correction unit 23 determines whether or not the pixel (i, j) is within the purple fringe region based on the obtained purple fringe region. When the pixel (i, j) is in the purple fringe region, the correction unit 23 proceeds to step S20 (YES side). On the other hand, when the pixel (i, j) is not in the purple fringe region, the correcting unit 23 proceeds to step S24 (NO side).

Step S20: correction unit 23, the correction value of the threshold epsilon ₂ or more at which the pixel is calculated in step 17 (i, j) is determined whether a difference of the correction value and the threshold value epsilon ₃ or more adjacent pixels To do. That is, the correction unit 23 determines whether the cause that the correction value of the pixel (i, j) is equal to or greater than the threshold ε ₂ is due to an isolated structure due to shot noise or the like. The value of the threshold ε ₃ is preferably determined and set according to the accuracy required for the axial chromatic aberration correction process, the processing capability of the CPU 1, and the like.

If the determination result is true, the correction unit 23 determines that the color structure of the pixel (i, j) is an isolated structure, and proceeds to step S24 (YES side). On the other hand, when the determination result is false, the correction unit 23 proceeds to step S21 (NO side).

Step S21: The correction unit 23 determines whether or not the color structure of the pixel (i, j) is a pseudo-axial chromatic aberration region based on the texture amount and the correction value in each pixel of the target image.

Here, as described above, the pseudo-axial chromatic aberration region shows a color structure that is affected by the axial chromatic aberration, but is a region to which the axial chromatic aberration correction process cannot be applied. The texture amount in the image area of axial chromatic aberration shown in FIG. 2A shows a distribution that varies greatly for each color component, whereas the texture amount of each color component has a large value as a feature of the pseudo-axial chromatic aberration area. Show similar value distribution. Accordingly, the correction unit 23 of the present embodiment determines whether or not the texture amounts of the respective color components of the pixel (i, j) are all equal to or greater than the threshold ε ₄ and the correction value of the pixel (i, j) is equal to or less than the threshold ε _5. To do. The values of the threshold ε ₄ and the threshold ε ₅ are preferably determined and set according to the accuracy required for the axial chromatic aberration correction process, the processing capability of the CPU 1, and the like.

Then, when the determination result is true, the correction unit 23 determines that the color structure of the pixel (i, j) is a pseudo-axial chromatic aberration region, and proceeds to step S24 (YES side). On the other hand, when the determination result is false, the correction unit 23 proceeds to step S22 (NO side).

Step S22: The correction unit 23 corrects the axial chromatic aberration of the pixel (i, j) using the correction value calculated in step S17.

For example, the correction unit 23 adjusts the sharpness of the R component of the pixel (i, j) based on the following equation (13) when the sharpness of the pixel (i, j) is the G plane. Correct axial chromatic aberration.
R ′ (i, j) = R0 (i, j) + (G0 (i, j) −G ′ (i, j)) (13)
Similarly, the correction unit 23 calculates a correction value G ″ (i, j) for the B component based on the distribution of the standard deviation DEVb of the color difference plane Cb between the B component and the G component, and the following equation (14) Based on the above, the sharpness of the B component in the pixel (i, j) is adjusted to correct the axial chromatic aberration.
B ′ (i, j) = B0 (i, j) + (G0 (i, j) −G ″ (i, j)) (14)
Step S23: The color difference correction unit 24 of the CPU 1 performs color difference correction on the pixel value of each color component of the pixel (i, j) that has been subjected to the axial chromatic aberration correction processing.

That is, in step S22, the pixel value of each color component of the pixel on which the axial chromatic aberration correction processing has been performed changes greatly in the direction of the color difference component particularly in the luminance color difference color space when compared with the pixel value before correction. In some cases, this causes discoloration. Therefore, in the present embodiment, in order to suppress the occurrence of the color change, the color difference correction unit 24 determines that the corrected color difference component in the pixel (i, j) is the direction of the color difference component before correction in the luminance color difference space. Correct to be the same as.

Specifically, the color difference correction unit 24 applies a known conversion process to the pixel values of the respective color components before and after the correction of the pixel (i, j), so that the RGB pixel values (R ′, G, B ′). Are converted into YCrCb luminance components and color difference components (Y ′, Cr ′, Cb ′). Here, it is assumed that the luminance component and the color difference component before correction are (Y0, Cr0, Cb0). Then, the color difference correction unit 24 corrects the direction of the color difference component of the pixel (i, j) to the direction before correction by the following equation (15). In the present embodiment, the luminance component Y ′ is not corrected.

The color difference correction unit 24 applies the above-described known conversion process again, and converts the luminance component and color difference components (Y ′, Cr ″, Cb ″) after the pixel color difference correction into RGB pixel values (R ₁ , Gb). ₁ , B ₁ ). The color difference correction unit 24 sets the pixel value (R ₁ , G ₁ , B ₁ ) as the pixel value of the pixel (i, j).

Step S24: The CPU 1 determines whether or not the processing has been completed for all the pixels of the target image. If the processing has not been completed for all the pixels, the CPU 1 proceeds to step S18 (NO side), and performs the processing from step S18 to step S23 for the next pixel. On the other hand, when the processing is completed for all the pixels, the CPU 1 records an image in which the axial chromatic aberration is corrected in the storage unit 2 or displays it on the output device 30. Then, the CPU 1 ends a series of processes.

As described above, in this embodiment, the correction of axial chromatic aberration is corrected with high accuracy by determining the color structure in each pixel based on the minimum blur index of each color difference surface, the texture amount of each color component, and the value of the correction value. can do.

Further, by not performing the axial chromatic aberration correction processing on the pixels determined to be color boundaries, pseudo-axial chromatic aberration regions, or the like, it is possible to avoid occurrence of discoloration or color loss.

Furthermore, by correcting the corrected pixel value in the direction of the chrominance component of the pixel value before correction in the chrominance space, axial chromatic aberration can be corrected more accurately while suppressing discoloration and color loss. It can be carried out.
<< Additional items of embodiment >>
(1) Although the image processing apparatus of the present invention is realized by causing the computer 10 to execute an image processing program, the present invention is not limited to this. The present invention is also applicable to a program for realizing the processing in the image processing apparatus according to the present invention by the computer 10 and a medium on which the program is recorded.

Also, the present invention can be applied to a digital camera as shown in FIG. 6 having the image processing program of the present invention. Note that in the digital camera shown in FIG. 6, the image sensor 102 and the DFE 103 of the digital front-end circuit that performs signal processing such as A / D conversion of the image signal input from the image sensor 102 and color interpolation processing are imaged. It is preferable to constitute a part.

When the digital camera is operated as the image processing apparatus of the present invention, the CPU 104 realizes each process of the image smoothing unit 20, the calculation unit 21, the adjustment unit 22, the correction unit 23, and the color difference correction unit 24 by software. Alternatively, these processes may be realized by hardware using an ASIC.

(2) In the above-described embodiment, the image smoothing unit 20 uses the N Gaussian filters to generate N smooth images from the target image, but the present invention is not limited to this. For example, when a point spread function (PSF) of an optical system such as an imaging lens 101 of a digital camera as shown in FIG. 6 is obtained, the image smoothing unit 20 uses a PSF instead of a Gaussian filter to generate a smooth image. It may be generated.

Further, the image smoothing unit 20 first generates a set of smooth images using, for example, at least two Gaussian filters, and mixes the set of smooth images at a predetermined ratio so that the blurring indices are mutually equal. N different smooth images may be generated. Thereby, it is possible to simplify and speed up the image processing. Further, by adjusting the mixing ratio of the smooth images within a range where the maximum radius of the Gaussian filter does not exceed the blur amount, a smooth image having an arbitrary blur index can be easily generated. That is, it is possible to easily generate a smooth image within a desired blur amount range even for an image region that requires a very large blur amount in correcting axial chromatic aberration.

(3) In the above embodiment, the calculation unit 21 calculates the texture amount by subtracting the target image and the smooth image of k = 1 having the smallest blurring index for each identical color component. It is not limited. It is preferable that the calculation unit 21 calculates the texture amount using a smoothed image of any blur index depending on how much of the spatial scale image structure is considered.

(4) In the above embodiment, the longitudinal chromatic aberration of the target image is based on the color difference surface Cr between the R component and the G component, the color difference surface Cb between the B component and the G component, and the color difference surface Crb between the R component and the B component. However, the present invention is not limited to this. For example, axial chromatic aberration of the target image may be corrected based on two of the three color difference surfaces. As a result, the speed of the correction process can be increased.

(5) In the above embodiment, the target image has pixel values of R component, G component, and B component in each pixel, but the present invention is not limited to this. For example, each pixel of the target image may have two or four or more color components.

Further, when R, G, and B color filters are arranged in accordance with a known Bayer array on each image element on the light receiving surface of the image sensor 102 of the digital camera shown in FIG. 6, the RAW imaged by the image sensor 102. The present invention can also be applied to images.

(6) In the above embodiment, in step S18, the correction unit 23, pixel (i, j) in the case and the correction value change rate threshold epsilon ₁ or more minimum blur index of each color difference threshold epsilon ₂ or more, the pixel Although the color structure of (i, j) is determined to be a color boundary, the present invention is not limited to this. For example, the correction unit 23 detects an area in which the change rate of the minimum blur index is smaller than the threshold ε ₁ and the correction value is equal to or larger than the threshold ε _{2 in} the image area around the pixel determined to be the color boundary. It is preferable to exclude the chromatic aberration correction target. This is because such an area around the color boundary is found to deteriorate the image quality of the target image when the axial chromatic aberration is corrected according to the present invention.

(7) In the present embodiment, in step S21, the correction unit 23 determines whether or not the color structure of the pixel is a pseudo-axial chromatic aberration region, but the present invention is not limited to this, and the process of step S21 is omitted. May be.

(8) In the above embodiment, the chrominance correction unit 24 performs chrominance correction on all the pixels that have been subjected to the axial chromatic aberration correction processing. However, the present invention is not limited to this, and after correction of those pixels. If the value of the color difference component size L ′ is smaller than the value of the color difference component size L before correction, the color difference correction may not be performed.

The color difference correction unit 24 is defined by FIG. 7 and the following equation (16) when the corrected color difference component size L ′ is larger than the uncorrected color difference component size L (predetermined size). Using the corrected output rate φ (L ′), the corrected color difference component output rate may be reduced based on the following equation (17) obtained by modifying the equation (15).

This makes it possible to more accurately suppress the occurrence of discoloration. Incidentally, the function clip _(V, U 1, _{U 2),} when the value of the parameter V is outside the range of values of the lower limit value _{U 1} and the upper limit value _{U 2,} is clipped to the lower limit value _{U 1} or the upper limit value _{U 2} . The coefficient WV indicates a width for changing the corrected output rate φ (L ′) from the upper limit value U ₂ (= 1) to the lower limit value U ₁ (= 0). In the present embodiment, an image of 255 gradations is displayed. In this case, for example, the coefficient WV is set to a value of 5 to 10. However, the value of the coefficient WV is preferably set as appropriate according to the required degree of suppression of discoloration.

(9) In the above embodiment, the correction unit 23 does not perform the correction process for the axial chromatic aberration when the color structure of the pixel is a color boundary or a pseudo-axial chromatic aberration region, but the present invention is not limited to this. For example, the correction unit 23 may perform the processes of step S22 and step S23 on the pixels in the color boundary and the pseudo-axial chromatic aberration region. However, it is preferable that the correction unit 23 performs the following equations (18) and (19) instead of the equations (13) and (14) in step S22.
R ′ (i, j) = R0 (i, j) + γ × (G0 (i, j) −G ′ (i, j)) (18)
B ′ (i, j) = B0 (i, j) + γ × (G0 (i, j) −G ″ (i, j)) (19)
Here, the coefficient γ is a value set between 0 and 1 according to the color structure. That is, when the color structure is a color boundary, a pseudo-axial chromatic aberration region, or the like, the coefficient γ = 0 or the minimum of each color difference surface such as 0.1 to 0.2 or less so that the influence thereof is not so noticeable. The coefficient γ is preferably set to a small value according to the blurring index, the correction value, the texture amount of each color component, or the like. On the other hand, when it is determined that the color structure is an axial chromatic aberration or purple fringe region, it is preferable to set the coefficient γ = 1 or the like.

From the above detailed description, the features and advantages of the embodiment will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 1 ... CPU; 2 ... Storage part; 3 ... Input / output I / F; 4 ... Bus; 10 ... Computer; 20 ... Image smoothing part; 21 ... Calculation part; 22 ... Adjustment part; 23 ... Correction part; Part: 30 ... output device; 40 ... input device

Claims

Image smoothing means for smoothing a target image having pixel values of a plurality of color components and generating a plurality of smooth images having different degrees of smoothing;
A calculation means for calculating a color difference that is a difference between a predetermined color component of the target image and a color component different from the predetermined color component of the smoothed image and a variance of the color difference in each pixel of the target image;
The color component having the highest sharpness is determined in the pixel of the target image based on the variance of the color difference, and the sharpness between the color components in the pixel of the target image is adjusted based on the determined color component having the highest sharpness. An adjustment means for calculating an adjustment amount;
Correction means for determining an image structure in a pixel of the target image based on the color difference and the adjustment amount, and correcting sharpness between color components in the pixel of the target image with the adjustment amount according to the determination result;
An image processing apparatus comprising:
The image processing apparatus according to claim 1.
The image processing apparatus according to claim 1, wherein the adjustment unit determines a color component having the highest sharpness in a pixel of the target image based on a minimum value of the variance of the color difference.
The image processing apparatus according to claim 1 or 2,
The calculation means, for each pixel of the target image, the difference between the target image and the one smooth image for each same color component, to calculate a texture amount indicating the number of image structures in the target image,
The image processing apparatus, wherein the correction unit determines an image structure in a pixel of the target image based on the color difference, the adjustment amount, and the texture amount.
The image processing apparatus according to claim 3.
The adjustment unit extracts a blurred image region in which the texture amount is equal to or less than a predetermined value, and smoothes the minimum value of the variance of the color difference in the pixels of the blurred image region. .
The image processing apparatus according to any one of claims 1 to 4,
The correction means determines whether the image structure in the pixel of the target image is a color boundary based on the change rate of the color difference in the pixel of the target image and the magnitude of the adjustment amount, and determines the color boundary In such a case, an image processing apparatus that excludes pixels of the target image from correction targets.
The image processing apparatus according to any one of claims 1 to 5,
The correction means compares the adjustment amount of the pixel of the target image with the adjustment amount of an adjacent pixel to determine whether or not the image structure of the pixel of the target image is an isolated structure. An image processing apparatus that excludes pixels of the target image from correction targets when it is determined that the structure has been corrected.
The image processing apparatus according to claim 3 or 4,
The correction unit determines whether the image structure of the pixel of the target image is a pseudo-axial chromatic aberration region similar to the axial chromatic aberration based on the texture amount and the magnitude of the correction amount of the pixel of the target image. And when it determines with the said quasi-axial chromatic aberration area | region, the pixel of the said target image is excluded from correction object, The image processing apparatus characterized by the above-mentioned.
The image processing apparatus according to claim 5.
The correction means determines, based on the distribution of each color component, whether or not the image structure of the pixel of the target image determined to be the color boundary is a color blur due to a density difference around a saturation region, and the color blur and An image processing apparatus characterized in that when it is determined, a pixel of the target image is the correction target.
The image processing apparatus according to any one of claims 1 to 8,
A color difference correction unit that corrects the pixel value of the pixel of the target image with the sharpness adjusted to be the same as the direction of the color difference of the pixel value before the sharpness adjustment in a color difference space. An image processing apparatus.
The image processing apparatus according to any one of claims 1 to 9,
The image smoothing means smoothes the target image with at least two different degrees of smoothing to generate a set of smoothed images, and mixes the set of smoothed images at a predetermined ratio to obtain the plurality of smoothed images. Generating an image processing apparatus.
Determining means for determining an image structure in a pixel of a target image having pixel values of a plurality of color components;
Determining the color component having the highest sharpness in the pixel of the target image, calculating an adjustment amount for adjusting the sharpness of the color component in the pixel of the target image based on the determined color component having the highest sharpness; Correction means for adjusting the sharpness of the color component in the pixel of the target image by the adjustment amount according to the determination result;
An image processing apparatus comprising:
Imaging means for imaging a subject and generating a target image having pixel values of a plurality of color components;
An image processing apparatus according to any one of claims 1 to 11,
An imaging apparatus comprising:
An input procedure for reading a target image having pixel values of a plurality of color components;
An image smoothing procedure for smoothing the target image and generating a plurality of smoothed images having different degrees of smoothing;
A calculation procedure for calculating a color difference that is a difference between a predetermined color component of the target image and a color component different from the predetermined color component of the smoothed image and a variance of the color difference in each pixel of the target image;
The color component having the highest sharpness is determined in the pixel of the target image based on the variance of the color difference, and the sharpness between the color components of the pixel of the target image is adjusted based on the determined color component having the highest sharpness. Adjustment procedure for calculating the adjustment amount,
A correction procedure for determining an image structure in a pixel of the target image based on the color difference and the adjustment amount, and correcting sharpness between color components in the pixel of the target image with the adjustment amount according to the determination result;
An image processing program for causing a computer to execute.
A determination procedure for determining an image structure in a pixel of a target image having pixel values of a plurality of color components;
Determining the color component having the highest sharpness in the pixel of the target image, calculating an adjustment amount for adjusting the sharpness of the color component in the pixel of the target image based on the determined color component having the highest sharpness; A correction procedure for adjusting the sharpness of the color component in the pixel of the target image by the adjustment amount according to the determination result,
An image processing program for causing a computer to execute.