JP5506528B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing technique for performing chromatic aberration correction.

  An imaging device using a photographing lens such as a digital camera is used for various purposes. However, the lateral chromatic aberration generated by the photographing lens is a problem as a factor that causes color shift in the image and lowers the quality. Image sensors used in digital cameras and the like are increasing in number of pixels year by year, and the unit pixel size tends to be reduced. Therefore, even with chromatic aberration of magnification that has not been a problem in the past, it has become a major factor in reducing image quality. Yes.

  As a technique for correcting such color misregistration by image processing, a technique for acquiring a color misregistration amount as a correction amount from an image has been proposed. As a technique for acquiring a color shift amount from an image, there is a method of paying attention to an edge portion of the image and using a similarity based on a correlation between each color component. For example, a method of acquiring a shift amount by searching for a position where the sum of differences between color components described in Patent Document 1 is minimized, or a sum of absolute differences of color components described in Patent Document 2 This is a method of acquiring the amount of deviation by searching for a position where the minimum is.

JP 2000-299874 A JP 2006-020275 A

  However, in the color misregistration amount acquisition method using the similarity based on the correlation of each color component as described above (determining the minimum position), it is correct at the edge where blurring occurs as shown in FIG. The amount of color misregistration may not be acquired. As a result, there is a problem that the hue of the edge portion changes greatly before and after the correction, and the effect of the correction cannot be obtained sufficiently.

  FIG. 8B shows a state after correction by the color misregistration amount acquisition method using the similarity based on the correlation of each color component. For example, when there is no color shift between the G (green) plane and the R (red) plane, and there is a color shift as shown in FIG. 8A between the G (green) plane and the B (blue) plane, FIG. The portion corresponding to the shaded area is yellow. Here, when correction is performed so as to be in the state of FIG. 8B, the portion corresponding to the shaded portion in FIG. 8B changes to blue, which is another color due to overcorrection, and therefore before and after the correction. If compared, it will give the observer a sense of incongruity. Although there is a method of reducing this uncomfortable feeling by adjusting the hue of the edge portion after correction, there is a case where the original color structure of the subject is lost.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to obtain an appropriate amount of color misalignment and correct color misalignment correction when correcting color misalignment due to lateral chromatic aberration by image processing. Is to be able to do.

  An image processing apparatus according to the present invention relates to an edge extraction means for extracting an edge portion from image data including color signals of a plurality of colors, and a first color signal and a second color signal among the color signals of the plurality of colors. , For each color signal, high-frequency component extraction means for extracting a high-frequency component in a direction crossing the edge portion, and the first high-frequency component extraction means extracted in the image data screen by the first high-frequency component extraction means. The first color signal and the second color signal based on the position in the screen where the high frequency component of the color signal and the second color signal indicate the maximum value and the position in the screen where the minimum value indicates the minimum value. A displacement amount acquisition means for acquiring a positional deviation amount of the image data, and based on the positional deviation amount acquired by the displacement amount acquisition means, at least one of the first color signal and the second color signal is converted into the image data. By moving within the screen Characterized in that it comprises a correction means for correcting the positional deviation of the first color signal and the second color signal.

  According to the present invention, when the color shift due to the chromatic aberration of magnification is corrected by image processing, it is possible to acquire an appropriate color shift amount and perform more appropriate color shift correction.

1 is a block diagram illustrating an imaging apparatus according to an embodiment of the present invention. 6 is a flowchart showing processing in an image processing apparatus in the imaging apparatus according to the embodiment. The figure which shows the area | region divided | segmented for every image height, and a color shift rate. 5 is a flowchart illustrating color misregistration detection processing. The figure which shows an attention edge and an attention line. The figure which shows the mode of color shift. The figure which shows the filter factor of HPF The figure which shows the color shift of an edge part.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the drawings. Hereinafter, for the target color planes, both the G plane and R plane color misalignment correction and the G plane and B plane color misregistration correction should be corrected by the following method. Shall be able to.

  In this embodiment, an example in which the present invention is applied to an imaging apparatus having a photographing system such as a digital camera will be described. However, the present invention is a plurality of images photographed by a photographing system using a photographing lens other than a digital camera. The present invention can also be applied to an image made up of color planes. Therefore, the image that is the subject of the present invention is not limited to either RAW data or JPEG data after development processing.

  First, an outline of a process from creating and correcting magnification chromatic aberration correction data from an image in an imaging apparatus having an image processing apparatus that performs magnification chromatic aberration correction will be described. FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus having an image processing apparatus according to the present embodiment.

  The imaging apparatus 100 includes an imaging optical system 10, an imaging element 20, an A / D conversion unit 30, a color separation unit 40, an edge extraction unit 50, an HPF (high pass filter) unit 61, a correction data creation unit 60, and an image correction unit. 70.

  In FIG. 1, light from a subject passes through an imaging optical system 10 and is imaged on an image sensor 20 that photoelectrically converts the subject image. The image pickup device 20 is a single-plate color image pickup device having a primary color filter with a general Bayer arrangement. The primary color filter is composed of three color filters each having a transmission main wavelength band in the vicinity of 650 nm, 550 nm, and 450 nm, and color planes corresponding to the bands of R (red), G (green), and B (blue), respectively. (Multi-color signal) is generated. In a single-plate color image sensor, this color filter is spatially arranged for each pixel, and each pixel can only obtain an intensity in a single color plane. For this reason, a color mosaic image is output from the image sensor.

  The A / D converter 30 converts the color mosaic image output as an analog voltage from the image sensor into digital data (image data) suitable for subsequent image processing. The color separation unit 40 interpolates the color mosaic image to generate a color image in which R, G, and B color information are aligned in all pixels. Although many methods such as bilinear interpolation and bicubic interpolation have been proposed for this interpolation method, the present invention does not limit the interpolation method, and any interpolation method may be used.

  The edge extraction unit 50 detects an edge (edge portion) from the generated image. Based on the detected edge information, the correction data creation unit 60 including the HPF unit 61 creates magnification chromatic aberration correction data from the image. The image correction unit 70 performs magnification chromatic aberration correction using the created magnification chromatic aberration correction data.

  FIG. 2 is a flowchart showing the flow of processing in the correction data creation unit 60 and the image correction unit 70 including the edge extraction unit 50 and the HPF unit 61, which are the magnification chromatic aberration correction part in the imaging apparatus of the present embodiment. Hereinafter, with reference to FIG. 2, correction of chromatic aberration of magnification in the imaging apparatus of the present embodiment will be described.

  First, in edge detection step S101, an edge where a color shift due to magnification chromatic aberration appears noticeably is detected from the image. A Y (luminance) plane is generated from the R, G, and B planes interpolated by the color separation unit 40, and this Y plane is used for edge detection. The edge detected here is limited to the edge where the pixel value changes greatly in the radial direction from the optical center, so that a highly accurate color misregistration amount can be obtained. Also, in the Y plane, the color shift due to the chromatic aberration of magnification appears as blurring, so it is desirable to target an edge that has a certain width such that a plurality of pixels with a monotonically increasing or decreasing pixel value continues. .

  In the color shift amount acquisition step S102, the color shift amount is acquired at each edge detected in the edge detection step S101. The color misregistration acquisition processing according to the present embodiment is applied to the acquisition of the color misregistration amount, and details will be described later. Depending on the positional relationship between the optical center and each edge, the color misregistration handled here can be any of up / down, left / right, right up / down left, left up / down right This simplifies the process.

  The color misregistration amount that is output in the color misregistration amount acquisition step S102 is a negative value when the R plane (or B plane) is shifted in the optical center direction with respect to the G plane. A positive value is set when the R plane (or B plane) is shifted in the direction opposite to the optical center with respect to the G plane.

  In the correction data creation step S103, the relationship between the image height and the color shift is obtained from the image height of each edge detected in the edge detection step S101 and the color shift amount of each edge acquired in the color shift amount acquisition step S102. Create correction data. The image height here is a distance from a pixel corresponding to the optical center (hereinafter simply referred to as the optical center).

The correction data creation procedure will be specifically described below.
(1) When the image height of the edge detected in the edge detection step S101 is L and the color shift amount acquired in the color shift amount acquisition step S102 is D, the color shift rate M with respect to the image height is obtained as follows. .

M = D / L
(2) As shown in FIG. 3A, the image is divided into eight regions (h1 to h8) for each image height, and the region to which the edge belongs is selected.
(3) The above-described (1) and (2) are performed on a plurality of edges detected in the image, and the color misregistration rate M is totalized for each of the eight regions divided for each image height, and the color misregistration rate M for each region. Is obtained, and the color misregistration ratio of each region is determined.
(4) As shown in FIG. 3B, a higher-order polynomial approximate expression F (L) representing the relationship between the image height and the color misregistration ratio is calculated from the image height and the obtained color misregistration ratio. The correction data. FIG. 3B shows an example in which correction data is calculated using a cubic polynomial.

  Note that edge detection and color misregistration acquisition may be performed for all edges in the screen. However, in each of the eight areas divided for each image height, edge detection and color shift amount acquisition are completed when the number of color shift ratios equal to or greater than a certain threshold is tabulated, improving processing efficiency while maintaining reliability. Can be achieved.

  Further, by using only the region where the corresponding edge is found among the eight regions divided for each image height for calculating the higher-order polynomial approximation, even if there is a region where the corresponding edge is not found, the correction data Can be created.

  In the image correction step S104, the color misregistration is corrected using the correction data created in the correction data creation step S103. First, in the pixel (X, Y) of the plane (R plane, B plane) to be corrected, the color misregistration rate M is obtained from the image height L of the pixel (X, Y). It is assumed that the optical system has a coordinate system of (0, 0).

M = F (L)
Next, the coordinates (X1, Y1) of the pixel generated by the color misregistration correction are obtained.

X1 = X + M × X
Y1 = Y + M × Y
A pixel value corresponding to the coordinates (X1, Y1) in the plane to be corrected is generated by a general interpolation process, and is set as the pixel value of the pixel (X, Y). Color misregistration correction is performed by performing these operations for all pixels.

  The above is the outline of the processing from creating and correcting magnification chromatic aberration correction data from an image. Details of the color misregistration amount acquisition step S102 in the present embodiment will be described below with reference to the flowchart of FIG.

  FIG. 4 shows the flow of obtaining the color shift amount (color component positional shift amount) in the G plane and B plane (first color signal and second color signal), but also in the G plane and R plane. The amount of color misregistration is acquired by the same process. First, in the target pixel acquisition step S201 of each color component, a pixel that is the target of the color misregistration amount acquisition process is acquired near the edge in the image detected in the edge detection step S101. As shown in FIG. 5, the pixel that is the target of the color misregistration acquisition process is one line (target line) in the color misregistration acquisition direction that crosses the target edge for acquiring the color misregistration amount. In the example of FIG. 5, since the edge of interest is located at the upper right of the image, the direction of obtaining the color misregistration is applied to the upper right / lower left direction. Hereinafter, an area for which a color misregistration amount is acquired is referred to as a target line.

  The size of the attention line is preferably set according to the width of the attention edge and the amount of change in luminance. For example, the length is preferably about twice the width of the edge. FIG. 6A shows an example of the G plane and B plane pixel signals acquired in the target pixel acquisition step S201 for each color component.

  In the next HPF step S202, the pixel signal acquired in the target pixel acquisition step S201 of each color component is subjected to high-pass filter processing to perform high-frequency component extraction. FIG. 7 shows the taps and filter coefficients of the high-pass filter used here. FIG. 6B shows the result of applying the filter shown in FIG. 7 to the G and B pixels shown in FIG. In the frequency domain, since the start point / end point of the edge includes many high-frequency components, the start point / end point (edge start point position and edge end point position) of the edge can be extracted by the high-pass filter. A portion where the absolute value of the output value of the high-pass filter is large represents the start and end points of the edge. In order to suppress the influence of noise or the like and improve the detection accuracy of the start point / end point of the edge, the pixel value may be smoothed on the target pixel before the high-pass filter.

  In the next edge start point / end point search step S203, the edge start point / end point is obtained from the output result of the high pass filter. The positions of the maximum value and the minimum value of the output value of the high pass filter are obtained for each color plane, and the positions are set as the start point and end point of the edge.

  The maximum value position of the high-pass filter output value of the G plane is Gmax, the minimum value position is Gmin, the maximum value position of the high-pass filter output value of the B plane is Bmax, and the minimum value position is Bmin. If there is a pixel that contains a lot of high-frequency components at a location other than the start point / end point of the edge in the line of interest, the maximum value / minimum value may not be the start point / end point of the edge. In order to cope with this, the start point / end point of the edge may be obtained by obtaining an inflection point near the center of the line of interest.

  Further, when the following equation is satisfied, it is determined that the edge is not suitable for the color misregistration amount acquisition process, and the color misregistration amount is not acquired at this edge.

(Gmax> Bmax AND Gmin <Bmin) OR
(Gmax <Bmax AND Gmin> Bmin)
In the next color misregistration amount determination step S204, first, two misregistration amounts (d1, d2) are obtained by the following equation. Here, d1 and d2 are expressed as follows.

d1 = | Gmax−Bmax |
d2 = | Gmin−Bmin |
Next, the two misregistration amounts d1 and d2 are compared, and the color misregistration amount is determined as follows.

    When d1 ≧ d2: d2 is the amount of color misregistration.

    When d1 <d2: d1 is the color misregistration amount.

  In the example of FIG. 6B, since d1 ≧ d2, d2 is determined as the color misregistration amount. FIG. 6C shows the result of correcting the color misregistration by moving at least one of the G signal and the B signal by d2. It can be seen that d2 is appropriate as an amount of shifting B without overcorrection.

  The above is the description of the color misregistration amount acquisition step S102. In this way, instead of determining the degree of similarity based on the sum of absolute differences, an appropriate color shift amount can be obtained by comparing the start and end positions of the edges obtained by applying a high-pass filter between the color components, and correction. Appropriate color misregistration correction with no sense of incongruity before and after can be performed.

  In the above embodiment, the unit of acquisition of the color misregistration amount is 1 pixel. However, by interpolating the output value of the high-pass filter performed in HPF step S202 by interpolation processing such as spline interpolation or low-pass filter, 1 The amount of color misregistration can be acquired in units less than pixels. Further, instead of the high pass filter, a high frequency component in a direction crossing the edge portion of the color signal may be extracted using wavelet transform.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. Moreover, you may combine suitably a part of above-mentioned embodiment.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (5)

Edge extraction means for extracting edge portions from image data including color signals of a plurality of colors;
High-frequency component extraction means for extracting a high-frequency component in a direction across the edge portion for each color signal for the first color signal and the second color signal of the color signals of the plurality of colors;
In the screen of the image data, the first color signal extracted by the high frequency component extraction means and the high frequency component of the second color signal indicate the maximum value and the position and minimum value in the screen. A deviation amount acquisition means for acquiring a positional deviation amount between the first color signal and the second color signal based on the position in the screen;
By moving at least one of the first color signal and the second color signal within the screen of the image data based on the positional shift amount acquired by the shift amount acquisition means, Correction means for correcting a positional deviation between the color signal and the second color signal;
An image processing apparatus comprising:   The deviation amount acquisition means is based on the position in the screen where the high frequency component of the first color signal and the second color signal shows the maximum value and the position in the screen where the minimum value shows the minimum value. An edge start point position and an edge end point position of the first color signal and an edge start point position and an edge end point position of the second color signal are obtained, and the edge start point position of the first color signal and the second color signal are obtained. The smaller value of the difference between the edge start point position of the first color signal and the difference between the edge end point position of the first color signal and the edge end point position of the second color signal is calculated as the first color signal and the second color signal. The image processing apparatus according to claim 1, wherein the image processing apparatus is acquired as a positional deviation amount of the color signal.   The deviation amount acquisition means interpolates a high frequency component of the first color signal and the second color signal, thereby obtaining an edge start point position and an edge end point position of the first color signal, and the second color signal. The image processing apparatus according to claim 2, wherein an edge start point position and an edge end point position of the color signal are obtained in units of less than one pixel. An image processing method in an image processing apparatus,
An edge extraction step in which the edge extraction means of the image processing device extracts an edge portion from image data including color signals of a plurality of colors;
The high-frequency component extracting means of the image processing device uses a high frequency in a direction crossing the edge portion for each color signal of the first color signal and the second color signal of the color signals of the plurality of colors. A high-frequency component extraction process for extracting components;
The deviation amount acquisition means of the image processing device has a maximum value in the high frequency component of the first color signal and the second color signal extracted in the high frequency component extraction step in the screen of the image data. A shift amount acquisition step of acquiring a shift amount between the first color signal and the second color signal based on the position in the screen and the position in the screen indicating the minimum value;
The correction means of the image processing device, based on the positional shift amount acquired in the shift amount acquisition step, at least one of the first color signal and the second color signal in the screen of the image data A correction step of correcting a positional deviation between the first color signal and the second color signal by moving the first color signal;
An image processing method comprising:   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 3.

Images