JP6907040B2 - Image processing equipment, imaging equipment, lens equipment, image processing methods, programs, and storage media - Google Patents

Description

The present invention relates to an image processing device that corrects chromatic aberration of magnification (color deviation).

In an imaging device such as a digital camera, various aberrations of an imaging lens used for forming a subject image cause a factor of deteriorating the image quality of the subject image. For example, chromatic aberration of magnification causes a color shift in the imaged subject image.

Since the number of pixels of the image sensor used in the image sensor is increasing year by year and the unit pixel size is shrinking, it has become a main factor for deteriorating the image quality even with a chromatic aberration of magnification that has hardly been a problem in the past. There is. Further, regarding the chromatic aberration of magnification, a color deviation different from the color deviation calculated by the design data of the imaging lens may occur due to the influence of individual variation due to the manufacturing error of the imaging lens. This is due to an error in the assembly position of the lens group constituting the image pickup lens and the influence of the variation in the material of each lens.

As a technique for correcting such a color shift by image processing, a technique for acquiring a color shift amount (correction amount) to be corrected from a captured image has been proposed. As a technique for obtaining the amount of color shift from an image, a method using the correlation between each color component at the edge portion of the image is known.

For example, in Patent Document 1, by calculating concentric aberration in which color shift occurs concentrically from the center of an image and uniform aberration in which the direction and amount of color shift occur uniformly in the entire image, individual variation of the lens can be obtained. An image processing device that corrects the corresponding chromatic aberration of magnification is disclosed.

Patent No. 5505135

The image processing apparatus of Patent Document 1 performs correction of uniform aberration that occurs in the same direction and the same amount in the screen and correction of concentric aberration in order to improve the correction accuracy of chromatic aberration of magnification when there is a manufacturing error. .. However, in the image processing device of Patent Document 1, an error component that is not uniform in the screen and is not unique in terms of image height due to manufacturing errors such as when eccentricity (shift) or tilt (tilt) occurs when the lens is attached. If this occurs, the chromatic aberration of magnification (color deviation) cannot be corrected with high accuracy.

Therefore, an object of the present invention is to provide an image processing device, an imaging device, a lens device, an image processing method, a program, and a storage medium capable of correcting color deviation due to a manufacturing error with high accuracy.

The image processing apparatus as one aspect of the present invention has a detection means for detecting the amount of color shift in the radial direction from the center of the image and the amount of color shift in the center of the image based on the amount of color shift detected by the detection means. The color deviation amount of the image is related to the color deviation amount calculating means for calculating the above, the color deviation amount detected by the detection means, and the color deviation amount of the image center calculated by the calculation means. It has a correction data calculation means for calculating correction data.

An image pickup device as another aspect of the present invention includes an image pickup device that photoelectrically converts an optical image formed via an image pickup optical system, and the image processing device.

The lens device as another aspect of the present invention includes an imaging optical system and the image processing device.

The image processing method as another aspect of the present invention is based on a color shift amount detection step for detecting the amount of color shift in the radial direction from the center of the image and the color shift amount detected by the color shift amount detection step. , The color shift amount calculation step for calculating the color shift amount at the center of the image, the color shift amount detected by the color shift amount detection step, and the color shift at the image center calculated by the color shift amount calculation step. It has a correction data calculation step of calculating correction data regarding the amount of color shift of the image based on the amount.

A program as another aspect of the present invention causes a computer to execute the image processing method.

A storage medium as another aspect of the present invention stores the program.

Other objects and features of the present invention will be described in the following embodiments.

According to the present invention, it is possible to provide an image processing device, an imaging device, a lens device, an image processing method, a program, and a storage medium capable of correcting color deviation due to a manufacturing error with high accuracy.

(A) A flowchart showing a color shift correction process according to the first embodiment, and (b) a block diagram of an image processing device according to the first embodiment. It is explanatory drawing of the amount of color deviation due to chromatic aberration of magnification. This is an example of a chart used for calculating the color shift correction value in each embodiment. It is explanatory drawing of the amount of color deviation in 1st Embodiment. It is explanatory drawing of the color deviation correction value in 1st Embodiment. It is explanatory drawing of the color deviation correction method in 1st Embodiment. It is explanatory drawing of the color deviation correction value in 1st Embodiment. It is a block diagram of the image pickup apparatus in the 2nd Embodiment. It is a flowchart which shows the color shift correction process in 2nd Embodiment. It is explanatory drawing of the color deviation acquisition area in 2nd Embodiment. It is explanatory drawing of the color deviation detection direction in 2nd Embodiment. It is explanatory drawing of the amount of color deviation in 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, with reference to FIG. 2, color shift due to chromatic aberration of magnification will be described. If there is no lens manufacturing error (individual variation), the color shift due to chromatic aberration of magnification can be known from the lens design data. In this case, the chromatic aberration of magnification generally shows a unique amount of color shift according to the image height, which is the distance from the center of the optical axis. However, in reality, due to lens manufacturing errors, the lens design data is often not met.

FIG. 2 is an explanatory diagram of color shift due to chromatic aberration of magnification, in which the vertical axis shows the amount of color shift and the horizontal axis shows the image height. FIG. 2A is a diagram showing the amount of color shift (color shift due to the design value) of the R component with respect to the G component imaged on the surface of the image sensor provided with RGB pixels, and is a horizontal view passing through the center of the captured image. Shown with respect to the area of direction. In FIG. 2A, as the direction of the amount of color deviation, the deviation in the right direction is positive and the deviation in the left direction is negative in the captured image. That is, as shown in FIG. 2A, the higher the image height, the more the R component shifts to the higher image height side with respect to the G component. Generally, the image pickup device (camera) and the lens device (lens) are designed so that the center of the optical axis coincides with the center of the image. A symmetrical color shift occurs with respect to the center of the image.

FIG. 2B is an example of a color shift when a lens mounting position error or a refractive index error due to a material occurs along the optical axis as a lens manufacturing error. In FIG. 2B, the solid line shows the actual color shift (color shift reflecting the influence of the manufacturing error), and the broken line shows the color shift due to the design value (design data). As shown in FIG. 2B, the color shift reflecting the influence of the manufacturing error is different from the amount of the color shift according to the design data, but it has a symmetrical shape on the left and right sides of the image, and the image height. It is a unique color shift according to.

FIG. 2C is an example of color shift when eccentricity (shift) or tilt (tilt) occurs when the lens is attached as a lens manufacturing error. As shown in FIG. 2C, color shift may occur even at the center of the image due to eccentricity or tilting of the lens. Further, due to the tilting of the lens, the imaging position differs depending on the position on the screen, and as a result, the amount of color shift differs between the left and right sides of the image. For example, the image is formed in front of the image sensor (closer to the lens) toward the high image height on the right side of the screen, and behind the image sensor (far side from the lens) toward the high image height on the left side of the screen. ) Is in the state of being imaged. In addition, the optical axis of the lens may be different from the center of the screen.

In this way, due to the influence of the optical axis shift due to the manufacturing error of the lens and the imaging position, the color shift differs depending on the position on the screen, is not uniquely determined according to the image height, and the lens design data. The deviation from the chromatic aberration of magnification due to the above may not be uniquely determined according to the image height.

Even if the image in which the color shift occurs in this state is corrected for the color shift by image processing based on the design data of the lens, the correction residue is generated as shown in FIG. 2D, and the rest remains. The amount depends on the image height. Ideally, when the correction is performed, as shown in FIG. 2 (e), the color shift does not occur in all the regions on the image. Each of the following embodiments solves such a problem.

<First Embodiment>
First, the first embodiment of the present invention will be described. In the present embodiment, a method of calculating correction data (magnification chromatic aberration correction value, color deviation correction value) regarding magnification chromatic aberration (color deviation amount) reflecting the influence of lens manufacturing error will be described in the manufacturing process of the lens apparatus. .. When the image pickup device is a digital camera with a built-in lens, the calculated correction data is stored in the digital camera body and used. Further, when the image pickup device is an interchangeable lens camera such as a single-lens reflex camera, the correction value is stored in the main body of the lens device and used. However, this embodiment is not limited to these.

The correction data is calculated by photographing a chart in the manufacturing process of the lens apparatus. Color shift due to chromatic aberration of magnification appears prominently at the edge of the image. Therefore, as shown in FIG. 3, as a chart used for calculating the correction value, there are eight directions (N, NW, W, SW, S, SE, E, NE directions) in the radial direction from the center of the image. Charts with edges are used. In the present embodiment, it is sufficient that the color shift can be detected in the radial direction from the center of the image, and the structure of the chart is not limited to the example shown in FIG.

Next, with reference to FIG. 1, a method of calculating the amount of color shift due to chromatic aberration of magnification generated in the lens device will be described. FIG. 1A is a flowchart showing a method of calculating the amount of color deviation due to chromatic aberration of magnification (color deviation correction processing). FIG. 1B is a block diagram of an image processing device. Each step of FIG. 1A is performed by the image processing apparatus 10 in the manufacturing process of the lens apparatus.

As shown in FIG. 1B, the image processing device 10 includes a detection means 11, a color shift amount calculation means 12, and a correction data calculation means 13. The detecting means 11 detects the amount of color shift in the radial direction from the image. The color shift amount calculation means 12 calculates the color shift amount at the center of the image based on the color shift amount detected by the detection means 11. The correction data calculation means 13 calculates the correction data regarding the color shift amount of the image based on the color shift amount detected by the detection means 11 and the color shift amount of the image center calculated by the color shift amount calculation means 12. do.

First, in step S101, the detection means 11 captures, for example, the chart shown in FIG. 3 by using an imaging device equipped with a lens that detects color shift due to chromatic aberration of magnification, and acquires a captured image. The image data acquired by the image pickup apparatus is an image composed of RGB components, and is processed in advance so that each pixel holds RGB information. Further, in order to acquire the amount of color shift described later using the acquired captured image, each process such as gamma correction, sharpening processing, and color reproduction processing generally performed by a digital camera is not performed here. Get an image of the state.

Subsequently, in step S102, the detection means 11 detects the color shift near the edge on the chart (color shift amount detection step). The target edge is an edge whose pixel value changes in the radial direction from the center of the image. This edge can be detected from the image, or a region on the image may be specified in advance from the position on the chart where the edge exists. When detecting an edge from an image, it is sufficient to detect an edge whose pixel value changes significantly in the radial direction from the center of the image.

The amount of color shift is obtained by searching for the position where the correlation between the R plane (R component) (or the B plane (B component)) with respect to the G plane (G component) is the largest. The edge detection direction is the radial direction from the center of the image. For example, as shown by the arrows in FIG. 3, the edge detection directions are left / right (W / E) direction, up / down (N / S) direction, and diagonally up / right depending on the position where the edge exists. It is set to either the diagonally lower left (NE / SW) direction or the diagonally upper left / downward right (NW / SE) direction.

FIG. 4 is an example of the amount of color shift of the R component with respect to the G component calculated by the edges existing in the horizontal direction (E direction and W direction in FIG. 3) passing through the center of the image using the chart shown in FIG. be. In FIG. 4, as for the amount of color deviation, the deviation in the right direction of the captured image is positive, and the deviation in the left direction is negative.

Subsequently, in step S103 of FIG. 1, the color shift amount calculation means 12 calculates the color shift amount generated in the center of the image based on the color shift amount detected in step S102 (color shift amount calculation step). ). This is performed using the detection result of the detected color shift amount near the center of the image and at a position symmetrical with respect to the center of the image. Specifically, among the color shift detection results shown in FIG. 4, the detection results at positions h1 and h1'symmetric with respect to the image center are used. Assuming that the amount of color shift at position h1 is d1 and the amount of color shift at position h1'is d1', the average value Ce of the two amounts of color shift d1 and d1'is calculated by the following equation (1).

Ce = (d1 + d1') / 2 ... (1)
Similarly, the amount of color shift calculated using the edges existing in the vertical directions (N and S directions in FIG. 3) through the center of the image is also near the center of the image and with respect to the center of the image. The average value Cn is calculated using the detection results at symmetrical positions.

The average value Ce thus calculated is a horizontal component of the amount of color shift occurring in the center of the image, and the average value Cn is a vertical component of the amount of color shift occurring in the center of the image. It should be noted that, among the edges existing in the eight directions, another one is used by using the detection result of the amount of color shift using the edges existing in the diagonal directions (NE direction and SW direction, NW direction and SE direction in FIG. 3). The average values Cne and Cnw of the amount of color shift (color shift component) of the set can be detected, respectively. Then, by averaging the average values Cne and Cnw and the previously calculated average values Ce and Cn, it is possible to uniquely determine the color shift vector at the center of the image.

The color shift amounts d1 and d1'are not the color shift amounts obtained from the edges completely located at the center of the image, but the color shift amounts obtained from the edges at a certain distance from the image center. Therefore, it is considered that chromatic aberration of magnification is generated according to the image height. However, by obtaining the average value from the amount of color shift acquired at a position symmetrical with respect to the center of the image in this way, the influence of chromatic aberration of magnification according to the image height is reduced or eliminated, and the color generated in the center of the image is obtained. The deviation can be estimated with high accuracy. Further, since the influence of the lens manufacturing error is large in the portion where the image height is high and the influence of the non-uniform deviation component in the screen may be large, the detection result of the amount of color deviation near the center of the image is used. This makes it possible to calculate the amount of color shift at the center of the image from the color shift detected in the radial direction.

Subsequently, in step S104 of FIG. 1, the correction data calculation means 13 is based on the amount of color shift in the radial direction of each edge detected in step S102 and the amount of color shift in the center of the image calculated in step S103. Then, the amount of color shift for each image height is calculated for each radiation direction. That is, the correction data calculation means 13 obtains correction data (correction value for chromatic aberration of magnification, color shift correction value) according to the image height for each direction from the center of the image (for each of the eight directions shown in FIG. 3). calculate. The amount of color shift calculated in step S104 (correction data calculation step) corresponds to the correction data.

FIG. 5 is an explanatory diagram of the correction data, and shows the relationship between the amount of color shift and the image height in the horizontal direction (E direction and W direction) passing through the center of the image. The average value Ce corresponding to the amount of color shift in the center of the image is a horizontal component of the amount of color shift in the center of the image calculated in step S103. Based on the respective color shift amounts of the image heights h1 to h7 acquired on the right side of the image, the approximate curve 1 in FIG. 5 passing through the average value Ce is calculated as a polynomial approximate expression such as a cubic function, and the calculated approximate curve 1 is calculated. Is the correction data (color deviation correction value) related to the magnification chromatic aberration in the E direction. Further, based on the respective color shift amounts of the image heights h1'to h7' acquired on the left side of the image, the approximate curve 2 in FIG. 5 passing through the average value Ce is calculated, and the calculated approximate curve 2 is used in the W direction. It is a correction value (color deviation correction value) related to chromatic aberration of magnification. Similarly, correction data (color shift correction value) relating to chromatic aberration of magnification in the N direction, NW direction, SW direction, S direction, SE direction, and NE direction is calculated. As described above, in the present embodiment, preferably, the correction data calculation means 13 calculates an approximate curve that passes through the amount of color shift (average value Ce) at the center of the image as the correction data.

Subsequently, in step S105 of FIG. 1, the image processing device 10 stores the calculated correction data (color shift correction value) in each direction as a storage means (such as a memory of the image pickup device (camera) or the lens device (lens body)). For example, it is stored in the storage unit 108) in FIG. The effect of the amount of color shift and manufacturing error due to chromatic aberration of magnification changes depending on the focal length of the lens, the object distance, and the optical conditions (imaging conditions) such as the aperture. Therefore, the correction value can be calculated for each optical condition. Alternatively, the correction value is calculated for each of a plurality of typical optical conditions, and the other optical conditions correspond to the optical conditions when the image is taken based on the calculated correction value when correcting the amount of color shift described later. The corrected correction value may be calculated. When shooting with an image pickup device, the correction values in eight directions according to the optical conditions at the time of shooting are acquired from the memory, or the correction values of typical optical conditions stored in the memory are changed to the optical conditions at the time of shooting. The corresponding correction value is calculated by interpolation or the like to correct the chromatic aberration of magnification (color deviation amount) of the image.

FIG. 6 is an explanatory diagram of a color shift correction method, and shows a positional relationship between a certain correction target pixel and an image center (center pixel) in an image. A case where the pixel P as the correction target pixel is corrected will be described with reference to FIG. FIG. 7 is an explanatory diagram of a color shift correction value. 7 (a) to 7 (c) show examples of correction values for the N direction, the NE direction, and the E direction shown in FIG. 6, respectively.

The correction values in each direction are stored in the memory of the image pickup apparatus or the lens apparatus as correction curves Fn (l), Fne (l), and Fe (l) with respect to the image height l. The pixel P is located so as to be sandwiched between the NE direction and the E direction. Therefore, first, the color shift rates K1 and K2 of the positions P1 and P2 in FIG. 6 corresponding to the image height lp of the pixel P are determined by the following equations (2) and (3) based on the correction values in the respective directions. Calculate as follows.

K1 = (Fe (lp) -Ce) / lp ... (2)
K2 = (Fne (lp) -Cne) / lp ... (3)
The calculated color shift rates K1 and K2 are linearly interpolated according to the angles θ and θ ′ with respect to the E direction and the NE direction, respectively, and the color shift rate Kp of the pixel P is calculated. When the coordinates of the pixel P are (x, y), the position (x', y') where the R component is imaged due to the color shift is calculated by the following equations (4) and (5).

x'= x × Rp + Ce ... (4)
y'= y × Rp + Cne… (5)
By replacing the R component at the position (x', y') calculated in this way with the R component in the pixel P, the color shift of the R component with respect to the G component can be corrected. Similarly, by performing such correction for each pixel in the image and also correcting the B component, it is possible to correct the color shift due to the chromatic aberration of magnification of the image.

As a result of performing the above-mentioned correction processing on the image in which the color shift shown in FIG. 2 (c) occurs, the color shift occurs in each region in the image as in the example shown in FIG. 2 (e). It is in a state of being corrected with high accuracy. According to the present embodiment, as shown in FIGS. 2C and 2D, the deviation between the lens design data and the actual amount of color deviation is not uniform in the image and depends on the direction from the center of the image. Even if the amount is different, it is possible to respond appropriately. In this way, by calculating the color shift correction value of the chromatic aberration of magnification including the variation due to the manufacturing error by using the amount of the color shift detected in the radial direction, the color shift due to the chromatic aberration of magnification can be corrected with high accuracy.

<Second embodiment>
Next, a second embodiment of the present invention will be described. In the present embodiment, an imaging device that detects and corrects chromatic aberration of magnification from an image at the time of shooting will be described.

First, the basic configuration of the image pickup apparatus according to the present embodiment will be described with reference to FIG. FIG. 8 is a block diagram of the image pickup apparatus 100. The image pickup optical system 101 has an aperture 101a and a focus lens 101b, and forms an image of a subject (not shown) on the image pickup element 102. The image pickup device 102 is a CCD sensor or a CMOS sensor, and photoelectrically converts an optical image (subject image) formed via the image pickup optical system 101. In the present embodiment, the image pickup device 102 is a single-plate color image pickup device in which a plurality of photoelectric conversion elements (a plurality of pixels) are arranged two-dimensionally and a general primary color filter is provided. In the primary color filter, three types of unit filters (R filter, G filter, B filter) having a transmission main wavelength band in the vicinity of wavelengths of 650 nm, 550 nm, and 450 nm are arranged two-dimensionally corresponding to each photoelectric conversion element. ing. As a result, each photoelectric conversion element receives light transmitted through one of the three types of unit filters and outputs an electric signal indicating the intensity thereof. Therefore, each photoelectric conversion element of the single plate color image sensor outputs the intensity of one of the R plane, the G plane, and the B plane. Therefore, the captured image output from the image sensor 102 is a color mosaic image.

The A / D conversion unit 103 converts a color mosaic image (analog image) output as an analog voltage from the image pickup element 102 into digital data (digital image) suitable for image processing in the subsequent stage, and outputs the color mosaic image (digital image) to the image processing unit 104. .. The image processing unit 104 includes a color separation unit 111, a color deviation detection unit 112, a central color deviation calculation unit 113, a color deviation calculation unit 114 according to image height, a color deviation correction unit 115, and a processing unit 116. The color shift detection unit 112, the central color shift calculation unit 113, and the image height-based color shift calculation unit 114 are the detection means 11, the color shift amount calculation means 12, and the correction data calculation means 13 in the first embodiment. Corresponds to each.

The color separation unit 111 interpolates the color mosaic image to generate a color image in which the color information of R, G, and B is aligned in all the pixels. As the interpolation method of the color mosaic image in the present embodiment, simple linear interpolation based on the values of pixels of adjacent different color planes, "E. Chang, S. Cheung, and D. Pan," Color variable array recovery " Complex methods such as those described in using a threshold-based variable number of pixels, "Proc. SPIE, vol. 3650, pp. 36-43, Jan. 1999" can be used, but are limited thereto. It's not a thing.

For each RGB plane interpolated by the color separation unit 111, the color deviation detection unit 112, the center color deviation calculation unit 113, the color deviation calculation unit 114 according to the image height, and the color deviation correction unit 115 The correction value is detected and corrected. The details of these will be described later. The processing unit 116 performs predetermined image processing such as luminance processing and color processing on the image data after color shift correction. The image recording medium 109 stores the image processed and output by the image processing unit 104 in a predetermined format. The display unit 105 can display the image after image processing.

A series of control of this embodiment is performed by the system controller 110. The image pickup optical system control unit 106 mechanically drives the image pickup optical system 101 based on the instruction of the system controller 110. Further, the imaging optical system control unit 106 controls the aperture diameter of the aperture 101a as an F-number shooting state setting. Further, in order to adjust the focus according to the subject distance, the imaging optical system control unit 106 controls the position of the focus lens 101b in the optical axis direction by an autofocus (AF) mechanism (not shown) or a manual manual focus mechanism.

The image processing unit 104 can acquire information (shooting condition information) regarding the shooting state of the imaging device 100 from the state detection unit 107 and use it for image processing. The state detection unit 107 can acquire information on the imaging state directly from the system controller 110, or may acquire information on the imaging state regarding the imaging optical system 101, for example, from the imaging optical system control unit 106. The storage unit 108 is a memory (storage means) for storing various parameters used by the image processing unit 104. The image processing unit 104 selects and uses necessary parameters from the storage unit 108. Further, the storage unit 108 stores the correction data (color deviation correction value) in the present embodiment.

In the present embodiment, the imaging optical system 101 is integrally configured with the imaging device 100. However, the present embodiment is not limited to this, and is configured to include an imaging device main body and an imaging optical system (interchangeable lens, lens device) that can be attached to and detached from the imaging device main body, like a single-lens reflex camera. It can also be applied to imaging systems. In this case, at least a part of the function of each part of the image processing unit 104 or at least a part of the function of the storage unit 108 (storage means for storing correction data) in the present embodiment may be provided in the interchangeable lens.

Next, the color shift correction process will be described with reference to FIG. FIG. 9 is a flowchart showing the color shift correction process. In each step of FIG. 9, based on the command of the system controller 110, the image processing unit 104 (color deviation detection unit 112, center color deviation calculation unit 113, image height-based color deviation calculation unit 114, or color deviation correction unit) It is executed by 115).

First, in step S201, the color shift detection unit 112 detects an edge (edge portion) from the color image generated by the color separation unit 111. This is because the color shift due to the chromatic aberration of magnification of the imaging optical system 101 appears remarkably at the edge portion of the image. A Y (luminance) plane is used for edge detection of the image. The Y plane can be calculated from the RGB plane using a known formula. Alternatively, the value of the G plane may be used as the Y plane. The edge detected in step S201 is limited to an edge whose pixel value changes significantly in the radial direction from the optical center, so that a highly accurate color shift amount can be obtained. Further, with respect to the Y plane, the color shift due to chromatic aberration of magnification appears as bleeding. Therefore, it is preferable that the color shift detecting unit 112 detects an edge having a certain width such that the monotonous increase or monotonous decrease of the pixel value continues for a predetermined number of times.

Subsequently, in step S202, the color shift detection unit 112 acquires the amount of color shift for each edge detected in step S201. The amount of color shift is acquired by searching for the position where the correlation between the G plane and the R plane (or B plane) is the largest. The color shift detection direction is set to one according to the region related to the direction from the optical center. FIG. 11 is an explanatory diagram of a color shift detection direction. For the pixels in each region shown by the diagonal lines in FIGS. 11A to 11D, the position where the correlation of the R plane (or B plane) with respect to the G plane is the largest is searched for in the direction indicated by the arrow. The color shift detection unit 112 totals the acquired color shift amount according to the position of the target edge in the screen.

FIG. 10 is an explanatory diagram of a color shift acquisition region, and shows a state in which the image is divided into a plurality of regions. The image is divided into eight regions (N, NW, W, SW, S, SE, E, NE) according to the direction from the image center, and the image height, which is the distance from the image center, is set for each region. It is further divided into eight areas according to the situation. The amount of color shift is totaled for each of the 64 regions divided in this way. When a plurality of edges exist in each region and a plurality of color shift amounts are acquired, the average value thereof is calculated to calculate one color shift amount in each region.

Subsequently, in step S203 of FIG. 9, the color shift detection unit 112 determines the reliability of the color shift amount calculated from the dispersion value of the color shift amount acquired in each region. This is because when the acquired color deviation amount varies greatly within the region, it is considered that the reliability of the acquired color deviation amount is low due to the influence of the color of the subject and the angle of the edge in the live-action image. If the reliability of the color shift amount is low, the acquired color shift amount is not used (adopted). The color shift detection unit 112 compares the dispersion value calculated from the amount of color shift for each region with a predetermined threshold value. Then, when the calculated dispersion value is larger than a predetermined threshold value, the color shift detection unit 112 invalidates the color shift amount acquired in that region.

FIG. 12 is an explanatory diagram of the amount of color deviation, and shows an example of the color deviation detection result in the region shown in FIG. 11A. The detection results of the E region on the right side of the image are shown as positions (image height) h1 to h7, and the detection results of the W region on the left side of the image are shown as positions (image height) h1'to h7'. In FIG. 12, as for the direction of the amount of color deviation, the deviation to the right is positive and the deviation to the left is negative in the captured image. In this example, at the position h5 in the E region and the position h4'in the W region of the image, the target edge is not detected, or the color deviation amount is determined to be invalid by the variation determination due to the dispersion in step S203. be.

Subsequently, in step S204 of FIG. 9, the color shift amount is calculated by interpolation from the surrounding color shift amount calculation result with respect to the position (region) where the color shift amount could not be calculated in this way. In FIG. 12B, the amount of color shift at position h5 is the average value of the amount of color shift at positions h4 and h6, and the amount of color shift at position h4'is the average value of the amount of color shift at positions h3'and h5', respectively. The calculated result is shown. Although the original amount of color shift cannot be calculated at positions h5 and h4', it is considered that there are no edges in the image or the color shift is inconspicuous depending on the subject color, so that the correction accuracy has an effect. It can be said that it is an area that is difficult to appear as image quality.

Further, the amount of color shift in the acquired region may be compared with the amount of chromatic aberration of magnification based on the design data of the lens. Here, when these deviations are large, it is also effective to invalidate the amount of color shift acquired in that region, or to correct the value so that it falls within a predetermined range from the amount of chromatic aberration of magnification according to the lens design data. When the variation range of the color shift due to the manufacturing error is estimated in advance, the reliability of the color shift amount acquired from the image can be further improved in consideration of the variation range and the amount of chromatic aberration of magnification based on the lens design data.

Subsequently, in step S205 of FIG. 9, the center color deviation calculation unit 113 calculates the color deviation amount generated in the image center portion based on the color deviation amount calculated in steps S202 and S204. This is performed using the detection result of the detected color shift amount near the center of the image and at a position symmetrical with respect to the center of the image. Specifically, in the color shift detection result shown in FIG. 4, assuming that the color shift amount at the position h1 is d1 and the color shift amount at the position h1'is d1', the two color shift amounts d1 and d1' The average value Ce1 is obtained by the following equation (6).

Ce1 = (d1 + d1') / 2 ... (6)
Further, assuming that the color shift amount at the position h2 is d2 and the color shift amount at the position h2'is d2', the average value Ce2 of the two color shift amounts d2 and d2'is obtained by the following equation (7).

Ce2 = (d2 + d2') / 2 ... (7)
Further, assuming that the color shift amount at the position h3 is d3 and the color shift amount at the position h3'is d3', the color shift amount can be obtained as in the equation (8) under the average value Ce3 of the two color shift amounts d3 and d3'.

Ce3 = (d3 + d3') / 2 ... (8)
Then, the central portion color deviation calculation unit 113 obtains the color deviation component Ce generated in the central portion of the image by calculating the average of the average values Ce1, Ce2, and Ce3.

Ce = (Ce1 + Ce2 + Ce3) / 3 ... (9)
Similarly, for the amount of color shift acquired in the region shown in FIG. 11B, the average value Cn is calculated using the detection results at a position near the center of the image and symmetrical with respect to the center of the image. do. The average value Ce thus calculated is a horizontal component of the amount of color shift occurring in the center of the image, and the average value Cn is a vertical component of the amount of color shift occurring in the center of the image. It should be noted that, among the edges existing in the eight directions, another one is used by using the detection result of the amount of color shift using the edges existing in the diagonal directions (NE direction and SW direction, NW direction and SE direction in FIG. 11). The average values Cne and Cnw of the amount of color shift (color shift component) of the set can be detected, respectively. Then, by averaging the average values Cne and Cnw and the previously calculated average values Ce and Cn, it is possible to uniquely determine the color shift vector at the center of the image.

In the present embodiment, the amount of color shift d1, d1', d2, d2', d3, d3'is not the result of color shift obtained from the edge completely located at the center of the image, but is a position at a considerable distance from the center of the image. The amount of color shift obtained from the edge of. Therefore, it is considered that chromatic aberration of magnification is generated according to the image height. However, by obtaining the average value from the color shifts acquired at positions symmetrical with respect to the center of the image in this way, the influence of chromatic aberration of magnification according to the image height is reduced or eliminated, and it occurs in the center of the image. Color deviation can be estimated with high accuracy. Further, since the influence of the lens manufacturing error is large in the portion where the image height is high and the influence of the non-uniform deviation component in the screen may be large, the detection result of the amount of color deviation near the center of the image is used. This makes it possible to calculate the amount of color shift at the center of the image from the color shift detected in the radial direction.

Subsequently, in step S206 of FIG. 9, the image height-based color shift calculation unit 114 performs correction data according to the image height for each direction from the image center (for each of the eight directions shown in FIG. 10). (Correction value related to chromatic aberration of magnification, color shift correction value) is calculated. Subsequently, in step S207, the color shift correction unit (correction means) 115 corrects the color shift due to chromatic aberration of magnification based on the correction data calculated in step S206. That is, the color shift correction unit 115 corrects the image using the correction data so as to reduce the amount of color shift of the image (photographed image) due to the chromatic aberration of magnification of the imaging optical system 101. Since these processes are the same as the processes of the first embodiment, the description thereof will be omitted. As described above, using the amount of color shift detected in the radial direction, it is possible to calculate the color shift correction value of the chromatic aberration of magnification including the variation due to the manufacturing error, and to correct the color shift with high accuracy.

Further, when a software program that realizes the functions of the above-described embodiment is supplied to a system or device having a computer capable of executing the program directly from a recording medium or by using wired / wireless communication, and the program is executed. Is also included in the present invention. Therefore, in order to realize the functional processing of the present invention on a computer, the program code itself supplied and installed on the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In that case, the form of the program does not matter, such as the object code, the program executed by the interpreter, and the script data supplied to the OS, as long as it has the function of the program. The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical / optical magnetic storage medium, or a non-volatile semiconductor memory. Further, as a method of supplying the program, it is conceivable that the computer program forming the present invention is stored in the server on the computer network, and the connected client computer downloads and programs the computer program.

According to each embodiment, it is possible to provide an image processing device, an imaging device, a lens device, an image processing method, a program, and a storage medium capable of correcting color deviation due to a manufacturing error with high accuracy.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof. Some of the above-described embodiments may be combined as appropriate.

104 Image processing unit (image processing device)
112 Color shift detection unit (detection means)
113 Center color shift calculation unit (color shift amount calculation means)
114 Color shift calculation unit by image height (correction data calculation means)

Claims (13)

A detection means that detects the amount of color shift in the radial direction from the center of the image,
A color shift amount calculation means for calculating the color shift amount at the center of the image based on the color shift amount detected by the detection means, and a color shift amount calculation means.
Correction for calculating correction data regarding the amount of color deviation of the image based on the amount of color deviation detected by the detection means and the amount of color deviation at the center of the image calculated by the means for calculating the amount of color deviation. An image processing apparatus comprising: a data calculation means. The image processing apparatus according to claim 1, wherein the correction data calculation means calculates, as the correction data, correction data regarding the amount of color shift according to the image height for each of a plurality of radiation directions of the image. .. The image processing apparatus according to claim 1 or 2, wherein the correction data calculation means calculates, as the correction data, an approximate curve passing through the color shift amount at the center of the image. The claim for calculating the amount of color shift is to calculate, as the amount of color shift at the center of the image, the average value of the amount of color shift detected by the detection means with respect to predetermined image heights in different radial directions. Item 2. The image processing apparatus according to any one of Items 1 to 3. The image processing apparatus according to any one of claims 1 to 4, further comprising a storage means for storing the correction data calculated by the correction data calculation means. Any one of claims 1 to 5, further comprising a correction means for correcting the image using the correction data so as to reduce the amount of color shift of the image due to chromatic aberration of magnification of the imaging optical system. The image processing apparatus according to the section. The image processing apparatus according to any one of claims 1 to 6, wherein the detection means detects the amount of color deviation in the radial direction at an edge portion included in the image. The detection means
The reliability of the detection result is determined based on the variation in the amount of color shift in the radial direction detected from the image.
The image processing apparatus according to any one of claims 1 to 7, wherein when it is determined that the reliability of the detection result is low, the detection result is not adopted. An imaging element that photoelectrically converts an optical image formed via an imaging optical system,
A detection means for detecting the amount of color shift in the radial direction from the center of the image acquired by the image sensor, and
A color shift amount calculation means for calculating the color shift amount at the center of the image based on the color shift amount detected by the detection means, and a color shift amount calculation means.
Correction data calculation means for calculating correction data regarding the amount of color deviation of the image based on the amount of color deviation detected by the detection means and the amount of color deviation at the center of the image calculated by the calculation means. An imaging device characterized by having. Imaging optics and
A detection means for detecting the amount of color shift in the radial direction from the center of an image formed via the imaging optical system, and
A color shift amount calculation means for calculating the color shift amount at the center of the image based on the color shift amount detected by the detection means, and a color shift amount calculation means.
Correction data calculation means for calculating correction data regarding the amount of color deviation of the image based on the amount of color deviation detected by the detection means and the amount of color deviation at the center of the image calculated by the calculation means. And, a lens device characterized by having. A color shift detection step that detects the amount of color shift in the radial direction from the center of the image,
A color shift amount calculation step for calculating the color shift amount at the center of the image based on the color shift amount detected by the color shift amount detection step, and a color shift amount calculation step.
Based on the color shift amount detected by the color shift amount detection step and the color shift amount of the image center calculated by the color shift amount calculation step, correction data regarding the color shift amount of the image is obtained. An image processing method characterized by having a correction data calculation step for calculation. A color shift detection step that detects the amount of color shift in the radial direction from the center of the image, and
A color shift amount calculation step for calculating the color shift amount at the center of the image based on the color shift amount detected by the color shift amount detection step, and a color shift amount calculation step.
Based on the color shift amount detected by the color shift amount detection step and the color shift amount of the image center calculated by the color shift amount calculation step, correction data regarding the color shift amount of the image is obtained. A program characterized by having a computer execute a correction data calculation step to be calculated. A storage medium for storing the program according to claim 12.

Images