JP6105960B2 - Image processing apparatus, image processing method, and imaging apparatus - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, and an imaging device, and more particularly to a technique for correcting the influence of chromatic aberration of an optical system in a captured image.

  A phenomenon in which the imaging position by the imaging lens is shifted depending on the wavelength (color) of the light due to the difference in the refractive index of the light depending on the wavelength is called chromatic aberration. For example, chromatic aberration of magnification among chromatic aberrations is observed as color blur especially around the edge of an image, and is a factor that degrades image quality. In addition, the magnitude of chromatic aberration (the amount of shift of the imaging position) varies according to the focal length, the aperture, the object distance (hereinafter referred to as an optical parameter), and the distance from the optical axis (hereinafter referred to as an image height). It is known.

  Japanese Patent Laid-Open No. 2004-228688 interpolates correction target color signal data with neighboring data based on a pre-stored shift amount of an imaging position between a reference color signal and a correction target color signal corresponding to an optical parameter and an image height. It is disclosed that the lateral chromatic aberration is corrected by doing so.

JP 2001-186533 A

  However, in the method of Patent Document 1, since the shift amount with respect to the optical parameter and the image height for which the shift amount is not stored needs to be obtained by interpolating the stored shift amount, an error due to interpolation occurs. In particular, since there are few combinations that can actually be stored among the enormous combinations of optical parameters, it is unlikely that a combination close to the desired combination of optical parameters is stored, and the possibility that an error will occur increases.

  In addition, the amount of deviation of the imaging position is generated from the design value of the lens, but because there are individual differences in manufacturing, the amount of deviation is not always the same as the design value of the lens. Arise. Furthermore, it cannot cope with a case where the characteristics of chromatic aberration of the imaging lens change, such as when a teleconverter lens or a wide converter lens is attached.

  The present invention has been made in view of the above-described problems, and an object thereof is to realize more flexible and highly accurate chromatic aberration correction while using a correction amount stored in advance.

An object of the present invention is to provide an image processing apparatus that corrects chromatic aberration of an optical system used to capture an image, and relates to at least information related to at least an aperture value and an object distance stored in advance. Based on a correction amount for each discrete combination of optical parameters including information, a correction amount for a discrete image height corresponding to the combination of optical parameters when an image is captured is calculated as a first correction amount. The reliability of the first correction amount is based on the first correction amount calculation means, the combination of the optical parameters when the image is captured, and the combination of the optical parameters corresponding to the correction amount stored in advance. a first reliability calculating means for calculating a degree of confidence based on the image, the correction amount for discrete image height corresponding to a combination of optical parameters at the time of capturing the image A second correction amount calculating unit that calculates a second correction amount; a second reliability calculating unit that calculates a second reliability that is a reliability of the second correction amount based on the image; A third correction amount generating means for generating a third correction amount by weighted addition of the first correction amount and the second correction amount using a weight based on the reliability of 1 and the second reliability; And an image processing apparatus having correction means for correcting chromatic aberration by applying a correction amount of 3 to an image.

  As described above, according to the present invention, it is possible to realize more flexible and accurate chromatic aberration correction while using a correction amount stored in advance.

1 is a block diagram showing a configuration of an imaging apparatus as an example of an image processing apparatus according to a first embodiment of the present invention. 1 is a block diagram showing a configuration example of a chromatic aberration evaluation unit 6 in FIG. The figure which shows the reliability calculation control from the image which concerns on the 1st Embodiment of this invention. 1 is a flowchart showing the control operation of the correction amount calculation unit 7 in FIG. The figure explaining the correction amount calculation part 7 which concerns on the 1st Embodiment of this invention. The figure explaining the correction amount set which concerns on the 1st Embodiment of this invention. Schematic diagram of chromatic aberration of magnification correction according to the first embodiment of the present invention The figure which shows the weight calculation control in the correction amount production | generation part 9 which concerns on the 1st Embodiment of this invention. The block diagram which shows the structure of the imaging device which concerns on the 2nd Embodiment of this invention. The flowchart which shows the control operation of the correction amount production | generation part 9 which concerns on the 2nd Embodiment of this invention. The flowchart which shows the control operation of the correction amount production | generation part 9 which concerns on the 3rd Embodiment of this invention. The block diagram which shows the control method of the data discard which concerns on the 3rd Embodiment of this invention

(First embodiment)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus as an example of an image processing apparatus according to the first embodiment of the present invention. The imaging device may be any electronic device (for example, a personal computer, a mobile phone, a game machine, etc.) having a built-in camera as well as a device having a main function of imaging such as a digital camera and a digital video camera.

  The optical system 1 includes a zoom lens, a focus lens, and a diaphragm, and forms a subject image on the imaging surface. The optical system drive unit 2 drives the zoom lens, focus lens, diaphragm, and the like of the optical system 1. The imaging device 3 is, for example, a CCD or CMOS image sensor in which a plurality of pixels are two-dimensionally arranged, and outputs a subject image formed by the optical system 1 as an electrical signal in pixel units. The image sensor driving unit 4 drives the image sensor 3. The chromatic aberration correction unit 5 applies chromatic aberration correction to the image signal output from the image sensor 3 using the correction amount from the correction amount generation unit 9.

  The chromatic aberration evaluating unit 6 as the second reliability calculating unit supplies the image signal to the correction amount calculating unit 7 and also calculates the reliability of the correction amount (second reliability) calculated by the correction amount calculating unit 7. Calculation is based on the image signal. A correction amount calculator 7 as a second correction amount calculator calculates a correction amount of chromatic aberration from the image signal. The image height calculation unit 8 calculates the image height (distance from the optical center). A correction amount generation unit 9 as a third correction amount calculation unit generates a correction amount (third correction amount) used in the chromatic aberration correction unit 5. In the optical database (DB) 10, chromatic aberration amounts (correction amounts) corresponding to a plurality of discrete image heights are stored in advance for each discrete combination of optical parameters (for example, focal length, stop, object distance). Yes.

  When the correction amount corresponding to the current optical parameter combination is not stored in the optical DB 10, the correction amount interpolation unit 11 as the first correction amount calculating unit interpolates the correction amount corresponding to the nearby optical parameter combination. To generate. The correction amount interpolation unit 11 as the first reliability calculation unit further includes the reliability of the correction amount generated by the correction amount interpolation unit 11 based on the difference between the current optical parameter and the optical parameter stored in the optical DB 10. (First reliability) is generated.

  The system control unit 12 controls the optical system driving unit 2, the image sensor driving unit 4, the correction amount generation unit 9, and the image height calculation unit 8, reads out a correction amount based on the optical parameter from the optical DB 10, and performs correction amount interpolation. To the unit 11. The system control unit 12 can be realized by a combination of a programmable processor such as a CPU or MPU and software. In addition, one or more functional blocks other than the optical system 1 and the image sensor 3 can be realized by software, hardware, or a combination thereof.

  Note that the system control unit 12 can acquire the optical parameters by an arbitrary method. For example, information regarding the current aperture value, focal length (field angle), and object distance can be acquired from the optical system driving unit 2. The information regarding the focal length may be the position of the zoom lens, and the information regarding the object distance may be the position of the focus lens. Further, when the present invention is implemented in an electronic device that does not have an imaging function, these optical parameters may be acquired from an external device, a header portion of image data, or the like. Needless to say, when it is known in advance that the angle of view is fixed, it is not necessary to acquire information on the focal length.

  An imaging operation of the imaging apparatus having the configuration of FIG. 1 will be described. First, the aperture or lens of the optical system 1 is driven through the optical system drive unit 2 by a control signal from the system control unit 12, and a subject image set to an appropriate brightness is formed on the image sensor 3. The image pickup device 3 is driven by a drive pulse output from the image pickup device drive unit 4 controlled by the system control unit 12, converts the subject image into an electrical signal by photoelectric conversion, and outputs it as an image signal. The image signal is usually subjected to preprocessing such as correlated double sampling, gain adjustment, and A / D conversion.

  The chromatic aberration evaluation unit 6 outputs the image signal from the image sensor 3 to the correction amount calculation unit 7. Further, the chromatic aberration evaluation unit 6 calculates (estimates) the reliability of the correction amount calculated by the correction amount calculation unit 7 based on the image signal from the image sensor 3, and outputs it to the correction amount generation unit 9.

FIG. 2 is a block diagram illustrating a functional configuration example of the chromatic aberration evaluation unit 6.
In the present embodiment, it is assumed that the image pickup device 3 is a single-plate image pickup device provided with a primary color filter having a Bayer arrangement. Therefore, the image signal output from the image sensor 3 corresponds to any one of red (R), green (G), and blue (B) for each pixel. Such an image signal in which each pixel has only one color information is called RAW data. Since RAW data is input to the chromatic aberration evaluation unit 6, the LPF 101 separates the RAW data into R, G, and B pixels, and then performs an interpolation operation (demosaic processing) so that all the pixels have RGB values. Do. Then, an R component image plane, a G component image plane, and a B component image plane are generated from the image data in which all pixels have RGB values obtained by the interpolation calculation, and are sent to the gradient detection units 102 to 104. Supply.

Each of the gradient detectors 102 to 104 detects a gradient or contrast as an example of an image feature in the image height direction from the R component image plane, the G component image plane, and the B component image plane, and outputs a gradient signal. To do.
In the reliability generation unit 105, the correction amount calculation unit 7 calculates (estimates) the reliability of the correction amount calculated from the image signal (RAW data) using the gradient signals from the gradient detection units 102 to 104, and the correction amount. Output to the generation unit 9. As will be described later, the correction amount calculation unit 7 calculates one correction amount in each of the plurality of evaluation regions. Therefore, the reliability generation unit 105 also calculates the reliability for each correction amount from the gradient signal in the image height direction of each evaluation region.

  Here, when calculating the correction amount of chromatic aberration based on the image signal, an image having a high peak value of the gradient signal of the reference color signal (G) and the correction target signal (R or B) among the RGB color components is suitable. Yes. The reliability generation unit 105 calculates the correction amount calculated by the correction amount calculation unit 7 as it is determined that the image is suitable for calculation of the correction amount of chromatic aberration due to the characteristics of the image detected from the RAW data. Is calculated (estimated) so as to increase the reliability. Specifically, the reliability generation unit 105 detects the detected RGB gradient peaks and, as shown in FIG. 3, the gradient peaks of the reference color signal (G) and the correction target signal (R or B). Are high and the sign is the same, the reliability is calculated so that a high reliability Tr is obtained.

  The correction amount calculation unit 7 calculates a correction amount of chromatic aberration corresponding to a plurality of discrete image heights from the RAW data supplied through the chromatic aberration evaluation unit 6, and outputs the correction amount to the correction amount generation unit 9.

  The operation of the correction amount calculation unit 7 will be described based on the flowchart shown in FIG. The correction amount calculation unit 7 performs the processing of S101 to S106 shown in FIG. 4 for each evaluation region, and the correction amount for the image height representing the evaluation region (here, the image height corresponding to the center of the evaluation region). Is calculated.

  In S101, the correction amount calculation unit 7 extracts an image signal (RAW data) corresponding to one evaluation area from among a plurality of evaluation areas set as shown in FIG. 5, for example. Here, as the image height direction, a plurality of evaluation areas are set in the vertical direction, horizontal direction, and diagonal direction from the center of the screen (intersection of the optical axis and the screen). In order to calculate correction values for different image heights, the evaluation areas are set so that the distances from the center of the screen to the evaluation area (for example, the center of the area) are different from each other. Further, the evaluation position is set so as to obtain a correction value for the same image height as the correction value stored in the optical DB 10. In the example of FIG. 5, all the evaluation areas are set in the upper right area when the screen is divided into two vertically and horizontally. However, the evaluation areas do not necessarily have to be set in a specific area. .

  In S102, the correction amount calculation unit 7 reads out one correction amount from a correction amount set having a certain range of resolution that is stored in advance as shown in FIG. 6, for example. The correction amount set is a correction amount with respect to the image height direction.

In step S103, the correction amount calculation unit 7 performs lateral chromatic aberration correction on the R and B planes in the evaluation region using the read correction amount.
FIG. 7 is a diagram schematically illustrating an example of magnification chromatic aberration correction in the correction amount calculation unit 7.
The correction amount of the lateral chromatic aberration is a phase shift amount obtained by separating the shift amount at the target pixel position as a phase shift component in the horizontal and vertical directions. The black pixel in FIG. 7A indicates the phase that the target pixel S should be originally, and the virtual pixel S ′ has a phase shift of Hp in the horizontal direction and Vp in the vertical direction due to the influence of the target pixel S. The imaged position is shown. In order to correct lateral chromatic aberration, the value of the virtual pixel S ′ may be obtained and the value of the pixel of interest S may be replaced. As shown in FIG. 7B, the value of the virtual pixel S ′ is the value of the pixels s1, s2, s3, s4 existing in the vicinity of the virtual image S ′, and the inter-pixel distance c1, It can be generated by performing weighted interpolation calculation with c2, c3, and c4. As shown in FIG. 7C, the value of the target pixel S is replaced with the value of the generated virtual pixel S ′, and the lateral chromatic aberration is corrected.

  Since the correction amount set shown in FIG. 6 is a correction amount in the image height direction, in S103, the correction amount calculation unit 7 decomposes the correction amount into a horizontal component and a vertical component according to the image height direction in the evaluation region, Correction is performed by obtaining the values of Hp and Vp in FIG. In the present embodiment, chromatic aberration correction is performed by commonly using one correction amount selected from the correction amount set for the R component and B component of each pixel in the evaluation region.

  In S104, the correction amount calculation unit 7 obtains a difference (G−R, G−B) between the G component as the reference signal and the R and B components after chromatic aberration correction. The difference may be the sum of differences for each pixel.

  In S105, the correction amount calculation unit 7 determines whether or not the difference extraction has been completed for all the correction amounts included in the correction amount set. If not completed, the process proceeds to S102, and another correction in the correction amount set. Set the amount, perform correction and difference extraction. If the difference extraction has been completed for all the correction values in S105, the correction amount calculation unit 7 in S106 determines the correction amount when the difference obtained in S104 is the minimum as the evaluation area (the image height representing the evaluation region). ) Is output to the correction amount generation unit 9 as a correction amount.

From the coordinates of the pixel of interest given from the system control unit 12 and the coordinates of the optical center set in advance (or given from the system control unit 12), the image height calculation unit 8 is expressed by the following equation (1), for example. The image height hgt of the pixel of interest is obtained using the calculation and output to the correction amount generation unit 9.

  Here, the coordinates are represented by, for example, an orthogonal coordinate system with the upper left corner as the origin, xadr indicates the horizontal coordinate of the pixel of interest, and yadr indicates the vertical coordinate of the pixel of interest. Hc represents the horizontal coordinate of the optical center, and Vc represents the horizontal coordinate of the optical center.

The correction amount generation unit 9 includes the correction amount and reliability from the correction amount interpolation unit 11, the reliability from the chromatic aberration evaluation unit 6, the image height from the image height calculation unit 8, and the correction amount from the correction amount calculation unit 7. From this, the correction amount is calculated and output to the chromatic aberration correction unit 5.
The correction amount from the correction amount interpolation unit 11 and the correction amount from the correction amount calculation unit 7 are correction amounts corresponding to discrete image heights. Therefore, the correction amount generation unit 9 obtains the correction amount for the image height of the pixel of interest given from the image height calculation unit 8 by interpolation of the correction amount for the neighboring image height.

  The correction amount interpolation unit 11 generates a correction amount corresponding to the current optical parameter from the correction amount (chromatic aberration amount) stored in the optical DB 10 and outputs the correction amount to the correction amount generation unit 9. Since an enormous storage capacity is required to store correction amounts separately for all combinations of optical parameters in the optical DB 10, it is practical to hold correction amounts corresponding to discrete combinations of optical parameters. . In this case, if the correction amount corresponding to the current optical parameter combination is stored in the optical DB 10, the correction amount interpolation unit 11 outputs the correction amount as it is. If the correction amount corresponding to the current combination of optical parameters is not stored in the optical DB 10, a plurality of correction amounts corresponding to the closest combination among the combinations of stored optical parameters are interpolated to obtain the current optical parameter value. A correction amount corresponding to the combination is generated.

  As described above, the optical DB 10 stores a plurality of correction amounts corresponding to different image heights for each combination of optical parameters. Therefore, the correction amount interpolation unit 11 also outputs a plurality of correction amounts corresponding to different image heights.

  When the correction amount is generated by interpolation, the accuracy of the generated correction amount decreases as the difference between the current combination of optical parameters and the combination of optical parameters corresponding to the correction amount used for interpolation increases. Therefore, the correction amount interpolation unit 11 outputs to the correction amount generation unit 9 the reliability DI that decreases as the difference between the combination of optical parameters corresponding to the correction amount used for interpolation and the current combination of optical parameters increases. . Note that the difference between the optical parameters can be evaluated by an arbitrary method such as numerically summing the differences for each of the optical parameters.

  The chromatic aberration correction unit 5 performs chromatic aberration correction on the image signal from the image sensor 3 using the correction amount from the correction amount generation unit 9. The method for correcting chromatic aberration may be the same as that described with reference to FIG.

Next, an example of the control operation of the correction amount generation unit 9 will be described with reference to FIG.
First, the correction amount generation unit 9 corresponds to the same image height among the correction amount calculated from the image by the correction amount calculation unit 7 and the correction amount (or the correction amount generated therefrom) stored in the optical DB 10. The correction amounts are weighted and added, and correction amounts for a plurality of discrete image heights are generated. When the image height supplied from the image height calculation unit 8 is different from a plurality of discrete image heights, a correction amount is generated by interpolating a plurality of correction amounts corresponding to close image heights.

The correction amount generation unit 9 as the third correction amount generation unit calculates the weight in the weighted addition from the reliability Tr from the chromatic aberration evaluation unit 6 and the reliability DI from the correction amount interpolation unit 11.
FIG. 8 schematically shows an example of the characteristic of the correction amount weight α calculated from the image by the correction amount calculation unit 7. The weight α of the correction amount calculated from the image increases as the reliability DI from the correction amount interpolation unit 11 decreases and the reliability Tr from the chromatic aberration evaluation unit 6 increases. More specifically, when the reliability DI from the correction amount interpolation unit 11 is the lowest and the reliability Tr from the chromatic aberration evaluation unit 6 is the highest (α = 1), the reliability DI is determined therefrom. It has a characteristic that it is high and becomes smaller as the reliability Tr becomes lower. When the reliability DI is maximum and the reliability Tr is minimum, the weight α is 0 (that is, only the correction amount based on the optical DB 10 is used).

  When the reliability DI and the reliability Tr are equal, the weight 0 <α <0.5. This is because the correction amount based on the design value is generally considered to have less error than the correction amount based on the image. For the same reason, the weight α does not change linearly with respect to the change in reliability, but has a characteristic that increases only when the reliability Tr is sufficiently high. In FIG. 8, Trmax indicates the maximum value of the reliability Tr from the chromatic aberration evaluation unit 6, and DImax indicates the maximum value of the reliability DI from the correction amount interpolation unit 11.

Next, the correction amount generation unit 9 as a third correction amount generation unit stores the correction amount calculated from the image by the correction amount calculation unit 7 using the calculation shown in the following formula (2), for example, in the optical DB 10. The correction amount cor is obtained by weighted addition of the corrected amount (or the correction amount generated therefrom).
cor = (1-α) * corDB + α * corCalc (2)
Here, corDB is the correction amount acquired from the optical DB 10 (or correction amount obtained by interpolating the correction amount acquired from the optical DB 10), and corCalc is calculated from the image by the correction amount calculation unit 7 (from the image data in the evaluation region). This represents the calculated correction amount.

  Based on the correction amounts obtained for a plurality of discrete image heights in this way, the correction amount generation unit 9 outputs a correction amount corresponding to the image height supplied from the image height calculation unit 8 to the chromatic aberration correction unit 5. .

  As described above, according to this embodiment, the correction amount calculated from the image captured by the current combination of optical parameters and the correction amount stored in advance are weighted and added based on the respective reliability. Then, the final correction amount is calculated. Therefore, it is possible to suppress the influence of an error caused by the difference between the current optical parameter and the optical parameter corresponding to the stored correction value, and to realize highly accurate chromatic aberration correction. Further, even when the chromatic aberration characteristics of the imaging lens change, it is possible to prevent the accuracy of chromatic aberration correction from being greatly reduced by adding the correction amount calculated from the image, and it is possible to realize chromatic aberration correction with high accuracy as a whole.

(Modification of the first embodiment)
In the first embodiment described above, as indicated by a dotted arrow in FIG. 1, the correction amount calculation unit 7 acquires the reliability calculated by the chromatic aberration evaluation unit 6, and the correction amount calculation accuracy is increased according to the reliability. By changing, the correction amount calculation load can be reduced.

  That is, if the reliability calculated by the chromatic aberration evaluation unit 6 is equal to or greater than a predetermined value, the correction amount calculation unit 7 performs the processing of S102 to S105 in FIG. 4 for all the correction amounts in the correction amount set shown in FIG. Against. On the other hand, if the reliability calculated by the chromatic aberration evaluation unit 6 is less than the predetermined value, the correction amount calculation unit 7 performs the correction amount obtained by thinning out a part of the correction amount set shown in FIG. The process of S105 is performed. Of course, the reliability may be divided into more levels, and the lower the level, the greater the thinning ratio. Thus, when the reliability is low, the calculation load may be reduced instead of reducing the accuracy of the correction amount. This modification can also be applied to second and third embodiments described later.

(Second Embodiment)
FIG. 9 is a block diagram showing a configuration example of an imaging apparatus as an example of an image processing apparatus according to the second embodiment of the present invention. The same blocks as those in FIG. The description to be omitted is omitted. In the present embodiment, a memory 13 connected to the system control unit 12 is added, and the reliability Tr from the chromatic aberration evaluation unit 6 and the correction amount corCalc calculated from the image by the correction amount calculation unit 7 are used as the optical at that time. Stored in association with parameters. Then, when the same combination of optical parameters is obtained thereafter, more accurate chromatic aberration correction is realized using the reliability and correction amount stored in the memory 13.

Hereinafter, the operation of the imaging apparatus according to the present embodiment will be described with emphasis on the differences from the first embodiment.
The memory 13 stores the reliability Tr from the chromatic aberration evaluation unit 6 and the correction amount corCalc from the correction amount calculation unit 7 in association with the optical parameters at that time in accordance with a recording command from the system control unit 12. Hereinafter, the reliability stored in the memory 13 is expressed as MTr, and the correction amount is expressed as McorCalc. The memory 13 also outputs the corresponding reliability MTr and correction amount McorCalc according to the read command from the system control unit 12 and the optical parameters.

FIG. 10 is a flowchart for explaining the operation of the correction amount generation unit 9 in the present embodiment.
In S201, the correction amount generation unit 9 checks whether the reliability MTr and the correction amount McorCalc corresponding to the same optical parameter as the current one are stored in the memory 13 through the system control unit 12, and if not, the correction amount generation unit 9 stores the processing in S202. If so, the process proceeds to S203.

  In S202, the correction amount generation unit 9 stores the reliability Tr from the chromatic aberration evaluation unit 6 and the correction amount corCalc from the correction amount calculation unit 7 in the memory 13 through the system control unit 12 in association with the optical parameters at that time. Then, the process proceeds to S207. On the other hand, in S203, the correction amount generation unit 9 reads the reliability MTr and the correction amount McorClac corresponding to the current optical parameter combination from the memory 13 through the system control unit 12, and advances the processing to S204.

  In S204, the correction amount generation unit 9 compares the reliability MTr read from the memory 13 with the reliability Tr calculated by the chromatic aberration evaluation unit 6, and if the reliability MTr is higher, the process advances to S205. If is higher, the process proceeds to S206.

  In S206, the correction amount generation unit 9 stores the reliability MTr and the correction amount McorCalc corresponding to the current optical parameter combination stored in the memory 13, and the reliability Tr and correction amount calculation unit 7 from the chromatic aberration evaluation unit 6. The correction amount from is updated with corCalc.

  In S207, the correction amount generation unit 9 uses the reliability Tr from the chromatic aberration evaluation unit 6 and the reliability DI from the correction amount interpolation unit 11, for example, to calculate the weight α according to the characteristics shown in FIG. Proceed to S208. In S208, the correction amount generation unit 9 obtains a correction amount according to the above equation (2) using the weight α calculated in S207.

  On the other hand, in S205, the correction amount generation unit 9 uses the reliability MTr from the memory 13 instead of the reliability Tr from the chromatic aberration evaluation unit 6, and calculates the weight α from the reliability DI from the correction amount interpolation unit 11. . In S209, the correction amount is obtained by applying the weight α calculated in S205, the correction amount McorCalc read from the memory 13, and the correction amount corDB based on the optical DB 10 to the above equation (2).

  As described above, in the present embodiment, the reliability and the correction amount calculated from the image are held together with the combination of the optical parameters, and can be used when the combination of the same optical parameters is obtained thereafter. Therefore, in addition to the effects of the first embodiment, even when the reliability of the correction amount calculated from the current image is low, highly accurate chromatic aberration correction is performed using the correction amount with high reliability obtained in the past. The effect that it can be performed is realizable.

  In the present embodiment, the case where the reliability and the correction amount corresponding to the current optical parameter combination are not stored in the memory 13 is recorded in the memory 13 regardless of the reliability. However, even if it is not stored in the memory 13, if the reliability does not reach a predetermined level, recording is not performed, so that the capacity of the memory 13 can be saved and the chromatic aberration can be corrected with high accuracy. Will be able to do.

(Third embodiment)
FIG. 11 is a flowchart for explaining the operation of the correction amount generation unit 9 according to the third embodiment of the present invention. Since the present embodiment can be implemented by the same imaging apparatus as that of the second embodiment except for the operation of the correction amount generation unit 9, the description regarding the configuration of the imaging apparatus is omitted.

  Compared to the second embodiment, the present embodiment increases the accuracy of the data stored in the memory 13 when it is necessary to discard at least part of the data in the memory 13, such as when the storage capacity is insufficient. This is to add control for discarding so as not to lower it.

  The operation of the correction amount generation unit 9 in this embodiment is determined between the free space of the memory 13 and the free space is small between S201 and S202 in the flowchart of FIG. 10 described in the second embodiment. In this case, processing for discarding data with low importance is inserted. Therefore, only the processes of S302 and S303 unique to this embodiment will be described below.

  In S302, the correction amount generation unit 9 checks whether there is a capacity that can be newly stored in the memory 13 through the system control unit 12. If the memory capacity is sufficient, the process proceeds to S304. If the memory capacity is not sufficient, the process proceeds to S303. Proceed with each process.

  In S303, the correction amount generation unit 9 selects and discards data to be discarded by an operation shown in FIG. 12 described later, and advances the process to S202.

  Next, selection of data to be discarded and discarding processing, which is a feature of the present embodiment, will be described with reference to the flowchart of FIG.

  First, in S 401, the correction amount generation unit 9 compares the data stored in the optical DB 10 with the data stored in the memory 13. Here, “data” is one or more data (one or more correction amounts and reliability) corresponding to the same combination of optical parameters.

  In S <b> 402, the correction amount generation unit 9 branches the process according to the presence / absence and number of data stored redundantly in both the optical DB 10 and the memory 13. Specifically, the correction amount generation unit 9 proceeds to S403 if there is one duplicate data, and proceeds to S404 if there is no duplicate data or there are a plurality of duplicate data.

In step S <b> 403, the correction amount generation unit 9 discards (deletes) the duplicated data stored in the memory 13 through the system control unit 12.
In S <b> 404, the correction amount generation unit 9 selects which data to discard, so that the difference between the optical parameter corresponding to the data stored in the memory 13 and the optical parameter corresponding to the data stored in the optical DB 10 is determined. Conduct a sparseness survey to determine the size.

  Of the data stored in the memory 13, if the number of data corresponding to a combination of optical parameters whose difference from the combination of optical parameters corresponding to the data stored in the optical DB 10 is less than a predetermined number is one, The correction amount generation unit 9 deletes the data in S406. Note that duplicate data is included in data corresponding to a combination of optical parameters whose difference is less than a predetermined value.

  On the other hand, when there are a plurality of such data, the correction amount generation unit 9 sorts the plurality of data in order of reliability in S407, and discards the data with the lowest reliability in S408.

  As described above, in the present embodiment, in addition to the effects of the second embodiment, when a part of the data held in the memory 13 is discarded, it is possible to discard the data in which the influence of the discard is suppressed. . In the present embodiment, data to be discarded is selected in consideration of data duplication, density, and reliability. However, data may be simply discarded in descending order of reliability.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (13)

An image processing apparatus that corrects chromatic aberration of an optical system used for capturing an image of a captured image,
Corresponds to a combination of optical parameters when the image is captured based on a correction amount for each discrete combination of optical parameters including information on at least the aperture value and information on an object distance stored in advance First correction amount calculating means for calculating a correction amount for the discrete image height as the first correction amount;
Based on a combination of optical parameters when the image is captured and a combination of optical parameters corresponding to the correction amount stored in advance, a first reliability that is a reliability of the first correction amount is calculated. First reliability calculation means for
A second correction amount calculating means for calculating a correction amount for a discrete image height corresponding to a combination of optical parameters when the image is captured based on the image as a second correction amount;
Second reliability calculation means for calculating a second reliability which is a reliability of the second correction amount based on the image;
A third correction amount that generates a third correction amount by weighted addition of the first correction amount and the second correction amount using weights based on the first reliability and the second reliability. Generating means;
An image processing apparatus comprising: correction means for applying the third correction amount to the image to perform the chromatic aberration correction.   The first reliability calculation means is configured to reduce the first reliability so that a difference between a combination of optical parameters when the image is captured and a combination of optical parameters corresponding to the correction amount stored in advance is increased. The image processing apparatus according to claim 1, wherein the reliability of the image processing apparatus is calculated.   The second reliability calculation means determines that the second reliability is higher when it is determined that the image is an image suitable for calculating the correction amount of chromatic aberration to some extent based on the feature of the image in the image height direction. The image processing apparatus according to claim 1, wherein the calculation is performed so that the value becomes higher.   The second reliability calculation means is configured to increase the reliability when the peak of the gradient of the reference color signal and the correction target signal in the image height direction of the image is both high and the sign is the same. The image processing apparatus according to claim 1, wherein the reliability is calculated.   The third correction amount generation means increases the weight of the second correction amount in the weighted addition as the first reliability decreases and as the second reliability increases. The image processing apparatus according to any one of claims 1 to 4. Furthermore, it has a memory | storage means to memorize | store the said 2nd correction amount and said 2nd reliability with the combination of a corresponding optical parameter,
The third correction amount generation means stores in the storage means the second correction amount and the second reliability corresponding to the same optical parameter combination as the optical parameter combination when the image was captured. And when the second reliability stored in the storage means is higher than the second reliability acquired from the second reliability calculation means, the second reliability is stored in the storage means. 6. The image processing apparatus according to claim 1, wherein the third correction amount is calculated using the second correction amount and the second reliability.   The third correction amount generating means further deletes, from the storage means, a second correction amount that overlaps with the previously stored correction amount and data associated with the second correction amount. The image processing apparatus according to claim 6.   When there are a plurality of second correction amounts that overlap the previously stored correction amount in the storage unit, the third correction amount generation unit uses the second correction amount with the lowest corresponding second reliability. The image processing apparatus according to claim 7, wherein the image data and the data associated with the second correction amount are deleted.   When there are a plurality of second correction amounts corresponding to combinations of optical parameters whose difference from the combination of optical parameters corresponding to the previously stored correction amounts is less than a predetermined amount in the storage unit, the third correction amount generation 8. The image processing apparatus according to claim 7, wherein the means deletes the second correction amount having the lowest corresponding second reliability and the data associated with the second correction amount.   The said 2nd correction amount calculation means reduces the calculation precision of a said 2nd correction amount, so that the said 2nd reliability is low, The any one of Claim 1 to 9 characterized by the above-mentioned. Image processing device. The optical system;
Imaging means for imaging a subject image formed by the optical system and outputting the image;
An image processing apparatus comprising: the image processing apparatus according to claim 1.   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1-10. An image processing method that is executed by an image processing apparatus that corrects chromatic aberration of an optical system used to capture an image of a captured image,
The first correction amount calculating means captures the image based on a correction amount for each discrete combination of optical parameters, which is stored in advance and includes at least information regarding the aperture value and information regarding the object distance. A first correction amount calculating step of calculating a correction amount for a discrete image height corresponding to a combination of optical parameters at the time as a first correction amount;
Based on the combination of the optical parameters when the first reliability calculation means captures the image and the combination of the optical parameters corresponding to the correction amount stored in advance, the reliability of the first correction amount A first reliability calculation step of calculating a first reliability that is:
Second correction amount calculation means calculates, based on the image, a correction amount for a discrete image height corresponding to a combination of optical parameters when the image is captured as a second correction amount. A correction amount calculating step;
A second reliability calculation step in which a second reliability calculation means calculates a second reliability which is a reliability of the second correction amount based on the image;
A third correction amount calculation means weights and adds the first correction amount and the second correction amount using a weight based on the first reliability and the second reliability, and a third correction A third correction amount generation step for generating an amount;
An image processing method, comprising: a correction step of correcting the chromatic aberration by applying the third correction amount to the image.

Images