JP2015226158A - Image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect a characteristic-caused color shift amount and an individual-difference-caused color shift amount.SOLUTION: In accordance with the relation among a color shift amount detected from a captured image, the characteristic-caused color shift amount being a color shift amount caused by the characteristic of the lens of the captured image, and the individual-difference-caused color shift amount being a color shift amount caused by the individual difference of the lens of the captured image, a detector detects the characteristic-caused color shift amount and the individual-difference-caused color shift amount from an inputted captured image. This invention can be applied to an image processing system, for example.

Description

  The present disclosure relates to an image processing apparatus and an image processing method, and more particularly, to an image processing apparatus and an image processing method capable of accurately detecting a characteristic-induced color shift amount and an individual-difference color shift amount.

  In the captured image, the chromatic aberration of magnification occurs because the size (magnification) of the image formed differs depending on the color wavelength. In recent years, the amount of correction to eliminate the effect of chromatic aberration of magnification on images that have developed chromatic aberration of magnification does not use information about the lens used to shoot the image or perform calibration in advance. In addition, a technique for calculating using only an image has been devised.

  As such a technique, for example, there is a technique for detecting a color shift that occurs symmetrically with respect to the center of the image (hereinafter referred to as a symmetric color shift) due to the characteristics of the lens used for shooting from the image ( For example, see Patent Document 1). In the technique described in Japanese Patent Application Laid-Open No. H10-228707, a detection process for detecting a symmetric color misregistration amount in the entire image by detecting local symmetric color misregistration amounts at a plurality of locations in the image and counting the detected color misregistration amounts. Is done.

  However, the technique described in Patent Document 1 does not take into account color misregistration caused by individual lens differences such as lens attachment errors, and aberrations in which the point symmetry of lateral chromatic aberration is lost due to individual lens differences. If this occurs, it cannot be corrected with high accuracy.

  On the other hand, there is also a technique for detecting both a symmetric color shift and a color shift that occurs uniformly in an image due to individual lens differences (hereinafter referred to as uniform color shift) from an image (see, for example, Patent Document 2). ). In the technique described in Patent Document 2, the detection processing of the symmetric color shift amount and the uniform color shift amount is performed independently.

  The detection process of the symmetric color shift amount is the same as the detection process described in Patent Document 1. In addition, the uniform color misregistration amount detection process detects a local uniform color misregistration amount in an area where no symmetric color misregistration occurs in the image, and calculates an average value of the detected local uniform color misregistration amount in the uniform color in the entire image. This is processing to detect as a deviation amount.

  The region where no symmetric color shift occurs in the image is a region on a straight line in the horizontal direction and the vertical direction passing through the center of the image. Therefore, in a region where no symmetric color shift has occurred, an edge having a large luminance change does not necessarily exist, and when there is no edge, a uniform color shift amount cannot be detected.

  It is also conceivable to detect a uniform color misregistration amount by enlarging an area in the image where no symmetric color misregistration has occurred into a predetermined range including horizontal and vertical straight lines passing through the center of the image. . However, in practice, a symmetric color shift occurs in a region other than the region on the straight line in the horizontal direction and the vertical direction passing through the center of the image. That is, the detected uniform color misregistration amount includes an outer product of the actual uniform color misregistration amount and the symmetric color misregistration amount as an offset. Therefore, the uniform color shift amount cannot be detected with high accuracy.

  Further, the detected symmetric color misregistration amount includes a uniform color misregistration amount, and the symmetric color misregistration amount cannot be detected with high accuracy.

JP 2010-219683 A JP 2012-015781

  As described above, from the image, the color resulting from the individual difference of the image lens such as the characteristic color misregistration amount such as the color misregistration amount such as the symmetric color misregistration amount and the uniform color misregistration amount. When the individual-deviation color shift amount that is the shift amount is detected independently, the detection accuracy is lowered.

  The present disclosure has been made in view of such a situation, and makes it possible to accurately detect a characteristic-induced color shift amount and an individual-difference color shift amount.

  An image processing apparatus according to one aspect of the present disclosure is caused by a color shift amount detected from an image, a characteristic-induced color shift amount that is a color shift amount caused by a lens characteristic of the image, and an individual difference of the lens of the image. The image processing apparatus includes a detection unit that detects the characteristic-induced color misregistration amount and the individual difference-induced color misregistration amount from an input image according to the relationship between the individual difference-induced color misregistration amounts that are the color misregistration amounts to be detected.

  An image processing method according to one aspect of the present disclosure corresponds to the image processing apparatus according to one aspect of the present disclosure.

  In one aspect of the present disclosure, a color misregistration amount detected from an image, a color misregistration amount that is a color misregistration amount caused by characteristics of the lens of the image, and a color misregistration caused by individual differences of the lenses of the image The characteristic-induced color misregistration amount and the individual difference-induced color misregistration amount are detected from the input image in accordance with the relationship between the individual difference-induced color misregistration amounts, which are amounts.

  Note that the image processing apparatus according to one aspect of the present disclosure can be realized by causing a computer to execute a program.

  In order to realize the image processing apparatus according to one aspect of the present disclosure, a program to be executed by a computer can be provided by being transmitted via a transmission medium or by being recorded on a recording medium.

  According to one aspect of the present disclosure, an image can be processed. Further, according to one aspect of the present disclosure, it is possible to accurately detect a characteristic-induced color shift amount and an individual-difference color shift amount.

  Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a block diagram showing an example of composition of a 1st embodiment of an image processing device to which this indication is applied. It is a figure showing the symmetrical color shift amount of each position of a captured image by a vector. It is a figure which represents the uniform color shift amount of each position of a captured image with a vector. It is a figure which represents the color shift amount which generate | occur | produces in each position of a captured image with a vector. It is a figure which shows the example of the detection direction in the symmetrical color shift amount detection part 31 of FIG. It is a figure which shows the example of the detection direction in the uniform color shift amount detection part of FIG. It is a figure explaining the symmetrical uniform relationship in case local symmetric color shift amount is detected from a captured image. It is a figure explaining the symmetrical uniform relationship in case a local uniform color shift amount is detected from a captured image. 2 is a flowchart illustrating image processing of the image processing apparatus 10 in FIG. 1. It is a block diagram which shows the structural example of 2nd Embodiment of the image processing apparatus to which this indication is applied. FIG. 20 is a block diagram illustrating a configuration example of a third embodiment of an image processing apparatus to which the present disclosure is applied. 12 is a flowchart illustrating image processing of the image processing apparatus 90 in FIG. 11. FIG. 20 is a block diagram illustrating a configuration example of a fourth embodiment of an image processing apparatus to which the present disclosure is applied. 14 is a flowchart illustrating image processing of the image processing apparatus 120 in FIG. 13. It is a block diagram which shows the structural example of the hardware of a computer.

Hereinafter, the premise of this indication and the form for implementing this indication (henceforth an embodiment) are explained. The description will be given in the following order.
1. First Embodiment: Image Processing Device (FIGS. 1 to 9)
2. Second Embodiment: Image Processing Device (FIG. 10)
3. Third embodiment: image processing apparatus (FIGS. 11 and 12)
4). Fourth embodiment: image processing apparatus (FIGS. 13 and 14)
5. Fifth embodiment: computer (FIG. 15)

<First embodiment>
(Configuration example of first embodiment of image processing apparatus)
FIG. 1 is a block diagram illustrating a configuration example of a first embodiment of an image processing apparatus to which the present disclosure is applied.

  The image processing apparatus 10 in FIG. 1 includes a detection unit 11, a uniform color shift amount totaling unit 12, and a symmetric color shift amount totaling unit 13. The image processing apparatus 10 detects a symmetric color shift amount as a characteristic-induced color shift amount and a uniform color shift amount as an individual-difference color shift amount from the input captured image.

  Specifically, the detection unit 11 of the image processing apparatus 10 includes a symmetric color misregistration amount detection unit 31, a symmetric color misregistration amount temporary aggregation unit 32, a uniform offset calculation unit 33, a uniform color misregistration amount detection unit 34, and a uniform color misregistration. The amount calculation unit 35, the symmetry offset calculation unit 36, and the symmetric color shift amount calculation unit 37 are configured. The detection unit 11 detects a symmetric color shift from an input image according to a relationship between a color shift amount detected from the captured image, a symmetric color shift amount of the captured image, and a uniform color shift amount of the captured image (hereinafter referred to as a symmetric uniform relationship). Detect the amount and uniform color shift amount.

  That is, the symmetric color misregistration amount detection unit 31 (characteristic-induced color misregistration amount calculation unit) of the detection unit 11 has an edge region (second region / first region) orthogonal to a predetermined detection direction of a captured image input from the outside. The color misregistration amount is detected by performing block matching in the area (1). The symmetric color misregistration amount detection unit 31 supplies the detected color misregistration amount as a local symmetric color misregistration amount to the symmetric color misregistration amount temporary aggregation unit 32 and the symmetric color misregistration amount calculation unit 37.

  The symmetric color misregistration amount temporary counting unit 32 assumes that the detected local symmetric color misregistration amount does not include the uniform color misregistration amount, and detects the symmetric color misregistration amount for each distance from the center of the captured image to the detection position. An average value of the local symmetric color shift amounts supplied from the unit 31 is calculated. The symmetric color misregistration amount provisional totaling unit 32 supplies the calculated average value to the uniform offset calculating unit 33 as the symmetric color misregistration amount (temporary value) in the entire image.

  The uniform offset calculation unit 33 uses the symmetric color misregistration amount supplied from the symmetric color misregistration amount temporary summation unit 32 or the symmetric color misregistration amount totalization unit 13 according to the symmetric uniformity relationship (individual difference offset). ) Is calculated. The uniform offset is a symmetric color misregistration amount included in the local uniform color misregistration amount detected by the uniform color misregistration amount detection unit 34. The uniform offset calculation unit 33 supplies the uniform offset to the uniform color misregistration amount calculation unit 35.

  The uniform color shift amount detection unit 34 (individual difference caused color shift amount detection unit) blocks in an edge region (first region / second region) orthogonal to a predetermined detection direction of a captured image input from the outside. By performing matching, the color misregistration amount is detected and supplied to the uniform color misregistration amount calculation unit 35 as a local uniform color misregistration amount.

  The uniform color misregistration amount calculation unit 35 (individual difference-induced color misregistration amount calculation unit) uses a uniform offset supplied from the uniform offset calculation unit 33 based on the local uniform color misregistration amount supplied from the uniform color misregistration amount detection unit 34. The local uniform color shift amount is calculated or updated by deleting. The uniform color shift amount calculation unit 35 supplies the calculated or updated local uniform color shift amount to the uniform color shift amount totaling unit 12.

  The symmetry offset calculation unit 36 calculates the symmetry offset (characteristic offset) according to the symmetry uniformity relationship using the uniform color misregistration amount supplied from the uniform color misregistration amount totaling unit 12. The symmetry offset is a uniform color shift amount included in the local symmetric color shift amount detected by the symmetric color shift amount detection unit 31. The symmetry offset calculation unit 36 supplies the symmetry offset to the symmetric color shift amount calculation unit 37.

  The symmetric color shift amount calculation unit 37 (characteristic-induced color shift amount calculation unit) calculates the symmetric offset supplied from the symmetric offset calculation unit 36 from the local symmetric color shift amount supplied from the symmetric color shift amount detection unit 31. By deleting, the local symmetric color shift amount is corrected. The symmetric color shift amount calculation unit 37 supplies the corrected local symmetric color shift amount to the symmetric color shift amount totaling unit 13.

  The uniform color misregistration amount totaling unit 12 obtains an average value of local uniform color misregistration amounts supplied from the uniform color misregistration amount calculating unit 35 as a uniform color misregistration amount in the entire image. The uniform color misregistration amount totaling unit 12 supplies the uniform color misregistration amount to the symmetry offset calculating unit 36. Further, when the obtained uniform color shift amount and the symmetric color shift amount converge, the uniform color shift amount totaling unit 12 outputs the uniform color shift amount as the final uniform color shift amount.

  The symmetric color misregistration amount calculation unit 13 calculates an average value of local symmetric color misregistration amounts supplied from the symmetric color misregistration amount calculation unit 37 for each distance from the center of the captured image to the detection position, and calculates this in the entire image. The amount of symmetric color shift is assumed. When the obtained uniform color shift amount and symmetric color shift amount converge, the symmetric color shift amount totaling unit 13 outputs the symmetric color shift amount as the final symmetric color shift amount.

  On the other hand, when the obtained uniform color shift amount and symmetric color shift amount have not converged, the symmetric color shift amount totaling unit 13 supplies the symmetric color shift amount to the uniform offset calculating unit 33. Accordingly, the uniform color shift amount and the symmetric color shift amount are updated using the symmetric color shift amount.

(Explanation of symmetrical color shift amount)
FIG. 2 is a diagram illustrating a symmetric color shift amount at each position of the captured image as a vector.

  As shown in FIG. 2, the symmetric color shift amount is a color shift amount that occurs symmetrically with respect to the center of the captured image.

(Description of uniform color misregistration)
FIG. 3 is a diagram illustrating a uniform color shift amount at each position of the captured image as a vector.

  As shown in FIG. 3, the uniform color misregistration amount is a color misregistration amount that occurs uniformly in the entire captured image.

(Explanation of color shift amount of captured image)
FIG. 4 is a diagram showing the amount of color shift occurring at each position of the captured image as a vector.

  As shown in FIG. 4, both symmetric color shift and uniform color shift occur at each position of the captured image, and the color shift amount at each position of the captured image is not point-symmetric. That is, the color misregistration amount detected from the captured image includes both a symmetric color misregistration amount and a uniform color misregistration amount.

(Example of detection direction in symmetric color shift detection unit)
FIG. 5 is a diagram illustrating an example of a detection direction in the symmetric color shift amount detection unit 31 in FIG. 1.

  As illustrated in FIG. 5, for example, the symmetric color shift amount detection unit 31 divides a captured image into eight regions, and sets an image height direction indicated by an arrow in the drawing as a detection direction in each of the divided regions. The symmetric color shift amount detection unit 31 detects a local symmetric color shift amount by detecting an edge region orthogonal to the set detection direction from the captured image and performing block matching (detection) in that region.

(Example of detection direction in uniform color shift detection unit)
FIG. 6 is a diagram illustrating an example of the detection direction in the uniform color shift amount detection unit 34 of FIG.

  As illustrated in FIG. 6, the uniform color shift amount detection unit 34 divides a captured image into eight regions, for example, and in four regions among the divided eight regions, the image height direction indicated by the arrows in the drawing. The direction orthogonal to is set as the detection direction. The uniform color shift amount detection unit 34 detects a local uniform color shift amount by detecting an edge region orthogonal to the set detection direction from the captured image and performing block matching (detection) in that region.

(Explanation of symmetric uniform relationship)
FIG. 7 is a diagram illustrating a symmetric uniform relationship when a local symmetric color shift amount is detected from a captured image.

  In FIG. 7, the horizontal axis is a horizontal straight line passing through the center of the captured image, and the vertical axis is a vertical straight line passing through the center of the captured image. The same applies to FIG. 8 described later.

  A thin dotted line arrow is a vector representing the amount of symmetric color misregistration, a thin dashed line arrow is a vector representing the amount of uniform color misregistration, and a thin solid line arrow is detected by the symmetric color misregistration amount detection unit 31. This is a vector representing the amount of local symmetric color shift. Further, a thick solid line arrow is a vector representing an actual color misregistration amount, and a thick dotted line arrow is a vector representing a target offset.

  As shown in FIG. 7, the local symmetric color shift amount detected by the symmetric color shift amount detection unit 31 is obtained by projecting a uniform color shift amount in the detection direction in addition to the actual symmetric color shift amount. Included as an offset for. Therefore, as shown in FIG. 7, even when the distance from the center of the captured image is the same, that is, when the image height is the same, the local symmetric color shift amount detected by the symmetric color shift amount detection unit 31. Is different.

  The symmetry offset is defined by the following formula (1).

In Expression (1), uniform_offset _α is a symmetry offset when the angle between the detection direction and the horizontal direction set by the symmetric color shift amount detection unit 31 is α, and the vector u represents the uniform color shift amount. It is a vector that represents. In addition, θ is an angle formed by a vector representing the uniform color shift amount and the horizontal direction.

Therefore, for example, the symmetry offset at the position where α is −45 degrees can be calculated by the following equation (2), where the vector u is (u _x , u _y ).

(Explanation of symmetric uniform relationship)
FIG. 8 is a diagram illustrating a symmetric uniform relationship when a local uniform color shift amount is detected from a captured image.

  In FIG. 8, the thin dotted arrow is a vector representing a symmetric color misregistration amount, the thin dashed line arrow is a vector representing a uniform color misregistration amount, and the thin solid line arrow is a uniform color misregistration amount detection. This is a vector representing the local uniform color shift amount detected by the unit 34. A thick solid line arrow is a vector representing an actual color misregistration amount, and a thick dotted line is a vector representing a uniform offset.

As shown in FIG. 8, when a vector u representing a uniform color shift amount is (u _x , u _y ), u _x is on a vertical straight line passing through the center of the captured image by the uniform color shift amount detection unit 34. This is equivalent to the amount of local uniform color shift occurring in the horizontal direction detected by. Further, u _y is equivalent to the local uniform color shift amount generated in the vertical direction detected by the uniform color shift amount detection unit 34 on the horizontal straight line passing through the center of the captured image.

  However, in addition to the actual uniform color shift amount, the local uniform color shift amount detected by the uniform color shift amount detection unit 34 at a position other than the vertical and horizontal straight lines passing through the center of the captured image includes A symmetric color shift amount projected in the detection direction is included as a uniform offset.

  The uniform offset is defined by the following equation (3).

In Equation (3), symmetric_offset _α is a uniform offset when the angle formed between the vector representing the symmetric color shift amount and the horizontal direction is α, and the vector s (p) is determined by the uniform color shift amount detection unit 34. It is a vector representing a symmetric color shift amount at a position p where a local uniform color shift amount is detected.

  Therefore, for example, the uniform offset at the position where α is −20 degrees can be calculated by the following equation (4).

(Description of processing of image processing apparatus)
FIG. 9 is a flowchart for explaining image processing of the image processing apparatus 10 of FIG. This image processing is started, for example, when a captured image is input to the image processing apparatus 10.

  In step S11 of FIG. 9, the symmetric color shift amount detection unit 31 of the image processing apparatus 10 performs local matching by performing block matching in the edge region orthogonal to the predetermined detection direction (FIG. 5) of the input captured image. A symmetric color shift amount is detected. The symmetric color misregistration amount detection unit 31 supplies the detected local symmetric color misregistration amount to the symmetric color misregistration amount temporary aggregation unit 32 and the symmetric color misregistration amount calculation unit 37.

  In step S <b> 12, the uniform color misregistration amount detection unit 34 detects the local uniform color misregistration amount by performing block matching in an edge region orthogonal to a predetermined detection direction (FIG. 6) of the input captured image. The uniform color shift amount detector 34 supplies the detected local uniform color shift amount to the uniform color shift amount calculator 35.

  In step S <b> 13, the symmetric color misregistration amount temporary aggregation unit 32 calculates an average value of local symmetric color misregistration amounts supplied from the symmetric color misregistration amount detection unit 31 for each distance from the center of the captured image to the detection position, A symmetrical color shift amount (a provisional value) in the entire image is determined. The symmetric color misregistration amount provisional totaling unit 32 supplies the determined symmetric color misregistration amount (a provisional value thereof) to the uniform offset calculating unit 33.

  In step S <b> 14, the uniform offset calculation unit 33 uses the symmetric color shift amount supplied from the symmetric color shift amount temporary summation unit 32 or the symmetric color shift amount totalization unit 13 according to the above-described equation (3). Calculate the offset. The uniform offset calculation unit 33 supplies the uniform offset to the uniform color misregistration amount calculation unit 35.

  In step S15, the uniform color misregistration amount calculation unit 35 deletes the uniform offset supplied from the uniform offset calculation unit 33 from the local uniform color misregistration amount supplied from the uniform color misregistration amount detection unit 34. The local uniform color shift amount is calculated. The uniform color misregistration amount calculation unit 35 supplies the calculated local uniform color misregistration amount to the uniform color misregistration amount totaling unit 12.

  In step S <b> 16, the uniform color misregistration amount totaling unit 12 obtains the average value of local uniform color misregistration amounts supplied from the uniform color misregistration amount calculating unit 35 as the uniform color misregistration amount in the entire image, and the symmetrical offset calculating unit 36. To supply.

  In step S <b> 17, the symmetry offset calculation unit 36 uses the uniform color shift amount supplied from the uniform color shift amount totaling unit 12 to calculate the symmetry offset by the above-described equation (1). The symmetry offset calculation unit 36 supplies the symmetry offset to the symmetric color shift amount calculation unit 37.

  In step S <b> 18, the symmetric color shift amount calculation unit 37 deletes the symmetric offset supplied from the symmetric offset calculation unit 36 from the local symmetric color shift amount supplied from the symmetric color shift amount detection unit 31. The local symmetric color shift amount is corrected and supplied to the symmetric color shift amount totaling unit 13.

  In step S19, the symmetric color misregistration amount totalizing unit 13 calculates an average value of local symmetric color misregistration amounts supplied from the symmetric color misregistration amount calculating unit 37 for each distance from the center of the captured image to the detection position. The total symmetrical color shift amount is obtained.

  In step S20, the image processing apparatus 10 determines whether the symmetric color shift amount and the uniform color shift amount have converged. For example, in the image processing apparatus 10, the difference between the uniform color shift amounts obtained in the previous step S16 and the previous step is smaller than the threshold value, and the difference between the symmetrical color shift amounts obtained in the previous step S19 and the current step step S19 is larger than the threshold value. If it is smaller, it is determined that the symmetrical color shift amount and the uniform color shift amount have converged.

  On the other hand, the image processing apparatus 10 determines that the difference between the uniform color shift amounts obtained in the previous step S16 and the current step is greater than or equal to the threshold value, or the difference between the symmetrical color shift amounts obtained in the previous step S19. If it is equal to or greater than the threshold value, it is determined that the symmetrical color shift amount and the uniform color shift amount have not converged.

  When it is determined in step S20 that the symmetric color shift amount and the uniform color shift amount have not converged, the symmetric color shift amount totaling unit 13 calculates the symmetric color shift amount obtained in the immediately preceding step S19 as a uniform offset calculation unit. 33. Then, the process returns to step S14, and the processes of steps S14 to S20 are repeated until the symmetrical color shift amount and the uniform color shift amount converge. As a result, the uniform color misregistration amount is updated using the symmetric color misregistration amount obtained in the immediately preceding step S19, and the symmetric color misregistration amount is updated using the uniform color misregistration amount obtained in the immediately preceding step S16. The

  On the other hand, if it is determined in step S20 that the symmetric color shift amount and the uniform color shift amount have converged, the process proceeds to step S21. In step S21, the uniform color misregistration amount totaling unit 12 outputs the uniform color misregistration amount obtained in the immediately preceding step S16 as the final uniform color misregistration amount.

  In step S22, the symmetric color shift amount totaling unit 13 outputs the symmetric color shift amount obtained in the immediately preceding step S19 as the final symmetric color shift amount. Then, the process ends.

  As described above, the image processing apparatus 10 detects the symmetric color shift amount and the uniform color shift amount from the captured image according to the symmetric uniform relationship. Therefore, it is possible to accurately detect the symmetric color shift amount and the uniform color shift amount of the captured image using only the captured image.

  Therefore, by performing magnification chromatic aberration correction on the captured image using the symmetric color shift amount and the uniform color shift amount detected by the image processing apparatus 10, a captured image without color shift can be obtained. In addition, even when the captured image is captured using an old lens that cannot acquire lens information, the symmetric color shift amount and the uniform color shift amount can be accurately detected. Furthermore, in the manufacturing process of an imaging device that captures a captured image, it is not necessary to perform calibration for each imaging device in order to detect a symmetrical color shift amount and a uniform color shift amount.

  Further, the image processing apparatus 10 updates the symmetric color shift amount and the uniform color shift amount until the symmetric color shift amount and the uniform color shift amount converge, thereby improving the detection accuracy of the symmetric color shift amount and the uniform color shift amount. Can be made.

<Second Embodiment>
(Configuration example of second embodiment of image processing apparatus)
FIG. 10 is a block diagram illustrating a configuration example of the second embodiment of the image processing apparatus to which the present disclosure is applied.

  Among the configurations shown in FIG. 10, the same reference numerals are given to the same configurations as those in FIG. 1. The overlapping description will be omitted as appropriate.

  The image processing apparatus 50 in FIG. 10 includes a detection unit 51, a symmetric color shift amount totaling unit 52, and a uniform color shift amount totaling unit 53. The image processing apparatus 50 calculates a symmetric color shift amount using a temporary value of the uniform color shift amount, and calculates a final uniform color shift amount using the symmetric color shift amount. That is, in the image processing device 50, the order of calculation of the uniform color shift amount and the symmetric color shift amount is opposite to that in the image processing device 10.

  Specifically, the detection unit 51 includes a symmetric color misregistration amount detection unit 31, a uniform color misregistration amount detection unit 34, a uniform color misregistration amount temporary aggregation unit 71, a symmetry offset calculation unit 72, a symmetric color misregistration amount calculation unit 73, The uniform offset calculation unit 74 and the uniform color misregistration amount calculation unit 75 are included.

  The uniform color misregistration amount temporary aggregation unit 71 assumes that the local uniform color misregistration amount detected by the uniform color misregistration amount detection unit 34 does not include a symmetric color misregistration amount, and averages the detected local uniform color misregistration amounts. The value is determined as a uniform color shift amount (a provisional value) in the entire image. The uniform color misregistration amount provisional total unit 71 supplies the uniform color misregistration amount (a provisional value thereof) to the symmetry offset calculation unit 72.

  The symmetry offset calculation unit 72 uses the uniform color misregistration amount output from the uniform color misregistration amount temporary aggregation unit 71 or the uniform color misregistration amount aggregation unit 53 to calculate the symmetry offset according to the symmetry uniform relationship. Specifically, the symmetry offset calculation unit 72 calculates the symmetry offset by the above-described equation (1) using the uniform color shift amount. The symmetry offset calculation unit 72 supplies the calculated symmetry offset to the symmetry color shift amount calculation unit 73.

  A symmetric color shift amount calculation unit 73 (characteristic-induced color shift amount calculation unit) calculates a symmetric offset supplied from the symmetric offset calculation unit 72 from the local symmetric color shift amount detected by the symmetric color shift amount detection unit 31. By deleting, the local symmetry color shift amount is calculated or updated. The symmetric color misregistration amount calculation unit 73 supplies the calculated or updated local color misregistration amount to the symmetric color misregistration amount totaling unit 52.

  The uniformity offset calculation unit 74 uses the symmetric color shift amount supplied from the symmetric color shift amount totaling unit 52 to calculate a uniform offset according to the symmetric uniformity relationship. Specifically, the uniform offset calculation unit 74 calculates the uniform offset by the above-described equation (3) using the symmetrical color shift amount. The uniformity offset calculation unit 74 supplies the calculated uniformity offset to the uniform color misregistration amount calculation unit 75.

  The uniform color misregistration amount calculation unit 75 deletes the uniform offset supplied from the uniform offset calculation unit 74 from the local uniform color misregistration amount supplied from the uniform color misregistration amount detection unit 34, thereby generating a local uniform color misregistration. The amount is corrected and supplied to the uniform color shift amount totaling unit 53.

  The symmetric color shift amount totaling unit 52 calculates the average value of the local symmetric color shift amounts supplied from the symmetric color shift amount calculating unit 73 for each distance from the center of the captured image to the detection position. The symmetric color shift amount totaling unit 52 supplies the calculated average value to the uniform offset calculating unit 74 as the symmetric color shift amount in the entire image. Further, when the obtained uniform color shift amount and the symmetric color shift amount converge, the symmetric color shift amount totaling unit 52 outputs the symmetric color shift amount as the final symmetric color shift amount.

  The uniform color shift amount totaling unit 53 obtains an average value of local uniform color shift amounts supplied from the uniform color shift amount calculating unit 75 as a uniform color shift amount in the entire image. When the obtained uniform color shift amount and the symmetric color shift amount converge, the uniform color shift amount totaling unit 53 outputs the uniform color shift amount as the final uniform color shift amount. On the other hand, when the obtained uniform color shift amount and symmetric color shift amount have not converged, the uniform color shift amount totaling unit 53 supplies the uniform color shift amount to the symmetric offset calculating unit 72. Thus, the uniform color shift amount and the symmetric color shift amount are updated using the uniform color shift amount.

  The image processing of the image processing apparatus 50 in FIG. 10 is the same as the image processing in FIG. 9 except that the symmetric color shift and the uniform color shift are reversed, and thus the description thereof is omitted.

  In the first and second embodiments, the update of the symmetric color shift amount and the uniform color shift amount is not performed until the symmetric color shift amount and the uniform color shift amount converge, but only a predetermined number of times. It may be performed.

<Third Embodiment>
FIG. 11 is a block diagram illustrating a configuration example of a third embodiment of an image processing apparatus to which the present disclosure is applied.

  Of the configurations shown in FIG. 11, the same configurations as those in FIG. The overlapping description will be omitted as appropriate.

  An image processing apparatus 90 in FIG. 11 includes a detection unit 91 and a uniform color shift amount totaling unit 92. The image processing apparatus 90 outputs the final symmetrical color misregistration amount without correcting the provisional value of the symmetrical color misregistration amount.

  Specifically, the configuration of the detection unit 91 is that a symmetric color deviation amount totaling unit 101 is provided instead of the symmetric color deviation amount temporary totaling unit 32, and that a symmetric offset calculation unit 36 and a symmetric color deviation amount calculation unit 37 are provided. The configuration is the same as that of the detection unit 11 in FIG.

  The symmetric color shift amount totaling unit 101 of the detection unit 91 assumes that the detected local symmetric color shift amount does not include the uniform color shift amount, and sets the symmetric color for each distance from the center of the captured image to the detection position. An average value of local symmetric color shift amounts supplied from the shift amount detection unit 31 is calculated. The symmetric color shift amount totaling unit 101 supplies the calculated average value as the symmetric color shift amount in the entire image to the uniform offset calculating unit 33 and outputs it as the final symmetric color shift amount.

  The uniform color misregistration amount totaling unit 92 obtains an average value of local uniform color misregistration amounts supplied from the uniform color misregistration amount calculating unit 35 as a uniform color misregistration amount in the entire image. The uniform color misregistration amount totaling unit 92 outputs the obtained uniform color misregistration amount.

(Description of processing of image processing apparatus)
FIG. 12 is a flowchart for explaining image processing of the image processing apparatus 90 of FIG. This image processing is started when a captured image is input to the image processing apparatus 90, for example.

  The processes in steps S31 and S32 in FIG. 12 are the same as the processes in steps S11 and S12 in FIG.

  In step S33, the symmetric color misregistration amount totalizing unit 101 calculates the average value of local symmetric color misregistration amounts supplied from the symmetric color misregistration amount detection unit 31 for each distance from the center of the captured image to the detection position, The amount of symmetric color shift in the whole is determined. The symmetrical color misregistration amount totaling unit 101 supplies the determined symmetrical color misregistration amount to the uniform offset calculating unit 33.

  In step S <b> 34, the uniform offset calculation unit 33 uses the symmetric color shift amount supplied from the symmetric color shift amount totaling unit 101 to calculate the uniform offset according to the above-described equation (3). The uniform offset calculation unit 33 supplies the uniform offset to the uniform color misregistration amount calculation unit 35.

  The processing in steps S35 and S36 is the same as the processing in steps S15 and S16 in FIG.

  In step S <b> 37, the uniform color misregistration amount totaling unit 92 outputs the uniform color misregistration amount supplied from the uniform color misregistration amount calculating unit 35 as the final uniform color misregistration amount.

  In step S38, the symmetric color shift amount totaling unit 101 outputs the symmetric color shift amount obtained in step S33 as the final symmetric color shift amount. Then, the process ends.

  As described above, the image processing apparatus 90 outputs the final symmetric color misregistration amount as it is without correcting the provisional value of the symmetric color misregistration amount, so that the processing amount is small. Further, the image processing apparatus 90 calculates a uniform offset using a provisional value of the symmetrical color shift amount, and obtains the uniform color shift amount by deleting the uniform offset from the local uniform color shift amount. Accordingly, it is possible to accurately detect the uniform color misregistration amount.

  In addition, although description is abbreviate | omitted, 2nd Embodiment and 3rd Embodiment can also be combined. In this case, the image processing apparatus outputs the final uniform color misregistration amount without correcting the temporary value of the uniform color misregistration amount.

<Fourth embodiment>
(Configuration example of fourth embodiment of image processing apparatus)
FIG. 13 is a block diagram illustrating a configuration example of the fourth embodiment of an image processing apparatus to which the present disclosure is applied.

  The image processing apparatus 120 in FIG. 13 includes a detection unit 121 and a totaling unit 122. The image processing device 120 represents the relationship between the color shift amount detected from the captured image, the symmetrical color shift amount of the captured image, the uniform color shift amount of the captured image, and the detection direction (detection direction) of the color shift amount of the captured image. Based on the symmetric uniform relationship model, a symmetric color shift amount and a uniform color shift amount are detected from the color shift amounts detected from the captured image.

  The detection unit 121 of the image processing device 120 detects the color misregistration amount of the captured image by performing block matching in an edge region orthogonal to a predetermined detection direction of the captured image input from the outside. The detection unit 121 detects the symmetric color shift so that the square error between the detected color shift amount of the captured image calculated based on the symmetric uniform relationship model and the detection direction and the detected color shift amount is minimized. Detect the amount and uniform color shift amount.

Here, the symmetric uniform relation model will be described. First, the distance r _i from the center of the captured image at the position p _i = (p _xi , p _yi ) in the captured image is expressed by the following equation (5).

The position p _i color shift amount Aberration _pi generated by symmetric color shift and uniform color shift in a uniform color shift and symmetric color shift amount s (r _i) the amount of _{u = (u x, u y} ) using, It can be formulated by the following formula (6).

Therefore, when a vector representing the detection direction is d _i, the color shift amount Aberration_model _pi detected from the captured image, as shown in the following equation (7), the position p _i of the color shift amount Aberration _pi and the vector d _i Is the inner product.

Therefore, the model represented by the above-described formula (7) is used as the symmetric uniform relation model. The detection unit 121 assigns a vector representing the detection direction to the above-described equation (7) so that the square error between the color misregistration amount aberration_model _pi in equation (7) and the color misregistration amount detected from the captured image is minimized. Then, the local symmetric color shift amount s (r _i ) and the local uniform color shift amount u are detected. The detection unit 121 supplies the detected local symmetric color shift amount and local uniform color shift amount to the totaling unit 122.

  The totaling unit 122 calculates an average value of local symmetric color shift amounts supplied from the detection unit 121 for each distance from the center of the captured image to the detection position, and sets this as the symmetric color shift amount in the entire image. The aggregation unit 122 outputs a symmetric color misregistration amount.

  Further, the totaling unit 122 obtains the average value of the local uniform color shift amount supplied from the detection unit 121 as the uniform color shift amount in the entire image. The aggregation unit 122 outputs a uniform color misregistration amount.

(Description of processing of image processing apparatus)
FIG. 14 is a flowchart illustrating image processing of the image processing apparatus 120 in FIG. This image processing is started, for example, when a captured image is input to the image processing apparatus 120.

  In step S51 of FIG. 14, the detection unit 121 of the image processing device 120 detects the color misregistration amount of the captured image by performing block matching in the edge region orthogonal to the predetermined detection direction of the input captured image. .

  In step S <b> 52, the detection unit 121 calculates the detected predicted value of the color shift amount of the captured image calculated based on the symmetrical uniform relationship model represented by the above-described equation (7) and the detection direction, and the detected color shift amount. The local symmetric color shift amount and the local uniform color shift amount are detected so that the square error is minimized. The detection unit 121 supplies the detected local symmetric color shift amount and local uniform color shift amount to the totaling unit 122.

  In step S53, the totaling unit 122 calculates the average value of the local symmetric color shift amounts supplied from the detection unit 121 for each distance from the center of the captured image to the detection position, and determines the symmetric color shift amount in the entire image. And output.

  In step S54, the totaling unit 122 determines the average value of the local uniform color shift amount supplied from the detection unit 121 as the uniform color shift amount in the entire image, and outputs it. Then, the process ends.

<Fifth embodiment>
(Description of computer to which the present disclosure is applied)
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.

  FIG. 15 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

  In the computer 200, a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are connected to each other via a bus 204.

  An input / output interface 205 is further connected to the bus 204. An input unit 206, an output unit 207, a storage unit 208, a communication unit 209, and a drive 210 are connected to the input / output interface 205.

  The input unit 206 includes a keyboard, a mouse, a microphone, and the like. The output unit 207 includes a display, a speaker, and the like. The storage unit 208 includes a hard disk, a nonvolatile memory, and the like. The communication unit 209 includes a network interface and the like. The drive 210 drives a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer 200 configured as described above, for example, the CPU 201 loads the program stored in the storage unit 208 to the RAM 203 via the input / output interface 205 and the bus 204 and executes the program. A series of processing is performed.

  The program executed by the computer 200 (CPU 201) can be provided by being recorded in the removable medium 211 as a package medium or the like, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer 200, the program can be installed in the storage unit 208 via the input / output interface 205 by attaching the removable medium 211 to the drive 210. The program can be received by the communication unit 209 via a wired or wireless transmission medium and installed in the storage unit 208. In addition, the program can be installed in the ROM 202 or the storage unit 208 in advance.

  Note that the program executed by the computer 200 may be a program that is processed in time series in the order described in this specification, or a necessary timing such as in parallel or when a call is made. It may be a program in which processing is performed.

  The effects described in this specification are merely examples and are not limited, and may have other effects.

  The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present disclosure.

  For example, the characteristic-induced color shift may be a color shift other than a symmetric color shift as long as the color shift is caused by lens characteristics. The individual-difference color misregistration amount may be a color misregistration other than the uniform color misregistration as long as it is a color misregistration due to an individual difference between lenses.

  Further, for example, the present disclosure can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is jointly processed.

  In addition, each step described in the above flowchart can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  In addition, this indication can also take the following structures.

(1)
Color shift amount detected from an image, characteristic-induced color shift amount that is a color shift amount caused by lens characteristics of the image, and individual difference-induced color shift that is a color shift amount caused by individual difference of the lens of the image An image processing apparatus comprising: a detection unit configured to detect the characteristic-induced color shift amount and the individual-difference color shift amount from an input image in accordance with a quantity relationship.
(2)
The detector is
By deleting the offset for individual difference calculated according to the relationship using the characteristic-induced color shift amount of the input image from the color shift amount detected in the first region of the input image, the input image The image processing apparatus according to (1), further including: an individual difference-induced color shift amount calculation unit that calculates the individual difference-induced color shift amount.
(3)
The detector is
A characteristic-induced color shift amount detection unit that detects a color shift amount detected in the second region of the input image as the characteristic-induced color shift amount of the input image;
The individual difference-induced color shift amount calculation unit uses the characteristic-induced color shift amount detected by the characteristic-induced shift amount detection unit from the color shift amount detected in the first region of the input image. The image processing apparatus according to (2), configured to delete the individual difference offset calculated according to a relationship.
(4)
The detector is
From the characteristic-induced color misregistration amount detected by the characteristic-induced color misregistration amount detection unit, calculated according to the relationship using the individual difference-induced color misregistration amount calculated by the individual difference-based color misregistration amount calculation unit. The image processing apparatus according to (3), further including: a characteristic-induced color shift amount calculation unit that corrects the characteristic-induced color shift amount by deleting the characteristic offset.
(5)
The individual difference-induced color misregistration amount calculation unit uses the characteristic-induced color misregistration amount corrected by the characteristic-induced color misregistration amount calculation unit from the color misregistration amount detected in the first region of the input image. The image processing apparatus according to (4), configured to update the individual-difference color shift amount by deleting the offset for individual difference calculated according to the relationship.
(6)
The detector is
By deleting the characteristic offset calculated according to the relationship using the individual difference-induced color shift amount of the input image from the color shift amount detected in the first region of the input image, the input image The image processing apparatus according to (1), further comprising: a characteristic-induced color shift amount calculation unit that calculates the characteristic-induced color shift amount.
(7)
The detector is
An individual difference-induced color shift amount detection unit that detects the color shift amount detected in the second region of the input image as the individual difference-induced color shift amount of the input image;
The characteristic-induced color shift amount calculation unit uses the individual difference-induced color shift amount detected by the characteristic-induced shift amount detection unit from the color shift amount detected in the first region of the input image. The image processing apparatus according to (6), configured to delete the characteristic offset calculated according to a relationship.
(8)
The detector is
Calculated according to the relationship using the characteristic-induced color misregistration amount calculated by the characteristic-induced color misregistration amount calculation unit from the individual difference-induced color misregistration amount detected by the individual difference-induced color misregistration amount detection unit. The image processing apparatus according to (7), further including: an individual difference-induced color shift amount calculation unit that corrects the individual difference-induced color shift amount by deleting the individual difference offset.
(9)
The characteristic-induced color misregistration amount calculation unit uses the individual difference-induced color misregistration amount corrected by the characteristic-induced color misregistration amount calculation unit from the color misregistration amount detected in the first region of the input image. The image processing apparatus according to (8), configured to update the characteristic-induced color misregistration amount by deleting the characteristic offset calculated according to the relationship.
(10)
The detection unit is configured to detect the characteristic-induced color shift amount and the individual-difference color shift amount from a color shift amount detected from the input image based on a model representing the relationship. The image processing apparatus according to 1).
(11)
The detection unit includes the characteristic-derived color so that a square error between a detection prediction value of a color shift amount of the input image calculated based on the model and a color shift amount detected from the input image is minimized. The image processing apparatus according to (10), configured to detect a shift amount and the individual-difference color shift amount.
(12)
The model includes a color shift amount detected from the input image, a characteristic-induced color shift amount of the input image, an individual difference-induced color shift amount of the input image, and a color shift amount of the input image. Represents a relationship,
The detection unit is configured to minimize a square error between a detection prediction value of a color shift amount of the input image calculated based on the model and the detection direction and a color shift amount detected from the input image. The image processing apparatus according to (11), configured to detect the characteristic-induced color shift amount and the individual-difference color shift amount.
(13)
The image processing device
Color shift amount detected from an image, characteristic-induced color shift amount that is a color shift amount caused by lens characteristics of the image, and individual difference-induced color shift that is a color shift amount caused by individual difference of the lens of the image An image processing method including a detecting step of detecting the characteristic-induced color shift amount and the individual-difference color shift amount from an input image in accordance with a relationship between the amounts.

  DESCRIPTION OF SYMBOLS 10 Image processing apparatus, 11 Detection part, 31 Symmetric color shift amount detection part, 34 Uniform color shift amount detection part, 35 Uniform color shift amount calculation part, 37 Symmetric color shift amount calculation part, 50 Image processing apparatus, 73 Symmetric color shift Amount calculation unit, 75 uniform color misregistration amount calculation unit, 120 image processing apparatus, 121 detection unit

Claims (13)

Color shift amount detected from an image, characteristic-induced color shift amount that is a color shift amount caused by lens characteristics of the image, and individual difference-induced color shift that is a color shift amount caused by individual difference of the lens of the image An image processing apparatus comprising: a detection unit configured to detect the characteristic-induced color shift amount and the individual-difference color shift amount from an input image in accordance with a quantity relationship. The detector is
By deleting the offset for individual difference calculated according to the relationship using the characteristic-induced color shift amount of the input image from the color shift amount detected in the first region of the input image, the input image The image processing apparatus according to claim 1, further comprising: an individual difference-induced color shift amount calculation unit that calculates the individual difference-induced color shift amount. The detector is
A characteristic-induced color shift amount detection unit that detects a color shift amount detected in the second region of the input image as the characteristic-induced color shift amount of the input image;
The individual difference-induced color shift amount calculation unit uses the characteristic-induced color shift amount detected by the characteristic-induced shift amount detection unit from the color shift amount detected in the first region of the input image. The image processing apparatus according to claim 2, configured to delete the individual difference offset calculated according to a relationship. The detector is
From the characteristic-induced color misregistration amount detected by the characteristic-induced color misregistration amount detection unit, calculated according to the relationship using the individual difference-induced color misregistration amount calculated by the individual difference-based color misregistration amount calculation unit. The image processing apparatus according to claim 3, further comprising: a characteristic-induced color misregistration amount calculation unit that corrects the characteristic-induced color misregistration amount by deleting the characteristic offset. The individual difference-induced color misregistration amount calculation unit uses the characteristic-induced color misregistration amount corrected by the characteristic-induced color misregistration amount calculation unit from the color misregistration amount detected in the first region of the input image. The image processing apparatus according to claim 4, wherein the individual-difference color shift amount is updated by deleting the individual-difference offset calculated according to the relationship. The detector is
By deleting the characteristic offset calculated according to the relationship using the individual difference-induced color shift amount of the input image from the color shift amount detected in the first region of the input image, the input image The image processing apparatus according to claim 1, further comprising: a characteristic-induced color shift amount calculation unit that calculates the characteristic-induced color shift amount. The detector is
An individual difference-induced color shift amount detection unit that detects the color shift amount detected in the second region of the input image as the individual difference-induced color shift amount of the input image;
The characteristic-induced color shift amount calculation unit uses the individual difference-induced color shift amount detected by the characteristic-induced shift amount detection unit from the color shift amount detected in the first region of the input image. The image processing apparatus according to claim 6, configured to delete the characteristic offset calculated according to a relationship. The detector is
Calculated according to the relationship using the characteristic-induced color misregistration amount calculated by the characteristic-induced color misregistration amount calculation unit from the individual difference-induced color misregistration amount detected by the individual difference-induced color misregistration amount detection unit. The image processing apparatus according to claim 7, further comprising: an individual difference-induced color shift amount calculation unit that corrects the individual difference-induced color shift amount by deleting an individual difference offset. The characteristic-induced color misregistration amount calculation unit uses the individual difference-induced color misregistration amount corrected by the characteristic-induced color misregistration amount calculation unit from the color misregistration amount detected in the first region of the input image. The image processing apparatus according to claim 8, wherein the characteristic-induced color shift amount is updated by deleting the characteristic offset calculated according to the relationship. The detection unit is configured to detect the characteristic-induced color shift amount and the individual-difference color shift amount from a color shift amount detected from the input image based on a model representing the relationship. The image processing apparatus according to 1. The detection unit includes the characteristic-derived color so that a square error between a detection prediction value of a color shift amount of the input image calculated based on the model and a color shift amount detected from the input image is minimized. The image processing apparatus according to claim 10, wherein the image processing apparatus is configured to detect a deviation amount and the individual-difference color deviation amount. The model includes a color shift amount detected from the input image, a characteristic-induced color shift amount of the input image, an individual difference-induced color shift amount of the input image, and a color shift amount of the input image. Represents a relationship,
The detection unit is configured to minimize a square error between a detection prediction value of a color shift amount of the input image calculated based on the model and the detection direction and a color shift amount detected from the input image. The image processing apparatus according to claim 11, configured to detect the characteristic-induced color shift amount and the individual-difference color shift amount. The image processing device
Color shift amount detected from an image, characteristic-induced color shift amount that is a color shift amount caused by lens characteristics of the image, and individual difference-induced color shift that is a color shift amount caused by individual difference of the lens of the image An image processing method including a detecting step of detecting the characteristic-induced color shift amount and the individual-difference color shift amount from an input image in accordance with a relationship between the amounts.

Images