JP2015088933A - Image processing device, imaging device, image processing method, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device for accurately detecting and correcting aberration, and also to provide an imaging device, an image processing method, and an image processing program.SOLUTION: A correction image and a detection image are created using lens data. A figure 40 is a figure in which an aberration region 61 calculated using the lens data is shown on a CbCr flat surface. Explanation about means for creating an aberration image is performed. First, a DSP 101 calculates the aberration region 61 using lens data. Next, a pixel having color difference information included in the aberration region 61 is abstracted from an imaging image. Then, saturation of the color difference information on the abstracted pixel is reduced. Concretely, the color difference information on the CbCr flat surface is moved parallel toward an original point. As a result, the color difference information on the pixel belonging to the aberration region 61 is moved to a region 62. Then, the saturation only of the fringe included in the imaging image is reduced and the fringe is suppressed.

Description

  The present invention relates to an image processing device, an imaging device, an image processing method, and an image processing program used for correcting aberration.

  2. Description of the Related Art An imaging device that suppresses fringes that are aberrations that occur when an image output from an imaging element is processed is known. The imaging device calculates a luminance difference between the target pixel and surrounding pixels around it, and detects and corrects a fringe according to the luminance difference (Patent Document 1).

JP 2006-014261 A

  However, a fringe may occur even when a luminance difference between the target pixel and surrounding pixels around it cannot be detected. In such a case, there is a possibility that an aberration such as fringe cannot be accurately detected. If the aberration cannot be detected accurately, the aberration cannot be corrected.

  The present invention has been made in view of these problems, and an object thereof is to obtain an image processing apparatus, an imaging apparatus, an image processing method, and an image processing program that accurately detect and correct aberrations.

  An image processing apparatus according to a first invention of the present application includes an image acquisition unit that acquires a captured image captured through an imaging lens, and a correction unit that corrects an aberration included in the captured image and creates a corrected image. The correction unit performs different processing on the captured image using the design data of the imaging lens, thereby creating one or more processed images, and performs predetermined processing on one of the captured image and the one or more processed images. To create a mask image, synthesize one of the captured image and one or more processed images that were not used when creating the mask image with the mask image to create a composite image, and then add the composite image to the captured image A corrected image is created by synthesis.

  The correction unit creates a first processed image obtained by performing a first process on the captured image using the design data of the imaging lens, creates a mask image by performing a predetermined process on the captured image, and generates a first image. Preferably, the processed image and the mask image are combined to create a combined image.

  The correction unit creates a second processed image obtained by performing the second process on the captured image, creates a mask image by performing a predetermined process on the second processed image, and differs from the second process. A third processed image obtained by performing the third process on the captured image may be created, and the third processed image and the mask image may be synthesized to create a composite image.

  The correction unit creates a first processed image obtained by performing the first process on the photographed image, and a second processed image obtained by performing a second process different from the first process on the photographed image. Then, a predetermined process may be performed on the second processed image to create a mask image, and the first processed image and the mask image may be synthesized to create a synthesized image.

  The first processing is preferably processing for acquiring color information that is likely to cause aberration based on the design data of the imaging lens and reducing the value of color information that is likely to cause aberration included in the captured image.

  The third process is preferably a process of acquiring color information that is likely to cause aberration based on the design data of the imaging lens and reducing the saturation of the color included in the color information that is likely to cause aberration in the captured image. .

  The second process is preferably a process of acquiring color information that is likely to cause aberration based on the design data of the imaging lens and emphasizing the color information that is likely to cause aberration included in the captured image.

  An image pickup apparatus according to a second invention of the present application includes the image processing apparatus and an image pickup device that picks up a picked-up image.

  The image processing method according to the third invention of the present application is one or more processes obtained by performing different processes on color information values included in a captured image captured through the imaging lens using design data of the imaging lens. Used to create an image, to create a mask image by applying predetermined processing to the value of color information contained in one of the captured image and one or more processed images, and to create a mask image A step of synthesizing a value of color information included in any one of the photographed image and the one or more processed images and a value of color information included in the mask image to create a composite image; And a step of creating a corrected image.

  An image processing program according to a fourth invention of the present application is one or more processes obtained by performing different processes on color information values included in a captured image captured through an imaging lens using design data of the imaging lens. Used to create an image, to create a mask image by applying predetermined processing to the value of color information contained in one of the captured image and one or more processed images, and to create a mask image A step of synthesizing a value of color information included in any one of the photographed image and the one or more processed images and a value of color information included in the mask image to create a composite image; And a step of creating a corrected image.

  According to the present invention, an image processing apparatus, an imaging apparatus, an image processing method, and an image processing program that accurately detect and correct aberrations are obtained.

1 is a block diagram schematically illustrating an image processing apparatus and an imaging apparatus according to an embodiment. It is the figure which showed the aberration area | region. It is a flowchart explaining an aberration correction process. It is the figure which showed the picked-up image. FIG. 6 is a diagram showing a first intermediate correction image. It is the figure which showed the 2nd intermediate correction image. It is the figure which showed the 3rd intermediate correction image. It is the figure which showed the relationship between a threshold value and color difference information. It is the figure which showed the 1st aberration image. It is the figure which showed the 4th intermediate correction image. It is the figure which showed the 2nd aberration image. It is the figure which showed the 5th intermediate correction image. It is the figure which showed the 3rd aberration image. It is the figure which showed the 4th aberration image. It is the figure which showed the 6th intermediate correction image. It is the figure which showed the 7th intermediate | middle correction image. It is the figure which showed the aberration detection image. It is the figure which showed the aberration relief image. It is a flowchart explaining a 2nd aberration correction process. It is the figure which showed the picked-up image. It is the figure which showed the image for correction | amendment. It is the figure which showed the mask image. It is the figure which showed the synthesized image. It is the figure which showed the correction | amendment image. It is a flowchart explaining a 3rd aberration correction process. It is the figure which showed the image for a detection. It is the figure which showed the image for correction | amendment. It is the figure which showed the mask image. It is the figure which showed the synthesized image. It is the figure which showed the correction | amendment image. It is a flowchart explaining a 4th aberration correction process. It is the figure which showed the synthesized image. It is the figure which showed the correction | amendment image. It is the graph which represented the color of fringe on the CbCr plane. It is the graph which represented the color of fringe on the CbCr plane. It is the graph which represented the color of fringe on the CbCr plane. It is the graph which represented the color of fringe on the CbCr plane. It is the flowchart which showed the aberration area | region determination process. It is the flowchart which showed the aberration area | region correction process. It is the graph which represented the color of fringe on the CbCr plane.

  Hereinafter, an image processing apparatus, an imaging lens, and an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 illustrates a digital camera 100 that is an embodiment of an imaging apparatus including the image processing apparatus according to the first embodiment. First, the configuration of the digital camera 100 will be described with reference to FIG.

  The digital camera 100 includes a DSP 101 that constitutes a correction unit, a CCD 111 that is an image sensor, a detachable image pickup lens 102, a timing generator (TG) 120, an AFE 130, a camera memory 103, an operation member 104, and a recording medium 105. The display medium 106 is mainly provided.

  The imaging lens 102 includes a plurality of lenses 107 and a lens memory 108 forming a storage unit, and forms an object image on the CCD 111. The imaging lens 102 has an aberration such as fringe. The lens memory 108 stores lens data that is design data indicating the characteristics of the imaging lens 102.

  The DSP 101, CCD 111, timing generator 120, AFE 130, camera memory 103, operation member 104, recording medium 105, and display medium 106 are stored in the camera body 110. In response to the exposure signal from the timing generator 120, the CCD 111 receives and captures the subject image from the imaging lens 102 and outputs an image signal. The AFE 130 processes the image signal and transmits it to the DSP 101. The DSP 101 further processes the image signal to generate a captured image and an image file, and transmits them to the display medium or the recording medium 105. The DSP 101 can execute a first aberration correction process to be described later.

  The recording medium 105 is an SD card (registered trademark), for example, which is detachably connected to the digital camera 100, and records an image file.

  The display medium 106 is a liquid crystal monitor provided on the back surface of the digital camera 100, for example, and displays an image.

  The camera memory 103 records the firmware and image files of the DSP 101, and is used as a temporary memory when the DSP 101 executes various processes.

  The operation member 104 is, for example, a shutter release button that is a two-stage switch, a cross key, a push-down switch, and the like, and transmits a signal to the DSP 101 in accordance with a user operation. The DSP 101 operates in accordance with a signal received from the operation member 104.

  The imaging lens 102 has an inherent aberration. The aberration can be calculated from the lens data. FIG. 2 is a diagram showing, in the CbCr color space, aberrations calculated using lens data, that is, colors that are highly likely to be fringes. The fringe color difference information tends to be included in the range of the predetermined color difference information. Therefore, a color difference area where fringes are likely to appear in the CbCr color space is determined using lens data. This color difference region is called an aberration region. The DSP 101 calculates the aberration area using the lens data.

  The DSP 101 can execute a first aberration correction process for correcting a fringe included in a captured image. In the first aberration correction process, two images are created from one captured image, different processes are performed to create an aberration image in which the fringe is emphasized, the created aberration images are synthesized, and this is combined with an LPF (low-pass filter). This is a process of correcting the fringe by applying a filter.

  Next, the first aberration correction process executed by the DSP 101 will be described with reference to FIG.

  In the first step S1, a captured image is acquired. The photographed image is the image shown in FIG. 4 and is composed of data expressed in the YCbCr color space. In the photographed image, a fringe 41 appears around the person. In addition, there is an elliptical pattern 42 having color difference information close to a fringe on the torso of the person. Then, the process proceeds to steps S21 and S31.

  In the next step S21, a CbCr image (first image) composed of CbCr data (first information) expressed in the CbCr color space is created using the photographed image.

  In the next step S22, lens data is acquired from the imaging lens 102, and an aberration region is calculated using the lens data. Then, a pixel having color difference information included in the aberration region in the CbCr image is detected. FIG. 5 shows a first intermediate correction image in which pixels having color difference information included in the aberration area are represented in white and pixels having color difference information not included in the aberration area are represented in black. In the image shown in FIG. 5, the fringe 41 and a part of the pattern 42 in the photographed image shown in FIG. 4 are represented in white, and the other parts are represented in black. That is, a part of the fringe 41 and the pattern 42 is detected as a fringe. A region represented in white is referred to as a first detection region. However, when the fringe and the subject are picked up, the color difference information of the overlapped part is a mixture of the color difference information of the fringe and the color difference information of the subject image, so it is separated from the original color difference information of the fringe. May not be included in the aberration region. The region having such color difference information appears as a black dot in the first detection region of the first intermediate correction image. In other words, in the first intermediate correction image, there are a plurality of black dots in a region corresponding to a part of the fringe 41 and the pattern 42 in the captured image. In the next step S23, these black dots are removed.

  In the next step S23, a median filter is applied to the first intermediate correction image. The median filter is a filter for smoothing an image, and removes point noise included in the image. As a result, a second intermediate corrected image from which the black dot region included in the first intermediate corrected image shown in FIG. 5 is removed is obtained. FIG. 6 shows a second intermediate correction image.

  In the next step S24, a low-pass filter (LPF) is applied to the second intermediate correction image to create a third intermediate correction image. Even though the pixel actually has a fringe, the color difference information of the pixel is slightly different from the color difference information of the fringe, so that the pixel may not be included in the aberration region. Thus, a region having slightly different color difference information is detected using a low-pass filter. By applying a low-pass filter, fringes existing in the vicinity of the aberration region are detected. FIG. 7 shows a third intermediate correction image.

  In the next step S25, the color difference information of the pixels included in the third intermediate correction image is binarized using a predetermined threshold value. That is, a pixel having color difference information equal to or greater than a threshold is detected as a pixel having fringe, and a pixel having color difference information less than the threshold is determined as a pixel having no fringe. The image created in this way is defined as a first aberration image. FIG. 8 is a diagram showing the relationship between the threshold value and the color difference information. When the threshold value is lowered, the region where it is determined that the fringe exists is widened, and the region where it is determined that no fringe is present is narrowed. When the threshold value is lowered, the region where the fringe is determined to be present becomes narrower and the region where it is determined that the fringe is not present becomes wider. By adjusting the threshold, it is possible to adjust the region where fringe is determined to exist. FIG. 9 is a first aberration image, in which pixels whose color difference information is greater than or equal to the threshold value are represented in white, and pixels whose color difference information is less than the threshold value are represented in black. The region where fringe is determined to be present by the low-pass filter may be expanded to include a region where fringe does not exist, but binarization using a threshold value may exclude regions where fringe does not exist. it can.

  By executing steps S21 to S25, a region having a color close to a fringe can be detected.

  On the other hand, in step S31, a Y image (second image) composed of Y data (second information) expressed by luminance, that is, Y is created using the photographed image. Generally, fringes tend to occur in a portion where a luminance difference is large between pixels. Therefore, in the subsequent processing, fringes are detected based on the luminance information. Processing proceeds to steps S32 and S41.

  In the next step S32, based on the luminance information, pixels whose luminance difference with neighboring pixels is greater than or equal to a predetermined value are extracted from the Y image. Here, extraction is performed using a Laplacian operator or a Sobel operator. As a result, a fourth intermediate correction image shown in FIG. 10 is created.

  In the next step S33, the fourth intermediate correction image is multiplied by nonlinear gamma. As a result, the luminance of a pixel having a small luminance difference from neighboring pixels is lowered, and the luminance of a pixel having a large luminance difference is increased. A fringe hardly occurs in a pixel having a small luminance difference from a neighboring pixel, but tends to occur strongly in a pixel having a large luminance difference from a neighboring pixel. By using a non-linear gamma, the luminance of a pixel having a high possibility of including a fringe is increased, and the luminance of a pixel having a low possibility of including a fringe is decreased. Thereby, the second aberration image shown in FIG. 11 is created.

  By executing steps S31 to S33, it is possible to detect a pixel having a large luminance difference from the adjacent pixel.

  On the other hand, in step S41, the luminance information of the pixels included in the Y image is binarized using a predetermined threshold value. That is, the edge of a saturated region is detected, and a saturated region and a non-saturated region are distinguished. The image created in this way is set as a third aberration image. Fringe tends to occur particularly strongly at the boundary between a region where luminance information is saturated and a region where saturation information is not saturated. Therefore, binarization is performed using a predetermined threshold value to detect the edge of the saturated region, and the non-saturated region is removed from the saturated region. FIG. 12 is a fifth intermediate correction image, in which pixels whose luminance information is greater than or equal to the threshold value are represented in white, and pixels whose luminance information is less than the threshold value are represented in black.

  In the next step S42, the contour of the subject image of the image included in the fifth intermediate correction image is detected. Thereby, the third aberration image shown in FIG. 13 is created.

  By executing steps S41 and S42, the boundary between the region where the luminance information is saturated and the region where the luminance information is not saturated is detected.

  In the next step S5, the logical sum of the second aberration image and the third aberration image is obtained. As a result, a fourth aberration image shown in FIG. 14 is created. ORing a second aberration image including a pixel having a large luminance difference from an adjacent pixel and a third aberration image including a boundary between a region where the luminance information is saturated and a region where the luminance information is not saturated Thus, it is possible to reliably detect a region where fringes are likely to occur.

  In the next step S6, a low-pass filter is applied to the fourth aberration image to create a sixth intermediate correction image shown in FIG. Thereby, the boundary of the area determined to contain the fringe is blurred.

  In the next step S7, the luminance information of the pixels included in the sixth intermediate correction image is binarized using a predetermined threshold value to create a seventh intermediate correction image shown in FIG. The width of the fringe appearing in the captured image varies depending on the imaging lens 102. By adjusting the threshold value, a fringe can be detected using a width corresponding to the imaging lens 102. Increasing the threshold value adapts to the imaging lens 102 having a small fringe width, and decreasing the threshold value adapts to the imaging lens 102 having a large fringe width.

  In the next step S8, a logical product of the first aberration image created in step S25 and the seventh intermediate correction image created in step S7 is obtained, and an aberration detection image is obtained.

  By taking the logical product of the first aberration image including a region having a color close to the fringe and the seventh intermediate correction image having a region where the fringe is likely to be obtained from the luminance, the region where the fringe is generated is obtained. It can be detected reliably.

  In the next step S9, the aberration detection image is created by applying a low-pass filter to the aberration detection image. As a result, the boundary between the region where the fringe occurs and the other region is blurred. In the aberration detection image, the boundary between the region where the fringe occurs and the other region is clear. Therefore, when the aberration detection image is combined with the captured image, the boundary between the region corrected by the combination and the region not corrected is clear. Become. However, if the aberration detection image is combined with the photographed image after applying a low-pass filter, the boundary between the region corrected by the combination and the region not corrected is blurred and becomes inconspicuous. FIG. 18 shows an aberration alleviated image.

  Then, the fringe is corrected by synthesizing the aberration-relieved image with the captured image.

  According to this embodiment, it is possible to accurately detect and correct the fringe using a single photographed image.

  In step S41, the contrast in the vicinity of the saturated portion may be increased by using gamma to enhance the gamma in the saturated portion of the luminance information without using binarization. In step S42, the contour of the subject image included in the fifth intermediate correction image can be easily detected.

  Note that, when the fringe is detected using the first aberration image, the second aberration image, and the third aberration image, all of the first aberration image, the second aberration image, and the third aberration image are detected. Alternatively, logical sum or logical product may be calculated after weighting or α blending is performed on any one or two of the images.

  Next, a second embodiment will be described with reference to FIGS. About the same structure as 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted. The digital camera 100 according to the second embodiment executes the second aberration correction process without executing the first aberration correction process. Hereinafter, the second aberration correction process will be described.

  The DSP 101 forms an image acquisition unit and a correction unit, and executes a second aberration correction process for correcting a fringe included in the captured image. The second aberration correction process is a process for correcting fringe by creating two images from one captured image, creating a mask image from one image, and synthesizing the mask image with the other image.

  Next, the second aberration correction process will be described with reference to FIG.

  In the first step S201, a captured image is acquired. The captured image is an image shown in FIG. In the photographed image, a fringe 41 appears around the person. Then, the process proceeds to steps S221 and S222.

  In the next step S221, a correction image (first processed image) is created using the captured image. The correction image is created using a parameter having a value determined so that the color difference information is not saturated. Thereby, the fringe included in the captured image is suppressed. The parameters are, for example, white balance, color matrix, gamma, saturation, sharpness, and the like. The value determined so as not to saturate the color difference information is, for example, a value that lowers the saturation or a value that weakens the color matrix of the color that is strongly generated by the fringe. FIG. 21 shows a correction image.

  Step S222 indicates that the captured image is used in the following processing, and no specific processing is performed.

  In the next step S203, a fringe included in the photographed image is detected, and a mask image from which the fringe portion is extracted is created. FIG. 22 shows a mask image. The mask image is an aberration detection image or an aberration alleviated image created by the first aberration correction process.

  In the next step S204, the mask image is combined with the correction image. Thereby, a composite image in which only the fringe portion is extracted from the correction image is obtained. FIG. 23 shows a composite image.

  In the next step S205, the composite image is combined with the captured image. More specifically, a corrected image in which the suppressed fringe portion is combined with the fringe portion included in the captured image is created. Thereby, a corrected image in which fringe is suppressed is obtained. FIG. 24 shows a corrected image.

  According to this embodiment, it is possible to accurately detect and correct the fringe using a single photographed image.

  Further, in the process of correcting by simply lowering the fringe saturation, there is a risk that the color of the fringe portion after correction may be darkened.However, in this embodiment, the original image is reduced after the fringe saturation is reduced. Since they are combined, the corrected fringe color does not become dark.

  Next, a third embodiment will be described with reference to FIGS. About the same structure as 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted. The digital camera 100 according to the third embodiment executes the third aberration correction process without executing the first aberration correction process. Hereinafter, the third aberration correction process will be described.

  The DSP 101 forms an image acquisition unit and a correction unit, and executes a third aberration correction process for correcting fringes included in the captured image. The third aberration correction process creates two images from one captured image, creates a mask image from one image, synthesizes the other image and the mask image, and further combines the synthesized image with the original image. This is a process of correcting the fringe by combining.

  Next, the third aberration correction process will be described with reference to FIG.

  In the first step S301, a captured image is acquired. The captured image is an image shown in FIG. In the photographed image, a fringe 41 appears around the person. Then, the process proceeds to steps S311 and S321.

  In the next step S311, a detection image (second processed image) is created using the captured image. The detection image is created using parameters having values determined so that fringes can be easily detected. Thereby, the fringe included in the captured image is emphasized. The parameters are, for example, white balance, color matrix, gamma, saturation, sharpness, and the like. The value determined so that the fringe can be easily detected is, for example, a value that increases the saturation of a color in a specific wavelength range, or a color matrix value that enhances a color that is strongly generated in the fringe. FIG. 26 shows a detection image. By increasing the saturation of a color in a specific wavelength range, and determining a portion where the color is saturated as a fringe, the fringe can be detected accurately and efficiently. Even when the fringe color is close to the actual subject color, the fringe can be easily determined by increasing the saturation.

  In the next step S312, a fringe included in the detection image is detected, and a mask image from which the fringe portion is extracted is created. FIG. 27 shows a mask image. The mask image is an aberration detection image or an aberration alleviated image created by the first aberration correction process.

  On the other hand, step S321 indicates that the captured image is used in the following processing, and no specific processing is performed.

  In the next step S322, a saturation-reduced image (third processed image) is created using the captured image. The saturation reduction image is an image created by reducing the saturation of the entire captured image. FIG. 28 shows a saturation reduction image.

  In the next step S334, the mask image is combined with the saturation reduction image. Thereby, a composite image in which only the fringe portion is extracted from the correction image is obtained. FIG. 29 shows a composite image.

  In the next step S335, the synthesized image is synthesized with the captured image. More specifically, a corrected image in which the suppressed fringe portion is combined with the fringe portion included in the captured image is created. Thereby, a corrected image in which fringe is suppressed is obtained. FIG. 30 shows a corrected image.

  According to this embodiment, the same effect as that of the second embodiment is obtained.

  Further, in step S311, the fringe can be detected accurately and efficiently by increasing the saturation of the color in the specific wavelength range and considering the portion where the color is saturated as a fringe. Even when the fringe color is close to the actual subject color, the fringe can be easily determined by increasing the saturation.

  Next, a fourth embodiment will be described with reference to FIGS. About the same structure as 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted. The digital camera 100 according to the fourth embodiment executes the fourth aberration correction process without executing the first aberration correction process. Hereinafter, the fourth aberration correction process will be described.

  The DSP 101 forms an image acquisition unit and a correction unit, and executes a fourth aberration correction process for correcting fringes included in the captured image. The fourth aberration correction process creates two images from one captured image, creates a mask image from one image, synthesizes the other image and the mask image, and further combines the synthesized image with the original image. This is a process of correcting the fringe by combining.

  Next, the fourth aberration correction process will be described with reference to FIG.

  In the first step S401, a captured image is acquired. The captured image is an image shown in FIG. In the photographed image, a fringe 41 appears around the person. Then, the process proceeds to steps S411, S412, and S415.

  In the next step S411, a correction image (first processed image) is created using the captured image. The correction image is created using a parameter having a value determined so that the color difference information is not saturated. Thereby, the fringe included in the captured image is suppressed. The parameters are, for example, white balance, color matrix, gamma, saturation, sharpness, and the like. The values determined so as not to saturate the color difference information are, for example, values that lower the saturation and values of the color matrix that increase the color that is strongly generated by the fringe. FIG. 21 shows a correction image.

  In the next step S412, a detection image (second processed image) is created using the captured image. The detection image is created using parameters having values determined so that fringes can be easily detected. Thereby, the fringe included in the captured image is emphasized. The parameters are, for example, white balance, color matrix, gamma, saturation, sharpness, and the like. The value determined so that the fringe can be easily detected is, for example, a value that increases the saturation of a color in a specific wavelength range, or a color matrix value that enhances a color that is strongly generated in the fringe. FIG. 26 shows a detection image. By increasing the saturation of a color in a specific wavelength range, and determining a portion where the color is saturated as a fringe, the fringe can be detected accurately and efficiently. Even when the fringe color is close to the actual subject color, the fringe can be easily determined by increasing the saturation.

  In the next step S413, a fringe included in the detection image is detected, and a mask image in which the fringe portion is extracted is created. FIG. 27 shows a mask image. The mask image is an aberration detection image or an aberration alleviated image created by the first aberration correction process.

  In the next step S414, the mask image is combined with the correction image. Thereby, a composite image in which only the fringe portion is extracted from the correction image is obtained. FIG. 32 shows a composite image.

  On the other hand, step S415 indicates that the captured image is used in the following processing, and no specific processing is performed.

  In the next step S416, the synthesized image is synthesized with the captured image. More specifically, a corrected image in which the suppressed fringe portion is combined with the fringe portion included in the captured image is created. Thereby, a corrected image in which fringe is suppressed is obtained. FIG. 33 shows a corrected image.

  According to this embodiment, the same effects as those of the second and third embodiments are obtained. In addition, the fringe portion can be corrected without a sense of incongruity by using the detection image and the correction image to generate a composite image in which only the fringe portion is extracted.

  Next, a fifth embodiment will be described with reference to FIGS. The same components as those in the first to fourth embodiments are denoted by the same reference numerals and description thereof is omitted. In the fifth embodiment, the DSP 101 executes the aberration region determination process for determining the aberration region shown in FIG.

  FIG. 34 is a diagram showing the fringe color of the imaging lens 102 on the CbCr plane. The fringes shown in FIG. 34 appear in the first and fourth quadrants of the CbCr plane. The fringe appearing in the first quadrant appears on the subject in a so-called rear pin state in which the imaging lens 102 is focused on the back side of the subject, and has a purple color or a magenta color. The fringe appearing in the fourth quadrant appears on the subject in a so-called front pin state in which the imaging lens 102 is in front of the subject and has a green color or a blue color.

  A fringe appearing in the first quadrant is approximated by a first aberration region 51 having an elliptical shape, and a fringe appearing in the second quadrant is approximated by a second aberration region 52 having an elliptical shape. The first aberration region and the second aberration region 52 are determined by parameters that determine the shape and position of each ellipse. These parameters include at least one of the length of the major axis of the ellipse, the length of the minor axis, the center coordinates, the tilt of the major axis, and the like.

  However, there is an imaging lens 102 having a fringe that occurs in an intermediate region between the first aberration region 51 and the second aberration region 52. A means for detecting the fringe generated in the intermediate region will be described with reference to FIG.

  When the DSP 101 determines that a fringe is generated in the intermediate region 60 between the first aberration region 51 and the second aberration region 52 based on the lens data, the first aberration region 51 and the second aberration region 52. An intermediate region 60 is defined using an arc connecting the two and the first aberration region 51, the second aberration region 52, and whether or not the color difference information is included in the intermediate region 60. The intermediate area 60 is determined using parameters. The parameters for determining the intermediate region 60 are, for example, parameters for determining the shapes and positions of the first arc 53 and the second arc 54. Among the center coordinates, radius, diameter, and center angle of the arc, Including at least one. The first arc 53 and the second arc 54 connect the first aberration region 51 and the second aberration region 52.

  On the other hand, a fringe may be detected after a predetermined image processing is performed on a captured image. In this case, since the color difference information of the fringe is changed by performing predetermined image processing, the color difference information of the fringe is not included in the aberration area created based only on the lens data, and the fringe cannot be detected. There is sex. Therefore, the following aberration region correction processing is performed so that the fringe can be detected even after the captured image is subjected to predetermined image processing.

  An example of the aberration region correction process will be described with reference to the drawings. FIG. 36 shows an aberration region used for detecting a fringe included in a captured image in which white balance is adjusted.

  The fringe included in the photographed image in which the white balance is adjusted tends to move in parallel on the CbCr plane. Therefore, the fringe is detected using the translated aberration region. The first aberration region 51 is an aberration region calculated using lens data. On the other hand, when the white balance is adjusted, the aberration region tends to move according to the vector 56. Therefore, the DSP 101 calculates the vector 56, translates the first aberration region 51 using the vector 56, and calculates the third aberration region 55. Then, it is determined whether or not the color difference information included in the captured image with the adjusted white balance is included in the third aberration region. Thereby, even if it is a picked-up image which adjusted white balance, a fringe can be detected appropriately.

  FIG. 37 shows an aberration region used for detecting a fringe included in a captured image in which the color matrix is changed.

  The fringe included in the captured image whose color matrix has been changed tends to rotate and move in the CbCr plane. Therefore, fringes are detected using the rotationally moved aberration region. The first aberration region 51 is an aberration region calculated using lens data. On the other hand, when the color matrix is changed, the aberration region tends to rotate about the rotation center point 58 by the angle θ. Therefore, the DSP 101 calculates the rotation center point 58 and the angle θ, rotates the first aberration region 51 using the rotation center point 58 and the angle θ, and calculates the fourth aberration region 57. Then, it is determined whether or not the color difference information included in the captured image whose color matrix has been changed is included in the fourth aberration region 57. As a result, fringe can be detected appropriately even in a captured image in which the color matrix is adjusted.

  The aberration region determination process will be described with reference to FIG. The aberration region determination process is executed when the imaging lens 102 is attached to the camera body 110.

  In the first step S381, fringe color difference information is calculated using lens data.

  In the next step S382, the color difference information calculated in step S381 is plotted on the CbCr plane.

  In the next step S383, an ellipse including the color difference information plotted in step S382 is created.

  In the next step S384, the parameters of the ellipse created in step S383 are stored in the camera memory 103.

  The aberration region correction process will be described with reference to FIGS. The aberration area correction process is executed by the DSP 101 when the digital camera 100 captures an image.

  In the first step S391, a captured image is acquired.

  In the next step S392, the color matrix used when the captured image is captured is compared with the color matrix used when the first aberration region 51 is created, and the difference between the two is calculated.

  In the next step S393, the rotation center point 58 and the angle θ described above are calculated using the difference calculated in step S392. Then, the fourth aberration region 57 is calculated by rotating the first aberration region 51 using the rotation center point 58 and the angle θ.

  In the next step S394, the white balance used when the captured image is captured is compared with the white balance used when the first aberration region 51 is created, and the difference between the two is calculated.

  In the next step S395, the vector 56 described above is calculated using the difference calculated in step S394. Then, the fifth aberration region 59 is calculated by translating the fourth aberration region 57 using the vector 56.

  As a result, the fringe can be detected appropriately by changing the aberration region in accordance with the color matrix and white balance used when the captured image is captured.

  According to the present embodiment, the aberration area is approximated by an ellipse, and parameters representing the shape and position of the ellipse are stored, so that the amount of data to be stored can be reduced.

  Even if the aberration region cannot be expressed by an ellipse, the approximation is performed using the intermediate region 60 and the parameters representing the shape and position of the intermediate region 60 are stored, so that the amount of data to be stored is small.

  Furthermore, since the aberration region is changed according to the image processing in which the photographed image is processed, the aberration can be accurately detected.

  The aberration area determination process and the aberration area correction process may be executed using lens data in advance by another apparatus instead of the DSP 101. At this time, the calculated parameters are stored in the lens memory 108 provided in the imaging lens 102. The DSP 101 reads parameters from the lens memory 108 and detects aberrations based on the parameters.

  The parameters in the present embodiment may be calculated in advance based on lens data and stored in the lens memory 108 included in the imaging lens 102. The DSP 101 reads out parameters from the lens memory 108 and detects aberrations based on the parameters.

  The intermediate region 60 between the first aberration region 51 and the second aberration region 52 is not a circular arc but is determined by a line segment (straight line) or chord (curve) having another shape, for example, a quadratic curve or the like. May be.

  Next, a sixth embodiment will be described with reference to FIG. The same components as those in the first to fifth embodiments are denoted by the same reference numerals and description thereof is omitted. In the sixth embodiment, the means for creating the correction image and the detection image is different from the second, third, and fourth aberration correction processes. Hereinafter, means for creating the correction image and the detection image will be described.

  In the present embodiment, a correction image and a detection image are created using lens data. FIG. 40 is a diagram showing the aberration region 61 calculated using the lens data on the CbCr plane.

  First, a means for creating a corrected image will be described. First, the DSP 101 calculates an aberration region 61 shown in FIG. 40 using lens data. Next, pixels having color difference information included in the aberration region 61 are extracted from the captured image. Then, the saturation of the color difference information of the extracted pixel is lowered. Specifically, the color difference information on the CbCr plane is translated toward the origin. As a result, the color difference information of the pixels belonging to the aberration area 61 moves to the area 62. Thereby, the saturation of only the fringe included in the captured image is lowered, and the fringe is suppressed. FIG. 21 shows a correction image.

  Next, a means for creating a detection image will be described. First, the DSP 101 calculates an aberration region 61 shown in FIG. 40 using lens data. Next, pixels having color difference information included in the aberration region 61 are extracted from the captured image. Then, the saturation of the color difference information of the extracted pixel is increased. Specifically, the color difference information on the CbCr plane is translated so as to be away from the origin. As a result, the color difference information of the pixels belonging to the aberration region 61 moves to the region 63. Thereby, the saturation of only the fringe included in the photographed image is increased, and the fringe is emphasized. FIG. 26 shows a correction image.

  It is also possible to suppress the fringe by changing the color matrix instead of the saturation. The means for creating a corrected image by changing the color matrix will be described below.

First, the DSP 101 calculates an aberration region 61 shown in FIG. 40 using lens data. Next, when the aberration region 61 exists, that is, when the fringe exists, RGB information obtained by converting the color difference information of the captured image into the RGB color space is acquired. Then, the RGB information is converted using the following equation (1).
Expression (1) is an expression that suppresses the blue color included in the fringe in the RGB color information. By using Formula (1), fringe can be suppressed.

On the other hand, when the aberration region 61 does not exist, that is, when the fringe does not exist, the RGB information is converted using the following equation (2).
Equation (2) multiplies RGB color information by a unit matrix and does not convert any RGB information.

  According to the present embodiment, it is possible to suppress or enhance only the fringe by using the lens data.

  In this embodiment, when creating a corrected image, the saturation of the pixels having the color difference information included in the aberration area 61 is not lowered, but the saturation of the pixels having the color difference information included in the aberration area 61 is lowered. Also good. Thereby, fringe is suppressed. Similarly, when creating a detection image, the sharpness of the pixels having the color difference information included in the aberration area 61 may be increased instead of increasing the saturation of the pixels having the color difference information included in the aberration area 61. Thereby, the fringe is emphasized.

  In this embodiment, the pixel having the color difference information included in the aberration region 61 is converted by using the equation (1) in the means for creating a corrected image by changing the color matrix, and is included in the aberration region 61. A pixel having no color difference information may be converted using Equation (2). At this time, the DSP 101 calculates the aberration region 61 shown in FIG. 40 using the lens data. Next, pixels having color difference information included in the aberration region 61 are extracted from the captured image. Next, RGB information obtained by converting the color difference information of the extracted pixels into the RGB color space is acquired. Then, the RGB information is converted using Expression (1). By using Formula (1), fringe can be suppressed. On the other hand, a pixel having color difference information that is not included in the aberration region 61 is converted using Expression (2).

  Further, the number 1 in this embodiment is an example, and when the fringe has another color such as green color or magenta color, another equation is used.

  In any of the embodiments, all images such as a photographed image may not be an image expressed in the YCbCr color space, but may be an image composed of data expressed in another color system. .

  In any embodiment, lens data may be stored in a captured image. A device such as a personal computer can read out lens data from a captured image, and can detect and correct aberrations using the read out lens data. Further, other data used for detecting the aberration may be stored in the captured image. A device such as a personal computer can read out other data from the captured image, and can detect and correct aberrations using the read data.

  In any of the embodiments, the lens data may be stored in the imaging lens 102, or the lens data corresponding to each photographing lens 102 may be stored in the digital camera 100.

  The lens data may be measured and stored not for each type of the photographing lens 102 but for each individual. The fringe can be detected and corrected with high accuracy.

  In any of the embodiments, the imaging lens 102 is not detachably provided on the digital camera 100 and may not be removable from the digital camera 100. The digital camera 100 may be a compact digital camera.

41 Fringe 42 Pattern 51 First Aberration Region 52 Second Aberration Region 53 First Arc 54 Second Arc 55 Third Aberration Region 56 Vector 57 Fourth Aberration Region 58 Center of Rotation 59 Fifth Aberration Region 60 Intermediate area 61 Aberration area 100 Digital camera 101 DSP
DESCRIPTION OF SYMBOLS 102 Imaging lens 103 Camera memory 104 Operation member 105 Recording medium 106 Display medium 107 Lens 108 Lens memory 110 Camera body 111 CCD
120 Timing generator 130 AFE

Claims (10)

An image acquisition unit that acquires a captured image captured through an imaging lens;
A correction unit that corrects aberrations included in the captured image and creates a corrected image;
The correction unit performs different processing on the captured image using design data of the imaging lens, thereby creating one or more processed images, and any one of the captured image and the one or more processed images. A mask image is created by performing predetermined processing on the mask image, and the mask image is synthesized with any one of the photographed image and the one or more processed images that are not used when creating the mask image, and a synthesized image is formed. An image processing apparatus that creates the corrected image by creating the synthesized image and synthesizing the synthesized image.   The correction unit creates a first processed image obtained by performing a first process on the captured image using design data of the imaging lens, performs a predetermined process on the captured image, and applies the mask image to the captured image. The image processing apparatus according to claim 1, wherein the image processing apparatus generates the combined image by combining the first processed image and the mask image.   The correction unit creates a second processed image obtained by performing a second process on the captured image, creates a mask image by performing a predetermined process on the second processed image, A third processed image obtained by performing a third process different from the process in the photographed image is created, and the synthesized image is created by synthesizing the third processed image and the mask image. The image processing apparatus according to 1.   The correction unit includes a first processed image obtained by performing a first process on the photographed image and a second process obtained by performing a second process different from the first process on the photographed image. An image is created, the mask image is created by performing a predetermined process on the second processed image, and the synthesized image is created by synthesizing the first processed image and the mask image. An image processing apparatus according to 1.   The first process is a process of acquiring color information that is likely to cause the aberration based on design data of the imaging lens and reducing a value of the color information that is likely to cause the aberration included in the captured image. Item 5. The image processing apparatus according to Item 2 or 4.   The third process is a process of acquiring color information that is likely to cause the aberration based on design data of the imaging lens and reducing the saturation of the color included in the color information that is likely to cause the aberration in the captured image. The image processing apparatus according to claim 3.   The second process is a process of acquiring color information that is likely to cause the aberration based on design data of the imaging lens and enhancing the color information that is likely to cause the aberration included in the captured image. Or the image processing apparatus of 4.   An imaging apparatus comprising: the image processing apparatus according to claim 1; and an imaging element that captures the captured image. Creating one or more processed images obtained by applying different processing to the value of color information included in the captured image captured through the imaging lens using design data of the imaging lens;
Creating a mask image by performing a predetermined process on the value of color information included in any one of the photographed image and the one or more processed images;
A color information value included in any one of the photographed image and the one or more processed images not used when creating the mask image and a color information value included in the mask image are combined and combined. Creating an image;
And a step of synthesizing the synthesized image with the photographed image to create a corrected image. Creating one or more processed images obtained by applying different processing to the value of color information included in the captured image captured through the imaging lens using design data of the imaging lens;
Creating a mask image by performing a predetermined process on the value of color information included in any one of the photographed image and the one or more processed images;
A color information value included in any one of the photographed image and the one or more processed images not used when creating the mask image and a color information value included in the mask image are combined and combined. Creating an image;
And a step of synthesizing the synthesized image with the photographed image to create a corrected image.