JP4804387B2 - Image processing apparatus, imaging apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing technique for performing processing for removing a ghost image from an image acquired by an imaging apparatus.

  In an imaging apparatus such as a digital still camera or a video camera, a part of light incident on the imaging optical system may be reflected in the imaging optical system and may be incident on the imaging surface as light that does not have sufficient image-forming properties. Such light forms an image different from the original subject image formed by light having sufficient image-forming properties (hereinafter referred to as imaging light), that is, a ghost image. As a result, the image quality of the image obtained by the imaging device is reduced.

  A ghost image is an image that is formed by light that does not have sufficient image-forming properties (hereinafter referred to as non-imaging light) and that can be visually regarded as being imaged on the image plane.

  Conventionally, as disclosed in Patent Document 1, light that forms a ghost image optically is excluded. Thereby, it is possible to obtain a high-quality image in which only the subject image formed by the imaging light appears.

On the other hand, recently, it has become possible to analyze a ghost image or a flare generation region and its shape by ray tracing simulation, and Patent Documents 2 and 3 disclose a technique for removing flare by an image processing technique using the result. It is disclosed. Flare means a state in which the non-imaging light described above does not form an image.
JP 2003-241130 A Japanese Patent No. 3372209 JP-A-9-238357

  In the ghost removal method using the optical system disclosed in Patent Document 1, there is a possibility that the imaging apparatus may be restricted in shape due to physical limitations in the optical system, and the imaging apparatus may be reduced in size and design freedom.

  Moreover, although the image processing techniques disclosed in Patent Documents 2 and 3 can reduce the influence of flare on the image, they have not yet removed the ghost image.

  The present invention provides an image processing apparatus, an imaging apparatus, and an image processing program capable of removing a ghost image from an image obtained by imaging using an image processing technique.

    An image processing apparatus according to an aspect of the present invention corresponds to a difference between a first original image and a second original image obtained by imaging with different F values from the first original image. A difference image generation unit that generates a second difference image corresponding to a difference between the first difference image and the second original image with respect to the first original image, and the second difference image, the second original image is used. A ghost region specifying unit that specifies a ghost region in which a ghost image exists in the image, a binary luminance image obtained by binarizing the luminance of the first difference image, a second original image, and data for specifying the ghost region are used. And a ghost correction processing unit that generates a corrected image corresponding to an image obtained by removing the ghost image from the second original image.

  An image processing apparatus according to another aspect of the present invention provides a difference between a first original image and a second original image obtained from first and second original images obtained by imaging with different F values. A difference image generation unit that generates a first difference image corresponding to the first difference image and a second difference image corresponding to a difference of the second original image with respect to the first original image, and a color of the first and second original images A saturation image generation unit that generates first and second saturation binarized images in which the degree is binarized, a first binarized luminance image that binarizes the first difference image, and the first A second binarized luminance image obtained by generating a first modified binarized luminance image that is a logical product image of the chroma binarized image and binarizing the second difference image; A second modified binarized luminance image that is a logical product image of the saturation binarized image is generated, and the second original binarized image is used in the second original image using the second modified binarized luminance image. A ghost image is generated from the second original image using a ghost region specifying unit that specifies a ghost region in which the ghost image exists, a first modified binary luminance image, a second original image, and data specifying the ghost region. And a ghost correction processing unit that generates a corrected image corresponding to the image from which the image is removed.

  Note that an imaging apparatus including an imaging system that photoelectrically converts a subject image formed by the imaging optical system to generate first and second original images and the image processing apparatus also constitutes another aspect of the present invention. .

  Furthermore, an image processing program according to another aspect of the present invention provides a computer with a second original of a first original image obtained from first and second original images obtained by imaging with different F values. A difference image generation step for generating a first difference image corresponding to a difference with respect to the image and a second difference image corresponding to a difference of the second original image with respect to the first original image, and a second difference image are used. A ghost region specifying step for specifying a ghost region in which a ghost image exists in the second original image, a binary luminance image obtained by binarizing the luminance of the first difference image, the second original image, and the ghost region And a ghost correction processing step of generating a corrected image corresponding to an image obtained by removing the ghost image from the second original image, using the data for specifying the image.

  An image processing program according to another aspect of the present invention provides a second original of a first original image from a first original image and a second original image obtained by imaging with different F values. A difference image generation step of generating a first difference image corresponding to a difference with respect to the image and a second difference image corresponding to a difference of the second original image with respect to the first original image; and first and second original images A saturation image generation step for generating first and second saturation binarized images in which the saturation of the image is binarized; a first binarized luminance image in which the first difference image is binarized; A second binarized luminance image obtained by generating a first corrected binarized luminance image that is a logical product image of the first chroma binarized image and binarizing the second difference image; A second modified binarized luminance image that is a logical product image of the second chroma binarized image is generated, and the second modified binarized luminance image is generated. A ghost region specifying step for specifying a ghost region in which a ghost image is present in the second original image using the image, and a first modified binary luminance image, a second original image, and data for specifying the ghost region are used. And a ghost correction processing step for generating a corrected image corresponding to an image obtained by removing the ghost image from the second original image.

  In general, as the F value increases, the spatial distribution of the ghost image becomes narrower and the peak intensity increases. For this reason, the present invention uses the change of the ghost image between the original images having different F values to identify the ghost image region by an image processing technique, and removes the ghost image from the original image. ADVANTAGE OF THE INVENTION According to this invention, a ghost image can be removed from an original image using an image processing technique, and the image processing apparatus, imaging device, and image processing program which can provide a quality image can be implement | achieved.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 shows a configuration of a digital still camera (hereinafter simply referred to as a camera) as an image pickup apparatus that is Embodiment 1 of the present invention. In this embodiment, a lens-integrated camera will be described. However, the present invention can also be applied to a lens-interchangeable camera, a lens-integrated camera, or a lens-interchangeable video camera.

  The camera has an imaging system 1 including an imaging optical system 2 including a plurality of lenses 2a and an aperture 2b and an imaging element 3 such as a CCD sensor or a CMOS sensor. The camera also includes an electric signal processing unit 4 including a digital signal processing unit 5, an image deterioration correction processing unit 6, and an image processing unit 7. Furthermore, it has a controller 9 such as a CPU for driving and controlling these components, and a memory unit 10 for storing image data and computer programs. The output image memory 8 includes a semiconductor memory that can be attached to and detached from the camera, an optical disk, and the like.

  Hereinafter, the operation of the camera and the flow of data in the camera will be described.

  Most of the light that has passed through the imaging optical system 2 forms an image on the light receiving surface (imaging surface or image surface) of the image sensor 3. The subject image formed on the image sensor 3 is photoelectrically converted by the image sensor 3 and output as an electrical signal (analog image signal). The analog image pickup signal output from the image pickup device 3 is transferred to the digital signal processing unit 5 in the electric signal processing unit 4.

  The digital signal processing unit 5 converts the transferred analog imaging signal into digital data in response to an instruction from the controller 9. Furthermore, an original image which is all pixel image data for each color of R, G, and B is generated by pixel interpolation processing.

  In the camera of the present embodiment, the aperture diameter of the diaphragm 2b in the imaging optical system 2 is changed by a signal from the controller 9, and the same subject is subjected to two different F values for the same subject in the same imaging environment. Imaging is performed to generate two original images (first and second original images). At this time, the controller 9 changes the exposure time (imaging time) of the image pickup device 3 so that the exposure amount of the image pickup device 3 at the time of two image pickups becomes the same. Specifically, when the F value of the imaging optical system 2 is set to n times, the exposure time is set to be a square of n. Since a general digital still camera has a function of automatically controlling the exposure time so that the exposure amount becomes constant even if the F value changes, the camera of this embodiment also uses this function.

  Note that the same (or constant) exposure amount includes not only completely the same amount but also a small difference that can be regarded as the same.

  Of the two generated original images, an original image (second original image: hereinafter referred to as a large F value original image) acquired in a state where the aperture diameter of the diaphragm 2b is small (F value is large) is finally obtained. It is stored in the memory unit 10 as a base image of the corrected image to be output. Also, an original image (first original image: hereinafter referred to as a small F value original image) acquired in a state where the aperture diameter of the aperture stop 2b is larger than when the large F value original image is acquired (F value is small) is also stored in the memory unit. 10 is stored.

  The large F value original image and the small F value original image are further transferred to the image deterioration correction processing unit 6. The image deterioration correction processing unit 6 performs the processing described below on the two original images in response to an instruction from the controller 9.

  As shown in FIG. 3, the image deterioration correction processing unit 6 includes a difference image data creation processing unit (difference image generation unit) 6a, a ghost region specifying unit 6b, and a ghost correction processing unit 6c. In the following description, a portion of an original image where a ghost image formed by non-imaging light exists is referred to as a ghost region, and a portion of a subject image formed by imaging light is referred to as a background image region.

  Here, general properties of a ghost image by the optical system will be described. As shown in FIGS. 1A and 1B, a part of the light incident on the optical system is internally reflected by the lens a plurality of times and does not have sufficient image quality, in other words, the image quality is lower than the image light. It reaches the light receiving surface of the image sensor as non-imaging light (also referred to as ghost light). Thereby, a ghost image that can be regarded as visually formed on the light receiving surface is formed.

  Under the condition that the exposure amount is constant, the brightness (intensity) of the background image does not change even if the aperture (F value) is changed. However, since the light that forms the ghost image does not have sufficient image-forming properties, the distribution range on the image plane of the ghost image decreases as the F value increases, as can be seen from the comparison between FIGS. 1A and 1B. On the other hand, since the exposure time becomes longer, the strength increases. In addition, the center of the ghost image may be considered to be substantially unchanged since the non-imaging light is only spatially limited by the diaphragm even if the F value changes. That is, the ghost image has a spatial distribution that decreases concentrically and increases in intensity as the F value increases. In addition, the ghost image has a greater intensity than the background image region regardless of the F value.

  In this embodiment, a ghost image is present in an area where luminance remains in a difference image corresponding to a difference between two original images having different F values by using such a change of the ghost image according to the F value. Specified as a ghost region.

  The difference image data creation processing unit 6a generates two difference images as image data corresponding to the difference between two original images having different F values.

  Further, the ghost area specifying unit 6b performs a process of specifying a ghost area from one difference image.

  Further, the ghost correction processing unit 6c performs a process of generating a corrected image corresponding to an image obtained by removing the ghost image in the specified ghost region from the large F-number original image as the base image. Note that the removal described here includes not only the case where the image is completely eliminated, but also the case where an afterimage is formed to such an extent that the ghost image can be regarded as absent (not noticeable) when the image is observed.

  Hereinafter, the processing in the image deterioration correction processing unit 6 will be described with reference to the flowchart of FIG. 4 and FIGS. The image deterioration correction processing unit 6 as an image processing microcomputer executes this processing according to an image processing program stored therein. In the following description, it is assumed that there is one ghost image included in the original image. The same applies to Example 2 described later. The following processing is performed for each color image of RGB.

  As described above, the two original images generated by the digital signal processing unit 5 are transferred to the difference image data creation processing unit 6a.

  In step M101, the difference image data creation processing unit 6a generates a difference image corresponding to the luminance difference between the two original images, as shown in FIG. Specifically, a difference image (first image) is obtained by subtracting the luminance value of the corresponding pixel of the large F-value original image, which is all pixel image data, from the luminance value of each pixel of the small F-value original image, which is all pixel image data. 1 difference image) a is generated. Similarly, a difference image (second difference image) b is generated by subtracting the luminance value of the corresponding pixel of the small F value original image from the luminance value of each pixel of the large F value original image.

  The difference image a is an image corresponding to the difference between the small F value original image and the large F value original image, and the difference image b is an image corresponding to the difference between the large F value original image and the small F value original image.

  Note that if the luminance difference between the corresponding pixels in the original image is negative, the luminance value of the corresponding pixel in the difference image is set to zero. The difference images a and b are stored in the memory unit 10 and also transferred to the ghost area specifying unit 6b.

  Since the background image portion in the original image does not change depending on the F value, it is not reflected in the difference image (that is, the luminance on the difference image is 0). In the difference image a, a finite luminance value remains in an area corresponding to the ghost area that existed in the small F value original image. However, among these ghost regions, the ghost region existing in the large F value original image has a larger image plane intensity than the ghost image in the small F value original image, and therefore the luminance difference becomes negative, and the difference image The luminance value on a is 0. As a result, the difference image a has a finite luminance value in a region obtained by subtracting the ghost region in the small F value original image from the ghost region in the large F value original image.

  On the other hand, in the difference image b, the pixels in the region corresponding to the ghost region in the large F value original image have a finite luminance value.

  Next, in step M102, the ghost area specifying unit 6b causes the memory unit 10 to store a set of data addresses (hereinafter referred to as ghost addresses) of pixels having a finite luminance value in the difference image as ghost areas. The ghost address is searched as shown in FIG.

  A pixel having a finite luminance value is searched from the entire difference image b, and the address of the pixel is set as a ghost address. In this way, a ghost region in the large F value original image is extracted and specified.

  Next, in Steps M103 to M106, a corrected image obtained by removing the ghost image from the small F-value original image is generated by the ghost correction processing unit 6c.

  First, in step M103, the pixel value of a pixel having no finite luminance value is set to 0 and the pixel value of a pixel having a finite luminance value is set to 1 for the entire difference image a stored in the memory unit 10. That is, the difference image a is binarized to obtain a difference converted image (binarized luminance image).

  Next, in step M104, as shown in FIG. 7, the product of the pixel value of each pixel of the difference converted image and the luminance value of the corresponding pixel of the large F value original image is obtained. Thus, only the area around the ghost area (corresponding to the area obtained by subtracting the ghost area of the large F-value original image from the ghost area of the small F-value original image) in the background image area of the large F-value original image has a finite luminance. A ghost peripheral background image having a value is generated.

  Next, in step M105, the entire ghost peripheral background image is searched, and an average value (luminance average value) of luminance values (pixel information) of pixels having a finite luminance value is calculated.

  Then, while referring to the data of the ghost address specified in step M102, the luminance values of all the pixels included in the ghost region of the large F value original image are replaced with the luminance average value. That is, processing is performed to replace the ghost region in the large F-number original image with an image corresponding to the background image around the ghost region. In this way, a corrected image corresponding to an image obtained by removing the ghost region (ghost image) from the large F value original image is generated.

  The corrected image is transferred to the image processing unit 7 and subjected to final image processing such as white balance adjustment, gray balance adjustment, density adjustment, color balance adjustment, and edge enhancement. The corrected image after the image processing is recorded in the output image memory 8 as an output image.

  As described above, in this embodiment, two difference images corresponding to a difference between two original images acquired by imaging with different F values are generated, and one of the difference images is used to generate one of the difference images. A ghost area (ghost address) in the original image is specified. Then, by correcting the luminance value of the pixel in the ghost region with the average luminance value of the range included in the background image region of the other original image, which is the peripheral region of the ghost region of the one original image, A corrected image corresponding to an image obtained by removing the ghost image from the original image is generated. At this time, an image obtained by binarizing one of the difference images is used to calculate a luminance value for ghost region correction for generating a corrected image.

  According to the present embodiment, it is possible to provide a user with a high-quality image (corrected image) from which a ghost image has been removed.

  In this embodiment, since the ghost area is corrected using the background image around the ghost area, a visually corrected image can be obtained. In this embodiment, since the average luminance value of the background image is used for correcting the ghost image, it is particularly effective when the luminance value of the background image region around the ghost image is not so different, such as a blue sky in fine weather.

  Furthermore, in this embodiment, simulation data regarding the position and shape of the ghost image is not required, and a large-capacity storage device for holding a large amount of simulation data can be eliminated.

  In addition, according to the present embodiment, since ghost removal (correction) processing is performed by image processing, the degree of freedom in designing the optical system is higher than when ghost removal is performed in the optical system, and the optical system is downsized. Also contributes.

  A digital still camera as an image pickup apparatus that is Embodiment 2 of the present invention will be described. The basic configuration of the imaging apparatus is the same as that described in the first embodiment with reference to FIG. For this reason, constituent elements common to the first embodiment are denoted by the same reference numerals as those of the first embodiment and are not described. The basic command and data flow in the image pickup apparatus are also the same as those in the first embodiment.

  However, in this embodiment, the configuration of the image deterioration correction processing unit 6 ′ is different from that of the image deterioration correction processing unit 6 of the first embodiment. That is, as shown in FIG. 8, in addition to the difference image data creation processing unit (difference image generation unit) 6a, the ghost region specifying unit 6b, and the ghost correction processing unit 6c, a hue conversion processing unit 6d and a high saturation region extraction processing unit ( A saturation image generation unit) 6e is provided. The hue conversion processing unit 6d and the high saturation region extraction processing unit 6e are provided in parallel with the difference image data creation processing unit 6a. In this embodiment, the saturation threshold for the ghost image is stored in the memory unit 10 in advance. The saturation threshold is an adjustable value.

  The hue conversion processing unit 6d separates the saturation image from the original image by hue conversion suitable for an image output system that allows the user to observe the output image through a monitor (not shown). In addition, since the ghost image generally has higher saturation than the background image, the high saturation region extraction processing unit 6e is provided in the present embodiment. An image region (high saturation image region) having a high saturation equal to or higher than the saturation threshold stored in the memory unit 10 is extracted from the saturation image separated from the original image. The ghost region specifying unit 6b discriminates a common region between the high saturation image region and the region having a finite luminance value in the difference image, so that the ghost region can be specified with higher accuracy than in the first embodiment.

  Hereinafter, the processing in the image deterioration correction processing unit 6 ′ in the present embodiment will be described with reference to the flowchart of FIG. 9 and FIGS. The processing of this embodiment is also performed for each color image of RGB.

Similar to the first embodiment, the two original images (large F value original image and small F value original image) generated by the digital signal processing unit 5 are sent to the difference image data creation processing unit 6a and the hue conversion processing unit 6d. Transferred. At this time, a set of the large F value original image and the small F value original image transferred to the difference image data creation processing unit 6a and the hue conversion processing unit 6d is referred to as an original image set.

  The large F value original image and the small F value original image are also stored in the memory unit 10.

  First, in step M201, the difference image data creation processing unit 6a uses the original image set, as in the first embodiment, from the luminance value of each pixel of the small F value original image to the luminance value of the corresponding pixel of the large F value original image. Is used to generate a differential image (first differential image) a. Further, the difference image (second difference image) b is generated by subtracting the luminance value of the corresponding pixel of the small F value original image from the luminance value of each pixel of the large F value original image.

  Also in this embodiment, when the luminance difference between mutually corresponding pixels in the original image becomes negative, the luminance value of the corresponding pixel in the difference image is set to 0.

  Next, in step M202, the difference image data creation processing unit 6a sets the pixel value of the pixel having no finite luminance value to 0 for the entire difference images a and b, and the pixel value of the pixel having the finite luminance value. Set to 1. That is, the difference images a and b are binarized to obtain difference converted images (first and second binarized luminance images) a ′ and b ′. These difference converted images a ′ and b ′ are transferred to the ghost area specifying unit 6b.

  On the other hand, in steps M203 to M205, the hue conversion processing unit 6d and the high saturation region extraction processing unit 6e perform high saturation image region extraction processing in parallel with the processing in steps M201 and M202 described above.

  First, in step M203, the hue conversion processing unit 6d performs HSL hue conversion processing on each of the large F-value original image and the small F-value original image in the original image set, as shown in FIG. Thereby, a large F value saturation image and a small F value saturation image as first and second saturation images acquired by imaging with different F values are obtained. The HSL hue conversion process is a process of normalizing color image data of 256 tones for each RGB to H (Hue) 255 stages, S (Saturation) and L (Luminosity: 256) stages. is there. The large F value saturation image and the small F value saturation image are transferred to the high saturation region extraction processing unit 6e.

  Next, in step M204, the high saturation area extraction processing unit 6e determines whether or not the saturation of each pixel is greater than or equal to the saturation threshold stored in the memory unit 10 for each of the two saturation images. To do.

  In step M205, as shown in FIG. 10, the high saturation area extraction processing unit 6e sets the pixel value to 1 for a pixel whose saturation is equal to or higher than the saturation threshold in the saturation image, and sets the pixel value equal to or lower than the saturation threshold. On the other hand, the pixel value is set to 0. As a result, a small F-value saturation binarized image (first chroma binarized image) and a large F-value chroma binary as first and second chroma binarized images having different F values from each other. A digitized image (second chroma binarized image) is generated. Note that the small F value saturation binarized image and the large F value saturation binarized image correspond to images obtained by binarizing the chroma of each of the small F value original image and the large F value original image. The large F value saturation binarized image and the small F value saturation binarized image are transferred to the ghost region specifying unit 6b.

  In step M206, the ghost area specifying unit 6b performs the two difference conversion images a ′ and b ′ generated in step M202, the small F-value saturation binary image generated in step M205, and the large F-value saturation 2 A process for specifying a ghost region is performed using the binarized image.

  Specifically, as shown in FIG. 11, first, a logical product image a ″ (first modified binary luminance image) is obtained from the logical product of the difference conversion image a ′ and the small F-value saturation binary image. Similarly, a logical product image b ″ (second modified binary luminance image) is obtained from the logical product of the difference conversion image b ′ and the large F-value saturation binary image. The logical product image a ″ is stored in the memory unit 10 to be used in a subsequent ghost correction process.

Next, in step M207, the ghost area specifying unit 6b specifies a set of ghost addresses corresponding to pixels having a pixel value of 1 in the logical product image b ″ as a ghost area in the large F value original image, and stores it in the memory unit 10. Remember me.

  Next, in step M208, the ghost correction processing unit 6c performs a ghost correction process similar to the process described with reference to FIG. 7 in the first embodiment on the large F value original image stored in the memory unit 10. That is, the product of the pixel value of each pixel of the logical product image a ″ stored in the memory unit 10 and the luminance value of the corresponding pixel of the large F value original image is taken, and the ghost periphery similar to that of the first embodiment (FIG. 7) is obtained. The background image of the ghost is obtained by subtracting the ghost area of the large F-number original image from the area around the ghost area in the background image area of the large F-number original image. Only an image having a finite luminance value.

  Next, in step M209, the entire ghost peripheral background image is searched, and an average value (luminance average value) of luminance values (pixel information) of pixels having a finite luminance value is calculated. Then, while referring to the data of the ghost address specified in step M207, the luminance values of all the pixels included in the ghost area in the large F value original image are replaced with the luminance average value. That is, processing is performed to replace the ghost region in the large F-number original image with an image corresponding to the background image around the ghost region. In this way, a corrected image corresponding to an image obtained by removing the ghost region (ghost image) from the large F value original image is generated.

  The corrected image is transferred to the image processing unit 7 and subjected to final image processing such as white balance adjustment, gray balance adjustment, density adjustment, color balance adjustment, and edge enhancement. The corrected image after the image processing is recorded in the output image memory 8 as an output image.

  As described above, in this embodiment, two difference images corresponding to a difference between two original images acquired with mutually different F values are generated, and two differences obtained by binarizing the two difference images are generated. A common area between the converted image and the saturation image obtained by extracting the high saturation areas of the two original images is obtained. Then, from this result, the ghost area (ghost address) in one original image is specified, and at the same time, the luminance of the range included in the background image area of the other original image that is the peripheral area of the ghost area of the one original image Find the average value. Thereafter, by correcting the luminance value of the pixel in the ghost region from the obtained luminance average value, a corrected image corresponding to an image obtained by removing the ghost image from the other original image is generated.

  According to the present embodiment, the same effects as in the first embodiment can be obtained, such as providing a user with a high-quality image (corrected image) from which a ghost image has been removed.

  In this embodiment, the high saturation region of the original image is extracted by hue conversion processing. The HSL hue conversion process is a conversion that is generally used when an image output system for evaluating an image is assumed as a display device, and the saturation of image data can be extracted independently. According to the present embodiment, the ghost image can be removed with higher accuracy than in the first embodiment.

  Note that a conversion method other than HSL hue conversion may be used as long as the hue conversion can appropriately perform image evaluation in the image output system.

  In each of the above-described embodiments, the case where the image processing apparatus (image deterioration correction processing units 6 and 6 ′) is built in the imaging apparatus has been described, but the present invention is not limited to this.

  For example, as shown in FIG. 12, two images (original images) captured by the imaging device 1401 with different F values are transmitted to the personal computer 1402. The transmission method may be either a cable method or a wireless method, and may be transmitted via the Internet or a LAN.

  Then, the personal computer 1402 may perform the image processing shown in the flowchart of FIG. 4 or FIG. In this case, the personal computer functions as an image processing apparatus.

The figure which represented the characteristic of the ghost image typically. The figure which represented the characteristic of the ghost image typically. 1 is a block diagram illustrating a configuration of an imaging apparatus that is Embodiment 1 of the present invention. FIG. 3 is a block diagram illustrating a configuration of an image deterioration correction processing unit according to the first embodiment. 7 is a flowchart illustrating the operation of the image deterioration correction processing unit according to the first embodiment. FIG. 3 is a conceptual diagram illustrating a difference image creation process according to the first embodiment. FIG. 3 is a conceptual diagram illustrating a ghost address specifying process according to the first embodiment. FIG. 3 is a conceptual diagram illustrating ghost correction processing according to the first embodiment. FIG. 6 is a block diagram illustrating a configuration of an image deterioration correction processing unit in an imaging apparatus that is Embodiment 2 of the present invention. 9 is a flowchart illustrating the operation of an image deterioration correction processing unit according to the second embodiment. FIG. 10 is a conceptual diagram illustrating hue conversion processing and high saturation region extraction processing according to the second embodiment. FIG. 9 is a conceptual diagram illustrating a process for creating a composite binarized image in the second embodiment. Schematic which shows the image processing apparatus which is Example 3 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up system 2 Image pick-up optical system 3 Image pick-up element 4 Electric signal processing part 5 Digital signal processing part 6, 6 'Image degradation correction process part 6a Difference image creation process part 6b Ghost area specific | specification part 6c Ghost correction process part 6d Hue conversion process part 6e High saturation region extraction processing unit 7 Image processing unit 8 Output image memory 9 Controller 10 Memory unit

Claims (8)

A first difference image corresponding to a difference between the first original image and the second original image obtained by imaging with different F values, and the second difference A difference image generation unit that generates a second difference image corresponding to a difference of the original image with respect to the first original image;
Using the second difference image, a ghost region specifying unit for specifying a ghost region in which a ghost image in the second original image exists;
The ghost image is removed from the second original image using the binarized luminance image obtained by binarizing the luminance of the first difference image, the second original image, and the data specifying the ghost region. An image processing apparatus comprising: a ghost correction processing unit that generates a corrected image corresponding to an image. A first difference image corresponding to a difference between the first original image and the second original image obtained by imaging with different F values, and the second difference A difference image generation unit that generates a second difference image corresponding to a difference of the original image with respect to the first original image;
A saturation image generation unit that generates first and second saturation binarized images obtained by binarizing the saturation of the first and second original images;
Generating a first modified binarized luminance image that is a logical product image of the first binarized luminance image obtained by binarizing the first difference image and the first chroma binarized image; And generating a second modified binarized luminance image that is a logical product image of the second binarized luminance image obtained by binarizing the second difference image and the second chroma binarized image. A ghost region specifying unit that specifies a ghost region in which the ghost image in the second original image exists using the second modified binarized luminance image;
A corrected image corresponding to an image obtained by removing the ghost image from the second original image using the first modified binarized luminance image, the second original image, and the data specifying the ghost region. An image processing apparatus comprising: a ghost correction processing unit to be generated.   3. The image processing apparatus according to claim 1, wherein the first original image is an image having a smaller F value at the time of imaging than the second original image.   The ghost correction processing unit performs a process for removing the ghost region by using pixel information of a region around the ghost region in the second original image. The image processing apparatus according to claim 1. An imaging system that photoelectrically converts a subject image formed by an imaging optical system to generate the first and second original images;
An imaging apparatus comprising: the image processing apparatus according to claim 1.   6. The imaging according to claim 5, wherein the imaging system varies the imaging time so that the F value is different and the exposure amount is the same when imaging the first and second original images. apparatus. On the computer,
A first difference image corresponding to a difference between the first original image and the second original image obtained by imaging with different F values, and the second difference A difference image generation step of generating a second difference image corresponding to a difference of the original image with respect to the first original image;
A ghost area specifying unit step for specifying a ghost area in which the ghost image in the second original image exists using the second difference image;
A binarized luminance image obtained by binarizing the luminance of the first difference image, the second original image,
A ghost correction processing step for generating a correction image corresponding to an image obtained by removing the ghost image from the second original image, using data specifying the ghost region;
An image processing program for executing image processing including: On the computer,
A first difference image corresponding to a difference between the first original image and the second original image obtained by imaging with different F values, and the second difference A difference image generation step of generating a second difference image corresponding to a difference of the original image with respect to the first original image;
A saturation image generation step of generating first and second saturation binarized images obtained by binarizing the saturation of the first and second original images;
Generating a first modified binarized luminance image that is a logical product image of the first binarized luminance image obtained by binarizing the first difference image and the first chroma binarized image; And generating a second modified binarized luminance image that is a logical product image of the second binarized luminance image obtained by binarizing the second difference image and the second chroma binarized image. A ghost region specifying step for specifying a ghost region in which the ghost image in the second original image exists using the second modified binarized luminance image;
A corrected image corresponding to an image obtained by removing the ghost image from the second original image using the first modified binarized luminance image, the second original image, and the data specifying the ghost region. A ghost correction processing step to be generated;
An image processing program for executing image processing including: