JP2015192338A - Image processing device and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce an influence of noise when a photographic image is corrected.SOLUTION: An image processing device includes: image acquisition means of acquiring a photographic image; first map generation means of generating a first map representing a distribution of degrees of low saturation and high lightness of the photographic image using saturation information associated with the photographic image and lightness information associated with lightness of the photographic image; and correction means of generating a correction image by correcting the photographic image using the first map.

Description

  The present invention relates to an image processing apparatus and an image processing program.

  When photographing outdoors, the photographed image may become whitish due to fog or haze. Therefore, a method for removing fog and haze from a photographed image is known (see Patent Document 1). In the method described in Patent Document 1, a dark channel image is generated from a captured image, and an image from which fog has been removed is generated using the dark channel image and the fog image model. Here, the dark channel image is an image composed of the minimum values of the intensity values of the RGB channels in each pixel of the captured image. Since the intensity of the dark channel image is higher in an image that is white and blurred by fog than in a clear image without fog, the dark channel image represents an approximate fog density.

JP2013-58202A

  In the dark channel image used in the above-described conventional technology, since the minimum value of the RGB channel intensity value is used, a dark level may be used, which may be greatly affected by noise.

(1) The image processing apparatus according to the first aspect uses the image acquisition means for acquiring the captured image, the saturation information regarding the saturation of the captured image, and the brightness information regarding the brightness of the captured image, and thereby the saturation of the captured image. Comprising: a first map creating means for creating a first map representing a distribution with a low brightness and a high brightness; and a correcting means for creating a corrected image by correcting a captured image using the first map. And
(2) The image processing program according to claim 9 uses a process for obtaining a captured image, saturation information regarding the saturation of the captured image, and brightness information regarding the brightness of the captured image, so that the saturation of the captured image is low. In addition, the processing is characterized in that the computer executes a process for creating a first map representing a distribution with a high degree of lightness and a process for creating a corrected image by correcting a captured image using the first map.
(3) The image processing apparatus according to claim 10 is an image acquisition unit that acquires a photographed image obtained by photographing a subject via a light-transmitting medium that exists in front of the subject, and saturation related to saturation of the photographed image. First map creating means for creating a first map representing a distribution of a degree of low saturation and high brightness in the photographed image using the information and the brightness information on the brightness of the photographed image, and the photographed image using the first map Correction means for generating a corrected image in which the influence of the light-transmitting medium is reduced by correcting the above.

  According to the present invention, it is possible to reduce the influence of noise when correcting a captured image.

It is a block diagram which shows the structural example of a digital camera. It is a schematic diagram explaining a fog image model. It is a flowchart explaining the flow of an image correction process. (A) is a schematic diagram of an example of a captured image, (b) is a schematic diagram of an example of a white intensity map, and (c) is a schematic diagram of an example of a transmittance map. It is a figure explaining the mode of provision of a program.

-First embodiment-
A first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of a digital camera equipped with an image processing apparatus according to the present embodiment. The digital camera 100 includes an operation member 101, a photographing lens 102, an image sensor 103, a control device 104, a memory card slot 105, and a monitor 106. The operation member 101 includes various input members operated by the user, for example, a power switch, a release button, a multi selector, a play button, a delete button, and the like.

  The photographing lens 102 is disposed so as to form a subject image on the imaging surface of the image sensor 103. The photographic lens 102 is represented by a single lens in FIG. 1 as a representative, but actually includes a plurality of optical lenses.

  The image sensor 103 is an image sensor such as a CMOS, for example, captures a subject image formed by the photographing lens 102, and outputs an image signal obtained by the imaging to the control device 104. A well-known color filter is provided on the light receiving surface of the image sensor 103. The color filter is a color separation filter in which primary color filters that pass any one of red (R), blue (B), and green (G) light are configured in a Bayer arrangement corresponding to pixel positions. . The imaging element 103 outputs a color image signal for each of the three primary colors of light by capturing a subject image through such a color filter. An image signal generated by the image sensor 103 is converted into a digital signal by an A / D converter (not shown) and output to the control device 104.

  The control device 104 includes a CPU, a memory, and other peripheral circuits, and controls the entire digital camera 100. Note that the memory constituting the control device 104 includes SDRAM and flash memory. The SDRAM is a volatile memory, and is used as a work memory for developing a program when the CPU executes a program, or as a buffer memory for temporarily recording data. The flash memory is a non-volatile memory in which data of a program executed by the control device 104, various parameters read during program execution, and the like are recorded.

  The memory card slot 105 is a slot for inserting a recording medium such as a memory card. The control device 104 writes and records the image file generated by performing the photographing process on the memory card inserted in the memory card slot 105. Further, the control device 104 reads an image file recorded in a memory card inserted in the memory card slot 105.

  The monitor 106 is a liquid crystal monitor (rear monitor) mounted on the back of the digital camera 100. The monitor 106 displays a reproduction image based on the image file recorded on the memory card, a setting menu for setting the digital camera 100, and the like. When the user operates the operation member 101 to set the digital camera 100 to the shooting mode, the control device 104 outputs image data for display of images acquired in time series from the image sensor 103 to the monitor 106. As a result, the live view image is displayed on the monitor 106.

  By the way, when shooting outdoors, subject light is scattered by airborne objects (for example, fog, haze, smoke, dust, etc.) existing between the subject and the digital camera 100, and the photographed image is blurred in white. May end up. Therefore, the digital camera 100 according to the present embodiment performs image correction processing for correcting such a whitish and blurred photographed image. Hereinafter, this image correction process will be described.

<Fog image model>
First, a model (fog image model) of a captured image (hereinafter referred to as a fog image) affected by the fog Fg will be described with reference to the schematic diagram of FIG. The fog image is considered to be a combination of direct light (subject light) from the subject (object) Tg and surrounding ambient light A (light from the light source Ls). Therefore, the fog image model can be expressed by the following equation (1).
I (x) = J (x) .t (x) + A. (1-t (x)) (1)

  In Expression (1), x indicates a pixel position. I (x) indicates the pixel value of each pixel of the captured image (fog image) captured by the digital camera 100. J (x) represents subject light at each pixel position, and represents the pixel value of each pixel of an image obtained by reducing the influence of the fog Fg from the fog image I (x), that is, a clear image of the subject Tg. A represents ambient light and is the same at all pixel positions. t (x) indicates the degree (transmittance) at which the subject light J (x) reaches the digital camera 100 through the atmosphere including the fog Fg (that is, not scattered by the fog Fg) at each pixel position. .

  As represented by Expression (1), the fog image I (x) includes a component J (x) · t (x) related to the subject light J (x) and a component A · (1-t (x) related to the ambient light A. )). J (x) · t (x) represents subject light that has reached the digital camera 100 without being scattered by the fog Fg. A · (1-t (x)) represents the degree of influence of ambient light on the captured image (fog image).

Further, when the formula (1) is modified, the following formula (2) is obtained.
J (x) = (I (x) −A) / t (x) + A (2)

  According to Expression (2), by obtaining the transmittance t (x) and the ambient light A, an image in which the influence of the fog Fg is reduced from the fog image (captured image) I (x), that is, a clear image J (X) can be obtained. In the present embodiment, image correction processing using such a fog image model is performed.

<Image correction processing>
Next, the flow of image correction processing of the present embodiment will be described using the flowchart shown in FIG. When execution of image correction processing is instructed by the user via the operation member 101, the control device 104 starts a program for performing image correction processing stored in a memory (not shown) and starts the image correction processing.

(Acquisition of captured images)
In step S1, the control device 104 acquires a captured image to be subjected to image correction processing, and proceeds to step S2. Here, it is assumed that the color space of the captured image is an RGB color space represented by three components of R (red), G (green), and B (blue). Note that the captured image to be subjected to the image correction processing may be read from a memory card, for example, or acquired from an external device by communication.

(Create white intensity map)
In step S2, the control device 104 creates a white intensity map that represents a distribution of a degree of low saturation and high brightness, that is, a whitish degree (hereinafter referred to as white intensity) in the captured image acquired in step S1. . In order to correct a photographed image that is whitish and blurry due to fog or haze, the photographed image can be made clear (clear) by reducing the whiteness of the whitish region of the photographed image and increasing the saturation. Based on this, in the image correction processing of the present embodiment, a white intensity map representing the distribution of the whitish degree (white intensity) of the photographed image is created, and the photographed image is corrected using this.

  In order to create a white intensity map of a photographed image, first, the control device 104 changes the color space of the photographed image acquired in step S1 from the RGB color space to H (hue), S (saturation), and V (lightness). ) To the HSV color space represented by the three components.

  Note that there are a cylindrical model and a conical model as a method for expressing the HSV color space. In this embodiment, a conical model is used. In the HSV color space of the cone model, the hue H varies along the outer circumference of the cone and is represented by an angle along the color circle where the hue is shown. Here, the hue H is expressed in a range of 0 to 360. The saturation S changes with the distance from the center of the cone. Here, it is assumed that the saturation S is expressed in the range of 0.0 to 1.0. The lightness V varies along the height direction of the cone. Here, it is assumed that the brightness V is expressed in the range of 0.0 to 1.0. A predetermined color is determined by a combination of these three components (H, S, V).

As a formula for converting the color space from the RGB color space to the HSV color space of the conical model, a known technique may be used. When the RGB value of the photographed image is normalized by setting the minimum value to 0.0 and the maximum value to 1.0, the color of the cone model in the HSV color space is calculated from the RGB values of the photographed image by the following equations (3) and (4). The degree S and the brightness V can be obtained. In the following expression, max (R, G, B) indicates the maximum value among the RGB values of each pixel, and min (R, G, B) indicates the minimum value among the RGB values of each pixel. .
S = max (R, G, B) −min (R, G, B) (3)
V = max (R, G, B) (4)

In this way, the control device 104 converts the color space of the captured image acquired in step S1 into the HSV color space of the conical model. Then, the control device 104 uses the S (saturation) and V (brightness) of the photographed image converted into the HSV color space, and the white intensity (that is, the low saturation and the high brightness) according to the following equation (5). ) To create a white intensity map representing the distribution of
W (x) = (1-S (x)). V (x) (5)
However,
x: Pixel position W (x): White intensity at each pixel position S (x): Saturation at each pixel position V (x): Lightness at each pixel position

Note that when the formulas (3) and (4) are substituted into the formula (5), the white intensity W (x) at each pixel position is expressed by the following formula (6).
W (x) = {1− (max (R, G, B) −min (R, G, B))} · max (R, G, B) = (1−max (R, G, B) + min (R, G, B)) · max (R, G, B) (6)

  As can be seen from Equation (5), the white intensity becomes higher as the saturation is lower or the brightness is higher, and the white intensity is lower as the saturation is higher or the brightness is lower. Here, the white intensity map will be described using the captured image of FIG. 4 as an example. FIG. 4A is a schematic diagram illustrating an example of a photographed image photographed outdoors where fog is generated. In the photographed image of FIG. 4A, a person 51 who is a foreground and a mountain 52 and a sky 53 which are a distant view are shown. In the captured image of FIG. 4A, it is assumed that the mountain 52 and the sky 53 which are distant views are reflected in whitish blur due to the influence of the fog, and the person 51 which is a close view is reflected relatively clearly with little influence of the fog. Suppose that FIG. 4B is a schematic diagram showing an example of a white intensity map in the photographed image of FIG. For example, in the white intensity map of FIG. 4B, the mountain area 62 and the sky area 63 that are greatly affected by fog are whitish, so the white intensity is high, and the person area 61 that is less affected by fog is clearly visible. Therefore, the white strength is lowered.

(Acquisition of ambient light color information)
In step S3 (FIG. 3), the control device 104 acquires color information of the ambient light in the captured image. Specifically, in the white intensity map created in step S2, the control device 104 sets the highest 0.1% of the white intensity in the photographed image as the top 0.1% of the entire area when the white intensity at each pixel position is arranged in descending order. Extract as a region. Then, the control device 104 acquires the color information (HSV) of the pixel position having the highest brightness V as the color information of the ambient light in the area of the captured image corresponding to the extracted white intensity maximum area. That is, the control device 104 acquires color information of a position where white intensity and brightness are high in the captured image as color information of ambient light.

  For example, in the case of the captured image of FIG. 4, in the sky region 63 with high white intensity, the brightness V is the highest at the position of the sun 54 that is the light source of the ambient light. Can be obtained as

  Since the maximum white intensity region may include not only a bright part of the sky but also a white object, the image correction of this embodiment is performed so that color information of a bright part of the sky can be acquired instead of a white object. In the processing, the color information of the pixel position with the highest brightness V in the region with high white intensity is acquired.

(Transparency map creation)
In step S4 (FIG. 3), the control device 104 uses the white intensity map created in step S2 and the brightness of the ambient light color information (HSV) acquired in step S3 to describe the fog image model described above. A transmittance map representing the distribution of transmittance t (x) corresponding to is created. Specifically, the control apparatus 104 calculates | requires the transmittance | permeability t (x) using the following formula | equation (7).
t (x) = 1−ω · W (x) / Av (7)
However,
x: pixel position t (x): transmittance at each pixel position ω: constant W (x): white intensity at each pixel position Av: brightness of ambient light

  As can be seen from Equation (7), the higher the white intensity W (x), the lower the transmittance t (x), and the lower the white intensity W (x), the higher the transmittance t (x). FIG. 4C is a schematic diagram illustrating an example of a transmittance map in the captured image of FIG. For example, in the transmittance map of FIG. 4C, the mountain region 72 and the sky region 73 where the white intensity W (x) is high have a low transmittance t (x) and the person region 71 where the white intensity W (x) is low. Then, the transmittance t (x) becomes low.

  Here, when the transmittance map having the same resolution as the original photographed image is used as it is and an image (corrected image) in which the influence of fog is reduced by the above-described fog image model is obtained, a ring-shaped ghost appears in the corrected image. A halo effect may occur. Therefore, in the image correction processing of the present embodiment, a transmittance map is created using the following first creation method or second creation method in order to prevent the halo effect.

(1) First Creation Method In the first creation method, first, the control device 104 creates a transmittance map having the same resolution as the captured image. Then, the control device 104 uses the created transmittance map to create one or more transmittance maps having resolutions lower than that of the captured image (that is, create a multi-resolution transmittance map). The control device 104 weights and adds the created transmittance maps to create a transmittance map (hereinafter referred to as a used transmittance map) used to create a corrected image. This first creation method is represented by the following equation (8).
T (x) = αt (x) + βt ₁ (x) + γt ₂ (x) +... + Ωt _n (x) (8)
However,
x: pixel position T (x): transmittance t (x) at each pixel position in the used transmittance map: transmittance t ₁ (x) to t _n (at each pixel position in the transmittance map having the same resolution as the captured image) x): transmittances α to ω at respective pixel positions in a transmittance map having a resolution sequentially lower than that of the captured image: weighting constants

Incidentally, for example, _t 1 (x) is half the resolution of the transmittance map of the captured _{image, t} 2 (x) is transmittance map ... 1/4 resolution of the captured image, as in, _{t 1} (X), t ₂ (x),..., T _n (x) indicate transmittance maps having different resolutions. An appropriate number of transmittance maps having a lower resolution than the captured image may be created, and at least one may be created.

  Thus, the halo effect in the corrected image can be reduced by creating a use transmittance map by converting the transmittance map into multiple resolutions and performing weighted addition.

(2) Second creation method It has also been experimentally found that the halo effect cannot be seen if the resolution is lowered without performing multi-resolution. Therefore, in the second creation method, a transmittance map having a resolution of 0.5 to 0.9 times the resolution of the captured image is created as the use transmittance map.

  Which of the first creation method and the second creation method is used to create the use transmittance map is appropriately selected. For example, the control device 104 selects a method for creating a use transmittance map by using any of the following first to fourth selection methods.

(1) First Selection Method In the first selection method, the control device 104 selects a method for creating a use transmittance map according to the frequency component of the captured image. When there are many high-frequency components in the photographed image, there are many edges. Therefore, if the use transmittance map is set to a low resolution, the resolution of the corrected image is lowered. Therefore, the control device 104 obtains the frequency component of the captured image. If there are many high-frequency components, the control device 104 selects the first creation method and creates a use transmittance map. If there are many low-frequency components, the control device 104 creates the second component. Select a creation method and create a usage transmittance map.

(2) Second Selection Method In the second selection method, the control device 104 creates a usage transmittance map in accordance with the noise generation status, specifically, the ISO sensitivity at the time of shooting a shot image. select. For ISO sensitivity at the time of shooting, refer to Exif (registered trademark) information added to the shot image. When there is a lot of noise, it is considered that the influence of noise can be reduced by creating a use transmittance map with as low a resolution as possible. Therefore, the control device 104 selects the second creation method when the noisy condition in the photographed image, that is, the ISO sensitivity at the time of photographing is high (for example, 1600 or more) and selects the use transmittance map. If the sensitivity is low and the sensitivity is low, the first transmission method is selected to create a use transmittance map.

(3) Third Selection Method In the third selection method, the control device 104 selects a method for creating a use transmittance map depending on whether the captured image is a still image or a moving image. Since it is necessary to increase the processing speed when the captured image is a moving image, the control device 104 selects the second generation method with a relatively low processing load and generates the use transmittance map. On the other hand, when the captured image is a still image, the control device 104 selects the first creation method and creates the use transmittance map because it is desired to improve the resolution.

(4) Fourth Selection Method In the fourth selection method, the user manually selects either the first creation method or the second creation method via the operation member 101. That is, the control device 104 selects either the first creation method or the second creation method in accordance with the operation of the operation member 101, and creates the use transmittance map.

  Note that any of the first to fourth selection methods described above may be used, and which one may be used may be set in advance, or may be set manually by the user.

(Creating a corrected image)
In step S5 (FIG. 3), the control device 104 corrects the photographed image using the ambient light color information acquired in step S3 and the use transmittance map created in step S4 (that is, the influence of fog). 3), a corrected image is created, and the process of FIG. 3 is terminated. Specifically, the control device 104 creates a corrected image represented in the HSV color space by the following equations (9) to (11).
Jh (x) = Ih (x) (9)
Js (x) = (Is (x) −α ₁ · As) / T (x) + α ₂ · As (10)
Jv (x) = (Iv (x) −β ₁ · Av) / T (x) + β ₂ · Av (11)
However,
x: Pixel position Ih (x), Is (x), Iv (x): Hue, saturation, brightness Jh (x), Js (x), Jv (x) of each pixel position of the captured image Hue, saturation, brightness α ₁ , α ₂ , β ₁ , β ₂ : constant As, Av: saturation of ambient light, brightness T (x): transmittance at each pixel position in the used transmittance map

Expressions (10) and (11) for obtaining saturation and lightness are expressions applying the above-described fog image model expression (2). For example, adjustment such as changing the degree of enhancement between saturation and lightness is performed. Therefore, the components of ambient light are multiplied by constants α ₁ , α ₂ , β ₁ , β ₂ .

  Also, in the HSV color space, whiteness reduction due to fog or haze only needs to consider saturation and lightness, so that the hue of the corrected image is the same as that of the photographed image as in equation (9). Good. Therefore, since the substantial calculation process is only required for the two channels of saturation and lightness, the processing load can be reduced as compared with the case where the calculation process is performed with three RGB channels.

  By performing the image correction processing shown in FIG. 3 as described above, a photographed image that is whitish and blurred due to the influence of floating objects in the atmosphere (for example, fog, haze, smoke, dust, etc.) is corrected to be clear. be able to. Further, in the above description, an example of reducing these effects by taking a floating object (mist, haze, etc.) in the atmosphere as an example has been described. However, the present invention is not limited to this, and the image correction process is also applicable to underwater photography. The photographed image that is whitish and blurred under the influence of water existing between the subject and the digital camera 100 can be corrected clearly.

According to the embodiment described above, the following operational effects can be obtained.
(1) In the digital camera 100, the control device 104 images the subject via a light-transmitting medium (for example, the atmosphere including fog or haze, water in the case of underwater photography) existing in front of the subject. Acquire a captured image. The control device 104 creates a white intensity map representing a distribution of the degree of low saturation and high brightness (white intensity) using the saturation information related to the saturation of the photographed image and the brightness information related to the brightness. The control device 104 corrects the photographed image using the white intensity map to reduce the influence of the light-transmitting medium (for example, the atmosphere including fog and haze, the influence of water in the case of underwater photography). Create an image. In this way, the captured image can be corrected without using a dark level as in the case of the conventional dark channel image, and the influence of noise when correcting the captured image can be reduced.

(2) In the digital camera 100, the control device 104 acquires the ambient light information related to the captured image, and creates a transmittance map that represents the distribution of the correction parameter (transmittance) using the white intensity map and the ambient light information. To do. The control device 104 corrects the captured image using the ambient light information and the transmittance map to create a corrected image. In this way, the captured image can be corrected in consideration of the influence of ambient light in addition to the degree of low saturation and high brightness (that is, the degree of whitishness).

(3) In the digital camera 100, the control device 104 represents the distribution of the correction parameter (transmittance) at a resolution lower than that of the captured image and the map representing the distribution of the correction parameter (transmittance) at the same resolution as the captured image. Add a map to create a transparency map. Thereby, the halo effect in the corrected image can be reduced.

(4) In the digital camera 100, the control device 104 creates a use transmittance map that represents the distribution of correction parameters (transmittance) at a lower resolution than the captured image. Thereby, the halo effect in the corrected image can be reduced.

(5) In the digital camera 100, the control device 104 represents the distribution of the correction parameter (transmittance) at a resolution lower than that of the captured image and the map representing the distribution of the correction parameter (transmittance) at the same resolution as the captured image. A first creation method for creating a used transmittance map by adding a map, and a second creation method for creating a used transmittance map representing a correction parameter (transmittance) distribution at a lower resolution than the captured image. And can be used. The control device 104 performs the first creation method and the second creation according to the frequency component of the shot image, the ISO sensitivity at the time of shooting the shot image, and whether the shot image is a moving image or a still image. Choose one of the methods to create a usage transmittance map. This makes it possible to properly use the first creation method that increases the resolution of the corrected image and the second creation method that has a low processing load.

(6) In the digital camera 100, the control device 104 converts the captured image expressed in the RGB color space into an HSV image expressed in the HSV color space of the conical model, and the S component (saturation) of the HSV image. And a white intensity map based on the V component (lightness). By using the cone model of the HSV color space in this way, it is easy to distinguish between “high saturation and low brightness” conditions and “low saturation and high brightness” conditions, such as fog, haze, smoke, etc. This makes it easier to extract a white and blurred area. Accordingly, it is possible to emphasize and process a region that is white and blurred due to the influence of fog, haze, smoke, or the like.

-Second Embodiment-
Next, a second embodiment of the present invention will be described. In the first embodiment described above, the example in which the image correction process is performed by converting the color space of the captured image from the RGB color space to the HSV color space has been described. In the second embodiment described below, the image correction processing is performed in the YCbCr color space represented by three components of Y (luminance) and Cb, Cr (color difference) without performing conversion to the HSV color space. An example of performing is described.

  Also in the second embodiment, the flow of image correction processing (FIG. 3) is the same as in the first embodiment, and will be described with reference to the flowchart of FIG. In step S1, the control device 104 acquires a captured image on which image correction processing is performed. Here, it is assumed that the captured image is expressed in the YCbCr color space.

In step S2, the control device 104 creates a white intensity map of the captured image. Here, the control device 104 calculates the saturation information S ′ of the captured image using CbCr (color difference) of the captured image expressed in the YCbCr color space. The saturation information S ′ can be obtained by the following equation (12), for example. In the formulas described below, it is assumed that Cb and Cr are expressed in a range of −0.5 to 0.5.

Since the above equation (12) involves calculation of the route, the saturation information S ′ may be obtained by simplifying the equation (13) below. In the following expression, normalize is a function that normalizes so that the minimum value is 0 and the maximum value is 1.
S ′ = normalize (Cb ² + Cr ² ) (13)

Further, the saturation information S ′ may be obtained by simplification as in the following formula (14) or (15).
S ′ = (min | Cb |, | Cr |) /0.5 (14)
S ′ = normalize (| Cb |) · normalize (| Cr |) (15)

  As described above, the control device 104 calculates the saturation information S ′ of the captured image expressed in the YCbCr color space using any of the above-described equations (12) to (15).

Further, the control device 104 uses Y (luminance) as brightness information of the captured image in the captured image represented in the YCbCr color space. The control device 104 uses the saturation information S ′ and brightness information Y (luminance) of the captured image to create a white intensity map of the captured image according to the following equation (16).
W (x) = (1−S ′ (x)) · Y (x) (16)
However,
x: pixel position W (x): white intensity S ′ (x) at each pixel position: saturation information Y (x) calculated from CbCr at each pixel position: luminance at each pixel position

  In step S3, the control device 104 acquires color information of ambient light in the captured image. Here, the lightness V is replaced with the luminance Y, and the color information of the ambient light may be obtained in the same manner as in the first embodiment described above.

  In step S4, the control device 104 creates a use transmittance map in the same manner as in the first embodiment described above.

In step S5, the control device 104 determines the saturation Js (x) and lightness Jv (x) of the corrected image using the above-described equations (10) and (11), as in the first embodiment. calculate. Further, since the hue Jh (x) of the corrected image is the same as the hue of the original photographed image, the hue H may be obtained from the CbCr component of the photographed image by the following equation (17).
H = tan ⁻¹ (Cr / Cb) (17)

Then, the control device 104 converts the obtained HSV color space correction image into a YCbCr color space correction image. Here, the brightness V of the corrected image may be Y (luminance) of the corrected image. Further, CbCr (color difference) of the corrected image is calculated from the saturation S and the hue H of the corrected image by the following equations (18) and (19).
Cb = S · cos (H) (18)
Cr = S · sin (H) (19)

According to the second embodiment described above, the following operational effects can be obtained.
The control device 104 calculates saturation information S ′ based on the Cb component and the Cr component of the captured image represented in the YCbCr color space, creates a white intensity map using the Y component of the captured image as the brightness information, and Image correction processing was performed. In this way, when performing the above image correction processing on a photographic image recorded in the YCbCr color space by directly performing calculation in the YCbCr color space also used for compression such as JPEG and MPEG, the color space Since the processing can be performed without conversion, the processing load can be reduced.

-Third embodiment-
Next, a third embodiment of the present invention will be described. In the above-described first embodiment, the example in which the image correction process is performed in the HSV color space has been described. In the third embodiment described below, an example will be described in which only the white intensity map is generated in the HSV color space and the subsequent processing (steps S3 to S5 in FIG. 3) is performed in the RGB color space.

  Also in the third embodiment, the flow of image correction processing (FIG. 3) is the same as in the first embodiment, and will be described with reference to the flowchart of FIG. In step S1, the control device 104 acquires a captured image represented in an RGB color space. In step S2, the control device 104 converts the color space of the photographed image from the RGB color space to the HSV color space, and uses the saturation S and the lightness V in the same manner as in the first embodiment described above. Create

In step S3, the control device 104 acquires color information of ambient light in the captured image. Specifically, as in the first embodiment, the control device 104 uses the white intensity map created in step S2 in the top 0.1% of the whole when the white intensity at each pixel position is arranged in descending order. The region is extracted as the maximum white intensity region in the captured image. Unlike the first embodiment, the control device 104 acquires the color information (RGB) of the pixel position having the highest gray scale intensity Gray in the extracted white intensity maximum area as the color information of the ambient light. . The gray scale intensity Gray is an average value of RGB values at each pixel position of the captured image, and is calculated by the following equation (20).
Gray = (R + G + B) / 3 (20)

In step S4, the control device 104 uses the white intensity map created in step S2 and the gray scale intensity calculated from the color information (RGB) of the ambient light acquired in step S3 to obtain a transmittance map of the captured image. Create The control apparatus 104 calculates | requires the transmittance | permeability t (x) using the following formula | equation (21).
t (x) = 1−ω · W (x) / A _gray (21)
However,
x: pixel position t (x): transmittance at each pixel position ω: constant W (x): white intensity at each pixel position A _gray : gray scale intensity of ambient light

  Then, in order to prevent the halo effect, the control device 104 creates a use transmittance map in the same manner as in the first embodiment described above.

In step S5 (FIG. 3), the control device 104 creates a corrected image obtained by correcting the captured image using the ambient light color information acquired in step S3 and the use transmittance map created in step S4. The process of 3 is finished. Specifically, the control device 104 creates a corrected image represented in the RGB color space by the following equations (22) to (24).
Jr (x) = {(Ir (x) -Ar) / T (x)}. Ar + Ar (22)
Jg (x) = {(Ig (x) −Ag) / T (x)} · Ag + Ag (23)
Jb (x) = {(Ib (x) −Ab) / T (x)} · Ab + Ab (24)
However,
x: Pixel position Ir (x), Ig (x), Ib (x): R, G, B at each pixel position of the captured image
Jh (x), Js (x), Jv (x): R, G, B at each pixel position of the corrected image
Ar, Ag, Ab: R, G, B of ambient light
T (x): transmittance at each pixel position in the used transmittance map

  Expressions (22) to (24) are expressions applying Expression (2) of the fog image model described above, and correspond to the first term (I (x) −A) / t (x) of Expression (2). By multiplying the component to be multiplied by the component of ambient light, it is possible to obtain a corrected image that sharpens the captured image more naturally.

The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.
(Modification 1)
The method for acquiring the color information of the ambient light in the captured image is not limited to the method described above. For example, an ambient light sensor that detects ambient light may be provided in the digital camera 100, and the detection result of the ambient light sensor at the time of shooting may be used, or information on white balance setting at the time of shooting may be used.

  In addition, instead of acquiring ambient light color information for each captured image, predetermined ambient light color information is stored in advance in a memory in the digital camera 100, and the ambient light color information is stored in the image correction process. May be read and used.

(Modification 2)
In the above-described embodiment, the example in which the image correction process is executed according to the user's instruction has been described. However, when the shooting mode is the landscape mode or the underwater mode, the control device 104 performs processing on the shot image. The image correction process may be automatically executed.

(Modification 3)
The formula for obtaining the transmittance t (x) is not limited to the formula (7) described above. For example, instead of the constant ω, the transmittance t (x) may be calculated by an equation using log as in the following equation (25).
t (x) = 1-log (W (x) / Av) (25)

(Modification 4)
In the above-described embodiment, the example in which the control device 104 of the digital camera 100 executes the above-described image correction processing (FIG. 3) has been described. However, an image processing device such as a personal computer executes a program for performing these processing. You may do it. In this case, the loading of the program to the personal computer (PC) 200 may be performed by setting a recording medium 201 such as a CD-ROM storing the program in the PC 200 as shown in FIG. It may be loaded into the PC 200 by a method via the line 202. When passing through the communication line 202, the program is stored in the hard disk device 204 of the server (computer) 203 connected to the communication line 202. The program can be supplied as various types of computer program products, such as provision via the recording medium 201 or the communication line 202.

(Modification 5)
The above-mentioned image correction processing is used to improve the visibility of captured images that are white and blurred due to the influence of fog, haze, smoke, etc., in surveillance cameras for disaster prevention and crime prevention, in-vehicle cameras for accident prevention, etc. Also suitable.

  Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 100 ... Digital camera, 101 ... Operation member, 102 ... Shooting lens, 103 ... Imaging element, 104 ... Control apparatus, 105 ... Memory card slot, 106 ... Monitor

Claims (10)

Image acquisition means for acquiring a captured image;
First map creation for creating a first map representing a distribution of a degree of low saturation and high brightness in the photographed image using saturation information relating to the saturation of the photographed image and brightness information relating to the brightness of the photographed image Means,
Correction means for correcting the captured image using the first map to create a corrected image;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
Ambient light information acquisition means for acquiring ambient light information related to the captured image;
Second map creating means for creating a second map representing a distribution of correction parameters in the photographed image using the first map and the ambient light information;
Further comprising
The image processing apparatus, wherein the correction unit corrects the captured image using the ambient light information and the second map to create the corrected image. The image processing apparatus according to claim 2,
The second map creating means adds the map representing the distribution of the correction parameters at the same resolution as the captured image and the map representing the distribution of the correction parameters at a lower resolution than the captured image. An image processing apparatus characterized by creating a map. The image processing apparatus according to claim 2,
The image processing apparatus, wherein the second map creating means creates the second map representing the correction parameter distribution at a lower resolution than the captured image. The image processing apparatus according to claim 2,
The second map creating means adds the map representing the distribution of the correction parameters at the same resolution as the captured image and the map representing the distribution of the correction parameters at a lower resolution than the captured image. A first creation unit that creates a map; and a second creation unit that creates the second map that represents the distribution of the correction parameters at a lower resolution than the captured image, and a frequency component of the captured image, Select either the first creation unit or the second creation unit according to the ISO sensitivity at the time of shooting the shot image, whether the shot image is a moving image or a still image, and user operation An image processing apparatus that creates the second map. In the image processing device according to any one of claims 2 to 5,
The ambient light information acquisition means acquires, as the ambient light information, image information of a pixel position where the brightness information is maximum in a region of the captured image corresponding to the region with the highest degree of the first map. An image processing apparatus. In the image processing apparatus according to any one of claims 1 to 6,
HSV conversion means for converting the captured image into an HSV image represented in an HSV color space;
The image processing apparatus, wherein the first map creating means creates the first map using the S component of the HSV image as the saturation information and the V component of the HSV image as the brightness information. In the image processing apparatus according to any one of claims 1 to 6,
The captured image is a YCbCr image expressed in a YCbCr color space,
The first map creating means calculates the saturation information using a Cb component and a Cr component of the YCbCr image, and creates the first map using the Y component of the YCbCr image as the brightness information. An image processing apparatus. Processing to acquire a captured image;
A process of creating a first map that represents a distribution of a degree of low saturation and high brightness in the captured image, using saturation information regarding the saturation of the captured image and brightness information regarding the brightness of the captured image;
A process of correcting the captured image using the first map to create a corrected image;
An image processing program for causing a computer to execute. Image acquisition means for acquiring a photographed image obtained by photographing the subject via a light-transmitting medium existing in front of the subject;
First map creation for creating a first map representing a distribution of a degree of low saturation and high brightness in the photographed image using saturation information relating to the saturation of the photographed image and brightness information relating to the brightness of the photographed image Means,
Correction means for correcting the captured image using the first map to create a corrected image in which the influence of the light-transmitting medium is reduced;
An image processing apparatus comprising:

Images