JP2015142357A - image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus which performs flare correction with high image quality.SOLUTION: The image processing apparatus includes: acquisition means for acquiring an original image represented by first to n-th (n≥3) color components; generation means for rearranging the first to n-th color components in the order of magnitudes to generate at least a maximum value surface, a minimum value surface and a median surface and applies gamma transformations of different properties to at least the maximum value surface and the minimum value surface to generate one lightness component on the basis of at least one of the maximum and minimum value surfaces to which the gamma transformations have been applied, and the median surface; and correction means for calculating a correction pixel value of the first color component in such a manner that a difference between the correction pixel value of the first color component and a local average value of the first color component in a correction target pixel of the first color component maintains a correlation to a difference between a pixel value of the lightness component and a local average value of the lightness component in the pixel.

Description

  The present invention relates to an image processing apparatus.

  It is known that when a subject image is photographed using an imaging optical system using a diffractive optical element (Diffraction Optical Element), a diffraction flare is generated around a bright light source. Since such a diffractive flare does not appear in a normal lens that does not use a diffractive optical element, attempts have been made to correct it by image processing.

  In Patent Document 1, the image formation characteristic data of the diffractive optical element is prepared in advance in correspondence with all shooting environments such as the subject distance, the zoom position of the optical system, the aperture value, the feed amount, and the angle of view. A correction method is disclosed in which the state of unnecessary diffracted light is estimated from imaging characteristic data while subtracting its component from the original image while using the luminance offset amount as a variable and observing the degree of coincidence with the image data.

  In Patent Document 2, an attempt is made to specify and correct unnecessary diffracted light from image data itself without using optical characteristic data as in Patent Document 1. In the method disclosed in Patent Document 2, unnecessary diffraction light is conspicuous because it is assumed that the light source is in the dark, and the unnecessary diffraction is determined from the distribution of the luminance histogram generated around the light source in that situation. The idea is to specify a range of luminance levels where light is generated, and to reduce the luminance distributed in that range to the background level of darkness to eliminate unwanted diffracted light.

JP 2005-167485 A International Publication No. 2010/032409

  The situation assumed in Patent Document 2 is a technique that is valid only when the light source is present in isolation in the dark. Similarly, Patent Document 1 only applies to situations where the background changes slowly. However, an arbitrary general subject image is usually not in such a convenient situation, and generally an object image having an arbitrary image structure is also present in a region where a diffraction flare of a light source occurs. doing. Under such circumstances, the methods shown in Patent Document 1 and Patent Document 2 may fail because the preconditions are not satisfied. That is, there is a risk that the image structure is lost due to the flare correction.

  In view of such a situation, an object of the present invention is to provide a general flare correction technique that can handle any subject image.

(1) An image processing apparatus according to a first aspect includes an acquisition unit that acquires an original image represented by first to nth color components (n ≧ 3), and values of the first to nth color components. By rearranging in order of size, at least the maximum value surface, the minimum value surface, and the median value surface are generated, and at least the maximum value surface and the minimum value surface are subjected to gamma conversion having different characteristics, and the gamma conversion is performed. Generating means for generating one brightness component based on at least one of the maximum value surface, the minimum value surface, and the median value surface; and the correction pixel value of the first color component in the correction target pixel of the first color component Correction of the first color component so that the difference between the local average value of the first color component and the local average value of the lightness component in the pixel is correlated with the difference between the pixel value of the lightness component and the local average value of the lightness component Correction means for obtaining a pixel value.
(2) An image processing apparatus according to a second aspect includes an acquisition unit that acquires an original image represented by first to nth color components (n ≧ 3), and values of the first to nth color components. By rearranging in order of size, at least the maximum value surface, the minimum value surface, and the median value surface are generated, and at least the maximum value surface and the minimum value surface are subjected to gamma conversion having different characteristics, and the gamma conversion is performed. Generating means for generating one brightness component based on at least one of the maximum value surface, the minimum value surface, and the median value surface; and the correction pixel value of the first color component in the correction target pixel of the first color component Correction of the first color component such that the ratio of the local average value of the first color component and the ratio of the pixel value of the lightness component and the local average value of the lightness component in the pixel is maintained. Correction means for obtaining a pixel value.
(3) The image processing apparatus according to claim 9 is configured to sequentially acquire an original image represented by first to nth color components (n ≧ 3), filter the first color components, and sequentially First generating means for generating a first high-frequency image having a plurality of resolutions, sequentially integrating the first high-frequency images from the lowest resolution, and generating a second high-frequency image integrated into one; The values of the 1st to nth color components are rearranged in order of size to generate at least three brightness planes including a maximum value plane, a minimum value plane, and a median plane, and the generated three brightness planes A second generation means for selectively weighting and synthesizing values based on a region unit to generate one brightness component, and filtering the brightness component to sequentially generate a third high-frequency image having a plurality of resolutions. The third high-frequency image is sequentially integrated from the lowest resolution to 1 And a third generation means for generating a fourth high-frequency image integrated with the first high-frequency image, subtracting the second high-frequency image integrated with one of the first color components from the first color component of the original image, And a fourth generation means for generating a corrected image of the first color component by adding the fourth high-frequency image integrated into one.
(4) The image processing apparatus according to claim 13 is an acquisition unit that acquires an original image composed of at least one color component, and a first that consists of a plurality of resolutions sequentially by filtering the one color component. First generation means for generating a high-frequency image, sequentially integrating the first high-frequency image from the lowest resolution to generate a second high-frequency image integrated into one, and a second integrated into the one Attenuating means for attenuating the amplitude of the value of the second high-frequency image integrated into the one based on the magnitude of the value itself in each pixel of the high-frequency image, and the attenuated first from one color component of the original image And a second generation unit that generates a corrected image of the one color component by subtracting two high-frequency images.
(5) An image processing apparatus according to claim 14, wherein an acquisition unit that acquires an original image composed of at least two color components, and each of the at least two color components are filtered, and sequentially from a plurality of resolutions. Generating a first high-frequency image, and sequentially integrating the first high-frequency image from the lowest resolution to generate a single second high-frequency image for each color component; A second color of the at least two color components based on a value magnitude at each pixel of the second high frequency image integrated with the one of the first color components of the at least two color components; Attenuating means for attenuating the amplitude of the value of the second high frequency image integrated into the one of the components, and subtracting the attenuated second high frequency image from the second color component of the original image. Correction of 2 color components Characterized by comprising second generating means for generating an image, a.
(6) An image processing apparatus according to claim 15 is an acquisition unit that acquires an original image composed of at least one color component, and a first that sequentially filters the one color component and has a plurality of resolutions. First generation means for generating a high-frequency image, sequentially integrating the first high-frequency image from the lowest resolution to generate a second high-frequency image integrated into one, and a plurality of the sequentially generated plural A third high-frequency image is generated at each resolution by further filtering the one color component in the first high-frequency image of resolution, and the third high-frequency image is sequentially integrated from the lowest resolution into one. Second generation means for generating the fourth high-frequency image, and the value of the value itself in each pixel of the fourth high-frequency image integrated into the one, the second high-frequency image integrated into the one Attenuating means for attenuating the amplitude of the original image, and third generating means for generating a corrected image of the one color component by subtracting the attenuated second high-frequency image from one color component of the original image. It is characterized by that.
(7) An image processing apparatus according to claim 16, wherein an acquisition unit that acquires an original image composed of at least two color components, and each of the at least two color components are filtered, and sequentially from a plurality of resolutions. Generating a first high-frequency image, and sequentially integrating the first high-frequency image from the lowest resolution to generate a single second high-frequency image for each color component; Filtering the first color component at each resolution by further filtering the first color component in the sequentially generated first high frequency image of the plurality of resolutions of the first color component of at least two color components. Generating and integrating the third high-frequency image sequentially from the lowest resolution to generate a fourth high-frequency image integrated into one, and integrating the first high-frequency image into the one of the first color components; Attenuating the amplitude of the value of the second high-frequency image integrated into the one of the at least two color components based on the magnitude of the value at each pixel of the fourth high-frequency image And a third generation means for generating a corrected image of the second color component by subtracting the attenuated second high-frequency image from the second color component of the original image. It is characterized by that.

  Even in a general image in which a diffractive flare generated around a light source and an arbitrary general subject image overlap each other, only a diffractive flare component can be appropriately extracted to provide a high-quality flare correction technique.

It is a block diagram which illustrates the composition of the camera carrying the control device as an image processing device in one embodiment of the present invention. It is a figure which illustrates the change of the pixel value according to the change of a pixel position. It is a figure which illustrates the gradation conversion of the V surface in a red-type and cyan-type hue area | region. It is a figure which shows a mode that a different V surface is produced | generated according to the hue and saturation of an original image. It is a figure showing the frequency distribution for every value of the integrated edge component E (x, y) of a secondary differentiation. It is a figure which shows the mode of a flare refining process. It is a block diagram which shows a flare correction process block. It is a flowchart which shows a flare correction process procedure. It is a figure which shows the provision form of the computer program product for a control apparatus to perform an image process.

  FIG. 1 is a block diagram illustrating the configuration of a camera 100 equipped with a control device 103 as an image processing device according to an embodiment of the invention. The camera 100 includes an operation member 101, an image sensor 102, a control device 103, a memory card slot 104, a monitor 105, and an optical system 106. The operation member 101 includes various input members operated by the user, such as a power button, a release button, a zoom button, a cross key, an enter button, a play button, and a delete button.

  The image sensor 102 is configured by a CMOS image sensor or the like. The image sensor 102 captures a subject image formed by the optical system 106 including a diffraction grating lens. An original image obtained by imaging is acquired from the imaging element 102 by the control device 103.

  The control device 103 includes a memory and other peripheral circuits, and controls a photographing operation performed by the camera 100. Memory constituting the control device 103 includes SDRAM and flash memory. The SDRAM is a volatile memory, and is used as a work memory for developing a program when the control device 103 executes a program for performing photographing processing and flare correction processing described later. The SDRAM is also used as a buffer memory for temporarily storing data. On the other hand, the flash memory is a non-volatile memory in which data of the above-described program executed by the control device 103, various parameters read during program execution, and the like are recorded.

  The memory card slot 104 is a slot for inserting a memory card as a recording medium. The control device 103 writes and records the image file generated by performing the photographing process and the flare correction process on the memory card inserted in the memory card slot 104. Further, the control device 103 acquires an image file recorded in the memory card inserted in the memory card slot 104.

  The monitor 105 is a liquid crystal monitor (rear monitor) mounted on the back surface of the camera 100. The monitor 105 displays a reproduction image based on the image file recorded on the memory card, a setting menu for setting the camera 100, and the like. When the user operates the operation member 101 to set the camera 100 to the shooting mode, the control device 103 outputs image data for display of images acquired in time series from the image sensor 102 to the monitor 105. As a result, a through image is displayed on the monitor 105.

  The image sensor 102 is a single-plate color image sensor, and R (red), G (green), and B (blue) color filters are arranged in accordance with the Bayer array on the imaging surface. That is, the image data acquired from the control device 103 from the image sensor 102 or the memory card inserted in the memory card slot 104 is shown in the RGB color system, and each pixel constituting the image data has RGB data. The color information of any one of the color components exists.

  As described above, the control device 103 reads and executes a program for performing flare correction processing from the flash memory, thereby executing an original image acquired from the image sensor 102 or the memory card inserted in the memory card slot 104. Perform flare correction processing. Before entering a specific example of an embodiment of an algorithm for flare correction processing performed by a program for performing flare correction processing, the nature of flare generated in a lens including a diffraction grating used in optical system 106 will be described, and a flare correction algorithm will be described. Describe the basic concept of.

<Properties of diffraction flare>
There are roughly two types of diffraction flare properties. One is a cyan flare and the other is a red flare. Although depending on the design of the diffraction grating, the way of producing the annular component by diffraction is roughly divided into two. Red in which the hue has a value near 0 degrees and cyan in which the hue has a value in the vicinity of 180 degrees are in an opposite color relationship in the hue circle.

  For example, cyan flare is easily generated by a headlight of an automobile with an HID lamp, etc., and low-order annular zones of the 0th order to the second order term of the diffraction flare are locally solidified around the light source, and are bright in a lump. A light bulb covers the area around the light source. Such flare mainly appears on the B and G planes of the RGB plane. The green light of a traffic light is covered with a similar massive diffractive flare and is covered until the shape of the hood around the green light is not visible.

  Green flare is likely to appear around mercury lamps such as street lamps. The appearance of green flare is very similar to that of cyan flare, and low-order diffracted light tends to appear locally round. The green flare mainly shows the shape of the ring zone on the G and B sides.

  Red flare, for example, is easily emitted around the light source of ordinary incandescent light bulbs and the headlights of passenger cars using the same, and the diffractive flare has light sources of several flares that range from 5th to 6th. It extends over a very far range. This is mainly on the R side.

  In addition, yellow flare often comes out over a wide area with many similar large rings. For example, when the sun is photographed directly, a large yellow or red zone is spread over the entire image. The yellow flare has an annular component mainly on the R and G planes.

<Principle of flare correction>
When the flare component described above is observed closely, it can be seen that the flare ring zone does not appear in any one of the RGB planes. Therefore, image correction is performed by actively utilizing the fact. Suppose that one brightness surface with reduced flare can be created with reference to any of RGB. This is assumed to be a V plane. If such a brightness plane can be created, the brightness plane does not contain the flare zonal component, and a brightness plane that can decompose the image structure to the pixel unit accuracy is created.

Therefore, it is preferable to correct each surface of RGB in which flare occurs so as to be compared with the V surface (lightness surface). That is, within each RGB color plane, correction values r ′, g ′, b ′ and local average values of the R component pixel value r, the G component pixel value g, and the B component pixel value b at a certain pixel position < The correction value r ′ for each color plane of RGB so that the ratio between r>, <g>, <b> is correlated with the ratio between the pixel value v at that position on the V plane and the local average <v>. , G ′, b ′. In other words, the correction values r ′, g ′, and b ′ are obtained so that the local variation of each RGB color plane behaves the same as the local variation of the V plane. This also means that the image structure is adjusted to the V plane while maintaining the average color. For example, as shown in FIG. 2, the change in the pixel value on the V plane according to the change in the pixel position is relatively gentle as represented by the solid line 210, whereas each of the RGB colors according to the change in the pixel position It is assumed that the fluctuation of the pixel value on the surface is repeatedly increased and decreased relatively rapidly as indicated by the broken line 220. In this case, the correction values r ′, g ′, and b ′ are represented by the alternate long and short dash line 230 as a result of correcting the fluctuation of the pixel values of the RGB color planes to behave the same as the local fluctuation of the V plane. As can be seen, the variation is similar to the pixel value on the V plane. When this is expressed by an equation, correction as in the following equation (1) is performed. Formula (1) is an example of correction formulas for RGB color planes in order from the top.

  <> Means an operation that performs local averaging. In order to eliminate the flare ring zone structure generated on the R plane, the area where <r> is smoothed is wide enough to mix the area where the flare ring zone is generated and the area where the flare ring zone is not generated. Smoothing is required. Normally, this range is a wide area image smoothing.

When this is shifted to the logarithmic gradation space, the above-described operation can be expressed by the sum and difference operations as in the following equations (2), (3), and (4). However, R = log (r), R ′ = log (r ′), G = log (g), G ′ = log (g ′), B = log (b), B ′ = log (b ′), V = log (v). The coefficient α takes a value in the range of 0 to 1, and α = α (x, y), which is a function depending on the location of the image. In the area where flare correction is performed, α = 1. In a region where flare correction is not performed, α = 0.

  Explaining the R plane as an example, the first term (R- <R>) of the correction term in the above equation (2) corresponds to the edge image of the R plane obtained by subtracting the smoothed image from the original image. . As shown in Equation (2), the first term of the correction term is subtracted from the R plane itself to eliminate the flare ring component, and instead corresponds to the edge image obtained on the V plane without flare. The second term (V- <V>) of the correction term is added to recover the edge.

  Since it is necessary to extract the edge structure by smoothing over a wide area, it is preferable to use multi-resolution representation. For example, there is a Laplacian pyramid expression. You can also use wavelet transform. These are all filtered by dividing the original image into a low-frequency image with a frequency lower than a predetermined frequency and a high-frequency image with a frequency equal to or higher than a predetermined frequency. The original image is equivalently expressed by a series of images and high-frequency images of each resolution. Therefore, when the low-frequency image with the lowest resolution is set to zero and inverse multi-resolution conversion is performed, a multi-resolution integrated edge component that equivalently represents the edge of the original image is generated. It can be considered that the multi-resolution integrated edge components of the RGB color planes and V planes generated in this way are represented by the terms in the parentheses of the correction terms of the above formulas (2), (3), and (4).

When the expression in the RGB space is converted to a YCbCr space expressed by luminance and color difference by color space conversion, the expressions (2), (3), (4) are equivalently expressed by the following expressions (5), ( 6) and (7). However, the coefficient β takes a value in the range of 0 to 1, β = β (x, y), and is a function depending on the location of the image. Even if the coefficient α that is a function that limits the correction area for the luminance component and the coefficient β that is a function that limits the correction area for the color difference component are not the same, it is only necessary to set a function suitable for each. Using.

  Therefore, when obtaining the corrected pixel value of the color difference component, as shown in the equations (6) and (7), the equivalent edge components (Cb− <Cb>), (Cr− By subtracting <Cr>), only the operation of bringing the flare color zone closer to the average color is performed. When obtaining the correction pixel value of the luminance component, the annular component generated on the luminance plane is eliminated by subtracting the equivalent edge (Y− <Y>) from the luminance component Y as shown in Equation (5). The edge of the image structure is recovered with reference to the lightness information (V- <V>) without flare.

<V-plane generation method>
Next, a method for generating a lightness surface (V surface) with suppressed flare assumed in the above description will be described. First, the three color component values of the (R, G, B) values existing in each pixel are rearranged in the order of the size of the values, and as shown in the equation (8), the maximum value plane V _max , the center The value surface V _med and the minimum value surface V _min are created from the RGB surfaces. Each of the maximum value plane, median value plane, and minimum value plane generated in this way has no discontinuity as an image structure, and can be recognized as a natural image structure plane. Therefore, the utility value as a surface capable of extracting the edge structure storing the image structure is generated.

  When the values are rearranged in this way, flare components are concentrated on the maximum value surface. On the other hand, a surface having no flare is basically generated on the minimum value surface. On the other hand, depending on the type of flare, a clear luminance structure without flare may appear on the median surface, or a flare structure may appear. In general, a flare mainly composed of a single color such as a red flare does not show a flare on the median plane, and a flare mainly composed of two colors such as a cyan flare also appears on the median plane. Therefore, a brightness component can be generated using the median value plane in a red-based region, but a brightness component cannot be generated using a median surface in a cyan region.

From the viewpoint of the structure of luminance, it is considered that the bright surface contains more luminance information as the V surface of the HSV space uses V _max . However, since flare gathers on the maximum value surface, much cannot be referred to for the purpose of flare correction. In that sense, the next important thing is the luminance structure of the median plane. Therefore, it is preferable to generate the V plane with reference to the median plane in the red hue region. On the other hand, in the cyan hue region, only the minimum value plane has no flare. Therefore, the V plane is generated while referring to the minimum value plane most dominantly.

  The minimum value surface referred to in the cyan hue region is the darkest surface, and may not represent the luminance structure of the image. In order to cope with such a situation, in the cyan hue region, the maximum value surface is referred to together with the minimum value surface, and the V surface is generated based on the average value. Here, it is necessary to devise in order to introduce the maximum value plane in which flare occurs. As described in the section on the principle of flare correction, the edge structure extracted from the V plane, that is, contrast information, is not extracted for performing edge recovery, but the luminance structure of the value as it is in the V plane. The flare is considered to be gathered in an area where the gradation is brightest among the maximum value planes, and is considered to represent an actual image structure in the other gradation areas. Therefore, the gradation area where the flare is gathered is subjected to contrast compression so that it is hardly extracted as an edge component, and gradation conversion is performed on the maximum value surface so that the contrast of only the existing edge is extracted from the maximum value surface. Do.

  Since the contrast structure of the light source region that is one of the flare generation sources is extracted only weakly from the maximum value plane, it is necessary to recover the contrast without flare in that region. For this purpose, the minimum value plane where no flare occurs is subjected to gamma conversion opposite to the maximum value plane, and the contrast of the gradation area that could not be extracted on the maximum value plane is amplified so as to recover from the minimum value plane. To extract. On the other hand, if the image structure of the part where there are many real edges in the region where the contrast is emphasized on the one of the maximum value planes is extracted in a contrast state that is weakened on the contrary on the minimum value plane, they cancel each other out. A luminance structure that maintains an appropriate level of contrast in all gradation ranges can be extracted from the V plane.

FIG. 3 is a diagram illustrating gradation conversion of the V plane in the red and cyan hue regions. In the median plane V _med used for the V plane in the red hue region, gradation conversion is not performed. In other words, as represented by the solid line 310, the median plane V _med has a linear transformation of 1 time. It can be said that it was done. In the maximum value surface V _max used for the V surface in the cyan hue region, the contrast in the high gradation region where the flare is gathered is compressed by the gamma conversion having the square root type gradation characteristic represented by the broken line 320. . When the contrast in the high gradation region is compressed, the real edge included in the region is also weakened. Therefore, in order to recover the real edge, the minimum value surface V _min with less flare is also used as the V surface in the cyan hue region. Used for. In this minimum value plane V _min , the contrast in the high gradation region is amplified by the gamma conversion having the square gradation characteristic represented by the alternate long and short dash line 330, and the emphasized real edge can be extracted.

Therefore, the V plane generation method is as shown in the following equations (9) and (10). The V plane in the red hue region is generated according to the equation (9), and the V plane in the normal cyan hue region is generated according to the equation (10). However, it is assumed that the value of each lightness component (brightness variable) V is standardized in the range of [0, 1].

In Expression (10), the square root type gradation conversion is performed on the maximum value plane, and the square type gradation conversion is performed on the minimum value plane. However, when experimentally examined, the flare component was more suppressed by reversing the tone conversion in the cyan hue area of the original image, particularly in the high saturation area where the saturation was a predetermined value or more. Since it has been found that the V plane can be generated, Expression (11) is used to generate the V plane in the region of the high saturation portion. Expression (10) is used in a cyan hue region in the original image where the saturation is less than a predetermined value. The predetermined value is 210 ± 10 in 256 gradations, for example.

  Therefore, in order to generate a V plane with reduced flare, the generation method is changed according to the hue and saturation of the original image. This is summarized in FIG. In FIG. 4, the flare is roughly classified into two types, a hue near 0 degree, that is, a red system with about 0 to 90 degrees and about 270 to 360 degrees, and a hue with a hue near 180 degrees, that is, about 90 to 270 degrees. Since the V plane generation method is switched only in accordance with this, it is a general concept as a method of generating a brightness plane without flare. In FIG. 4, for example, when a flare correction different from a red flare or a cyan flare is performed for a green flare or a yellow flare, the hue of the red flare is about 0 to 60 degrees and about 300 to 360 degrees. In addition, the cyan flare may have a hue of about 120 to 240 degrees.

Although it is not possible to classify as described above, according to the formula (12), it is experimentally shown that the brightness surface in which the flare ring disappears can be generated effectively according to the equation (12). I have confirmed. For yellow flares, when Equation (12) is compared with Equation (9), about half of the red flares are generated on the R surface, and the other half of the flares are on the G surface. Is generated in antiphase.

<Flare refining>
In formulas (5) to (7), formulas for setting the coefficients α and β, which are functions in units of regions, are presented for locations where flare correction is performed. Here, the concept of concretely realizing the coefficient β, which is a correction area specifying function for the color difference component, will be described. In flare correction, the luminance component has an edge recovery function based on the V plane, but the chrominance component does not have that function, so that only the flare component is left from the edge component extracted from the chrominance plane, and the actual edge component is excluded. Otherwise, the color will be lost. Here, how to distinguish between the flare component and the actual edge component on the color difference plane will be described.

Normally, the diffractive flare component changes very slowly compared to the real edge structure of the image. If this property is used, if the edge strength is large with respect to the integrated edge component E (x, y) that equivalently represents the original image, the possibility of being an actual edge is very high. On the other hand, if the edge strength is small, the possibility of a flare component is high. Therefore, if the coring process is performed on the extracted edge component, the existing edge component can be eliminated. That is, the coring process attenuates the edge strength of the integrated edge component E (x, y) equivalently representing the original image (the amplitude of the value of the high-frequency image of the integrated edge component). A high-frequency image generated by multi-resolution conversion is generally generated using a second-order differential edge detection filter. The name of the Laplacian pyramid is also derived from it. Therefore, this coring process can be called a secondary flare refining in the sense of a refining process that leaves a flare component by the integrated edge component E (x, y) of the secondary differential. Flare refining refers to removing flare by removing real edges from edge components. The edge component E ′ (x, y) is extracted as a flare by the secondary flare refining shown in Expression (13).

FIG. 5 shows how the real edge and the flare are identified by the flare refining coring shown in Expression (13). As described above, the flare changes much more slowly than the actual edge with respect to the change in the pixel position. For this reason, in the pixel corresponding to the flare, the value of the integrated edge component E (x, y) of the second-order differential tends to take 0 or a value close to 0. That is, when flare occurs, as shown in FIG. 5, the number of pixels in which the value of the integrated edge component E (x, y) of the secondary differentiation becomes 0 or a value close to 0 increases. The set value of the fluctuation width σ _2nd of the value of the integrated edge component E (x, y) in the pixel corresponding to the flare is a value determined in advance using a sample image, and usually takes a large value. If the value of the fluctuation width σ _2nd is too small, a part of the flare component is not included in the edge component E ′ (x, y). If the value of the fluctuation width σ _2nd is too large, a part of the actual edge component is an edge. It will be contained in component E '(x, y). When an image is represented with 256 gradations, for example, a fluctuation value of ± 64 gradation width, which is about ¼ of the image, is set. That is, the flare fluctuation width ± σ _2nd extracted by the second derivative is ± 64.

  Among the edge components E ′ (x, y) that have undergone the second-order flare refining according to the above equation (13), real edges (real edges with small contrast) having an amplitude smaller than the set fluctuation width are flare components. There is a possibility of remaining together. If the subtraction process is performed as it is, although the countermeasure for suppressing the harmful effect is taken, the negative effect remains.

What should be considered next is a method of more actively using the property that the flare changes slowly and the edge is not easily detected. That is, when edge extraction is performed once again on a high-frequency image at each resolution generated by multi-resolution conversion, the fine structure of the actual edge is extracted, whereas the flare component is extracted by the high-pass high-pass. It becomes harder to be layered. Therefore, flare refinement is performed by referring to the high-pass high-pass, that is, the fourth-order differential edge and coring the second-order differential edge again. This can also be called fourth-order flare refining. Using the edge component E ′ (x, y) obtained by the equation (13) as a new integrated edge component E (x, y), the equation (14) using the new integrated edge component E (x, y). The edge component E ′ (x, y) is newly extracted as a flare by the fourth-order flare refining shown in FIG. ∇ ² E (x, y) represents the secondary differential value of E (x, y).

The flare fluctuation width σ _4th extracted by the fourth derivative can be set smaller than the flare fluctuation width σ _2nd extracted by the second derivative. In other words, this is equivalent to performing flare refining more accurately and increasing the separation ability of real edges and flares. The flare fluctuation width ± σ _4th extracted by the fourth derivative is normally about half of the value ± 64 of the flare fluctuation width ± σ _2nd extracted by the second derivative, as shown in the equation (15). It can be set to ± 32.

In addition, since the red flare spreads over a large area compared to the cyan flare, the fact that the contrast of the individual zonal components is small can also be used. As a result, in the reddish hue, the real edge and the flare can be further distinguished, and the image structure can be preserved. Therefore, the fluctuation widths σ _2nd and σ _4th are preferably set as a function of hue. This difference in the nature of the flare causes a difference of about half in the fluctuation width setting. That is, as shown in Expression (16), the fluctuation width σ _2nd (R) of the red flare is about half of the fluctuation width σ _2nd (Cy) of the cyan flare, and as shown in Expression (17) The fluctuation width σ _4th (R) of the red flare is about half of the fluctuation width σ _4th (Cy) of the cyan flare.

  The state of the second flare refining and the fourth flare refining is shown in FIG. The idea is to continue with 6th flare refining and 8th flare refining.

On the other hand, the idea of 0th-order flare refining can also be introduced. The 0th order represents not the edge component but the pixel value itself. That is, the intensity for performing flare correction is set according to the brightness level. This is essentially the concept introduced to the coefficient β in the equations (5) to (7). A map for performing 0th-order flare refining of the color difference component, that is, a region where the color difference component should be protected may be referred to the V _min plane. This is because diffraction flare hardly appears on the V _min surface, and even if it is colorful neon or the like, there is a high possibility that the value of the minimum value surface is small, and the dark region is considered to be a region where the image structure should be protected. Because. Therefore, when the 0th-order map of the color difference component is expressed by β ⁽⁰⁾ , the following expression (18) is obtained. That is, β ⁽⁰⁾ is a function of V _min .

A similar idea can be introduced as a map for performing 0th-order flare refining of luminance components. For the region where the luminance component should be protected, for example, the V _max plane may be referred to. This is because an area to be corrected for diffraction flare appears on the V _max plane. Therefore, when the 0th-order map of the luminance component is represented by α ⁽⁰⁾ , the following equation (19) is obtained. That is, α ⁽⁰⁾ is a function of V _max .

  A specific example of the flare correction algorithm in the present embodiment will be described with reference to FIGS. FIG. 7 is a block diagram showing a flare correction processing block. Each processing block of the flare correction processing block shown in FIG. 7 is functionally realized in the control device 103. FIG. 8 is a flowchart showing a flare correction processing procedure. Each step of the flare correction processing procedure shown in FIG. 8 is executed by the control device 103.

1) Input original image to be corrected (block B701 in FIG. 7 and step S801 in FIG. 8)
An example will be described in which an original image that has been subjected to user gamma correction for which an image formed by the optical system 106 has been imaged by the image sensor 102 and has been subjected to user gamma correction is input as a correction target image. For example, a JPEG image output by a camera corresponds to this.

2) Tone conversion (block B702 in FIG. 7 and step S802 in FIG. 8)
2-1) Inverse gamma conversion The user gamma of the input original image is solved and returned to linear gradation data. Assuming a gamma characteristic defined by sRGB and performing gradation conversion of the inverse gamma characteristic, RGB data of linear gradation is created.

2-2) Logarithmic Gamma Conversion Each linear gradation RGB data is subjected to a logarithmic gamma conversion as shown in the following equation (20) to shift the working gradation space to the logarithmic gradation space. Here, k is a parameter that adjusts the logarithmic characteristic with a constant. The value of k is set to a value sufficiently larger than 1. For example, the order is 10,000.

  In equation (20), all the variables t when performing gradation conversion are defined by being normalized within the range of t = [0, 1]. That is, it is assumed that t = R, G, B, but each RGB value is rescaled to a range of [0, 1]. In addition, since the transition process to the logarithmic gradation space is not essential, the subsequent processes may be performed with the linear gradation as it is.

3) Generation of V plane (blocks B703 to B705 in FIG. 7 and step S803 in FIG. 8)
3-1) Creation of HSVVV surface In order to generate one lightness surface (V surface) with reduced flare, the RGB values are set as shown in equations (21), (22), and (23). Each surface of V _max , V _med , and V _min rearranged in order is generated.

Further, as shown in the equations (24) and (25), a hue H plane and a saturation S plane to be referred to when integrating into one V plane are generated. This is the same as the definition of HSV space.

3-2) V-plane Gamma Conversion Two types of gamma conversion are performed on each of the V _max and V _min planes, Γ conversion shown in Expression (26) and Γ- ¹ conversion shown in Expression (27). Γ is a square root type gradation characteristic, and Γ ⁻¹ is a square type gradation characteristic. It is defined by a slightly modified function so that the derivative at the origin of Γ does not diverge. The value of the constant ε is a value sufficiently smaller than 1.

The V _max and V _min surfaces after gamma conversion obtained in this way are expressed by the following equations (28), (29), (30) and (31).

3-3) Synthesis of V-plane 3-3-1) V-plane generation of reddish hue V-plane used in reddish hues with hues in the vicinity of 0 degrees (about 0 to 90 degrees and about 270 to 360 degrees) Define (V _R ). For this, V _med is used as shown in equation (32).

3-3-2) V-plane generation of cyan hues For cyan hues with hues in the vicinity of 180 degrees (about 90 to 270 degrees), they range from low saturation to medium saturation and a fairly high saturation range. V _a used in the normal saturation region where the saturation is less than the predetermined value and V _c used in the super high saturation region where the saturation is the predetermined value or more are defined as the following equations (33) and (34).

In order to integrate the equations (33) and (34) with respect to the saturation, the V plane (V _Cy ) used in the cyan hue by weighted synthesis is defined as shown in the equation (35). The coefficient η in the equation (35) is expressed by the equation (36) and takes a value in the range of [0, 1]. Here, V _a and V _c are gently connected by adopting a Fermi distribution function for the coefficient η to be weighted and synthesized. With respect to the saturation value S to take at each pixel, to switch between _{V a} and _{V c} around the set value _{S F} ± _{S T.}

3-3-3) V-plane synthesis related to hue The V-plane (V _R ) mainly used in the red hue obtained above and the V-plane (V _Cy ) mainly used in the cyan hue are shown in Expression (37). In this way, weighted synthesis is performed to integrate the V plane as one brightness plane. The coefficient ξ takes a value in the range [0, 1].

As a coefficient for use in weighted synthesis shown in equation (37) xi], in order to loosely connect the V _R and V _Cy, showing the two types of the formula (38) and (39). As shown in the equation (38), the first changes linearly, and as shown in the equation (39), the second is Lorenchan (the same function as the resonance curve). Normally, H ₀ = 180 (Cy) and H _w = 90 are taken. In the case of Lorenchan, the value is 0 in the interval of H ₀ ± H _w , and is 1 outside the range.

4) Conversion to YCbCr space (block B706 in FIG. 7 and step S804 in FIG. 8)
Shift to the luminance / color difference space as a space where flare correction is actually performed. Conversion from RGB space to YCbCr space is performed. As shown in equation (40), the conversion equation uses a matrix of JPEG standard.

  If the above formula (40) is abbreviated, Y = 0.3R + 0.6G + 0.1B, Cb = (10/9) * (BY) / 2, Cr = (10/7) * (RY) / 2.

5) Edge extraction of Y, V, Cb, and Cr planes (blocks B707 to B712 in FIG. 7 and step S805 in FIG. 8)
5-1) Multi-resolution conversion Each color plane of YCbCr and V is subjected to multi-resolution conversion, and a low-frequency image less than a predetermined frequency of the lowest resolution that is sequentially generated and a high frequency equal to or higher than a predetermined frequency composed of a plurality of resolutions. Equivalent expressions are represented by a series of images. Typical examples of multi-resolution conversion include Laplacian pyramid expression and wavelet conversion, and either one may be used. The specific processing method in the case of wavelet transform is well known since it is described in detail in, for example, US Patent Application Publication No. 2010/0074523, up to the method of creating an integrated edge in the following processing 5-2). Not to mention. An example of the Laplacian pyramid will be described.

In the example shown in FIG. 6, the original image G ₀ is subjected to a low-pass filter (Gaussian blur) of about 5 × 5 to remove aliasing components, sub-sampled to ½ resolution, and this is a low-frequency image with low one-stage resolution. and G _1. The low-frequency image is enlarged twice, and the difference from the original image G ₀ is set as a high-frequency image L ₁ having a predetermined frequency or higher corresponding to the low-frequency image. When this process is repeated, a high-frequency image L _i is obtained as a difference between the low-frequency image G _{i + 1} and the low-frequency image G _i, and a series of the low-resolution image G ₆ having the lowest resolution and the high-frequency images L _{5 to} L ₀ having the respective resolutions. Is generated. Thus, the original image is equivalently expressed. Number of resolution is determined corresponding to the number of pixels the original image G _0, be about 1 million pixels, the number of stages of resolution is usually 5-6 stages take. That is, i = 0 to 5 or 0 to 6.

5-2) Edge integration by inverse multi-resolution conversion (second-order differential edge)
As shown in FIG. 6, of the multi-resolution transformed images in the process 5-1), after setting all the pixel values of the low-frequency image G ₆ of the lowest resolution to zero, each of the remaining resolution high-frequency image Multi-resolution integrated edge images E ₅ ^{(2) to} E ₀ ⁽²⁾ are generated by performing inverse multi-resolution conversion only on L _{5 to} L ₀ . In the integrated edge E ₀ ⁽²⁾ thus obtained, the annular component of the diffraction flare included in the original image G ₀ is expressed as it is. Since the high-frequency image L _i is obtained as a difference between the low-frequency image G _i and the low-frequency image G _i−1 , the high-pass filter that generates the high-frequency images L _{5 to} L ₀ is second-order differentiation. Therefore, the integrated edge E ₀ ⁽²⁾ is also a description representative of the second order differentiation in the entire image of the original image G ₀ . Using the Laplacian symbol Δ, the integrated edges ΔY, ΔV, ΔCb and ΔCr corresponding to the integrated edge E ₀ ⁽²⁾ of each color plane are expressed as the following equations (41) to (44).

In the equation (41) to (44), the symbol ∇ ² representing the second derivative instead of the Laplacian symbol △ may be used. The calculation of <> representing the local average can be obtained by multiplying a huge Gaussian blur filter at the actual resolution without using multi-resolution conversion.

5-3) Edge extraction from multi-resolution high-pass plane As shown in FIG. 6, for each of the series of high-frequency images L _{5 to} L ₀ of each resolution obtained by multi-resolution conversion in process 5-1), As shown in Expressions (45) to (48), a high-pass component, that is, fourth-order differential edges e _{5 to} e ₀ are extracted from the high-pass surface by applying a second-order differential edge filter. The edge filter may be subtracted from the original image by applying 5 × 5 Gaussian blur to the high-frequency images L _{5 to} L ₀ as the original image.

5-4) Edge integration by inverse multi-resolution conversion (fourth derivative edge)
As shown in FIG. 6, integrated edges E ₅ ^{(4) to} E ₀ ⁽⁴⁾ are similarly generated for the fourth-order differential edges e _{5 to} e ₀ . Using the Laplacian symbol Δ, integrated edges Δ (ΔY), Δ (ΔV), Δ (ΔCb) and Δ (ΔCr) corresponding to the integrated edge E ₀ ⁽⁴⁾ of each color plane are expressed by the following equation (49 ) To (52). The integrated edges Δ (ΔY), Δ (ΔV), Δ (ΔCb) and Δ (ΔCr) are ∇ ² (∇ ² Y), ∇ ² (∇ ² V), ∇ ² (∇ ² Cb). And ∇ ² (∇ ² Cr).

6) Flare refining of Cb and Cr edges (blocks B713 to B715 and B717 in FIG. 7 and step S806 in FIG. 8)
6-1) Fourth-order flare refining Flare refining is performed in which edges are excluded from the integrated edges of the second-order differentials of the color-difference surface equivalently representing the original image, leaving only the flare. Perform fourth-order flare refining with reference to the edge of fourth-order derivative. In addition to self-refining that refers to your own color plane, you can also introduce the idea of mutual refining that refers to other color planes. Thus, the feature that the flare changes slowly can be derived using any information for distinguishing from the edge. Only the formulas (53) to (55) for flare refining of the Cr surface are shown below. The same applies to the Cb surface. As self-refining, the calculation shown in Formula (53) is performed. Although mutual refining may not be performed, when mutual refining is also performed, the operations shown in the equations (54) and (55) are performed. It is assumed that the prime symbol 'on the left side, that is, the corrected integrated edge is sequentially substituted into the right side of the following equation. The prime symbol on the right side at that time is omitted. Moreover, you may skip the 4th refining represented by Formula (53)-(55).

Here, the values of the fluctuation widths σ _{4th_C} and σ _{4th_Y} are approximately the same. However, as explained using the equations (16) and (17), the fluctuation width of the red flare is about half of the fluctuation width of the cyan flare. Further, for example, the difference in amplification gain coefficient according to the definition formula of YCbCr conversion shown in Expression (40) needs to be considered.

6-2) Secondary flare refining Secondary flare refining is also defined in the same way as the fourth flare refining. Of the Cb and Cr components, equations (56) to (58) of only the Cr component are shown. As self-refining, the calculation shown in Formula (56) is performed. Although mutual refining may not be performed, when mutual refining is also performed, the operations shown in the equations (57) and (58) are performed. It is assumed that the prime symbol 'on the left side, that is, the corrected integrated edge is sequentially substituted into the right side of the following expression. The prime symbol on the right side at that time is omitted. In the secondary refining represented by the formulas (56) to (58), the value after the above-described fourth-order flare refining is preferably used, but such secondary refining may be skipped.

Here, the values of the fluctuation widths σ _{2nd_C} and σ _{2nd_Y} are approximately the same. The difference is the same as the precautions of the fourth-order flare refining. That is, as described using the equations (16) and (17), the fluctuation width of the red flare is about half of the fluctuation width of the cyan flare. As described using Equation (15), the flare fluctuation widths σ _{4th_C} and σ _{4th_Y} extracted by the fourth derivative are about half of the flare fluctuation widths σ _{2nd_C} and σ _{2nd_Y} extracted by the second derivative. is there. Further, for example, the difference in amplification gain coefficient according to the definition formula of YCbCr conversion shown in Expression (40) needs to be considered.

6-3) Zero-order flare refining The flare correction exclusion area of the color difference component is specified with reference to the V _min plane. A 0th-order flare refining map β (x, y) is defined as follows. That is, β returns a value in the range [0, 1] based on the brightness of V _min (x, y). The definition of β is the same as in equations (6), (7), and (18), and is 1 at a location where flare correction is performed and 0 at a location where flare correction is not performed.

Switching between 0 and 1 is performed near the constant V _F ^1/2 ± V _T ^1/2 . When V _T ^1/2 = 0, switching is performed like a step function. Therefore, it is preferable that V _T ^1/2 ≠ 0. In addition, it is preferable that flare correction is not performed when the brightness of the minimum value surface _Vmin is extremely dark. Therefore, the value of V _F ^½ takes a dark level value such that the flare is usually not conspicuous. However, since the work is performed in the logarithmic gradation space, the value takes a large value even in the dark part. In the case of an image with 256 gradations, for example, V _F ^1/2 = 150, and V _T ^1/2 is a value of about 10. The calculation of the 0th-order flare refining using the coefficient β obtained by the equation (59) is performed by the following equations (60) and (61) for each pixel. In the 0th-order refining represented by the formulas (60) and (61), it is preferable to use the value after the above-described quaternary or secondary flare refining. Equations (60) and (61) are equivalent to equations (6) and (7).

7) Flare refining of Y and V edges (blocks B716 and B717 in FIG. 7 and step S807 in FIG. 8)
7-1) Zeroth-order flare refining The calculation of the zeroth-order flare refining using the coefficient α is performed for each pixel by the following equations (62) and (63).

  The following i) to iii) are shown as three possible examples of the coefficient α in the equations (62) and (63). In the case of ii) or iii), the calculation of the following equation (64) or (65) is performed for each pixel.

i) No processing (α = 0)

ii) Specification of luminance component flare correction exclusion area by V _max surface, part 1

iii) Luminance component flare correction exclusion area specification by V _max plane 2
In the equation (65), switching between 0 and 1 is performed in the vicinity of the constant V _F ² ± V _T ² , and the value is similar to the switching of the coefficient β.

8) Flare correction (block B718 in FIG. 7 and step S808 in FIG. 8)
As shown in the equations (66) to (68), the edge components ΔY ′, ΔCb ′, and ΔCr ′ obtained by refining the flare component from the integrated edge component equivalently representing the original image are used as the color of the original image. Flare correction is performed by subtracting from components Y, Cb and Cr, respectively. Regarding the color difference component, Expressions (67) and (68) correspond to Expressions (6) and (7). Regarding the luminance component, since edge recovery has not yet been performed, Expression (66) lacks the edge recovery component ΔV ′ on the right side as compared with Expression (5).

9) Inverse color space conversion (block B719 in FIG. 7 and step S809 in FIG. 8)
The Y′Cb′Cr ′ surface is inversely converted to a flare-corrected RGB surface. As a result, R ′, G ′, and B ′ are generated.

10) Edge recovery (block B720 in FIG. 7 and step S810 in FIG. 8)
As shown in Expressions (69) to (71), edge recovery is performed for each of R ′, G ′, and B ′ in each of the RGB color planes to obtain R ″, G ″, and B ″. . Note that R ″, G ″, B ″ on the left side in the equations (69) to (71) are the corrected pixel values R ′, G ′ of the three color components in the above equations (2) to (4). , B ′.

11) Inverse gradation conversion (block B721 in FIG. 7 and step S811 in FIG. 8)
11-1) Inverse logarithmic gamma transformation The inverse transformation of the logarithmic gamma transformation performed in the process 2-2) is performed.

11-2) Gamma conversion The inverse gamma conversion performed in the process 2-1) is performed.

12) Corrected image output (block B722 in FIG. 7 and step S812 in FIG. 8)
An image with corrected diffraction flare is output.

The control apparatus 103 as the image processing apparatus in the present embodiment described above has the following operations and effects.
(1) The control device 103 acquires an original image G _{0 in} which each pixel is represented by three color components. The control device 103 rearranges the values of the three color components for each pixel in order of size to generate at least the maximum value surface V _max , the minimum value surface V _min, and the median value surface V _med , and at least the maximum value plane V _max and the minimum value plane V _min subjected to gamma conversion of gamma conversion and square-type tone characteristic of the square root type gradation characteristics with respect to the gamma converted maximum surface V _max and the minimum value plane V _min, and central One brightness component V is generated based on at least one of the value planes V _med . The control device 103 uses the correction pixel values R ′, G ′, and B ′ of the three color components and the local average values <R>, <G>, and <B of the three color components in the correction target pixels of the three color components. The correction pixel value R of the three color components is kept in correlation with the difference or ratio between the pixel value V of the lightness component and the local average value <V> of the lightness component. Find ', G' and B '. By doing so, both the cyan diffraction flare and the red diffraction flare are corrected cleanly. Even if it is in the image structure like a car headlight, the diffraction flare can be eliminated and suppressed while preserving the image structure. That is, even if the diffraction flare is superimposed on a large number of image structures as a large annular zone, the diffraction flare and the image structure can be accurately discriminated and the diffraction flare can be eliminated or suppressed.

(2) The control unit 103 obtains the original image G ₀ which each pixel consists of three color components. The control device 103 filters the three color components of each pixel to sequentially generate high-frequency images L _{5 to} L ₀ having a predetermined frequency or more having i kinds of resolutions, and the highest frequency images L _{5 to} L ₀ are obtained. The integrated edge component E (x, y) = E ₀ ⁽²⁾ corresponding to the high-frequency image integrated sequentially from the low resolution and integrated into one is generated. The size of the value of each color component in each pixel of the integrated edge component E (x, y) corresponding to the integrated high-frequency image and / or the size of the value of another color component of the three color components Based on the above, the amplitude of the value of the integrated edge component E (x, y) corresponding to the high-frequency image integrated into one is attenuated by Expression (13). A process of subtracting the edge component E ′ (x, y) corresponding to the attenuated high-frequency image from each of the three color components of the original image G ₀ is performed in the YCbCr space using equations (56) to (58). By doing so, corrected pixel values R ′, G ′ and B ′ of the corrected image of the three color components are generated. When shooting at sunset, etc., when the light source intensity is high and the exposure time is long, red and yellow diffractive flares are usually conspicuous as high-order zone components that are not conspicuous as tail parts, and these are superimposed. Is recognized as a flare that is brightly covered as a whole, but even if it is such a flare cover due to higher-order diffracted light, the effect of correction is exhibited. By taking 5 to 6 resolution steps of the high frequency images L _{5 to} L ₀ , the effect is particularly good. The present invention can be used not only for diffraction flare caused by a DOE lens (an optical system including a diffraction grating lens) but also for flare correction of flare caused by internal reflection or irregular reflection of a normal lens.

  In the above-described embodiment, an RGB color image is dealt with. However, when emerald green is present in addition to RGB such as a four-color sensor, the maximum value surface, the minimum value surface, and the two center points are also considered in the same way. A value plane can be generated to correct diffraction flare due to the DOE lens.

  As shown in FIG. 9, a computer program product for the control apparatus 103 of the camera 100 to execute image processing in the above-described embodiment is a program storage medium 2000 (for example, CD-ROM) or a communication network 4000 (for example, the Internet). Is provided to the data processing apparatus 1000 (for example, a PC) through a data signal transmitted by. The camera 100 reads an image processing program from the data processing apparatus 1000 via communication.

  The data processing apparatus 1000 is provided with an image processing program via the program storage medium 2000. Alternatively, the image processing program may be provided in the following supply form. The data processing apparatus 1000 has a connection function with the communication network 4000. The server 3000 is a computer that provides an image processing program, and stores the image processing program in a storage medium such as a hard disk. The server 3000 places the image processing program read from the hard disk on a carrier wave as a data signal, and transfers it to the data processing apparatus 1000 via the communication network 4000. As described above, the image processing program is preferably supplied by various forms of computer program products including provision as data signals via the program storage medium 2000 and the communication network 4000.

100 camera, 101 operation member, 102 image sensor, 103 control device,
104 memory card slot, 105 monitor, 106 optical system,
1000 data processing device, 2000 program storage medium, 3000 server,
4000 communication network

Claims (22)

Acquisition means for acquiring an original image represented by first to nth color components (n ≧ 3);
The values of the first to nth color components are rearranged in order of size to generate at least a maximum value surface, a minimum value surface, and a median value surface, and at least characteristics with respect to the maximum value surface and the minimum value surface Generating means for generating a lightness component based on at least one of the gamma-converted maximum value surface and minimum value surface, and the median value surface;
In the correction target pixel of the first color component, the difference between the corrected pixel value of the first color component and the local average value of the first color component is the pixel value of the lightness component and the local average value of the lightness component in the pixel. An image processing apparatus comprising: correction means for obtaining a correction pixel value of the first color component so as to maintain a correlation with respect to the difference between the first and second color components. Acquisition means for acquiring an original image represented by first to nth color components (n ≧ 3);
The values of the first to nth color components are rearranged in order of size to generate at least a maximum value surface, a minimum value surface, and a median value surface, and at least characteristics with respect to the maximum value surface and the minimum value surface Generating means for generating a lightness component based on at least one of the gamma-converted maximum value surface and minimum value surface, and the median value surface;
In the correction target pixel of the first color component, the ratio between the corrected pixel value of the first color component and the local average value of the first color component is determined by the pixel value of the lightness component and the local average value of the lightness component in the pixel. An image processing apparatus comprising: correction means for obtaining a correction pixel value of the first color component so as to maintain a correlation with the ratio of the first color component. The image processing apparatus according to claim 1 or 2,
The image processing apparatus according to claim 1, wherein the generation unit performs gamma conversion of a square root gradation on the maximum value plane and performs a square gradation gamma conversion on the minimum value plane. The image processing apparatus according to claim 1 or 2,
The image processing apparatus according to claim 1, wherein the generation unit performs a gamma conversion of a square gradation on the maximum value surface and performs a gamma conversion of a square root gradation on the minimum value surface. The image processing apparatus according to claim 1 or 2,
The generating means mainly uses the median value plane in a red hue area in the original image, and averages the gamma-converted maximum value face and minimum value face in a cyan hue area in the original image. An image processing apparatus that generates the lightness component mainly using a value. The image processing apparatus according to any one of claims 3 to 5,
The generating means performs a gamma conversion of a square tone on the maximum value plane and a square root floor on the minimum value plane in a high saturation area where the saturation in the original image is a predetermined value or more. In other areas, the maximum value plane is subjected to square root gamma conversion, and the minimum value plane is subjected to square tone gamma conversion. An image processing apparatus. The image processing apparatus according to claim 1.
The image processing apparatus according to claim 1, wherein the original image is an image represented in a logarithmic gradation space. The image processing apparatus according to claim 2,
An image processing apparatus, wherein the original image is an image expressed in a linear gradation space. Acquisition means for acquiring an original image represented by first to nth color components (n ≧ 3);
A first high-frequency image having a plurality of resolutions is sequentially generated by filtering the first color component, and the first high-frequency image is sequentially integrated from the lowest resolution to be integrated into a second high-frequency image. First generation means for generating an image;
The values of the first to nth color components are rearranged in order of size to generate at least three brightness planes including a maximum value plane, a minimum value plane, and a median plane, and the generated three brightness planes A second generation unit that selectively weights and synthesizes values based on the region unit to generate one brightness component;
The brightness component is filtered to generate a third high-frequency image having a plurality of resolutions sequentially, and the third high-frequency image is sequentially integrated from the lowest resolution to be integrated into one. Third generation means for generating
By subtracting the second high frequency image integrated into one of the first color components from the first color component of the original image and adding the fourth high frequency image integrated into one of the lightness components, the first An image processing apparatus comprising: a fourth generation unit configured to generate a color component correction image. The image processing apparatus according to claim 9.
The image processing apparatus according to claim 1, wherein the first color component is a luminance component. The image processing apparatus according to claim 9.
The second generation means uses a value based on the maximum value plane and the minimum value plane in a cyan hue region in the original image, and uses a value based on the median value plane in a red hue region in the original image. An image processing apparatus that generates the one brightness component. The image processing apparatus according to claim 9.
The image processing apparatus according to claim 1, wherein the value based on the maximum value surface and the minimum value surface is a value based on a surface obtained by performing different gamma conversion on the maximum value surface and the minimum value surface. Obtaining means for obtaining an original image comprising at least one color component;
Filtering the one color component to sequentially generate a first high-frequency image having a plurality of resolutions, and sequentially integrating the first high-frequency images from the lowest resolution into a second one. First generation means for generating a high-frequency image;
Attenuating means for attenuating the amplitude of the value of the second high-frequency image integrated into one based on the magnitude of the value itself in each pixel of the second high-frequency image integrated into the one;
An image processing apparatus comprising: second generation means for generating a corrected image of the one color component by subtracting the attenuated second high-frequency image from one color component of the original image. . Acquisition means for acquiring an original image comprising at least two color components;
Each of the at least two color components is filtered to sequentially generate a first high-frequency image having a plurality of resolutions, and the first high-frequency images are sequentially integrated from the lowest resolution to be integrated into one. First generating means for generating a second high-frequency image for each color component;
The second of the at least two color components is based on the magnitude of the value at each pixel of the second high frequency image integrated with the one of the first color components of the at least two color components. Attenuating means for attenuating the amplitude of the value of the second high frequency image integrated into said one of the color components;
And a second generation unit configured to generate a corrected image of the second color component by subtracting the attenuated second high-frequency image from the second color component of the original image. Image processing device. Obtaining means for obtaining an original image comprising at least one color component;
Filtering the one color component to sequentially generate a first high-frequency image having a plurality of resolutions, and sequentially integrating the first high-frequency images from the lowest resolution into a second one. First generation means for generating a high-frequency image;
A third high-frequency image is generated at each resolution by further filtering the one color component in the sequentially generated first high-frequency images having a plurality of resolutions, and the third high-frequency image is sequentially generated from the lowest resolution. And a second generation means for generating a fourth high-frequency image integrated into one,
Attenuating means for attenuating the amplitude of the value of the second high-frequency image integrated into one based on the magnitude of the value itself in each pixel of the fourth high-frequency image integrated into the one;
An image processing apparatus comprising: a third generation unit configured to generate a corrected image of the one color component by subtracting the attenuated second high-frequency image from one color component of the original image. . Acquisition means for acquiring an original image comprising at least two color components;
Each of the at least two color components is filtered to sequentially generate a first high-frequency image having a plurality of resolutions, and the first high-frequency images are sequentially integrated from the lowest resolution to be integrated into one. First generating means for generating a second high-frequency image for each color component;
In each of the sequentially generated first high-frequency images of a plurality of resolutions of the first color component of the at least two color components, the first high-frequency image is further filtered to obtain a third high-frequency image at each resolution. A second generation means for generating the fourth high-frequency image integrated in one by sequentially integrating the third high-frequency image from the lowest resolution;
Based on the magnitude of the value at each pixel of the fourth high-frequency image integrated into the one of the first color components, integrated into the one of the second color components of the at least two color components. Attenuating means for attenuating the amplitude of the value of the second high-frequency image obtained;
And third generation means for generating a corrected image of the second color component by subtracting the attenuated second high-frequency image from the second color component of the original image. Image processing device. The image processing apparatus according to any one of claims 13 to 16,
An image processing apparatus according to claim 1, wherein the color component of the generated corrected image is a color difference component. The image processing apparatus according to claim 14 or 16,
The image processing apparatus according to claim 1, wherein the first color component is a color difference component or a luminance component. The image processing apparatus according to any one of claims 13 to 16,
The image processing apparatus, wherein the attenuation means increases the degree of attenuation as the size increases. The image processing apparatus according to claim 13 or 14,
The attenuation means attenuates the amplitude of the value of the second high-frequency image integrated into the one by a Gaussian distribution function having the pixel value of the second high-frequency image integrated into the one as an argument, and An image processing apparatus characterized by increasing the degree of attenuation as the value of is increased. The image processing apparatus according to any one of claims 13 to 20,
The attenuation means attenuates the amplitude of the value of the second high-frequency image integrated into one smaller in the cyan hue region than in the red hue region in the original image. Processing equipment. The image processing apparatus according to any one of claims 1, 2, 9, and 13 to 16.
The image processing apparatus according to claim 1, wherein the original image is an image obtained by capturing an image formed by an optical system including a diffraction grating lens.

Images