JP4719559B2 - Image quality improving apparatus and program - Google Patents

Description

  The present invention automatically adjusts the image quality of a color image with color, particularly the brightness and color of a color image input by a digital camera, a mobile phone camera, a scanner device, a surveillance camera, etc., with an effective intensity, The present invention relates to a method, an apparatus, and a program for improving an image so that the image looks beautiful while maintaining the atmosphere of the image.

  In the field of image processing and visual interface technology using images, together with a high-accuracy image input mechanism, the input image is converted into an image quality that humans feel beautiful without going through special human operations. There is a growing need.

  A clean image considered by the present invention is an image that has been naturally corrected while improving the visibility of the brightness and color of the image while maintaining the atmosphere of the original image.

  As this type of image quality improvement method, there are a method of linearly converting each pixel value of an image, an exponential conversion method represented by gamma correction, and a logarithmic conversion method. In addition, when the image is to be extremely enhanced, there is a method of enhancing the contrast of the pixel value of the image by histogram flattening.

  Furthermore, the range is divided into several sections according to the luminance sensitivity characteristic curve of humans, line segments that approximate the curve are obtained for each section, and these line segments are combined to convert pixel values. Linear transformation methods (Takagi, Shimoda (supervised) “Image Analysis Handbook”, The University of Tokyo Press, 1991).

  FIG. 1 is a diagram for explaining an image contrast enhancement method using linear transformation, FIG. 2 is a diagram for explaining an image contrast enhancement method using exponential transformation, and FIG. 3 is a logarithmic transformation. It is a figure for demonstrating the method of the contrast enhancement of the image by A.

  The conventional method will be described with reference to FIGS.

のように定義し、この線形変換によって入力画像の画素値ｖをｆ（ｖ）に変換する。この際、利用者は強調したい画素値の区間（例えばｂ≦ｖ≦ａ）での１次変換式の傾きを大きく設定することが、行われている。 As shown in FIG. 1, the image contrast enhancement method by linear transformation divides the interval of the pixel value v of the input image into several parts such as v ≦ b, b ≦ v ≦ a, and a ≦ v. A linear conversion formula that converts pixel values in the interval

The pixel value v of the input image is converted to f (v) by this linear conversion. At this time, the user sets a large slope of the primary conversion equation in a pixel value interval (for example, b ≦ v ≦ a) to be emphasized.

  The method of image contrast enhancement by exponential conversion is called gamma correction in the field of television technology, and converts the pixel value v of the input image to g (v) based on the exponential function as seen in FIG. At this time, the degree of contrast enhancement is determined by the slope of the curve at each point of the exponential function in FIG. 2, and the contrast is enhanced in a large range of pixel values v (b ≦ v ≦ a in the example of FIG. 2). The exponential transformation formula is applied to the entire input image in the same manner.

  As seen in FIG. 3, the image contrast enhancement technique using logarithmic conversion converts the pixel value v of the input image into h (v) based on a logarithmic function. At this time, the degree of contrast enhancement is determined by the slope of the curve at each point of the logarithmic function in FIG. 3, and the contrast is enhanced in a small range of pixel values v (b ≦ v ≦ a in the example of FIG. 3). The curve shape, that is, the logarithmic transformation formula, is applied in the same manner to the entire input image.

  The method based on histogram flattening is a method in which a cluster of pixel value distributions are appropriately distributed around the image and contrast enhancement is executed according to the distribution. It has become.

Japanese Patent Publication No. 7-37084 Japanese Patent Laid-Open No. 10-198801 Takagi, Shimoda (supervised) “Image Analysis Handbook”, The University of Tokyo Press, 1991

  However, in the method based on the linear conversion, the mapping of the output pixel value with respect to the input pixel value is converted into the primary conversion equation as shown by the broken line shown in FIG. 1 according to the image quality while the user observes each input image. The generality of the conversion formula cannot be obtained.

  In the method based on exponential conversion, pixel values are converted according to a predefined conversion formula, but the contrast of a section with a large pixel value (b ≦ v ≦ a in the example of FIG. 2) is enhanced, The contrast in the interval between the small values (v ≦ b in the example of FIG. 2) deteriorates.

  Similarly, in the method using logarithmic conversion, the pixel value is converted according to a predefined conversion formula, but the contrast of a section having a small pixel value (b ≦ v ≦ a in the example of FIG. 3) is enhanced. On the other hand, the contrast of a section having a large pixel value (a ≦ v in the example of FIG. 3) deteriorates.

  In the method based on histogram flattening, the global contrast enhancement effect is remarkable, while there is a problem that a relatively small local contrast indicating a change in the region is suppressed. Therefore, the processing result may be unnatural and the original image may be lost.

  To solve these problems, non-linear correction that could not be obtained with conventional conversion functions, and enhanced contrast without emphasis on humans by emphasizing the contrast of locally dark and locally bright areas that are difficult for humans to see An image quality improvement method and apparatus have been proposed (Japanese Patent Publication No. 7-37084).

  Also proposed are an image quality improvement method, an edge strength calculation method, and an apparatus for realizing the above nonlinear correction and enhancing not only the local contrast but also the global contrast, and automatically and adaptively processing the image. Yes (Japanese Patent Application No. 9-2773)

  The above method is intended for correction of luminance information, and does not consider chromaticity information, so the above method cannot be applied to a color image. Also, in the LMS color space corresponding to the R, G, and B values of human photoreceptor cells, the processing for brightness contrast and the processing for color contrast are different, so we tried to make it applicable to color images using the same method as above. Even effective image correction cannot be performed.

  In addition, the function that draws an inverse S-shaped curve in which the correspondence of the output value to the input value used in this method (Fig. 4) has the effect of helping the sensitivity of dark and bright areas where human visual sensitivity is reduced. Although it is very effective for black and white image processing, it is difficult to apply the same method to color images due to the problem of dropping colors when the method is applied to color images that contain color. .

  In the LMS space of human visual cells, processing for luminance and processing for hue are performed in the receptive field as shown in FIG. Humans have a visual characteristic function called lateral inhibition that is generated by the action of this receptive field. Humans naturally adjust brightness and hue contrast by the action of side suppression. The image improvement method of the present invention pays attention to the function of visual characteristics called human side suppression, and makes the function a mathematical model so that a color image input by a digital sensor can be changed according to the distribution of the image. Thus, while maintaining the atmosphere of the original image, the brightness contrast (luminance contrast) and the color contrast (color contrast) are automatically enhanced with an effective intensity.

The receptive field for luminance is called brightness contrast type. The receptive field for hue is called color contrast type. Both receptive fields have On-center type and Off-center type, and they interact. For example, the brightness contrast type works by suppressing the ambient brightness when bright light enters a certain point by the action of the on-center type. On the other hand, when the light around a certain point is excited by the off-center function, the light at a certain point is suppressed. On the other hand, in the color contrast type, when the red light enters a certain point, the surrounding green light is suppressed instead of the surrounding red light by the action of the On-center type. On the other hand, when the red light around a point is excited by the off-center action, the red light at a point is not suppressed, but the green light at a point is suppressed. Thus, with regard to color contrast, human visual cells do not process LMS cone cells corresponding to RGB values independently, but L cones and M cones (that is, R values: red and G values: Green) and L cone / M cone and S cone (ie, Y value: yellow and B value: blue). In the present invention, the V circuit, the RG circuit, and the YB circuit are corrected independently, and the V circuit has a function that draws an inverted S-shaped curve that helps the sensitivity of dark and bright areas where human visual sensitivity decreases. When used (Fig. 4), the contrast and sharpness of brightness are emphasized, and the RG and YB circuits have S-shaped curves that increase the contrast between the R and G values and the contrast between the Y and B values, respectively. Is used (FIG. 6) to enhance hue contrast and vividness. Therefore, in the present invention, the RGB value is converted into a V value representing brightness, a C _RG value representing the color difference between R and G, and a C _YB value representing the color difference between Y and B, and correction is performed for each circuit. .

FIG. _6A shows an S-shaped curve used for emphasizing the color difference (C _RG value), the horizontal axis is the C _RG value of the original image, and the vertical axis is corrected. C _RG value (both ordinate and abscissa are -255 for green (G) and 255 for red (R)). FIG. 6B shows an S-shaped curve used for emphasizing the color difference (C _YB value), the horizontal axis is the C _YB value of the original image, and the vertical axis is the corrected C _YB. (-255 corresponds to blue (B) and 255 corresponds to yellow (Y) on both the vertical and horizontal axes).

  That is, the present invention provides a color difference deriving step for deriving a color difference from an original image, and a correction color difference deriving step for deriving a corrected color difference by correcting the color difference derived in the color difference deriving step using an S-shaped curve. And an image quality improvement image generation step for generating an image with improved image quality using the correction color difference derived in the correction color difference deriving step.

  The above invention is a luminance deriving step for deriving the luminance V from the original image, and a corrected luminance for deriving the corrected luminance by correcting the luminance V derived in the luminance deriving step using an inverse S-shaped curve. A deriving step, and the image quality improved image generating step may generate an image with improved image quality using the corrected color difference and the corrected luminance.

The color difference in the above invention may be a C _RG value representing a color difference between R and G and a C _YB value representing a color difference between Y and B. In this case, the color difference deriving step may derive the C _RG value and the C _YB value by conversion from the RGB value of the original image. Similarly, the luminance V can be derived by conversion from RGB values of the original image. In this case, it is also possible to derive and correct color differences and / or luminances for all RGB values extracted from all pixels of the original image, and RGB values extracted from some pixels of the original image. It is also possible to derive and correct color differences and / or brightness only for.

  The correction color difference deriving step in the above invention may further comprise the step of determining the S-shaped curve using the upper and lower limits of the color difference range.

In the correction color difference deriving step in the above invention, after correcting the color difference using an S-shaped curve, for example, the correction color difference is derived by normalizing the color difference from the minimum value to the maximum value range of the original image. Can be. Further, correction variables (R _{On + Off} , G _{On + Off} , B _On , Y _Off ) are determined based on the RGB values obtained from the pixel of interest and its neighboring pixels, and the values normalized as described above are obtained. It is also possible to derive the correction color difference by further fine adjustment using the correction variables (R _{On + Off} , G _{On + Off} , B _On , Y _Off ).

Corrected luminance deriving step in the invention derives image pixel value variable alpha _On and alpha _Off, further comprising the step of determining the inverse S-shaped curve using the image pixel value variable alpha _On and alpha _Off Can do.

  The corrected luminance deriving step in the above invention derives the corrected luminance by normalizing from the lower limit to the upper limit of the range that the luminance can take, for example, after correcting the luminance using an inverted S-shaped curve. Can be.

  The present invention can also be configured as an image quality improvement program that causes a processing device to execute each step in each of the above inventions.

The present invention can also be configured as an image quality improvement apparatus provided with respective mechanisms for executing the respective steps in the above inventions.
Note that the S-shaped curve in the present invention is a curve (U = f (u)) for converting the minimum value of the gamut of the color difference (u) to the maximum value of the corrected color difference (U). In the range where (u) is zero from the minimum value, the second derivative (d ² f (u) / d ² u) of the curve is non-negative (ie, zero or positive), and the color difference (u) is zero. To the maximum value range, it refers to a curve in which the second derivative (d ² f (u) / d ² u) of the curve is non-positive (that is, zero or negative).

The inverted S-shaped curve in the present invention is a curve (V = g (v)) for converting the minimum value of the range of luminance (v) into the corrected luminance (V) (V = g (v)). In the range of the intermediate value v) from the minimum value, the second derivative (d ² g (v) / d ² v) of the curve is non-positive, and the luminance (v) is In the range that is the maximum value, it means a curve in which the second derivative (d ² g (v) / d ² v) of the curve is non-negative.

The image improvement apparatus of the present invention includes (a) an input mechanism for inputting image information, (b) a V input mechanism for inputting a V value representing luminance from a value obtained by the input image, and (c). An RG input mechanism for inputting a C _RG value, which is a color difference between the R value and the G value, from the value obtained from the input image, and (d) a color difference between the Y value and the B value from the value obtained from the input image. A YB input mechanism for inputting a C _YB value; (e) a V-neighbor image extraction mechanism that receives an image from the V-input mechanism and extracts a pixel value of a nearby V including the pixel for each pixel; An RG neighborhood image extraction mechanism that receives an image from the RG input mechanism and extracts a pixel value of a neighboring _CRG including the pixel for each pixel; and (g) receives an image from the YB input mechanism, and receives the pixel for each pixel. receiving and YB neighboring images extracted mechanism to retrieve the pixel values of C _YB near, the neighboring images from (h) the V neighboring images extracted mechanism including, the picture And V neighboring image pixel value variance range analysis mechanism for measuring the local domain of the V in the vicinity of, the (i) receives the neighboring images from the RG neighboring images extracted mechanism, local domain of C _RG value of the pixel neighboring and RG neighboring image pixel value variance range analysis mechanism for measuring, (j) the YB receive neighboring images from the vicinity extraction mechanism, YB neighboring image pixel value variance range analysis to measure the local domain of C _YB value of the pixel neighborhood And (k) based on the results of the V input mechanism and the V neighborhood image pixel value domain analysis mechanism, the lower limit value of the image, the standard deviation value, the average value of the entire image, and the average value of the neighborhood of the image V _On-center image pixel value variable determination mechanism for determining the correction amount of the pixel value at the center of the neighborhood, and (l) the upper limit value of the image based on the results of the V input mechanism and the V neighborhood image pixel value domain analysis mechanism , Standard deviation value, average value of entire image, correction value of pixel value at the center of the V neighborhood by the average value of the neighborhood of the image V _off-center image pixel value variable determining mechanism for determining V, and (m) a variable obtained by the V _On-center image pixel value variable determining mechanism and the V _Off-center image pixel value variable determining mechanism. Correct the _On-center image pixel value and the V _Off-center image pixel value, and determine the pixel value of the V-neighbor center pixel with a function that draws an inverse S-shaped curve that can be obtained by the logarithm of these ratios. A V pixel value correcting mechanism that performs (n) a V pixel value normalizing mechanism that normalizes the pixel value obtained by the V pixel value correcting mechanism, and (o) the RG input mechanism and the RG neighboring image pixel value variable. An RG pixel value correction mechanism for determining an overall correction amount with an S-shaped curve such as a sigmoid function based on the result of the area analysis mechanism, and (p) a pixel value obtained by the RG pixel value correction mechanism. decisions and RG pixel value normalizer to normalize the correction variables R _{On + Off-center} of R by the variable obtained from (q) the RG neighboring images extracted mechanism An R-neighbor image pixel value adjustment mechanism that finely adjusts the value obtained by the RG normalization mechanism, and (r) a G-neighbor center pixel value obtained from the RG-neighbor image extraction mechanism and a neighboring R average value. G correction variable G _{On + Off-center} is determined based on G neighboring image pixel value adjustment mechanism for finely adjusting the G value obtained from the RG pixel value normalization mechanism, and (s) the R pixel value adjustment A C _rg conversion mechanism for converting the R and G pixel values obtained by the mechanism and the G pixel value adjustment mechanism into a color difference between the R and G values, and (t) the YB input mechanism and the YB neighboring image pixel value range. Based on the result of the analysis mechanism, a YB pixel value correction mechanism that determines an overall correction amount with an S-shaped curve such as a sigmoid function, and (u) a pixel value obtained by the YB pixel value correction mechanism. YB pixel value normalization mechanism to normalize, and (v) the local Y average pixel value including the pixel value of the Y neighborhood center obtained from the YB neighborhood image extraction mechanism and the pixel value of the B neighborhood center Based on the B average between local, including, to determine the correction variable amount B _On-center of B values, and B neighboring elementary image pixel value determining mechanism for adjusting the B value obtained from the YB pixel value normalizer, ( u) Y value correction variable amount Y _Off based on the local Y average including the pixel value of the Y neighborhood center obtained from the YB neighborhood image extraction mechanism and the local B average including the pixel value of the B neighborhood center a Y neighborhood elementary image pixel value adjustment mechanism that determines _-center and finely adjusts the Y value obtained from the YB pixel value normalization mechanism; (w) the B neighborhood image elementary value decision mechanism and the Y neighborhood image pixel _Cyb conversion mechanism that converts Y and B pixel values obtained by the value adjustment mechanism into Y and B color differences, and (x) V pixel value and C obtained by the V pixel value normalization mechanism integrating pixel values of C _yb obtained in the pixel value and C _yb conversion mechanism of C _rg obtained by _rg conversion mechanism, and the rgb conversion mechanism for obtaining the modified rgb values, from (y) year rgb conversion mechanism Each pixel image Receives the value, the image quality improving device characterized by comprising an output mechanism for outputting to form a complete image.

  According to the above image quality improvement method and apparatus, the input color image is adapted to the distribution of the pixel values of the brightness of the input image, the color difference between the R value and the G value, and the color difference between the Y value and the B value. It is possible to adjust both the distribution of the pixel values of the entire image and the distribution of the local pixel values, and the brightness contrast and the color contrast of the section having a small pixel value are enhanced. The brightness contrast and color contrast of the large value section can also be automatically enhanced.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 7 is a flowchart for explaining the image quality improvement method according to the first embodiment of the present invention, and FIG. 8 is a block diagram of an apparatus for carrying out this image quality improvement method.

First, an image is input by an input mechanism such as an image sensor or a monitor camera. This input mechanism converts R, G, B values into V, C _RG , C _YB values, for example, as in the following equation.

The V input mechanism receives an image from the input mechanism. Each pixel value of this V input image is represented by V _L as the lower limit of the possible range, V _{U as} the upper limit, Vgmean as the average value of the V value of the entire input image, and Vhensa as the standard deviation of the V value of the entire input image Shall.

The V neighborhood image extraction mechanism receives an image from the V input image, extracts a partial image including the neighborhood area for each image in the image, and sets it as a nearby V _On-center image. As an example of the neighborhood, the pixel value of the pixel of interest is V _0, and the pixel values of the 8 neighborhood pixels (FIG. 9) are V ₁ , V ₂ , V ₃ , V ₄ , V ₅ , V ₆ , V _{7, respectively.} the maximum value Vmax = max in _{V 8 (V 0, ···,} V 8), the minimum value _{Vmin = min (V 0, ···} , V 8), the average value Vlmean = (V ₀ + ··· + V ₈₎ / 9 is used for explanation.

The V _On-center image pixel value variable determination mechanism receives the minimum value Vmin of the local domain of the pixel value from the V neighborhood image pixel value domain analysis mechanism. For example, the V _On-center image pixel value variable α _On is calculated by the following equation.

The V _Off-center image pixel value variable determination mechanism receives the maximum value Vmax of the local domain of the pixel value from the V neighborhood image pixel value domain analysis mechanism. For example, the V _Off-center image pixel value variable α _Off is calculated by the following equation.

によって注目する画素の画素値V₀を補正する。この式は、V_On-center画像画素値変量α_Onによって補正されたV_On-center画像の画素値と、V_Off-center画像画素値変量α_Offによって補正されたV_Off-center画像画素値の比の対数を求めていることに相当している。

同様に、以下の諸量、すなわち、各画素値が取り得る変域の下限V_Lおよび上限V_Uに関してもk(V_L),k(V_U)を求める。

The V pixel value correction mechanism receives the pixel value V ₀ of the pixel of interest from the V neighborhood image pixel value domain analysis mechanism, and the V _On-center image pixel value variable α _On from the V _Off-center image pixel value variable determination mechanism. to receive, it receives the V _Off-center image pixel value variable alpha _Off than V _Off-center image pixel value variable determining mechanism. For example, the pixel value function k (V) defined by

To correct the pixel value V ₀ of the pixel of interest. This equation, V _On-center and the image pixel value variable α pixel value of V _On-center image corrected by _On, V _Off-center image pixel value variable α of V _Off-center image pixel value corrected by _Off This is equivalent to finding the logarithm of the ratio.

Similarly, k (V _L ) and k (V _U ) are also obtained for the following various quantities, that is, the lower limit V _L and the upper limit V _U of the domain that each pixel value can take.

V画素値正規化機構は、上式によって正規化された画素値K(V₀)を次の３式によってさらに正規化することも可能であり、これにより、画素値K(V₀)がより確実にV_LからV_Uの範囲内に収まようにできる。得られる値によって補正されたV値（v値）が求まる。

The V pixel value normalization mechanism receives the pixel value K (V ₀ ) of each pixel value normalized to the range of V _L and V _U from the V pixel value correction mechanism, and the pixel value k (V ₀ ) is normalized to a range from V _L to V _U and is set to K (V ₀ ).

The V pixel value normalization mechanism can further normalize the pixel value K (V ₀ ) normalized by the above equation using the following three equations, and thus the pixel value K (V ₀ ) can be further increased. It can be surely kept within the range of V _L to V _U. A V value (v value) corrected by the obtained value is obtained.

The RG input mechanism receives an image from the input mechanism. Each pixel value of the RG input image, the lower limit of the C _{RG L} for the variable area that can be taken, denote the C _{RG U} limit.

A neighboring image is received from the RG neighboring image extraction mechanism, and is determined as a neighboring RG image by the RG neighboring image pixel value domain analysis mechanism that measures the local domain of the C _RG value near the pixel. As an example of the neighborhood, the pixel value of the image of interest is C _R-G0, and the _R pixel value of the eight neighboring images is R o, R ₂ , R ₃ , R ₄ , R ₅ , R ₆ , R _7, R ₈ , maximum value Rmax = max (R ₀ , ..., R ₈ ), minimum value Rmin = min (R ₀ , .., R ₈ ), average value Rlmean = (R ₁ +... + R ₈₎ / 8 and G pixel values of images in the vicinity of 8 are respectively G ₁ , G ₂ , G ₃ , G ₄ , G ₅ , G ₆ , G _7, G ₈ maximum value Gmax = max (G ₀ , ..., G ₈ ), minimum value Gmin = min (G ₀ , ..., G ₈ ), average value Glmean = Explanation will be made using (G ₁ +... + G ₈₎ / 8.

（λは定数）
によって注目する画素の画素値C_R-G0を補正する。この式は、R-G近傍画像画素値変域解析機構より、注目する画素の画素値C_R-G0を受け取り、C_{R-G L,} C_{R-G Un} により、関数の曲率が定めている。

（λは定数）
同様に、以下の諸量、すなわち、各画素値が取り得る変域の下限C_{R-G L} ,および上限C_{R-G U}に関してもl(C_{R-G L}),l(C_{R-G U}),l(C_{R-G 0})を求める。

（λは定数） The RG pixel value correction mechanism is a function l (C _RG ) that draws an S-shaped curve like the sigmoid function for the overall correction strength of the image.

(Λ is a constant)
To correct the pixel value C _R-G0 of the pixel of interest. This expression receives the pixel value C _R-G0 of the pixel of interest from the RG neighborhood image pixel value domain analysis mechanism, and the curvature of the function is determined by C _{RG L,} C _{RG Un} .

(Λ is a constant)
Similarly, with respect to the following various quantities, that is, the lower limit C _{RG L} and the upper limit C _{RG U} that each pixel value can take, l (C _{RG L} ), l (C _{RG U} ), l (C _{RG 0} )

(Λ is a constant)

RG pixel value normalization mechanism
The RG pixel value normalization mechanism receives the pixel value l (C _RG ) of each pixel value normalized to the range of C _{RG L,} C _{RG Un} from the RG pixel value correction mechanism. l (C _{RG 0} ) is normalized to a range from C _{RG L} to C _{RG U} and is set to L (C _{RG 0} ).

The R neighborhood image pixel value adjustment mechanism determines a correction variable of R based on the pixel value of the R neighborhood center obtained from the RG neighborhood image extraction mechanism and the G average value of the neighborhood, and determines the R of the color contrast type receptive field. The overall correction of the image obtained by the RG pixel value normalization mechanism is finely adjusted from a local surface by an expression representing the function. For example, the R neighborhood image pixel value variable R _{On + Off} value is determined by the following equation. By adding the R _{ON + OFF} value to the original R value obtained by the input image, the R value (r ′ value) corrected by the RG circuit is obtained.

The G neighborhood image pixel value adjustment mechanism determines the G correction variable based on the G neighborhood center pixel value and the neighborhood R average value obtained from the RG neighborhood image extraction mechanism, and the function of G in the color contrast type field The overall correction of the image obtained by the RG pixel value normalization mechanism is finely adjusted from a local surface by an expression such that For example, the G neighborhood image pixel value variable G _{On + Off} value is determined by the following equation. The G value (g ′ value) corrected by the RG circuit is obtained by adding the G _{ON + OFF} value to the original G value obtained by the input image.

The C _rg conversion mechanism converts the color difference between the R and G values from the values obtained from the R neighborhood image pixel value variable and the G neighborhood image pixel value variable using the following equation.

The YB input mechanism receives an image from the input mechanism. Assume that each pixel value of this YB input image represents the lower limit of the range that can be taken as C _{YB L} and the upper limit as C _{YB U.}

Receive neighboring images from YB neighboring images extracted mechanism, by the pixel near the C _YB values of YB neighboring image pixel value variance range analysis mechanism for measuring the local domain, and YB image neighborhood. As an example of the neighborhood, since the number of cells in the YB circuit is very small compared to the cells in the RG circuit, the pixel value of the image of interest is C _{YB 0} , the B pixel value and the eight neighboring pixels. The pixel values of B are B ₀ , B ₁ , B ₂ , B ₃ , B ₄ , B ₅ , B ₆ , B _7, B ₈ and their average values Blmean = (B ₀ + ... + B ₈₎ / 9 and its Y pixel values and Y pixel values of its 8 neighboring pixels are Y ₀ , Y ₁ , Y ₂ , Y ₃ , Y ₄ , Y ₅ , Y ₆ , Y _7, Y ₈ and their average values, respectively. This will be described using Ylmean = (Y ₀ +... + Y ₈₎ / 9.

（λは定数）
同様に、以下の諸量、すなわち、各画素値が取り得る変域の下限C_{R-G L} ,および上限C_{R-G U}に関してもl(C_{R-G L}),l(C_{R-G U}),l(C_{R-G 0})を求める。

（λは定数） The YB pixel value correction mechanism determines the overall correction strength of an image by a function that draws an S-shaped curve such as a sigmoid function. The pixel value C _{YB 0} of the pixel of interest is received from the YB neighborhood image pixel value domain analysis mechanism, and the curvature of the function is determined by C _{YB L and} C _{YB U.} For example, the pixel value function l (C _YB ) defined by

(Λ is a constant)
Similarly, with respect to the following various quantities, that is, the lower limit C _{RG L} and the upper limit C _{RG U} that each pixel value can take, l (C _{RG L} ), l (C _{RG U} ), l (C _{RG 0} )

(Λ is a constant)

YB pixel value normalization mechanism
The YB pixel value normalization mechanism receives the pixel value l (C _YB ) of each pixel value normalized to the C _{YB L,} C _{YB U} range from the YB pixel value correction mechanism. (C _{YB 0} ) is normalized to a range from C _{YB L} to C _{YB U} and is set to L (C _{YB 0} ).

The B neighborhood image pixel value adjustment mechanism determines the B correction variable based on the B neighborhood average pixel value including the B neighborhood center and the Y neighborhood average pixel value including the Y neighborhood center obtained from the YB neighborhood image extraction mechanism. The overall correction of the image obtained by the YB pixel value normalization mechanism is finely adjusted from the local surface by an expression that expresses the function of B of the contrast type receptive field. For example, the B value corrected by the YB circuit (g ′ value) is obtained by adding the B _ON value that determines the B neighborhood image pixel value variable B _On value to the original B value obtained by the input image by the following equation: .

The Y neighborhood image pixel value adjustment mechanism determines the Y correction variable based on the Y neighborhood average pixel value including the Y neighborhood center and the B neighborhood average pixel value including the B neighborhood center obtained from the YB neighborhood image extraction mechanism. The overall correction of the image obtained by the YB pixel value normalization mechanism is finely adjusted from a local surface by an expression representing the function of Y of the contrast type receptive field. For example, the Y neighborhood image pixel value variable Y _Off value is determined by the following equation. The Y value corrected by the YB circuit (y ′ value) is obtained by adding the Y _OFF value to the original Y value obtained from the input image.

The _Cyb conversion mechanism converts the color difference between the Y and B values from the values obtained from the B neighborhood image pixel value variable and the Y neighborhood image pixel value variable using, for example, the following equation.

The rgb conversion mechanism converts the corrected luminance (v value) and color difference (C _rg , _Cyb value) obtained by the V pixel value normalization mechanism, C _RG conversion mechanism, and C _YB conversion mechanism into an rgb value. . For example, it is obtained by simultaneous equations as shown below.

  The image output mechanism receives the normalized pixel values (r value, g value, b value) of each pixel from the rgb conversion mechanism, and configures and outputs the entire image.

For reference, FIG. 10 shows how the brightness contrast, sharpness, hue contrast, and vividness of the image are improved in this embodiment compared to the result of processing using histogram smoothing. Show. FIG. 10 is a photograph of a part of a poster taken with a mobile phone camera. 8A is an original image, FIG. 8B is an output image corrected by the apparatus proposed in the present invention, and FIG. 8C is an output image processed using histogram smoothing.
In (c), the blue color that did not exist in (a) appears in the image of the girl on the right side, while (b) shows the outline of the building in the upper left and the overall It can be seen that the vividness also increases, the visibility increases, and natural correction is made while maintaining the original atmosphere.

  As described above, according to this embodiment, by building a model based on the function of side suppression, the input original image can be naturally corrected while maintaining the atmosphere of the original image. There are effects such as instant correction of photos taken with a mobile phone and automatic correction of images with poor brightness and hue contrast.

  Since the V circuit, RG circuit, and YB circuit are each independently corrected, it is possible to change only the brightness, change only a specific hue, or change the brightness and hue at the same time. There are effects such as freely changing the brightness, hue, and saturation.

  As described above, according to the present invention, for example, the brightness and hue contrast of a color image including colors input from an image sensor, a monitor camera, or the like can be naturally corrected without changing the original atmosphere.

The figure for demonstrating the method of the contrast emphasis of the image by linear transformation. The figure for demonstrating the method of the contrast emphasis of the image by an exponential transformation. The figure for demonstrating the method of the contrast enhancement of the image by logarithmic transformation. A function that draws an inverse S-shaped curve in which the correspondence between the output value and the input value used for luminance value correction. Explanatory diagram showing the mechanism for adjusting the contrast of brightness and hue performed in human visual cells Explanatory drawing which shows the function which draws an S-shaped curve with the correspondence of the output value with respect to the input value used for correction _| amendment of a color difference (C _RG value and C _YB value). 5 is a flowchart for explaining an image quality improvement method according to an embodiment of the present invention. 1 is a block diagram showing a configuration of an image quality improvement apparatus according to an embodiment of the present invention. Explanatory view showing an example of how to take the pixel of interest (V ₀₎ and neighboring pixels _{(V 0, ···, V 8} ). Explanatory drawing which shows a comparison with the processing image by this invention, and the processing image by histogram smoothing.

Claims (2)

(a) an input mechanism for inputting image information;
(b) a V input mechanism for inputting a V value representing luminance from a value obtained from an image received from the input mechanism;
(c) an RG input mechanism for inputting a C _RG value, which is a color difference between the R value and the G value, from a value obtained from an image received from the input mechanism;
(d) a YB input mechanism that inputs a C _YB value that is a color difference between the Y value and the B value from a value obtained from an image received from the input mechanism;
(e) a V-neighbor image extraction mechanism that receives an image from the V-input mechanism and extracts a neighboring V pixel value including the pixel for each pixel;
(f) an RG neighboring image extraction mechanism that receives an image from the RG input mechanism and extracts a pixel value of a neighboring C _RG including the pixel for each pixel;
(g) a YB neighborhood image extraction mechanism that receives an image from the YB input mechanism and extracts a pixel value of a neighboring _CYB including the pixel for each pixel;
(h) a V neighborhood image pixel value domain analysis mechanism that receives a neighborhood image from the V neighborhood image extraction mechanism and measures a local domain of V near the pixel; and
(i) an RG neighborhood image pixel value domain analysis mechanism that receives a neighborhood image from the RG neighborhood image extraction mechanism and measures a local domain of the C _RG value near the pixel;
(j) a YB neighborhood image pixel value domain analysis mechanism that receives a neighborhood image from the YB neighborhood extraction mechanism and measures a local domain of the C _YB value near the pixel;
(k) Based on the results of the V input mechanism and the V neighborhood image pixel value domain analysis mechanism, the lower limit value of the image, the standard deviation value, the average value of the entire image, the average value of the neighborhood of the image, V _On-center image pixel value variable determination mechanism that determines pixel value correction amount,
(l) Based on the results of the V input mechanism and the V vicinity image pixel value domain analysis mechanism, the upper limit value of the image, the standard deviation value, the average value of the entire image, the average value of the vicinity of the image, V _Off-center image pixel value variable determination mechanism that determines the pixel value correction amount,
(m) by a variable obtained by the V _On-center image pixel value variable determining mechanism and V _Off-center image pixel value variable determining mechanism, said V _On-center image pixel value and V _Off-center image pixel value modified, and these function to draw a reverse S-shaped curve that is obtained by logarithm of the ratio, V pixel value correction mechanism for determining the pixel value of the V near the center pixel,
(n) a V pixel value normalization mechanism that normalizes the pixel value obtained by the V pixel value correction mechanism;
(o) based on the results of the RG input mechanism and the RG neighborhood image pixel value domain analysis mechanism, an RG pixel value correction mechanism that determines an overall correction amount with a curve that draws an S -shape;
(p) an RG pixel value normalization mechanism that normalizes pixel values obtained by the RG pixel value correction mechanism;
(q) R correction variable R _{On + Off} is determined by a variable obtained from the RG neighborhood image extraction mechanism, and an R neighborhood image pixel value adjustment mechanism that finely adjusts the value obtained by the RG normalization mechanism;
(r) Based on the pixel value of the G neighborhood center obtained from the RG neighborhood image extraction mechanism and the R average value of the neighborhood, a correction variable G _{On + Off} of G is determined and obtained from the RG pixel value normalization mechanism. G neighboring image pixel value adjustment mechanism for finely adjusting the G value,
(s) a C _rg conversion mechanism for converting the R and G pixel values obtained by the R vicinity image pixel value adjustment mechanism and the G vicinity image pixel value adjustment mechanism into a color difference between the R and G values;
(t) based on the results of the YB input mechanism and the YB neighborhood image pixel value domain analysis mechanism, a YB pixel value correction mechanism that determines an overall correction amount with a curve that draws an S -shape;
(u) a YB pixel value normalization mechanism that normalizes pixel values obtained by the YB pixel value correction mechanism;
(v) Based on the Y average between the locals including the pixel value of the Y neighborhood center obtained from the YB neighborhood image extraction mechanism and the B average between the locals including the pixel value of the B neighborhood center, the correction variable amount B of the B value B near image pixel value adjusting mechanism for finely adjusting the B value obtained by determining _On and YB pixel value normalizing mechanism,
(w) Y value correction variable amount Y based on the Y average between the locals including the pixel value of the Y neighborhood center obtained from the YB neighborhood image extraction mechanism and the B average between the locals including the pixel value of the B neighborhood center Determining _Off , Y neighborhood image pixel value adjustment mechanism for finely adjusting the Y value obtained from the YB pixel value normalization mechanism,

(x) from the pixel values of the obtained Y and B values in the said B neighboring images image pixel value adjustment mechanism Y neighboring image pixel value adjusting mechanism, and C _yb conversion mechanism for converting the color difference of the Y and B values,
(y) integrating pixel values of the V pixel value C _yb obtained pixel value of V obtained in the normalizer and the C _rg pixel values of the obtained C _rg conversion mechanism by the C _yb converting mechanism An rgb conversion mechanism for obtaining a corrected rgb value;
(z) An image quality improvement apparatus comprising: an output mechanism that receives a pixel value of each pixel from the rgb conversion mechanism, and configures and outputs the entire image. Computer
(a) an input mechanism for inputting image information;
(b) a V input mechanism for inputting a V value representing luminance from a value obtained from an image received from the input mechanism;
(c) an RG input mechanism for inputting a C _RG value, which is a color difference between the R value and the G value, from a value obtained from an image received from the input mechanism;
(d) a YB input mechanism that inputs a C _YB value that is a color difference between the Y value and the B value from a value obtained from an image received from the input mechanism;
(e) a V-neighbor image extraction mechanism that receives an image from the V-input mechanism and extracts a neighboring V pixel value including the pixel for each pixel;
(f) an RG neighboring image extraction mechanism that receives an image from the RG input mechanism and extracts a pixel value of a neighboring C _RG including the pixel for each pixel;
(g) a YB neighborhood image extraction mechanism that receives an image from the YB input mechanism and extracts a pixel value of a neighboring _CYB including the pixel for each pixel;
(h) a V neighborhood image pixel value domain analysis mechanism that receives a neighborhood image from the V neighborhood image extraction mechanism and measures a local domain of V near the pixel; and
(i) an RG neighborhood image pixel value domain analysis mechanism that receives a neighborhood image from the RG neighborhood image extraction mechanism and measures a local domain of the C _RG value near the pixel;
(j) a YB neighborhood image pixel value domain analysis mechanism that receives a neighborhood image from the YB neighborhood extraction mechanism and measures a local domain of the C _YB value near the pixel;
(k) Based on the results of the V input mechanism and the V neighborhood image pixel value domain analysis mechanism, the lower limit value of the image, the standard deviation value, the average value of the entire image, the average value of the neighborhood of the image, V _On-center image pixel value variable determination mechanism that determines pixel value correction amount,
(l) Based on the results of the V input mechanism and the V vicinity image pixel value domain analysis mechanism, the upper limit value of the image, the standard deviation value, the average value of the entire image, the average value of the vicinity of the image, V _Off-center image pixel value variable determination mechanism that determines the pixel value correction amount,
(m) by a variable obtained by the V _On-center image pixel value variable determining mechanism and V _Off-center image pixel value variable determining mechanism, said V _On-center image pixel value and V _Off-center image pixel value modified, and these function to draw a reverse S-shaped curve that is obtained by logarithm of the ratio, V pixel value correction mechanism for determining the pixel value of the V near the center pixel,
(n) a V pixel value normalization mechanism that normalizes the pixel value obtained by the V pixel value correction mechanism;
(o) based on the results of the RG input mechanism and the RG neighborhood image pixel value domain analysis mechanism, an RG pixel value correction mechanism that determines an overall correction amount with a curve that draws an S -shape;
(p) an RG pixel value normalization mechanism that normalizes pixel values obtained by the RG pixel value correction mechanism;
(q) R correction variable R _{On + Off} is determined by a variable obtained from the RG neighborhood image extraction mechanism, and an R neighborhood image pixel value adjustment mechanism that finely adjusts the value obtained by the RG normalization mechanism;
(r) Based on the pixel value of the G neighborhood center obtained from the RG neighborhood image extraction mechanism and the R average value of the neighborhood, a correction variable G _{On + Off} of G is determined and obtained from the RG pixel value normalization mechanism. G neighboring image pixel value adjustment mechanism for finely adjusting the G value,
(s) a C _rg conversion mechanism for converting the R and G pixel values obtained by the R vicinity image pixel value adjustment mechanism and the G vicinity image pixel value adjustment mechanism into a color difference between the R and G values;
(t) based on the results of the YB input mechanism and the YB neighborhood image pixel value domain analysis mechanism, a YB pixel value correction mechanism that determines an overall correction amount with a curve that draws an S -shape;
(u) a YB pixel value normalization mechanism that normalizes pixel values obtained by the YB pixel value correction mechanism;
(v) Based on the Y average between the locals including the pixel value of the Y neighborhood center obtained from the YB neighborhood image extraction mechanism and the B average between the locals including the pixel value of the B neighborhood center, the correction variable amount B of the B value B near image pixel value adjusting mechanism for finely adjusting the B value obtained by determining _On and YB pixel value normalizing mechanism,
(w) Y value correction variable amount Y based on the Y average between the locals including the pixel value of the Y neighborhood center obtained from the YB neighborhood image extraction mechanism and the B average between the locals including the pixel value of the B neighborhood center Determining _Off , Y neighborhood image pixel value adjustment mechanism for finely adjusting the Y value obtained from the YB pixel value normalization mechanism,

(x) from the pixel values of the obtained Y and B values in the said B neighboring images image pixel value adjustment mechanism Y neighboring image pixel value adjusting mechanism, and C _yb conversion mechanism for converting the color difference of the Y and B values,
(y) integrating pixel values of the V pixel value C _yb obtained pixel value of V obtained in the normalizer and the C _rg pixel values of the obtained C _rg conversion mechanism by the C _yb converting mechanism An rgb conversion mechanism for obtaining a corrected rgb value;
(z) A program that receives a pixel value of each pixel from the rgb conversion mechanism, and functions as an output mechanism that configures and outputs the entire image.

Images