JP2006053859A - Image processor, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently confirm the effect of beauty skin treatment for removing a wrinkle component performed to a photographic image. <P>SOLUTION: A processed image D1 of removing the wrinkle component is generated by extracting the wrinkle component from an image D0. A wrinkle quantity detecting means 13 detects a wrinkle component quantity on respective areas of dividing the image D0 into 9 areas, and an area determining means 15 determines the largest area of the wrinkle component quantity as a confirming area. An image extracting means 17 extracts a postprocessing confirming image Db and a processed confirming image Da corresponding to the confirming area from the respective image D0 and processed image D1, and a display control means 21 displays a synthetic image G0 of these images on a monitor 19. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method and apparatus for performing beautiful skin processing on a photographic image including a face, specifically, processing for suppressing and removing wrinkle components such as noise, wrinkles on the skin, and wrinkles contained in the face. The present invention also relates to a program for causing a computer to execute an image processing method.

  2. Description of the Related Art Conventionally, skin beautification processing for suppressing and removing wrinkles and spots from photographic images including human faces (hereinafter simply referred to as images) has been performed. For example, skin beautification can be performed by applying a low pass filter (LPF) that is usually used for noise removal. However, LPF can suppress wrinkles, wrinkles, and wrinkle components from an image, but has a drawback of degrading an edge portion in an image signal and blurring the entire image.

  Also, it was devised to separate and suppress high-frequency wrinkle components of small amplitude in the image by utilizing the fact that many components such as wrinkles and spots exist as small-amplitude signals in the high-frequency components of the image. In addition, an ε-filter (ε-separated nonlinear digital filter) is also applied to remove wrinkles and spots (Non-Patent Document 1). Since the ε-filter has a characteristic of flattening only a small amplitude level change in an image signal, an image having a beautifying skin using the ε-filter stores an edge having a steep level change, The overall sharpness is hardly impaired.

  The ε-filter basically serves to subtract a value obtained by applying a nonlinear function to the amount of change in amplitude level from the original image signal. This nonlinear function is a function that sets the output to 0 when the amplitude of the signal is larger than a predetermined threshold. That is, when the ε-filter is applied, the output of the non-linear function is 0 in the part where the amplitude is larger than the threshold value in the image, and the original signal of the part is retained in the image after the skin beautification process. In the region where the amplitude is equal to or less than the threshold value, in the image after the skin beautification process, the signal value of the region is a value obtained by subtracting the output of the nonlinear function (the absolute value is greater than 0) from the original signal value. By doing this, although it is not so-called noise, the change in brightness is smoothed in wrinkles or spots such as wrinkles or spots with small amplitude, and the wrinkles and spots can be made inconspicuous and the amplitude is large. The edge portion can be held.

  Further, in Patent Document 1, in order to maintain the texture and texture of the skin while removing wrinkles and spots, a nonlinear function having a large threshold value ε0 is applied to extract components such as wrinkles. Applying a nonlinear function with a small value to extract the skin texture and texture components, subtracting components such as wrinkles from the original signal, and adding the skin texture and texture components to the wrinkles. A method of retaining the texture and texture of the skin while removing spots and the like has been proposed.

  In order to confirm the effect of the skin beautification process performed on the image as described above, the processed image is displayed on a display screen such as a monitor to be confirmed by the operator. However, for example, in the case of a portable camera, in order to display a processed image on an attached monitor with a small resolution, the processed image must be reduced, but even if the reduced image is confirmed on the attached monitor, The effect of skin treatment cannot be confirmed. Therefore, among commercially available digital cameras, the operator can designate a display area for an image obtained by imaging, and the designated area can be displayed on the attached monitor at the same magnification or enlarged. If such a digital camera is used, the stored processed image can be read from the storage means, and the area specified by the operator can be displayed at the same magnification or enlarged. By this, the effect of the skin beautification can be confirmed.

In addition, as described in Patent Document 2, a method of extracting an image at a predetermined position such as a central region of an image and displaying it at the same magnification, or dividing the image into a plurality of regions and sequentially displaying the same magnification is proposed. ing.
Japanese Patent Laid-Open No. 2001-118084 JP 2003-283888 A Arakawa et al., "Color face image processing with vector ε-filter-Removal of flaw components" March 1998 Proceedings of the IEICE General Conference, D-11-43, PP143-

  However, the method for causing the operator to specify the display area requires the operator to specify the area to be confirmed, and the operation is troublesome. Further, in a method of extracting and displaying an image at a predetermined position, such as the method described in Patent Document 2, the image at a predetermined position is not a region suitable for confirming the processing effect. In this case, there is a problem that an appropriate judgment cannot be made.

  SUMMARY OF THE INVENTION In view of the above circumstances, an object of the present invention is to make it possible to efficiently confirm the effect of skin beautification processing for removing wrinkle components such as noise, wrinkles, and stains contained in a face performed on a photographic image. .

The image processing apparatus of the present invention includes a wrinkle confirmation region determination unit that determines a region where wrinkle components are concentrated as a wrinkle confirmation region in a photographic image including a face,
Skin-beautifying means for obtaining a processed image that has been subjected to skin-beautifying treatment by removing the wrinkle component from the photographic image;
Confirmation image acquisition means for acquiring an image of a region corresponding to the wrinkle confirmation region in the processed image as a processed confirmation image;
It is characterized by comprising display control means for displaying the extracted processed confirmation image with a size matching the resolution of the display device.

  Here, the “photo image” is obtained by reading an image on a silver halide photographic film or printed matter (for example, a print) with a reading device such as a scanner as well as a digital image obtained by imaging a subject with a digital camera or the like. Including digital images. Hereinafter, for convenience of explanation, an image simply means a photographic image.

  In addition, “wrinkle component” is included not only in the wrinkles contained in the face but also in the face of noise, stains, etc., to improve the texture and texture of the skin of the face, and to make the person look younger Means a component capable of

In the image processing apparatus of the present invention, the confirmation image acquisition means is a means for extracting an image of a region corresponding to the wrinkle confirmation region in the photographic image as a confirmation image before processing,
The display control means may be means for causing the display device to display the processed confirmation image together with the pre-processing confirmation image.

Further, in the image processing apparatus of the present invention, the wrinkle confirmation region determining unit detects the amount of wrinkle components for each of a plurality of region images of the same size obtained by dividing the photographic image. When,
An area determination unit that determines an area corresponding to the area image with the largest wrinkle amount as the wrinkle confirmation area may be provided.

Further, in the image processing apparatus of the present invention, the skin beautification processing means, based on the photographic image, a means for creating a plurality of band limited images representing components for a plurality of frequency bands of the photographic image;
For an input value in which the absolute value of the output value is less than or equal to the absolute value of the input value and the absolute value is less than or equal to a predetermined threshold value, the absolute value of the output value increases as the absolute value of the input value increases. On the other hand, for an input value whose absolute value is larger than the predetermined threshold, nonlinear conversion processing in which the absolute value of the output value is equal to or smaller than the absolute value of the output value corresponding to the predetermined threshold is performed in the band-limited image. Means for applying to each pixel value to obtain a plurality of converted images;
Means for multiplying a pixel value of the plurality of converted images by a predetermined subtraction coefficient;
Means for subtracting pixel values of the plurality of converted images multiplied by the subtraction coefficient from pixel values of the photographic image to obtain pixel values of the processed image.

  Here, the predetermined subtraction coefficient may be the same for each pixel of the photographic image, but is preferably determined according to the pixel value of the photographic image.

  In an image, wrinkle components such as noise, wrinkles, and spots have different forms of existence corresponding to the pixel values of the image. For example, noise is often generated in a dark part (that is, a part having a small luminance value) in an image. Therefore, when a subtraction coefficient is used that is smaller as the pixel value is larger (conversely, the smaller the image value is), noise removal is performed. The effect can be enhanced.

  In addition, since components such as wrinkles and stains are present in skin parts that are lighter than the hair or the like, if a pixel value, for example, a subtraction coefficient that is larger as the luminance value is larger (conversely smaller as the pixel value is smaller) is used. In addition to removing components such as stains, an effect of suppressing component components such as wrinkles and stains, in particular, a so-called skin-beautifying effect can be obtained.

  The nonlinear conversion process is preferably a process in which the absolute value of the output value is substantially constant with respect to an input value whose absolute value is greater than the predetermined threshold.

  Further, it is preferable that the predetermined threshold value is determined according to a frequency band of the band limited image to be processed.

The image processing method according to the present invention determines an area where wrinkle components are concentrated in a photographic image including a face as a wrinkle confirmation area,
Applying a skin beautifying process to remove the wrinkle component on the photographic image to obtain a processed image subjected to the skin beautifying process,
Obtaining an image of a region corresponding to the wrinkle confirmation region in the processed image as a processed confirmation image;
The extracted processed confirmation image is displayed in a size that matches the resolution of the display device.

  The image processing method according to the present invention may be provided as a program for causing a computer to execute the image processing method.

  According to the present invention, in order to confirm the effect of the skin beautification process performed on the photographic image, a region where wrinkle components are concentrated in the photographic image before processing is determined as a wrinkle confirmation region, and this determination is performed from the processed image. An image of an area corresponding to the wrinkle confirmation area is extracted as a processed confirmation image and displayed in a size that matches the display device. If the effect of beautiful skin treatment is good, the wrinkles contained in the face of the photographic image will disappear or be reduced, but wrinkles such as wrinkles such as cheeks on the face, spots and other wrinkle components will not appear or are difficult to appear The effect of the processing can be reflected in the image of the region in the processed image corresponding to the region in which wrinkle components such as the part or between the eyes appear remarkably. Therefore, by extracting an image of a region corresponding to a region where wrinkle components are concentrated in a photographic image from the processed image and confirming the effect of the skin beautifying effect, the operator can accurately determine the effect of the skin beautifying treatment. Can do. In addition, since only a part of the processed image (processed confirmation image) is displayed in a size that matches the display device, even in a display device with a small resolution such as a monitor attached to a portable camera, the operator can effectively perform the skin-beautifying effect. Confirmation can be performed correctly.

  In addition, since the processed confirmation image is automatically extracted from the processed image and displayed, there is no need for processing such as region designation by the operator or sequential display of the image for each region. Good.

  Furthermore, if the image of the region corresponding to the wrinkle confirmation region is extracted as the pre-processing confirmation image from the pre-processing photographic image and displayed together with the processed confirmation image, the operator can use the pre-processing and post-processing wrinkle components. Therefore, the effect of beautifying skin can be confirmed more accurately.

  In addition, a plurality of band limited images representing components of a plurality of different frequency bands of the photographic image created based on the photographic image are subjected to nonlinear conversion processing to obtain a plurality of converted images, and the plurality of converted images are converted into a plurality of converted images. A pixel value of the processed image may be obtained by multiplying a predetermined subtraction coefficient and subtracting the pixel values of a plurality of converted images multiplied by the subtraction coefficient from the pixel value of the photographic image. Here, the non-linear conversion process for obtaining a plurality of converted images is a process for setting the absolute value of the output value to be equal to or smaller than the absolute value of the input value, and for an input value having an absolute value equal to or smaller than a predetermined threshold value. As the absolute value of the input value increases, the absolute value of the output value increases. On the other hand, for an input value whose absolute value is greater than the predetermined threshold value, the absolute value of the output value corresponds to the output value corresponding to the predetermined threshold value. This is processing that is less than the absolute value. The converted image obtained by this non-linear conversion process represents a wrinkle component having a small amplitude such as wrinkles, spots or noises in the frequency band corresponding to the converted image. Therefore, by multiplying the pixel values of the plurality of converted images by a predetermined subtraction coefficient and subtracting from the pixel values of the original photographic image to obtain the pixel values of the processed image, the photographic image in a plurality of frequency bands is obtained. Wrinkle components such as wrinkles, stains or noise can be effectively removed.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus according to the present embodiment performs a skin beautification process on a photographic image (hereinafter referred to as an image for short), and is used for causing an operator to check the effect of the skin beautifying process, and is read into an auxiliary storage device. This is realized by executing the beautifying skin treatment program on a computer (for example, a personal computer). The skin beautification processing program is stored in an information storage medium such as a CD-ROM, or distributed via a network such as the Internet and installed in a computer.

  Further, since the image data represents an image, the following description will be given without particularly distinguishing the image from the image data.

  As shown in FIG. 1, the image processing apparatus according to the present embodiment creates a plurality of blurred images S1, S2,... Sn (n: integer greater than or equal to 2) with different frequency response characteristics of the original image S0. Image creating means 3, band limited image creating means 5 for creating a plurality of band limited images T1, T2,... Tn using original image S0 and blurred images S1, S2,. Wrinkle component extraction means 7 for performing non-linear transformation processing on T1, T2,... Tn and extracting wrinkle components Q1, Q2,. And a wrinkle component removing unit 9 that removes the wrinkle component from S0 to obtain a wrinkle-removed image S′0 (Y1).

  In the image processing apparatus according to the present embodiment, since each of the above units performs processing in the luminance space, the luminance of the image D0 is obtained by performing YCC conversion on the input image D0 (R0, G0, B0). The wrinkle-removed image S′0 obtained by the YCC conversion unit 1 that obtains the value Y0 (the luminance value Y0 constitutes the original image S0 described above), the color difference values Cb0 and Cr0, and the wrinkle component removal unit 9 And an output unit 11 that outputs an image composed of the pixel value Y1 and the color difference values Cb0 and Cr0 obtained by the YCC conversion unit 1 as a processed image D1 (R1, G1, B1).

  Furthermore, the image processing apparatus of the present embodiment detects the amount of wrinkle components for each of the areas A1 to A9 obtained by dividing the image D0 into nine as shown in FIG. 2 based on the band limited image generated by the band limited image generating unit 5. A wrinkle amount detecting means 13 for performing the determination, an area determining means 15 for determining the area having the largest amount of the wrinkle component among the nine areas A1 to A9 as the confirmation area Ak, and the confirmation area Ak from the image D0 and the processed image D1. The image extraction means 17 which extracts the image of a corresponding area as the pre-processing confirmation image Db and the processed confirmation image Da, the monitor 19, and the display of the pre-processing confirmation image and the processed confirmation image on the monitor 19 are controlled. Display control means 21.

  Hereinafter, details of each component of the present embodiment will be described.

  The YCC conversion means 1 converts the R, G, and B values of each pixel of the image D0 into a luminance value Y and color difference values Cb and Cr according to the following equation (1).

Y = 0.2990 × R + 0.5870 × G + 0.1140 × B
Cb = −0.1687 × R−0.3313 × G + 0.5000 × B + 128 (1)
Cr = 0.5000 × R−0.4187 × G−0.0813 × B + 128
The blur image creation unit 3 creates a plurality of blur images using the luminance value Y0 obtained by the YCC conversion unit 1. FIG. 3 is a block diagram showing the configuration of the blurred image creation means 3. In the image processing apparatus of the present embodiment, the blurred image creation unit 3 performs filtering processing and thinning processing on the luminance value Y0 to obtain filtering images B1, B2,... Bn, and for each filtering image Interpolation means 33 for performing interpolation processing, and filtering means 31 and control means 35 for controlling interpolation means 33 are provided.

The filtering means 31 performs a filtering process using a low-pass filter. As this low-pass filter, for example, a filter F substantially corresponding to a 5 × 1 grid-like one-dimensional Gaussian distribution as shown in FIG. 4 is used. be able to. This filter F has σ = 1 in the following equation (2).

  The filtering unit 31 performs the filtering process and the half-thinning process on the image in the x direction and the y direction with respect to the image to be processed using the filter F as described above, so that the entire image to be processed is processed. Perform the filtering process.

FIG. 5 shows details of processing that the control unit 35 of the blurred image creation unit 3 causes the filtering unit 31 and the interpolation unit 33 to perform. As shown in the figure, the filtering means 31 first performs a filtering process on the original image S0 (Y0) every other pixel using the filter F shown in FIG. 4, and then thins out the unfiltered pixels. Process. A filtering image B1 (Y1) is obtained by this filtering process. The size of the filtering image B1 is 1/4 of the size of the original image S0 (1/2 in the x direction and y direction). Next, the filtering unit 31 applies the filtering process by the filter F every other pixel to the filtering image B1 (Y1), and performs the same thinning process to obtain the filtering image B2 (Y2). The filtering unit 31 repeats the filtering process by the filter F and the 1/2 thinning process to obtain n filtered images Bk (k = 1 to n). The size of the filtering image Bk is 1/2 ^2K of the size of the original image S0. FIG. 6 shows the frequency characteristics of each filtered image Bk obtained by the filtering means 31 when n = 3 as an example. As shown in the figure, the response of the filtered image Bk is such that the higher the k, the higher the frequency component is removed.

  In the image processing apparatus according to the present embodiment, the filtering unit 31 performs the filtering process on the x direction and the y direction of the image by the filter F shown in FIG. A filtering process may be performed on the original image S0 and the filtered image Bk at a time using a × 5 two-dimensional filter.

The interpolation unit 33 performs an interpolation process on each filtering image Bk obtained by the filtering unit 31 so that each filtering image Bk has the same size as the original image S0. There are various interpolation processing methods such as a B-spline method. In this embodiment, the filtering unit 31 uses a filter F based on a Gaussian signal as a low-pass filter. A Gaussian signal is also used as an interpolation coefficient for performing an interpolation calculation. This interpolation coefficient is approximated to σ = 2 ^k−1 in the following equation (3).

  Since k = 1 when interpolating the filtered image B1, σ = 1. The filter for performing interpolation when σ = 1 in the above equation (3) is a 5 × 1 one-dimensional filter F1 as shown in FIG. The interpolation unit 33 first expands the filtering image B1 to the same size as the original image S0 by inserting one pixel at a time every other pixel into the filtering image B1 to the same size as the original image S0. On the other hand, a filtering process is performed by the filter F1 shown in FIG. 8 to obtain a blurred image S1. The blurred image S1 has the same number of pixels as the original image S0, that is, the same size as the original image S0.

  Here, the filter F1 shown in FIG. 8 is a 5 × 1 filter. However, before applying the filter F1, pixels with a value of 0 are inserted into the filtered image B1 every other pixel, and therefore interpolation is performed. The interpolation processing by the means 33 is substantially performed by two types of filters: a 2 × 1 filter (0.5, 0.5) and a 3 × 1 filter (0.1, 0.8, 0.1). Equivalent to filtering.

  When the interpolation unit 33 performs interpolation on the filtered image B2, k = 2, so σ = 2. In the above equation (3), the filter corresponding to σ = 2 is the 11 × 1 one-dimensional filter F2 shown in FIG. The interpolation unit 33 first expands the filtering image B2 to the same size as the original image S0 by inserting three pixels each having a value of 0 every other pixel into the filtering image B2, and then enlarges the image. Is subjected to a filtering process by a filter F2 shown in FIG. 9 to obtain a blurred image S2. The blurred image S2 has the same number of pixels as the original image S0, that is, the same size as the original image S0.

  Similarly, the filter F2 shown in FIG. 9 is an 11 × 1 filter, but before applying the filter F2, three pixels each having a value of 0 are inserted into the filtered image B2 every three pixels. Therefore, the interpolation processing by the interpolation means 33 is substantially performed by a 2 × 1 filter (0.5, 0.5) and a 3 × 1 filter (0.3, 0.65, 0.05), ( This is equivalent to filter processing using four types of filters (0.3, 0.74, 0.13) and (0.05, 0.65, 0.3).

In this way, the interpolation means 33 inserts (2 ^k −1) pixels each having a value of 0 every other pixel into each filtering image Bk, so that the filtering image Bk is the same as the original image S0. For a filtered image Bk that is enlarged to size and in which a pixel having a value of 0 is inserted, a length created based on the above equation (3) is (3 × 2 ^k −1). A blurred image Sk is obtained by performing a filtering process.

  FIG. 10 shows the frequency characteristics of each blurred image Sk obtained by the blurred image creation means 3 when n = 3 as an example. As shown in the figure, the blurred image Sk is such that the higher the k, the higher frequency components of the original image S0 are removed.

The band-limited image creating means 5 uses the blurred images S1, S2,... Sn obtained by the blurred image creating means 3 and components for each of a plurality of frequency bands of the original image S0 according to the following equation (4). .., Tn are generated.
Tm = S (m−1) −Sm (4)
Where m is an integer between 1 and n

  FIG. 11 shows the frequency characteristics of each band limited image Tm obtained by the band limited image creating means 5 when n = 3 as an example. As illustrated, the band limited image Tm represents a component in the low frequency region of the original image S0 as m increases.

  The wrinkle component extraction unit 7 performs non-linear transformation on each band limited image Tm (m = 1 to n) obtained by the band limited image creating unit 5 and generates wrinkles in the frequency band corresponding to each band limited image Tm. , Stains, noise and the like (hereinafter referred to as wrinkle components) Q1, Q2,..., Qn. This non-linear transformation is a process for making an output value equal to or less than an input value, and for an input value less than or equal to a predetermined threshold value, the larger the input value, the larger the output value, while an input value greater than the predetermined threshold value. Is a process in which the output value is equal to or less than the output value corresponding to the predetermined threshold value. In this embodiment, the process is performed by the function f as shown in FIG. In the figure, the broken line indicates a function in which the output value = the input value, that is, the slope is 1. As shown in the figure, the nonlinear transformation function f used in the wrinkle component extraction means 7 of this embodiment has a slope of 1 when the absolute value of the input value is smaller than the first threshold Th1, and the absolute value of the input value. Is greater than or equal to the first threshold and less than or equal to the second threshold Th2, the slope is smaller than 1, and when the absolute value of the input value is greater than the second threshold Th2, the output value is the absolute value of the input value. This is a function having a constant value M smaller than the absolute value. The function f may be the same for each band limited image, but may be different for each band limited image.

  The wrinkle component extracting means 7 uses the luminance value of each band limited image as an input value, performs nonlinear conversion on each band limited image using the nonlinear conversion function f shown in FIG. The wrinkle component Qm (m = 1 to n) in the frequency band corresponding to each configured band-limited image is extracted.

The wrinkle component removal unit 9 multiplies each wrinkle component Qm extracted by the wrinkle component extraction unit 7 by a subtraction coefficient β, and subtracts the wrinkle component Qm multiplied by the subtraction coefficient from the original image S0 (Y0). The wrinkle-removed image S′0 (Y1) is obtained. The following equation (5) shows the processing performed by the wrinkle component removing unit 9.

  Here, the subtraction coefficient β is determined according to the luminance value Y0 of the pixel of the original image S0. Specifically, when the luminance value Y0 is larger, the pixel value Y1 of this pixel is obtained. The value of the subtraction coefficient β used is increased. The wrinkle component Qm extracted by the wrinkle component extraction means 7 may contain components such as hair, and if components such as hair are removed in the same way as true wrinkle components, There is a problem that the hair is blurred and the quality of the processed image is not good.

  In the present embodiment, focusing on the fact that the skin portion where wrinkles or the like are generally generated is bright (that is, the luminance value is large) and the hair portion is dark (that is, the luminance value is small), the higher the luminance value, the higher the value of the pixel. The subtraction coefficient β is large, the degree of subtraction (ie, wrinkle removal) of the hair portion is weakened, and the degree of subtraction (ie, wrinkle removal) of the skin portion is increased. As a result, wrinkle components such as true wrinkles, stains or noise can be removed, and components representing the hair can be left, and since the hair portion is not blurred, the image quality of the processed pixel is improved.

  The output means 11 outputs an image (Y1, Cr0, Cb0) composed of the luminance value Y1 obtained by the wrinkle component removing means 9 and the color difference values Cb0, Cr0 of the image D0 obtained by the YCC conversion means 1. RGB conversion is performed to obtain a processed image D1 (R1, G1, B1) in which each pixel value is an RGB value, and this is output.

  The wrinkle amount detecting means 13 is obtained by dividing the image D0 into nine as shown in FIG. 2 using the band limited image T1 of the highest frequency band among the band limited images Tm generated by the band limited image generating means 5. The amount of the wrinkle component is detected for each of the nine areas A1 to A9. FIG. 13 is a diagram for explaining the detection of the amount of wrinkle components. Here, for the sake of simplicity, the description will be made assuming that the signal value of the band limited image T1 is one-dimensionally distributed. In the band limited image T1, the wrinkle component Qm appears as a signal value having a small contrast. For this reason, as shown in FIG. 13, the wrinkle amount detection means 13 assigns a pixel in which the signal value of the band limited image T1 is equal to or greater than the threshold Th3 and equal to or less than the threshold Th4 in each of the areas A1 to A9. The number of counted pixels is detected as the amount of wrinkle components in each of the areas A1 to A9.

  The threshold value Th3 is set to a value that is smaller by a predetermined amount with reference to the median value C0 of the signal value of the band limited image T1, and the threshold value Th4 is set to be a value that is larger than the median value C0 by a predetermined amount. Here, when the band-limited image T1 takes a value of 0 to 255, the median value C0 is 128. Therefore, for example, 113 may be used as the threshold Th3 and 140 may be used as the threshold Th4.

  The area determination unit 15 compares the amount of the wrinkle component in each of the areas A1 to A9 detected by the wrinkle amount detection unit 13, and determines the area having the largest amount of the wrinkle component as the confirmation area Ak. For example, when the area A5 has the largest amount of wrinkle components in the image shown in FIG. 2, the area A5 is determined as the confirmation area Ak.

  The image extraction means 17 extracts an image of a region corresponding to the confirmation region Ak from the image D0 as a pre-processing confirmation image Db. In addition, an image of a region corresponding to the confirmation region Ak is extracted from the processed image D1 as a processed confirmation image Da.

  The display control unit 21 combines the pre-processing confirmation image Db and the processed confirmation image Da so as to line up to generate a composite image G0, and displays the composite image G0 on the monitor 19. At this time, the display control means 21 changes the size of the composite image G0 according to the resolution of the monitor 19. For example, when the size of the monitor 19 is QVGA, that is, having a resolution of 320 × 240, the size of the synthesized image G0 is changed so that the synthesized image G0 is maximally within the size of the monitor 19.

  FIG. 14 is a view showing the composite image G0 displayed on the monitor 19. As shown in FIG. As shown in FIG. 14, the composite image G0 is displayed on the monitor 19 so that the composite image G0 fits within the size of the monitor 19 to the maximum extent. The pre-processing confirmation image Db and the processed forehead person image Da included in the composite image G0 are also displayed as “before processing” and “after processing” so that it can be seen that they have been processed and processed.

  As described above, according to the image processing apparatus of the present embodiment, when the image of FIG. 2 is taken as an example, a region where wrinkles such as wrinkles and spots such as cheeks of the face do not appear or are difficult to appear has been processed. The confirmation image Da is not extracted from the processed image D1, and a region in which wrinkle components such as the corners of the eyes and between the eyes appear remarkably is extracted as the processed confirmation image Da and displayed on the monitor 19. It will be. For this reason, the operator can efficiently check the effect of the skin beautifying process on the image.

  In addition, since the confirmation image Db before processing in the same region in the image D0 before processing is displayed together with the processing confirmation image Da extracted from the processed image D1, the operator can display the same region before processing and after processing. Images can be compared, and thereby, the effect of skin beautification can be confirmed better.

  Further, a plurality of band limited images Tm (m = 1 to n, n ≧ 2) representing components for a plurality of different frequency bands of the original image S0 (Y0) are created, and nonlinear conversion is performed on these band limited images. A plurality of converted images (wrinkle components) are obtained by performing processing, and the pixel values of the plurality of converted images are multiplied by a subtraction coefficient β determined according to the luminance value of the original image S0 to obtain the original image Since the pixel value (luminance value Y1) of the processed image is obtained by subtracting from the pixel value (luminance value Y0) of S0, the wrinkle component in a plurality of frequency bands of the photographic image is effectively removed and the skin is beautiful Processing can be performed. In addition, since wrinkle components are removed in a plurality of frequency bands, in order to enhance the skin beautifying effect as in the prior art, the generation of artifacts caused by increasing the removal of wrinkle components in one frequency band is generated. This can prevent the image quality of the processed image.

  The preferred embodiment of the present invention has been described above, but the image processing method and apparatus of the present invention and the program therefor are not limited to the above-described embodiment, and various increases and decreases may be made without departing from the gist of the present invention. , Can make changes.

  For example, the image processing method and apparatus of the present invention can be effective for confirming the effect of skin beautification processing with a display device having a particularly small resolution, but the effect of skin beautifying treatment with a display device having a high resolution such as VGA can be obtained. In the case of confirmation, when an image having the same size as the resolution of the display device is extracted from the processed image D1 and displayed as the processed confirmation image Da, the size of the processed confirmation image Da is large. There is a possibility that a region where wrinkle components are concentrated and a region where wrinkle components are not concentrated, that is, a region suitable for confirming the effect of skin beautification and a region not suitable for blending may be mixed, which may hinder confirmation of the effect of skin beautification. Absent.

  For this reason, the present invention is applied to the case where the effect of the skin beautification process is confirmed on a display device having a high resolution, and the processed confirmation image Da is extracted so as to be smaller than the display device, and the extracted processed confirmation image Da is extracted. By enlarging the image to a size that matches the resolution of the display device, the effect of blur correction can be accurately confirmed even when the resolution of the display device is large. In this case, the extracted processed confirmation image Da is not enlarged to the same size as the resolution of the display device, but is enlarged as it is or to a size smaller than the resolution of the display device and only a part of the display screen of the display device. You may make it display on.

  Further, the wrinkle amount detection means 13 detects the amount of wrinkle components in the areas A1 to A9 using the band limited image T1 in the highest frequency band. However, the band limited image in a frequency band other than the highest frequency band is detected. It may be used to detect the amount of the wrinkle component.

In the image processing apparatus of the present embodiment, the band limited image creating unit 5 generates a band limited image according to the above equation (4) using the original image S0 and the blurred image Sk (k = 1 to n, n ≧ 2). The processing performed in the band-limited image creation means 5, the wrinkle component extraction means 7, and the wrinkle component removal means 9 can be expressed by the following formula (6). For example, the following formula (7) The processing of the band-limited image creating unit, the wrinkle component extracting unit, and the wrinkle component removing unit 9 may be performed according to Equation (8) or Equation (9). That is, as in the processing in the image processing apparatus of the present embodiment represented by Expression (6), using the original image and each blurred image, the images in the frequency bands adjacent to each other (the original image S0 is the frequency of the blurred image S1 and the frequency The band-limited image may be obtained by subtracting the band-limited images), but as shown in Expression (7), the band-limited image is obtained by subtracting all the blurred images and the original image. Alternatively, as shown in Expression (8), the band-limited image may be obtained by subtracting the blurred images in the frequency bands adjacent to each other without using the original image, or Expression (9). ), The band-limited image is obtained by subtraction of the blurred image S1 and all the blurred images Sm (m = 2 to n, n ≧ 3) excluding the blurred image S1 without using the original image. May be.

  Also, the method for creating the band limited image is to create a blurred image of the original image and to create the blurred image of the original image and / or the blurred image as in the methods represented by the above formulas (4) and (6) to (9). The method is not limited to the method of creating using an image, and any method may be used as long as an image representing a plurality of different frequency band components of the original image can be created.

  In this embodiment, the entire photographic image is processed. However, since the purpose of the processing is beautifying skin, for example, only the skin color portion is extracted from the photographic image, and the extracted portion is processed. Only beautifying skin treatment may be performed. Furthermore, in the case where the purpose is skin beautification of the face part, the face part may be extracted using various conventionally proposed face extraction techniques, and the skin beautification process may be performed only on the face part. By doing so, it is possible to reduce the amount of calculation for beautifying skin and to prevent blurring of parts other than the skin color or parts other than the face. In this case, the confirmation area Ak may be determined in the extracted face portion.

  In this embodiment, the subtraction coefficient β used when removing the wrinkle component is determined according to the luminance value Y0 of the original image S0. For example, in addition to the luminance value of the original image, You may make it change a subtraction coefficient according to the information of the object of the photographic image used as object. For example, the older the subject is, the larger the subtraction coefficient is to enhance the skin-beautifying effect, or vice versa, the older the subject is, the smaller the subtraction coefficient is set so that the processed image does not become unnatural. It may be weakened.

  Similarly, the threshold for performing nonlinear conversion on the band limited image, particularly the threshold Th2, may be changed according to the subject information. The subject information may be acquired by recognition processing or by referring to tag information given to the image D0, or may be acquired by manual input by the operator.

  In addition to the subject information, parameters such as a subtraction coefficient and a threshold value for nonlinear conversion may be changed according to the strength of the skin effect desired by the operator.

  Furthermore, the above-described parameters may be changed according to the use of the photographic image, such as for a remains, for a wedding, or for an automatic photographing printing apparatus (so-called “printer”).

  In the image processing apparatus of the present embodiment, the wrinkle component is a small-amplitude component that represents a change in brightness and has no color amplitude, and by removing the wrinkle component only in the luminance space, Although efficient skin-beautifying processing is realized, for example, processing may be performed on all of the R, G, and B components, and not only the wrinkle component but also the color wrinkle component may be removed.

  In addition, the image processing apparatus according to the present embodiment uses a subtraction coefficient in which the value increases as the pixel value of the original image increases in order to suppress wrinkles and stain components and to obtain a skin-beautifying effect. For example, when noise removal is mainly used, the larger the pixel value of the original image, the smaller the value may be used.

  The skin beautification process is not limited to the technique of the above embodiment, and any technique such as the technique described in Non-Patent Document 1 or Patent Document 1 can be used.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. The figure which shows the example of the image used for this embodiment FIG. 2 is a block diagram showing a configuration of a blurred image creation unit in the image processing apparatus shown in FIG. The figure which shows the example of the one-dimensional filter F which the filtering means in the blur image creation means shown in FIG. 2 uses The figure which shows the process performed in the blurred image preparation means shown in FIG. The figure which shows the frequency characteristic of the filtering image produced by the filtering means in the blurred image production means shown in FIG. The figure which shows the example of the two-dimensional filter which the filtering means in the blur image creation means shown in FIG. 2 uses The figure which shows the example of the filter F1 which the interpolation means in the blurred image preparation means shown in FIG. 2 uses for interpolation of filtering image B1 The figure which shows the example of the filter F2 which the interpolation means in the blurred image preparation means shown in FIG. 2 uses for interpolation of filtering image B2 The figure which shows the frequency characteristic of the blur image produced by the blur image creation means shown in FIG. The figure which shows the frequency characteristic of the band limited image produced by the band limited image preparation means in the image processing apparatus of embodiment of this invention shown in FIG. The figure which shows the example of the nonlinear function which the wrinkle component extraction means uses in the image processing apparatus of embodiment of this invention shown in FIG. The figure for demonstrating the detection of the quantity of a wrinkle component The figure which shows the synthesized image of the processed confirmation image and the confirmation image before processing displayed on the monitor

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 YCC conversion means 3 Blurred image creation means 5 Band-limited image creation means 7 Wrinkle component extraction means 9 Wrinkle component removal means 11 Output means 13 Wrinkle amount detection means 15 Area determination means 17 Image extraction means 19 Monitor 21 Display control means 31 Filtering means 33 Interpolating means 35 Control means

Claims (6)

In a photographic image including a face, a wrinkle confirmation region determining means for determining a region where wrinkle components are concentrated as a wrinkle confirmation region;
Skin-beautifying means for obtaining a processed image that has been subjected to skin-beautifying treatment by removing the wrinkle component from the photographic image;
Confirmation image acquisition means for acquiring an image of a region corresponding to the wrinkle confirmation region in the processed image as a processed confirmation image;
An image processing apparatus comprising: a display control unit configured to display the extracted processed confirmation image with a size that matches a resolution of the display apparatus. The confirmation image acquisition means is a means for extracting an image of a region corresponding to the wrinkle confirmation region in the photographic image as a pre-processing confirmation image;
The image processing apparatus according to claim 1, wherein the display control unit is a unit that causes the display device to display the processed confirmation image together with the pre-processing confirmation image. The wrinkle confirmation area determining means detects the amount of wrinkle components for each of a plurality of area images of the same size obtained by dividing the photographic image;
The image processing apparatus according to claim 1, further comprising an area determination unit that determines an area corresponding to the area image having the largest wrinkle amount as the wrinkle confirmation area. Means for creating a plurality of band limited images representing components for each of a plurality of frequency bands of the photographic image based on the photographic image;
For an input value in which the absolute value of the output value is less than or equal to the absolute value of the input value and the absolute value is less than or equal to a predetermined threshold value, the absolute value of the output value increases as the absolute value of the input value increases. On the other hand, for an input value whose absolute value is larger than the predetermined threshold, nonlinear conversion processing in which the absolute value of the output value is equal to or smaller than the absolute value of the output value corresponding to the predetermined threshold is performed in the band-limited image. Means for applying to each pixel value to obtain a plurality of converted images;
Means for multiplying a pixel value of the plurality of converted images by a predetermined subtraction coefficient;
2. A device for obtaining a pixel value of the processed image by subtracting a pixel value of the plurality of converted images multiplied by the subtraction coefficient from a pixel value of the photographic image. 4. The image processing apparatus according to any one of items 1 to 3. In a photographic image including a face, an area where wrinkle components are concentrated is determined as a wrinkle confirmation area,
Applying a skin beautifying process to remove the wrinkle component on the photographic image to obtain a processed image subjected to the skin beautifying process,
Obtaining an image of a region corresponding to the wrinkle confirmation region in the processed image as a processed confirmation image;
An image processing method, wherein the extracted processed confirmation image is displayed in a size that matches a resolution of a display device. In a photographic image including a face, a procedure for determining a wrinkle component concentration area as a wrinkle confirmation area;
A procedure for obtaining a processed image subjected to the skin beautification process by removing the wrinkle component from the photographic image;
A procedure for acquiring an image of a region corresponding to the wrinkle confirmation region in the processed image as a processed confirmation image;
A program for causing a computer to execute an image processing method including displaying the extracted processed confirmation image in a size that matches a resolution of a display device.

Images