JP4984247B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for performing image correction on an image obtained by photographing a person.

Currently, various methods are used to improve the appearance of a photograph taken of a person. In particular, it has become common to make skin spots and wrinkles inconspicuous by photographing methods and image processing. As a specific method, first of all, by adjusting the position where the lighting equipment is arranged at the time of taking a picture and optimizing the way the lighting is applied to the subject, blots and wrinkles existing on the skin of the person are made inconspicuous, A method for improving the quality of a photograph (for example, see Patent Document 1) and a method for simultaneously capturing and synthesizing a focused image and a defocused (blurred) image at the time of photographing (for example, see Patent Document 2) Can be mentioned. Secondly, there is a method in which a user designates the position of a spot or wrinkle on a captured image and performs correction by image processing (see, for example, Patent Document 3).
JP 2004-234571 A JP 2005-159561 A JP 2003-209683 A

However, in the first method, there is a problem that some preparation or a special photographing method must be performed at the time of photographing, and an already photographed image cannot be corrected. In the second method, since the user needs to process individual wrinkles and parts using an application dedicated to image processing, the user acquires advanced knowledge and skills to master such an exclusive application. There is a problem of having to.
The present invention has been made in view of the above points, and an object of the present invention is to provide an image processing apparatus and an image processing that can correct a person image after photographing and allow a user to perform correction by a simple operation. It is to provide a method.

The present invention has been made to solve the above-described problems, and an image information acquisition unit that acquires an original image that is image data of a person image to be corrected, and the original image acquired by the image information acquisition unit A face skin region extraction image generating unit that generates a masked image in which position information of the face skin region is added to the original image by extracting a face skin region that is a skin portion of the face from the face, and the face skin region From the background image generation unit that generates a background image by smoothing processing from the masked image generated by the extracted image generation unit, the background image generated by the background image generation unit, and the image information acquisition unit possess the acquired the original image combining unit for generating Aesthetic image by synthesizing, wherein the combining unit compares color information of corresponding pixels of the background image and the original image, The difference is preset The threshold or more pixels are an image processing apparatus for changing the synthesized value in the proportions set the pixel values of the original image in advance and the pixel value of the background image. Here, as the skin beautification image, it is possible to apply an image or the like having a good appearance by making the wrinkles and spots on the face of the person photograph inconspicuous by image processing.

  According to the present invention, an original image is obtained, and a base image is generated by performing a smoothing process for smoothing an image of a facial skin region that is a facial skin portion based on the original image. Since the images are combined, it is possible for the user to perform correction by a simple operation after shooting a person image.

  In the image processing apparatus according to the aspect of the invention, the background image generation unit may perform a smoothing process using a smoothing filter to generate the background image.

  According to the present invention, since the smoothing process is performed using the smoothing filter, it is possible to generate a background image in which the density of the image is reduced, including spots and wrinkles.

  Further, the present invention is characterized in that, in the above-described image processing apparatus, the background image generation unit previously sets a smoothing filter used for the smoothing process.

  According to the present invention, since the smoothing filter used for the smoothing process is set in advance, the user does not need to input parameters for setting the smoothing filter, and correction can be performed with a simple operation. .

  Further, according to the present invention, in the image processing apparatus, the combining unit compares color information of corresponding pixels of the background image and the original image, and a difference between the pixels is equal to or greater than a preset threshold value. Is characterized in that the pixel value of the original image is changed to a value synthesized with the pixel value of the background image at a preset ratio.

  According to the present invention, since the color information of each pixel is compared using a threshold value, wrinkles that are correction targets such as whether minor smears and wrinkles are to be corrected or only severe smears and wrinkles are to be corrected. It becomes possible to adjust the degree of blur. In addition, the pixel value at the position of the spot or wrinkle in the original image is synthesized with the pixel value of the background image at a preset ratio and approaches the pixel value of the background image. Can be generated.

  In the image processing apparatus according to the present invention, the background image generation unit selects a smoothing filter used for a smoothing process based on a parameter input to the input setting unit.

  According to the present invention, the user can select the degree of smoothing the background image by inputting the parameter for selecting the smoothing filter.

  Further, according to the present invention, in the image processing apparatus, the parameter input to the input setting unit is a parameter that specifies at least one range of a stain or a wrinkle to be corrected, and is based on the parameter Then, a threshold value is set for the comparison of the corresponding pixels of the background image and the original image in the combining unit.

  According to this invention, by inputting a parameter, a range of spots and wrinkles to be corrected is specified, and a correction process can be performed by setting a threshold value according to a user instruction. As a result, the user can specify the range in an intuitively understandable manner when performing the correction process.

  Further, according to the present invention, in the above image processing apparatus, the parameter input to the input setting unit is a parameter that specifies a degree of correction, and based on the parameter, the synthesis unit and the original image A ratio for combining pixel values of pixels of the background image is set.

  According to the present invention, by inputting parameters, the degree of correction of blots and wrinkles, such as making blotches and wrinkles less noticeable or slightly less noticeable, is specified, and synthesis according to user instructions The correction processing can be performed by setting the ratio of. Thereby, when performing the correction process, the user can specify the degree of correction by an intuitively easy-to-understand method.

  In addition, the present invention is characterized in that the image processing apparatus includes a printing unit that prints the skin beautification image generated by the combining unit.

  According to this invention, the user can print a skin-beautified image that has been corrected.

According to the present invention, the image processing apparatus described above acquires an original image that is image data of a person image to be corrected, and the image processing apparatus is a facial skin portion from the acquired original image. A step of generating a masked image in which position information of the face skin region is added to the original image by extracting a face skin region; and an image processing device converts a background image from the masked image by a smoothing process. The generating step and the image processing device compare the color information of the corresponding pixels of the background image and the original image, and pixels whose difference is equal to or greater than a preset threshold value are set to the pixel value of the original image. And changing the pixel value of the background image to a value synthesized at a preset ratio .

Further, the present invention obtains, in a computer, an original image that is image data of a human image to be corrected, and extracts a facial skin area that is a skin portion of the face from the obtained original image. Generating a masked image obtained by adding position information of the face skin region to the original image, generating a ground image from the masked image by a smoothing process, the ground image and the original image Compare the color information of each corresponding pixel, and change the pixel value of the original image to the value synthesized with the pixel value of the background image with the pixel value of the background image if the difference is greater than a preset threshold value Is a program for executing steps.

  According to the present invention, an original image is acquired, and an image subjected to smoothing processing for smoothing an image of a facial skin area that is a facial skin portion based on the original image is generated. Later, it becomes possible for the user to perform correction with a simple operation.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram for explaining the concept of processing executed by the image processing apparatus according to the present embodiment. (A) of this figure is the original image 100 of the person image to be corrected. Further, (b) in the figure is a background image 300 in which the density of the image is reduced, including spots and wrinkles, and this image is the skin that is a part of the facial skin from the original image 100 in (a). It is generated by performing correction by a smoothing process to be described later on the image from which the region has been extracted. In this background image 300, spots and wrinkles are not noticeable by performing the correction. The base image 300 obtained in this way and the original image 100 are combined to generate a skin-beautifying image 400 of FIG. 5C with good appearance, in which spots and wrinkles are not noticeable.

  Next, a specific configuration of the present embodiment will be described. FIG. 2 is a block diagram showing the configuration of the image processing apparatus according to the embodiment of the present invention. The image processing apparatus 1 includes an image information acquisition unit 2, an input setting unit 3, an image processing unit 4, and a printing unit 5. The image processing unit 4 includes a facial skin region extracted image generation unit 41, a background image generation unit 42, and a synthesis unit 43.

  The image information acquisition unit 2 acquires an original image 100 (see FIG. 1A), which is image data of a person image to be corrected, input to the image processing apparatus 1, and extracts the face image from the original image 100. The data is output to the image generation unit 41 and the synthesis unit 43.

  The image processing unit 4 performs the following correction on the original image 100 input from the image information acquisition unit 2 by the face skin region extraction image generation unit 41, the background image generation unit 42, and the synthesis unit 43.

  The face skin region extraction image generation unit 41 extracts a face skin region that is a skin portion of the face from the original image 100 acquired from the image information acquisition unit 2, and adds an image with a mask to which position information of the face skin region is added. 200 is generated, and the masked image 200 is output to the background image generation unit 42. As a method for extracting a face skin region, for example, a method of extracting only a portion corresponding to a pixel whose pixel value is in a predetermined range (skin color) from the original image 100 is used. When extracting a facial skin area, feature areas such as eyes, nose, and mouth are excluded as skin areas in order not to perform correction described later. Here, an area corresponding to the face is specified based on the position of the feature area. Also, if you want to leave a large mole as a feature and want to remove it from the facial skin area, set a parameter that specifies the size of the feature area to be excluded in the input setting unit 3, and based on that parameter It is also possible to recognize a feature region having a specified size and generate a facial skin region by excluding the region.

  The background image generation unit 42 generates the background image 300 by performing correction by a smoothing process on the masked image 200 input from the facial skin region extraction image generation unit 41. The smoothing process is a process of smoothing the color change depending on the location. For example, the process of averaging the pixel values of pixels in a predetermined range, for example, removing random noise on the image or for each pixel. This is a process that reduces fine changes to make a smooth image. By removing random noise and reducing fine changes for each image, the density of the image including spots and wrinkles is reduced. For example, a smoothing filter is applied to the smoothing process. In this way, the background image 300 is generated (see FIG. 1B), and the background image 300 is output to the synthesis unit 43.

The synthesizing unit 43 synthesizes the original image 100 acquired by the image information acquisition unit 2 and the background image 300 generated by the background image generation unit 42 to generate a skin beautifying image 400 (see FIG. 1C).
That is, first, the color information of each pixel corresponding to each of the original image 100 and the background image 300 is compared, and pixels whose difference is equal to or greater than a preset threshold value are compared with the pixel value of the original image 100 and the background image 300. The pixel values are combined at a set ratio to generate an image of the face skin area. To combine at a set ratio is to change the pixel value of the original image 100 to a value close to the pixel value of the background image 300. The pixel value to be changed is determined by the correction level of the parameter set by the input setting unit 3. For example, when the correction level is set high, the pixel value is changed to a pixel value closer to the pixel value of the background image 300. In this way, spots and wrinkles are less noticeable. Further, the threshold value is a parameter set by the user in the input setting unit 3, and the smaller the threshold value, the more pixels whose pixel value is changed.

となる。またこの式以外にも、下記数式２

を使用してもよい。なお、数式１を使用する場合は、構成が簡易になるため、家庭用の画像処理装置１に適しており、数式２を使用する場合は、より厳密な合成が行われるので、業務用の画像処理装置１に適している。この様な処理を各画素について繰り返し、変更された画素値に基づいて画像が生成される。 FIG. 3 shows a specific example for explaining the processing of the synthesis unit 42. The horizontal axis is the pixel position of the image data, and the vertical axis is the pixel value, for example, color information such as RGB. A solid line is data of the original image 100. Since the color information is greatly different from the skin at the spot or wrinkle position, the solid line has a dent at the corresponding pixel position. When the smoothing process is performed on the data of the image with mask 200, the pixel values of the pixels in a predetermined range are smoothed with the peripheral pixel values, so that smooth curved line data such as a dotted line is obtained. This is the background image 300. Then, the pixel value of the background image 300 thus obtained and the pixel value of the original image 100 are compared for each pixel, and it is determined whether or not the difference is equal to or greater than a threshold value. For example, the difference between the pixel value a (x, y) of the original image 100 and the pixel value a ′ (x, y) of the background image 300 at the pixel position (x, y) is large. If it is greater than or equal to s, the pixel value at the pixel position (x, y) of the skin beautification image is changed to the pixel value A (x, y) calculated based on the parameters. The pixel value A (x, y) is expressed by the following mathematical formula 1 where α is a ratio of combining.

It becomes. In addition to this formula, the following formula 2

May be used. In addition, since the structure becomes simple when using Formula 1, it is suitable for the home image processing apparatus 1, and when using Formula 2, since stricter composition is performed, the image for business use is used. Suitable for the processing apparatus 1. Such processing is repeated for each pixel, and an image is generated based on the changed pixel value.

  Finally, the image of the facial skin area generated is fitted into the original image 100, that is, the skin beautification image is obtained by changing the pixel value of the facial skin area of the original image 100 to the pixel value of the generated facial skin area image. 400 is generated. By this composition, it is possible to generate a good-looking photograph with no spots or wrinkles. The skin beautification image 400 is output to the printing unit 5.

  The printing unit 5 prints the skin beautification image 400 generated by the image processing unit 4.

  The input setting unit 3 receives input from the user and sets parameters of the image processing unit 4. These parameters include a correction level parameter and a threshold parameter, which are used by the synthesis unit 42. The threshold parameter is a parameter that specifies a range of at least one of a stain and a wrinkle to be corrected. Based on this parameter, the corresponding pixels of the background image 300 and the original image 100 are compared. Threshold is set. The correction level parameter is a parameter for designating the degree of correction. Based on this parameter, the ratio for combining the pixel values of the pixels of the original image 100 and the background image 300 is set. As described above, by setting these parameters, it is possible to make the spots and wrinkles less noticeable as the correction level becomes higher, or to correct minor spots and wrinkles as the threshold value becomes smaller. Further, as necessary, as described above, a parameter for specifying the size of a feature region to be excluded such as a large mole that is a feature of a person may be set as a parameter used in the facial skin region extracted image generation unit 41. Is possible. The user can change these parameters according to the scene. In this embodiment, a case where parameters are set in advance will be described, and a case where a user sets parameters will be described later.

  FIG. 4 is a flowchart illustrating an example of a process flow in the image processing apparatus 1 described above. First, the image information acquisition unit 2 acquires an original image 100 that is image data of a person image to be corrected (step S1), and outputs it to the facial skin region extracted image generation unit 41. Next, the face skin region extracted image generation unit 41 generates a masked image 200 by extracting a face skin region that is a skin portion of the face from the input original image 100 (step S2), and a background image 300 output to the generator 42.

  Next, the background image generation unit 42 generates a background image 300 by performing a smoothing process on the input masked image 200 (step S <b> 3), and outputs it to the synthesis unit 43. The synthesizing unit 43 first compares the color information of each corresponding pixel in the skin region image and the background image 300, and determines whether or not the difference is greater than or equal to a threshold value (step S4). If the difference is equal to or greater than the threshold, the pixels of the original image 100 and the pixels of the background image 300 are combined at a set ratio (step S5). Thereafter, it is determined whether or not the processing from step S4 to step S5 has been completed for all the pixels in the facial skin area (step S6). If not completed (step S7), the processing from step S4 is repeated. If the difference is less than the threshold value in step S4, the process in step S5 is skipped and the processes in and after step S6 are performed.

  When all the pixels are completed, the synthesis unit 43 generates the skin beautification image 400 by fitting the generated image of the facial skin area into the original image 100 acquired by the image information acquisition unit 2 (Step S8), The converted image 400 is output to the printing unit 5. In this way, a good-looking photograph in which spots and wrinkles are not noticeable is generated. Finally, the printing unit 5 prints the skin beautification image 400 output from the synthesis unit 43 (step S9).

  As described above, according to the present embodiment, correction can be performed after a person image is captured, and correction can be performed by a user with a simple operation. In the embodiment described above, parameters are set in advance. That is, when an administrator of the image processing apparatus or the like sets standard parameters in advance, correction processing can be performed only by designating a human image as a correction target without the user inputting parameters.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to
For example, instead of automatically generating the face skin area, the user may select the area and generate the face skin area. A configuration in which the printing unit 5 is replaced with a display unit is also possible.

In the above-described embodiment, the case where the parameters are set in advance has been described. However, the parameters may be input by the user.
FIG. 5 is a diagram illustrating an example of a screen on which a user inputs parameters. On the screen in this figure, the user can specify intuitively easy-to-understand contents. In this figure, a slide bar (symbol a) for determining whether to include even a large one in the correction range sets the size of a feature region to be excluded from the correction range, such as a large mole that is a feature of a person. When the slide bar is brought close to “include”, the feature region to be excluded becomes large, and when the slide bar is brought close to “exclude”, the feature region to be excluded becomes small. The slide bar (symbol b) for determining whether to include even a thin object in the correction range sets the severity of wrinkles and spots to be corrected. When the slide bar is brought close to “include”, the thin wrinkles and spots are subject to correction. When the slide bar is brought close to “exclude”, the thin ones are not subject to correction. The slide bar (symbol c) that determines the composition ratio sets the strength of the composition. When the slide bar is moved closer to “Strong”, the correction strength becomes stronger (the degree that the spots and wrinkles are inconspicuous) becomes stronger. .
In this way, the user can set parameters simply by moving the slide bar. Therefore, parameters can be set by a designation method that is intuitively easy to understand without any special knowledge. Note that a plurality of smoothing filters are provided in the image processing apparatus, and a smoothing filter used in the smoothing process can be selected by input from the user.

2 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to perform image processing. May be. Here, the “computer system” may include an OS and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

It is a figure for demonstrating the concept of the process which the image processing apparatus of this embodiment performs. 1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. It is the figure which showed the specific example explaining the process of the base image generation part. It is a flowchart showing an example of the flow of a process in the image processing apparatus of this embodiment. It is a figure showing an example of the screen in which a user inputs a parameter.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 2 ... Image information acquisition part 3 ... Input setting part 4 ... Image processing part 41 ... Face skin area extraction image generation part 42 ... Background image generation part 43 ... Composition part 5 ... Printing part 100 ... Original image 200 ... Image with mask 300 ... Image for background 400 ... Skin beautification image

Claims (9)

An image information acquisition unit that acquires an original image that is image data of a person image to be corrected;
Facial skin that generates a masked image in which position information of the facial skin region is added to the original image by extracting a facial skin region that is a part of the facial skin from the original image acquired by the image information acquisition unit An area extraction image generation unit;
A background image generation unit that generates a background image by a smoothing process from the masked image generated by the face skin region extraction image generation unit;
A combining unit that generates a beautifying image by combining the background image generated by the background image generating unit and the original image acquired from the image information acquisition unit;
I have a,
The synthesizing unit compares the color information of each pixel corresponding to the background image and the original image, and a pixel whose difference is equal to or greater than a preset threshold value sets the pixel value of the original image to the background image. An image processing apparatus that changes a pixel value to a value synthesized at a preset ratio . The image processing apparatus according to claim 1, wherein the ground image generation unit performs a smoothing process using a smoothing filter to generate the ground image. The image processing apparatus according to claim 2, wherein the ground image generation unit sets in advance a smoothing filter used for smoothing processing. The said background image generation part selects the smoothing filter used for a smoothing process based on the parameter input into the said input setting part, The Claim 1 characterized by the above-mentioned Image processing device. The parameter input to the input setting unit is a parameter that specifies a range of at least one of a stain and a wrinkle to be corrected, and based on the parameter, the background image and the original image are converted by the combining unit. The image processing apparatus according to claim 4 , wherein a threshold value for comparing each corresponding pixel of the image is set . The parameter input to the input setting unit is a parameter that specifies the degree of correction, and based on the parameter, the combining unit is configured to combine pixel values of the pixels of the original image and the background image. 6. The image processing apparatus according to claim 4, wherein a ratio is set . The image processing apparatus according to claim 1, further comprising a printing unit that prints the skin beautification image generated by the combining unit . An image processing apparatus obtaining an original image that is image data of a person image to be corrected;
Generating an image with a mask in which position information of the face skin region is added to the original image by extracting a face skin region that is a part of the skin of the face from the acquired original image;
An image processing apparatus generating a background image from the image with mask by a smoothing process;
An image processing apparatus compares the color information of each pixel corresponding to the background image and the original image, and a pixel whose difference is equal to or greater than a preset threshold value sets the pixel value of the original image to that of the background image. A step of changing to a value synthesized with a pixel value and a preset ratio;
An image processing method. On the computer,
Obtaining an original image that is image data of a person image to be corrected;
Generating a masked image in which position information of the facial skin region is added to the original image by extracting a facial skin region that is a part of the skin of the face from the acquired original image;
Generating a background image from the masked image by a smoothing process;
The color information of each pixel corresponding to the background image and the original image is compared, and for the pixel whose difference is greater than or equal to a preset threshold value, the pixel value of the original image is set in advance as the pixel value of the background image. Generating a skin beautifying image by changing to a value synthesized at a proportion,
A program for running

Images