JP2017111595A - Image processing device, image processing method, image processing program and image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device, an image processing method, program and system capable of generating an appropriate image by only one smoothing operation from one image, without requiring manual work or a skill, naturally reducing skin noises such as blotches, freckles, wrinkles or pimples of a human image while maintaining minute unevenness such as a personal texture or luster.SOLUTION: An information processing device includes: a guide image generation part 110 configured to generate a guide image from an input image; and a noise reduction part 120 configured to apply a reference type edge preservation smoothing filter to the input image according to the guide image to generate a processed image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus, an image processing method, an image processing program, and an image processing system.

  Conventionally, various proposals have been made to beautifully express a person's skin in an image by performing skin-beautifying processing for removing skin noise such as spots, freckles, wrinkles, and acne on an image including a person.

  For example, a method has been proposed in which skin noise is removed by designating a position such as a spot, freckles, or acne using a pointing device, and replacing the designated position with surrounding pixels (see, for example, Patent Document 1). reference).

  In addition, a method has been proposed in which a statistical model is constructed in advance from a facial skin image without skin noise, and skin noise is removed based on the constructed statistical model (see, for example, Patent Document 2).

  In addition, by applying an ε-filter that separates and removes high-frequency noise components with small amplitude superimposed on the signal waveform, smoothing small-gradient gradients such as wrinkles and spots while retaining edges with large gradients A technique has been proposed (see, for example, Patent Document 3).

  In addition, a technique has been proposed in which skin noise is removed by optimizing the strength of the smoothing process according to the face detection size so as not to blur the area other than the person as much as possible (see, for example, Patent Document 4). .

  However, the skin beautification process described in Patent Document 1 is a manual operation and has a problem that it takes time and effort.

  The skin beautifying process described in Patent Document 2 has a problem that a smoothing process ignoring fine irregularities peculiar to individuals is performed by using statistical information, resulting in an uncomfortable feeling.

  The skin beautification described in Patent Documents 3 and 4 can retain edges such as contours, but has a problem that fine irregularities peculiar to individuals such as texture and gloss are lost together with noise such as spots.

  On the other hand, a reference type edge preserving smoothing filter that corrects noise while leaving the texture of a subject using two images having different characteristics has been proposed (for example, see Non-Patent Documents 1 and 2).

  Non-Patent Document 1 discloses a technique for reducing noise in a captured image without damaging the contours and fine irregularities by using, as reference information, a flash image with less noise captured with the same composition as the captured image including noise. Has been. However, in order to apply this method, a captured image and another image with the same composition are required, and a special optical system is required to achieve one shot, or a general snapshot And cannot be applied to videos.

  In order to solve this problem, Non-Patent Document 2 proposes a technique for reducing noise by applying a reference-type edge preserving smoothing filter from one image by using an image obtained by smoothing a noise image as a guide image. Has been. However, since the guide image has already been smoothed, there is a problem that fine irregularities peculiar to individuals such as texture and luster are lost and processing time is required because the smoothing process is repeated.

  As described above, there are various problems in the conventional method for removing skin noise, and solutions to these problems are required.

  The present invention has been proposed in view of the above-described conventional problems, and its object is to make fine irregularities such as texture and gloss peculiar to an individual from one image without requiring manual work or skill. It is intended to naturally reduce skin noise such as stains, freckles, wrinkles, and acne in a human image while holding the image, and to generate a suitable image by only one smoothing process.

  In order to solve the above problems, in the present invention, a guide image generation unit that generates a guide image from an input image, and a reference-type edge-preserving smoothing filter is applied to the input image based on the guide image. And noise reduction means for generating a processed image.

  In the present invention, skin noise such as stains, freckles, wrinkles, acne, etc. of a person image is retained from a single image while maintaining fine irregularities such as texture and gloss peculiar to individuals without requiring manual work or skill. Can be reduced naturally, and a suitable image can be generated by a single smoothing process.

1 is a diagram illustrating a hardware configuration example of an image processing apparatus according to a first embodiment. It is a figure which shows the function structural example of the image processing apparatus concerning 1st Embodiment. 5 is a flowchart illustrating an example of image processing in the first embodiment. It is a figure which shows the example of a part of input image. It is a figure which shows the example of pixel value distribution. It is a figure which shows the example of noise reduction. It is a figure which shows the example of an overemphasis area | region restoration. It is a figure which shows the function structural example of the image processing apparatus concerning 2nd Embodiment. It is a flowchart which shows the example of the image process in 2nd Embodiment. It is a flowchart which shows the process example of noise size estimation and filter parameter determination. It is a figure which shows the example of a dimension table. It is a figure which shows the structural example of the image processing system concerning 3rd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described.

<First Embodiment>
FIG. 1 is a diagram illustrating a hardware configuration example of an image processing apparatus 1 according to the first embodiment. In FIG. 1, an image processing apparatus 1 includes a control unit 11, a main storage unit 12, an auxiliary storage unit 13, an external storage device I / F unit 14, and a network I / F unit 15 that are connected to each other via a bus B. And a display unit 16 and an operation unit 17. The image processing apparatus 1 is composed of, for example, a personal computer.

  The control unit 11 is a CPU (Central Processing Unit) that performs control of each device, calculation of data, and processing in the computer. The control unit 11 is an arithmetic device that executes a program stored in the main storage unit 12 or the auxiliary storage unit 13. The control unit 11 receives data from the input device or the storage device, calculates, and processes the output device or the storage device. Output to the device.

  The main storage unit 12 is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), or the like, and stores programs or data such as an OS (Operating System) or application software that is basic software executed by the control unit 11 or This is a storage device for temporary storage.

  The auxiliary storage unit 13 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and is a storage device that stores data related to application software and the like.

  The external storage device I / F unit 14 is an interface between the image processing apparatus 1 and a storage medium 18 such as a flash memory connected via a data transmission path such as USB (Universal Serial Bus). The program stored in the storage medium 18 is installed via the external storage device I / F unit 14 and can be executed by the image processing apparatus 1.

  The network I / F unit 15 has a communication function connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network) constructed by a data transmission path such as a wired and / or wireless line. This is an interface between the device and the image processing apparatus 1.

  The display unit 16 is, for example, a display such as a liquid crystal or an organic EL (Electro Luminescence), displays an image, an operation icon, and the like, and performs various settings when the user uses the functions of the image processing apparatus 1. It is a device that functions as an interface.

  The operation unit 17 is, for example, a key switch composed of hard keys or a mouse. Further, the operation unit 17 may be a touch panel or the like provided on the screen of the display unit 16.

  FIG. 2 is a diagram illustrating a functional configuration example of the image processing apparatus 1 according to the first embodiment. In FIG. 2, the image processing apparatus 1 includes a guide image generation unit 110, a noise reduction unit 120, and an overemphasized region restoration unit 130. The guide image generation unit 110 has a function of generating a guide image from an input image. The noise reduction unit 120 has a function of generating a processed image by applying a reference type edge preserving smoothing filter to an input image based on the guide image generated by the guide image generation unit 110. The overemphasized area restoration unit 130 has a function of restoring the overemphasized area of the processed image from the difference information between the input image and the processed image and generating an output image. The guide image generation unit 110, the noise reduction unit 120, and the overemphasized region restoration unit 130 are realized by the control unit 11 illustrated in FIG. 1 executing programs stored in the main storage unit 12, the auxiliary storage unit 13, and the like. Function.

  FIG. 3 is a flowchart showing an example of image processing in the first embodiment. In FIG. 3, when the process is started, first, in step S11, the guide image generation unit 110 generates a guide image I with less skin noise from the input image p from which the skin region in the image is extracted.

  Subsequently, in step S12, the noise reduction unit 120 applies the reference-type edge preserving smoothing filter to the input image p using the guide image I as a reference image, and generates a processed image q with less skin noise.

  Finally, in step S13, the overemphasized region restoration unit 130 identifies the overemphasized region generated by the noise reduction processing in step S12 from the input image p and the processed image q, and determines the overemphasized region of the processed image q. The restored output image d is generated, and the process ends.

  Hereinafter, the processing contents of each step in the process of correcting the skin area from the input image from which the skin area in the image has been extracted will be described in detail together with data used for the image processing.

[Step S11: Guide Image Generation]
When an input image p in which a skin region in an image is extracted using an existing skin region extraction method or an object detection method is input to the image processing apparatus 1, the guide image generation unit 110 detects skin noise from the input image p. A few guide images I are generated. In the skin region, the longer the wavelength, the higher the light straightness and the less skin noise information is recorded. Therefore, the guide image I can be generated by extracting the channel value of the longest wavelength of the input image. it can. For example, in the case of an RGB image, it can be performed by extracting Rch (Red channel) information as it is.

  The skin is a translucent object, and complex light scattering called subsurface scattering occurs inside the skin. In the visible region, light has higher straightness as the wavelength is longer, and tends to go around the incident part. In addition, the melanin pigment and hemoglobin pigment that influence the skin color have strong absorption in the low to medium wavelength band, and as a result, skin noise such as spots and acne is hardly recorded on the Rch of the image due to the effect of light wrapping. Therefore, it can be used as the guide image I. In addition, since the guide image I does not spatially smooth the information of the input image p, fine unevenness such as texture can be maintained.

  Further, the generation of the guide image I may be performed by the following formula (1), for example.

  Here, the subscript i represents the pixel position in the image, and the subscript c represents each channel of the multi-channel input image. For example, in the case of an RGB image, take the maximum value of Rch, Gch, and Bch in each pixel. A guide image I is generated. In the skin region, the guide image I mainly reflects information on long-wavelength Rch. Therefore, the guide image I with less skin noise can be obtained from the equation (1).

[Step S12: Noise reduction]
When a guide image I with less skin noise is generated from the input image p by the guide image generation unit 110, the noise reduction unit 120 applies a reference type edge preserving smoothing filter to the input image p using the guide image I as a reference image. Then, a processed image q with less skin noise is generated.

  In this embodiment, the reference type edge preserving filter is disclosed in Kaiming He, Jian Sun, and Xiaoou Tang .: Guided Image Filtering, IEEE TPAMI, Vol.35, No.6, pp.1397-1409 (2013). An example using a guided filter will be described.

  The guided filter is a filter that assumes that the processed image q with less skin noise of the input image p is a linear transformation of the guide image I, and is represented by the following equation (2), for example.

  Here, ω is a kernel representing the pixel i in the pixel i and surrounding pixels, and | ω | represents the number of pixels in the kernel. a and b are coefficients, and a and b with a horizontal line on them are their averages.

  a and b are represented by Formula (3) and Formula (4), respectively. Note that ω is set to a prescribed shape and size.

  Here, p with a horizontal line drawn thereon is the average of the pixel values of the input image p in the kernel ω, and μ and σ are the average and variance of the pixel values of the guide image I in the kernel ω. ε is a control parameter of a and is set to a specified value.

  In this embodiment, each channel of the input image p is filtered with the guide image I with less skin noise. As shown in the equations (2) to (4), the guided filter can be regarded as a transfer of the gradient of the guide image I with less skin noise instead of the gradient of the input image p including skin noise. That is, in the skin region, it is possible to generate a processed image q with less skin noise that is consistent with the brightness of the input image p.

  FIG. 4A shows an example of a part of an input skin region (input image p), and shows a portion from the nose to the cheek. In the following description, as shown in FIG. 4B, the effect of noise reduction will be described by paying attention to a specific spot 101 of the skin region 100 and a pixel value distribution of a broken line portion 102 that crosses the spot 101.

  FIG. 5 exemplifies the pixel value distribution of the broken line portion 102 shown in FIG. 4B in each of the input image p, the guide image I, and the processed image q, and each channel of the color image as a pixel value. (For example, in the case of RGB, GRAY = 0.299 * R + 0.587 * G + 0.114 * B) is used. The reason why the level of the pixel value of the guide image I is different from the other is that the guide image I is mainly composed of R channel information having the largest pixel value in the skin.

  In FIG. 5, a hatched area 103 corresponds to the spot 101 in FIG. 4B, and it can be seen that the input image p is recorded with a steep gradient. On the other hand, in the guide image I, the stain in the region 103 has a gentler gradient than the input image p due to the light sneaking action, and the processed image q to which the gradient of the guide image I is transferred results in the steepness in the input image p. Smooth slopes and stains are removed.

  On the other hand, in FIG. 5, a region 104 indicated by a wavy line is the contour of the nose on the left side of FIG. 4, and since the gradient is large in the guide image I, the contour is also retained in the processed image q. Further, since the guide image I uses the channel information recorded by the image sensor (camera) as it is without being smoothed, the fine unevenness indicating the skin texture of the person in the area 105 indicated by the halftone dots in FIG. 5 is also retained. The As a result, by applying the guided filter shown in the equations (2) to (4) to the input image p using the guide image I that takes the Rch value or the channel maximum value of the input image p as a reference image, the contours and fine irregularities It is possible to obtain a processed image q from which only skin noise such as a stain has been removed.

  6A shows the same input image p as FIG. 4A, and FIG. 6B shows a processed image q. For comparison, FIG. 6C shows the result of noise removal using a bilateral filter that is an edge-preserving smoothing filter. It can be seen from FIG. 6C that the bilateral filter reduces skin noise such as spots and maintains the outline of the nose, but the fine unevenness of the skin is lost by the smoothing process and the skin is smooth. On the other hand, as shown in FIG. 6B, the processed image q obtained by the noise reduction unit 120 of the present embodiment is not only the outline of the nose but also the skin such as a stain while maintaining the fine unevenness of the skin. Since the noise can be reduced, more natural skin noise removal can be realized.

  Further, as the reference type edge preserving smoothing filter, a joint bilateral filter disclosed in Non-Patent Document 1 can be used instead of the guided filter shown in the equations (2) to (4). The joint bilateral filter is expressed by, for example, the following formula (5).

As shown in equation (5), the joint bilateral filter determines the weight of the pixel mixture in the kernel ω _k by two Gaussian functions. The first Gaussian function is a weight related to the closeness of the distance, and the weight increases as the pixel i and the pixel j are spatially closer. The second Gaussian function is a weight related to the proximity of pixel values, and the weight increases as the pixel values of the pixel i and the pixel j in the guide image I are closer. By having the second Gaussian function, skin noise such as a stain can be reduced while maintaining the outline of the nose and the fine unevenness of the skin as in the guided filter. Here, σ _s and σ _r are parameters for controlling the weight of each Gaussian function, and are set to prescribed values. K _i is a weight normalization term and is expressed by the following equation (6).

[Step S13: Overemphasis region restoration]
When the processed image q with less skin noise of the input image p is generated by the noise reducing unit 120, the over-emphasized region restoration unit 130 finally generates an over-emphasized region generated by noise reduction from the input image p and the processed image q. And an output image d in which the overemphasized region of the processed image q is restored is generated.

  In the example shown in FIG. 5, fine unevenness representing the skinness of the person appears in the halftone dot area 105, but the pixel value of the processed image q is slightly smaller than the input image p in the area 105. This is over-emphasis due to the transfer of the gradient of the guide image I in which the gradient between the gloss and the normal skin portion becomes gentle, and in the processed image q, the fineness such as the gloss representing the skinness of the person is displayed. Unevenness is also reduced.

  Similarly, when the expression (1) is used to generate the guide image I, if the pixel value is locally saturated even in one of the channels due to an exposure setting error or illumination, the region has no gradient. Therefore, over-emphasis is applied in reducing noise. The overemphasized region restoration unit 130 is a process for restoring such a guide image and an overemphasized region caused by noise reduction, and an output image d obtained by restoring the overemphasized region according to the following equation (7). Is generated.

  In Expression (7), the second term on the right side indicates the restoration information of the overemphasized region. t is a predetermined threshold value. In FIG. 5, the halftone dot region 105 representing the overemphasized region has a smaller pixel value in the processed image q than in the input image p. On the other hand, skin noise such as spots, freckles, acne, wrinkles, etc. as shown in the hatched area 103 appears darker than the normal skin part, so that the processed image q has a larger pixel value than the input image p. That is, as shown in the second term on the right side of Expression (7), when the input image q is subtracted from the input image p, the difference information includes positive and skinned areas such as gloss, gloss, and saturated pixel values. The noise region appears as negative, and the over-emphasis region and the skin noise region can be separated. Here, for example, by setting t = 0, it is possible to extract the original information of the input image p in the overemphasized region where the difference information is not negative, and the processed image q is the source of the input image p in the overemphasized region. By adding this information, the overemphasized area can be restored.

  FIG. 7 shows an example in which an overemphasized region caused by noise reduction is restored. FIG. 7A shows an input image p, FIG. 7B shows a processed image q, and FIG. 7C shows an output image d. Is shown. When the processed image q in FIG. 7B and the output image d in FIG. 7C are compared with the input image p in FIG. 7A, the output image d has a fine gloss with the stains removed. It can be seen that has been restored.

  The restoration of the overemphasized region is not limited to the equation (7), and the following equation (8) can also be used. In other words, any method may be used as long as it uses the difference between the pixel values of the overemphasized region and the skin noise to separate and reflect the difference.

  According to the image processing shown in the present embodiment, it is possible to naturally reduce skin noise such as spots and acne while maintaining fine irregularities such as texture and gloss representing the skinness of the person.

  Note that the final output image of the present embodiment is the output image of the skin area in the image when the skin area in the image extracted using the existing skin area extraction method as described above is used as the input image. May be used as the final output image, or when the human region extracted using the existing object detection method is used as the input image, the human image in the image is replaced with the output image. It is good. In addition, when the image itself is an input image, the obtained output image may be the final output image, or a skin region with less skin noise obtained by performing extraction of an existing skin region on the output image. May be combined with an image to obtain a final output image.

<Second Embodiment>
In this embodiment, a method for adaptively realizing skin noise removal regardless of the image magnification of the subject will be described. In other words, in noise reduction processing, it is necessary to set an appropriate filter parameter according to the target skin noise level. If the subject image magnification is not constant, the filter parameter is set each time. It must be done and is not practical. In the present embodiment, optimization is achieved by dynamically changing the filter parameter according to the image magnification of the subject.

  FIG. 8 is a diagram illustrating a functional configuration example of the image processing apparatus 1 according to the second embodiment. In FIG. 8, the image processing apparatus 1 includes a guide image generation unit 110, a noise reduction unit 120, an overemphasized region restoration unit 130, and a filter parameter optimization unit 140. The configuration is the same as that shown in FIG. 2 except that a filter parameter optimization unit 140 that optimizes the filter parameters of the noise reduction unit 120 based on the input image is newly added. The hardware configuration of the image processing apparatus 1 is the same as that shown in FIG. In the guide image generation unit 110, the noise reduction unit 120, the overemphasis region restoration unit 130, and the filter parameter optimization unit 140, the control unit 11 illustrated in FIG. 1 is stored in the main storage unit 12, the auxiliary storage unit 13, and the like. It is a function realized by executing a program.

  FIG. 9 is a flowchart showing an example of image processing in the second embodiment. In FIG. 9, when the process is started, first, in step S21, the guide image generation unit 110 generates a guide image I with less skin noise from the input image p from which the skin region in the image is extracted.

  Subsequently, in step S22, the size of the skin noise in the image is estimated, and the filter parameter of the reference type edge enhancement smoothing filter used in the subsequent step S23 is determined.

  Subsequently, in step S23, the noise reduction unit 120 applies a reference type edge preserving smoothing filter to the input image p using the guide image I as a reference image, and generates a processed image q with less skin noise.

  Finally, in step S24, the overemphasized region restoration unit 130 identifies the overemphasized region generated by the noise reduction processing in step S23 from the input image p and the processed image q, and determines the overemphasized region of the processed image q. The restored output image d is generated, and the process ends.

  Below, the processing content of each step in the process which corrects a skin area | region from the input image from which the skin area | region in the image was extracted is demonstrated with the data etc. which are used for an image process. Note that processing similar to that of the first embodiment will be briefly described.

[Step S21: Guide Image Generation]
When an input image p in which a skin region in an image is extracted using an existing skin region extraction method or an object detection method is input to the image processing apparatus 1, the guide image generation unit 110 detects skin noise from the input image p. A few guide images I are generated.

[Step S22: Filter parameter optimization]
When the guide image generation unit 110 generates a guide image I with less skin noise from the input image p, the filter parameter optimization unit 140 then estimates the size of the skin noise from the input image p, and the optimal filter parameter To decide.

  The guided filters shown in the equations (2) to (6) are for transferring the gradient of the guide image I based on the average / dispersion in the kernel ω of the input image p, and filter processing is performed according to the size of the kernel ω. The effect of changes. If the kernel ω is too small, the effect of reducing skin noise such as stains is weak even if the guide image I with less skin noise is used, and if the kernel ω is too large, fine effects caused by capillaries that are not skin noise Color unevenness disappears and the natural appearance of the skin is impaired. On the other hand, since the appearance size of the skin area of a person in the image changes depending on the shooting distance and the angle of view, the appearance size of skin noise such as a stain in the skin area also changes accordingly. Therefore, it is desirable to estimate the apparent size of skin noise from the image and set the optimum filter parameters.

  FIG. 10 is a flowchart illustrating a processing example of noise size estimation and filter parameter determination (step S22 in FIG. 9) by the filter parameter optimization unit 140.

  When the process is started, first, in step S221, organs such as a face, a mouth, and a nose are detected from the image. The detection of the organ can be performed using an object detection method described in Paul Viola and Michael J. Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”, IEEE CVPR, 2001.

  Subsequently, in step S222, the subsequent processing branches depending on whether or not the organ has been detected. If not detected, a prescribed kernel size is adopted in step S225. That is, the same processing as in the first embodiment is performed.

  If the organ can be detected, then in step S223, the detected organ size is determined from the previously stored organ dimension table.

Is read. FIG. 11 is a diagram illustrating an example of a dimension table. For example, when an eye is detected in step S221, FIG.

Is read out.

Subsequently, returning to FIG. 10, in step S223, the detected organ size s _o [px] (pixels) is set to the corresponding organ size.

To determine the optimum kernel size _sω . The following equation (9) is an example of a formula for calculating the magnitude s _omega optimal kernel omega.

Here, λ _o is a kernel size determination coefficient, and is a coefficient set in advance so that skin noise can be effectively removed from the correspondence relationship between the dimensional statistical value of the detection organ and the dimensional statistical value of the skin noise. In equation (9), since the detected organ size s _o [px] is normalized by the dimensional statistical value, one representative coefficient λ is set instead of setting λ _o for each organ. May be. However, it is desirable to set λ _o individually because the dimensional statistical values of the organs also have different standard deviations depending on the organs (individual differences are different).

When multiple organs are detected, the average or median value of the individual _sω may be taken, the reliability of the detected organs (detection accuracy, standard deviation, etc.), and the importance of the subject in the image it may be utilized with the s _omega selection based on the (size of the detection organ and the skin area). Since s _ω is the size of the kernel ω, it is generally set to be a positive number. When an organ is not detected, a prescribed kernel size is used as in the first embodiment.

[Step S23: Noise Reduction]
Subsequently, returning to FIG. 9, the noise reduction unit 120 applies the reference type edge preserving smoothing filter to the input image p using the guide image I as a reference image, and generates a processed image q with less skin noise. Here, the kernel size of the reference type edge-preserving smoothing filter utilizes s _omega obtained by the filter parameter optimization unit 140.

[Step S24: Restoring Overemphasized Region]
When the processed image q with less skin noise of the input image p is generated by the noise reducing unit 120, the over-emphasized region restoration unit 130 finally generates an over-emphasized region generated by noise reduction from the input image p and the processed image q. And an output image d in which the overemphasized region of the processed image q is restored is generated.

  The first embodiment is different from the first embodiment in that the filter parameter optimization in step S22 is added, and thereby, an adaptive skin beautification process according to the skin region in the image can be realized.

  Note that the final output image of the present embodiment is the output image of the skin area in the image when the skin area in the image extracted using the existing skin area extraction method as described above is used as the input image. May be used as the final output image, or when the human region extracted using the existing object detection method is used as the input image, the human image in the image is replaced with the output image. It is good. In addition, when the image itself is an input image, the obtained output image may be the final output image, or a skin region with less skin noise obtained by performing extraction of an existing skin region on the output image. May be combined with an image to obtain a final output image.

If there are multiple skin regions, the optimal kernel size s _ω is set for each of the skin regions, an output image is generated individually, and each output image is combined with the image as a final output image. Good.

<Third Embodiment>
In the present embodiment, a configuration example in which actual services such as image acquisition, display / printing, and the like are performed using the image processing apparatus 1 in the first and second embodiments described above will be described.

  FIG. 12 is a diagram illustrating a configuration example of an image processing system according to the third embodiment. In FIG. 12, the image processing system includes an image processing apparatus 1, an image generation apparatus 2, a server apparatus 4 and an image output apparatus 5 connected via a network 3.

  The image generation device 2 is, for example, a digital camera or a scanner, and generates an input image to be processed by the image processing device 1. The input image is sent to the network I / F unit 15 (FIG. 1) or the external storage device I / F unit 14 (FIG. 1) of the image processing device 1 via the storage medium 18 such as the WIFI communication module 21 of the image generation device 2 or the flash memory. Input from 1).

  The server device 4 can provide an input image to be processed by the image processing device 1 via the network 3 such as the Internet instead of the image generating device 2. Further, the server device 4 can accept and store the output image of the image processing device 1 via the network 3.

  The image output device 5 can display or print the output image output from the image processing device 1 directly or by acquiring the output image accumulated in the server device 4.

  The functions of the image processing apparatus 1 may be configured to be incorporated in the image generation apparatus 2, the server apparatus 4, and the image output apparatus 5.

<Summary>
As described above, according to the present embodiment, from one image, it is not necessary to perform manual work or skill, and while maintaining fine irregularities such as texture and gloss peculiar to an individual, -It is possible to naturally reduce skin noise such as acne, and to generate a suitable image by only one smoothing process.

  The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments, various modifications and changes may be made to the embodiments without departing from the broad spirit and scope of the invention as defined in the claims. Obviously you can. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings.

<Correspondence between Terms in Embodiment and Terms in Claims>
The guide image generation unit 110 is an example of “guide image generation means”. The noise reduction unit 120 is an example of “noise reduction means”. The overemphasized region restoring unit 130 is an example of “overemphasized region restoring means”.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 11 Control part 12 Main storage part 13 Auxiliary storage part 14 External storage apparatus I / F part 15 Network I / F part 16 Display part 17 Operation part 18 Storage medium B bus 110 Guide image generation part 120 Noise reduction part 130 Overemphasis region restoration unit 140 Filter parameter optimization unit 2 Image generation device 3 Network 4 Server device 5 Image output device

JP 2003-331306 A JP 2007-087234 A Japanese Patent Laid-Open No. 2001-118084 JP 2009-020834 A

G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Cohen and K. Toyama .: Digital photography with flash and no-flash image pairs, ACM TOG, Vol. 23, No. 3, pp. 664 -672, 2004. Tsune Sen, Kohei Inoue, Kiichi Urahama. : Bootstrap noise elimination by self-cross bilateral filter, Journal of the Institute of Image Information and Television Engineers Vol.33, No.51,2009.

Claims (13)

Guide image generation means for generating a guide image from an input image;
An image processing apparatus comprising: noise reduction means for generating a processed image by applying a reference-type edge preserving smoothing filter to the input image based on the guide image. The image processing apparatus according to claim 1, further comprising an overemphasized region restoring unit that restores an overemphasized region of the processed image from difference information between the input image and the processed image. The overemphasized area restoring means separates the overemphasized area of the processed image from the other from the difference information between the input image and the processed image, and adds the input image information of the overemphasized area to the processed image. The image processing apparatus according to claim 2, wherein the overemphasized region is restored. The image processing apparatus according to claim 1, wherein the reference type edge preserving smoothing filter is a guided filter or a joint bilateral filter. 5. The apparatus according to claim 1, further comprising means for estimating a magnitude of noise to be corrected and determining a filter parameter of the reference-type edge-preserving smoothing filter based on the magnitude. The image processing apparatus according to item. The image processing apparatus according to claim 1, wherein the input image is a multi-channel image in which information having different wavelengths is recorded. The image according to any one of claims 1 to 6, wherein the guide image generation unit generates a guide image by selecting a channel with less noise among channels constituting the input image. Processing equipment. The image processing apparatus according to claim 1, wherein the guide image generation unit generates a guide image by selecting a maximum value of a channel in each pixel of the input image. The image processing apparatus according to claim 1, wherein the input image includes an image of a person's skin. A guide image generation step for generating a guide image from the input image;
An image processing method, wherein a computer executes a noise reduction step of applying a reference-type edge preserving smoothing filter to the input image based on the guide image and generating a processed image. Computer
Guide image generation means for generating a guide image from an input image,
An image processing program for applying a reference-type edge preserving smoothing filter to the input image based on the guide image and causing it to function as noise reduction means for generating a processed image. An image generation device that generates and outputs an input image; and
An image processing apparatus that performs image processing on the input image and outputs a processed image;
An image output device for outputting the processed image,
The image processing apparatus includes:
Guide image generation means for generating a guide image from an input image;
An image processing system comprising: a noise reduction unit that applies a reference-type edge preserving smoothing filter to the input image based on the guide image to generate a processed image. A server device that provides the input image to the image processing device, receives and stores the processed image from the image processing device, and provides the processed image stored in response to a request from the image output device; The image processing system according to claim 12.