WO2023143129A1 - 图像处理方法、装置、电子设备及存储介质 - Google Patents

图像处理方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023143129A1
WO2023143129A1 PCT/CN2023/072098 CN2023072098W WO2023143129A1 WO 2023143129 A1 WO2023143129 A1 WO 2023143129A1 CN 2023072098 W CN2023072098 W CN 2023072098W WO 2023143129 A1 WO2023143129 A1 WO 2023143129A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sample
skin
target
training
Prior art date
Application number
PCT/CN2023/072098
Other languages
English (en)
French (fr)
Inventor
陈朗
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2023143129A1 publication Critical patent/WO2023143129A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map

Definitions

  • the present disclosure relates to the technical field of image processing, for example, to an image processing method, device, electronic equipment, and storage medium.
  • the present disclosure provides an image processing method, device, electronic equipment and storage medium, so as to realize the effect of simplifying the image processing flow, improving the adaptability of the image processing and the naturalness of the image processing results.
  • the present disclosure provides an image processing method, the method comprising:
  • the image to be processed includes a facial area, and the skin state of the facial area of the image to be processed is at a first skin age;
  • the target skin image processing model is based on the sample original image and the sample image
  • the sample effect image corresponding to the original image is obtained by training the initial skin image processing model, the skin state of the facial area in the target effect image is at a second skin age, and the second skin age is less than or equal to the first skin age.
  • an image processing device which includes:
  • An image acquisition module configured to acquire an image to be processed, wherein the image to be processed includes a facial area, and the skin state of the facial area of the image to be processed is at a first skin age;
  • a skin processing module configured to input the image to be processed into a pre-trained target skin image processing model to obtain a target effect image corresponding to the image to be processed, wherein the target skin image processing model is based on the original sample
  • the image and the sample effect image corresponding to the sample original image are obtained by training the initial skin image processing model, and the skin state of the facial area in the target effect image is at a second skin age, and the second skin age is less than or equal to the specified skin age. Describe the first skin age.
  • the present disclosure also provides an electronic device, which includes:
  • processors one or more processors
  • a storage device configured to store one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method provided in the present disclosure.
  • the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the image processing method provided in the present disclosure is implemented.
  • the present disclosure further provides a computer program product, including a computer program carried on a non-transitory computer readable medium, the computer program including program code for executing the image processing method provided in the present disclosure.
  • FIG. 1 is a schematic flow chart of an image processing method provided in Embodiment 1 of the present disclosure
  • FIG. 2 is a schematic flowchart of a training method for a target skin image processing model provided in Embodiment 2 of the present disclosure
  • FIG. 3 is a schematic flowchart of a training method for a target skin image processing model provided by Embodiment 3 of the present disclosure
  • FIG. 4 is a schematic diagram of an image processing and model training method provided by Embodiment 4 of the present disclosure.
  • FIG. 5 is a schematic structural diagram of an image processing device provided in Embodiment 5 of the present disclosure.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by Embodiment 6 of the present disclosure.
  • the term “comprise” and its variations are open-ended, ie “including but not limited to”.
  • the term “based on” is “based at least in part on”.
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one further embodiment”; the term “some embodiments” means “at least some embodiments.” Relevant definitions of other terms will be given in the description below.
  • FIG. 1 is a schematic flow chart of an image processing method provided by Embodiment 1 of the present disclosure. This embodiment is applicable to the situation of adjusting the skin state of the user's facial area.
  • the method can be executed by an image processing device, and the device It can be implemented by software and/or hardware, and can be configured in a terminal and/or server to implement the image processing method in the embodiments of the present disclosure.
  • the method of this embodiment may include:
  • the image to be processed includes a face area, and the skin state of the face area in the image to be processed is at the first skin age.
  • the image to be processed may be an image to be subjected to subsequent skin rejuvenation processing, and the image to be processed may be an image including a facial area captured by a camera, or an uploaded image including a facial area, etc.
  • the skin age can be the age of the skin, for example, it can be determined according to the values of wrinkles, pores, pigmentation, elasticity, smoothness and hydration of the skin, that is, the age determined according to the state of the facial skin.
  • Skin age and real age can be the same or different. Generally, the skin age corresponding to each real age is the average skin quality corresponding to the general population in the age group of the real age. but, There are also cases where the skin age does not match the actual age. True age is calculated from date of birth.
  • the first skin age may be information used to describe the skin state of the face area in the image to be processed.
  • an image including the face area may be captured, and the captured image may be used as an image to be processed. It is also possible to upload an image containing a face area based on the image upload function, and use the uploaded image as an image to be processed.
  • the image to be processed can be acquired in the following manner:
  • Step 1 When receiving a special effect trigger operation for enabling a preset skin processing special effect, display at least one image acquisition control.
  • the preset skin treatment effects can be special effects related to the adjustment of the skin state on the facial area, for example: special effects for reducing skin age, or various skin treatment effects, as long as the skin rejuvenation for the facial area is included in these special effects In the processing part, these special effects can be determined as preset skin processing special effects.
  • the special effect triggering operation may be clicking a control associated with the special effect triggering, or it may be a voice triggering, motion triggering, gesture triggering and other triggering operations.
  • the image acquisition control can be a control corresponding to different image acquisition methods, the image acquisition control can be a virtual sign set on the application software interface, and the image acquisition control can have various forms, such as buttons or icons.
  • a trigger operation for triggering preset skin processing special effects may be performed.
  • the special effect trigger operation for enabling the preset skin processing special effect is received, at least one image acquisition control may be displayed, so that the user can determine the image to be subsequently subjected to the preset skin processing special effect by triggering the image acquisition control.
  • Step 2 Receive a control trigger operation for at least one image acquisition control, and use an image acquisition method corresponding to the triggered image acquisition control to acquire an image to be processed.
  • an image acquisition mode corresponding to the control trigger operation is determined, and an image to be processed is acquired according to the determined image acquisition mode.
  • the triggered image acquisition control is "photograph and upload”
  • the shooting function can be started to acquire images to be processed.
  • the triggered image acquisition control is "local upload”
  • the photo album can be opened so that the user can select the pictures in the photo album to upload.
  • the target skin image processing model may be a model for performing skin image processing on the image to be processed,
  • the target skin image processing model is obtained by training the initial skin image processing model according to the sample original image and the sample effect image corresponding to the sample original image.
  • the skin state of the facial area in the target effect image is at the second skin age, and the second skin age is less than or Equal to first skin age.
  • the second skin age may be information describing a skin state of a face area in the target effect image.
  • the sample original image may be an unprocessed facial image
  • the sample effect image may be a facial image after skin image processing, for example: a facial image after skin lifting processing.
  • the initial skin image processing model can be a model whose model parameters are default parameters, and is used as a basic model for subsequent model training.
  • the target effect image is an image obtained after the image to be processed is processed by the target skin image processing model, that is, an image corresponding to the image to be processed after skin image processing.
  • the sample effect image can be the image processed by image processing software (such as: PhotoShop, etc.) on the sample original image, for example: the processing can be to remove dark circles, remove nasolabial folds, remove wrinkles, sag filling, etc. Skin rejuvenation deal with.
  • the sample original image and sample effect image may also be images in the image set before and after skin image processing, the image before skin lifting is determined as the sample original image, and the image after skin image processing is determined as the sample effect image.
  • the initial skin image processing model can be a neural network model, a generation confrontation network model, and the like.
  • target effect images can be displayed in a preset display area.
  • the target effect image itself can be displayed statically or dynamically, and can also be displayed by superimposing other display effects on the target effect image.
  • the preset display area may be the entire display area or a part of the entire display area.
  • the display method of the target effect image can be to display the image to be processed and the target effect image in different regions in the display area, so as to facilitate the viewing of the difference between the image to be processed and the target effect image, which is intuitive Understand the effect of processing the image to be processed.
  • the target effect image can be pushed to the user, so that the user can intuitively see the processed image of the skin image.
  • the target effect image can be displayed in the preset display area while the image to be processed is displayed.
  • the skin age corresponding to the target rendering image is less than or equal to the skin age corresponding to the image to be processed. Since the user's skin age may not match the real age, in this case, the skin age corresponding to the target effect image may still be greater than the real age, or not much different from the real age.
  • the real age of user A is 20 years old, and the skin of user A's image to be processed age is 40 years old, then the skin age of the target effect image is less than or equal to 40 years old, but may still be older than 20 years old, or less than or equal to 20 years old; the real age of user B is 60 years old, and the If the skin age is 40 years old, then the skin age of the target effect image is less than or equal to 40 years old, and may be much younger than 60 years old.
  • the skin age corresponding to the target effect image still needs to be determined according to the image processing conditions.
  • the technical solution of the embodiment of the present disclosure by acquiring the image to be processed, inputting the image to be processed into the pre-trained target skin image processing model, and obtaining the target effect image corresponding to the image to be processed, it solves the problem of homogeneity when the user's face is beautified.
  • the problem of severe blurring, serious loss of details, and low resolution can be solved, and the image processing process can be simplified, the adaptability of image processing can be improved, and the naturalness of image processing results can be improved.
  • Fig. 2 is a schematic flowchart of a training method for a target skin image processing model provided by Embodiment 2 of the present disclosure.
  • the training method can refer to the technical solution of this embodiment. Wherein, explanations of terms that are the same as or corresponding to those in the foregoing embodiments will not be repeated here.
  • the method of this embodiment may include:
  • the sample original image may be an unprocessed facial image
  • the sample rendering image may be a facial image after skin image processing.
  • the skin state of the face area of the sample original image is at the first skin age
  • the skin state of the face area of the sample effect image is at the second skin age
  • the second skin age is less than or equal to the first skin age.
  • sample original images including facial regions may be obtained by shooting, downloading or uploading. Furthermore, each sample original image can be processed step by step, mainly for skin rejuvenation.
  • the processed image is determined as a sample effect image corresponding to the sample original image, so that the sample effect image is younger than the sample original image.
  • the sample effect image corresponding to the sample original image can be determined in the following manner:
  • a preliminary effect image obtained by performing skin age conversion processing on the facial area in the sample original image is acquired, and the sample effect image is determined according to the preliminary effect image.
  • Skin age transformation treatment includes wrinkle lightening treatment, dark circle lightening treatment and sunken filling treatment. one less.
  • the skin age conversion processing corresponds to the skin image processing performed by the target skin image processing model obtained through subsequent training.
  • the preliminary effect image may be an image after skin age conversion processing on the sample original image, that is, an image after skin rejuvenation processing.
  • Skin age conversion processing is performed on the facial area in each sample original image, so that the facial area in the sample original image is younger and the skin age is smaller.
  • the image processed by skin age conversion is determined as the preliminary effect image.
  • the preliminary effect image can be used as a sample effect image, or adjustment processing can be performed on the preliminary effect image to obtain a sample effect image, wherein the adjustment processing can be brightness adjustment, saturation adjustment, color adjustment, sharpness adjustment, etc.
  • the sample effect image can be determined according to the preliminary effect image in the following manner:
  • the skin color correction processing may be a processing method of adjusting the color of the skin area in the preliminary effect image so that the color of the skin area is close to the sample original image.
  • the skin color of the face area in the preliminary effect image is adjusted so that the skin color of the face area in the preliminary effect image is closer to the reference, so as to improve the authenticity of image processing.
  • the skin color correction process can be performed on the face area in the preliminary effect image according to the sample original image through the following steps to obtain the sample effect image:
  • Step 1 Calculate the average value of the first skin color of the face area in the sample original image and the average value of the second skin color of the face area in the preliminary effect image.
  • the first skin color average value may be an average value of color values of multiple pixel points corresponding to the face area in the sample original image.
  • the second mean value of skin color may be the mean value of color values of multiple pixel points corresponding to the face area in the preliminary effect image.
  • an average value of multiple color values is calculated to obtain a first skin color average value.
  • the mean value of the plurality of color values is calculated to obtain the second skin color mean value.
  • the mean value of the first skin color of the face region in the sample original image can be calculated as follows:
  • the skin area may be the remaining area of the facial area excluding non-skin areas such as the eyebrow area, the eye area, and the lip area.
  • the face area is identified in the sample original image, and the skin area is determined in the face area, and then the color values of multiple pixels in the skin area are superimposed and averaged to obtain a first skin color average value.
  • the method for determining the mean value of the second skin color is similar to the above method, and will not be repeated here.
  • Step 2 According to the first skin color average value, the second skin color average value, and the original color values corresponding to the multiple pixel points of the facial area in the preliminary effect image, respectively determine the targets corresponding to the multiple pixels of the facial area in the preliminary effect image Color value, generate a sample effect image based on the target color value.
  • the original color value may be a color value corresponding to each pixel of the face area in the preliminary effect image.
  • the target color value may be the color value to which each pixel of the face area in the preliminary effect image is to be adjusted.
  • the color difference between the sample original image and the preliminary effect image can be determined. Furthermore, the original color value corresponding to each pixel of the face area in the preliminary effect image is processed according to the color difference between the sample original image and the preliminary effect image, and a target color value corresponding to each pixel is determined. Generate a sample effect image according to the target color value corresponding to each pixel, so as to adjust the preliminary effect image to a sample effect image in which the skin color matches the sample original image.
  • the original color value corresponding to each pixel of the face area in the preliminary effect image may be subtracted from the second skin color mean value to obtain a de-averaged color value corresponding to each pixel. Furthermore, adding the de-averaged color value corresponding to each pixel point to the first skin color mean value to obtain the target color value corresponding to each pixel point.
  • the average value of the skin color can be adjusted to make the image more realistic.
  • S220 Construct a training sample set based on multiple sample original images and sample effect images corresponding to the sample original images, and train the generative adversarial network according to the sample original images in the training sample set and the sample effect images corresponding to the sample original images, and train the The completed skin image processing generator is used as the target skin image processing model.
  • the initial skin image processing model includes a generative adversarial network, which includes a skin image processing generator and a skin image processing discriminator.
  • the skin image processing generator can be a fully connected neural network, deconvolution network, etc.
  • the skin image processing discriminator can be any discriminator model, such as a fully connected network, a network containing convolution, etc.
  • the training sample set may be an image set composed of a plurality of sample original images and a sample effect image corresponding to each sample original image, and is used for subsequent training to obtain a target skin image processing model.
  • a training sample set is obtained by combining multiple sample original images and sample effect images corresponding to the sample original images.
  • the generation confrontation network is trained, which can be to train the skin image processing generator and the skin image respectively. Skin image processing discriminator, and then, the trained skin image processing generator is used as the target skin image processing model.
  • the generation confrontation network is trained in the following manner:
  • the target iterative training times of the sample original image according to the sample original image, the sample effect image corresponding to the sample original image and
  • the target iteration training number is used to train the generative adversarial network.
  • the skin age may be the age corresponding to the skin state of the face area of the person corresponding to the face area in the sample original image, not the real age. Exemplarily, if it is difficult to determine the value of the skin age based on image processing and analysis, and only a certain value range can be determined, then this value range can be used as the skin age.
  • the target iterative training times may be the times of training the generative confrontation network according to the sample original image.
  • a target iterative training number corresponding to each sample original image corresponding to each skin age may be determined. Furthermore, the original image of each sample is input into the generative adversarial network for training according to the target iterative training times corresponding to the original image of the sample.
  • the number of sample original images corresponding to skin age A is 300, and the number of sample original images corresponding to skin age B is 600. Then, the target iterative training number of each sample original image corresponding to skin age A can be determined as 2, and the target iterative training number of each sample original image corresponding to skin age B is still 1; it is also possible Determine the target iterative training times of random 150 samples of original images corresponding to skin age A as 3, and determine the target iterative training times of the remaining 150 sample original images as 1, and keep each The target iterative training times of the sample original images is still 1; the target iterative training times of one of the sample original images corresponding to the skin age A can also be determined as 301, and the target iterative training times of the remaining 299 sample original images It is determined as 1, and the target iteration training times of each sample original image corresponding to skin age B is still 1.
  • grouping processing can be performed according to skin age, so as to improve the efficiency of sample equalization.
  • multiple sample original images in the training sample set are grouped to obtain at least two sample training groups of age groups, and according to the image of the sample original image corresponding to each sample training group Quantity determines the target number of training iterations for each training set of samples.
  • the target number of iterative training for the sample training group with a small number of images is not lower than the target iterative training number for the sample training group with a large number of images.
  • the skin age corresponding to each sample original image in the training sample set is determined, and the plurality of sample original images are grouped according to a preset grouping requirement to be divided into at least two sample training groups corresponding to age groups. Furthermore, according to the image quantity of the sample original image corresponding to each sample training group, increase the target iterative training times of the sample training group with a relatively small number of images, so as to balance the multiple sample training groups.
  • the sample training group corresponding to the first age group is 400
  • the number of sample original images corresponding to the second sample training group is 200
  • the number of sample original images corresponding to the third sample training group is 100. Accordingly, it can be determined that the target number of iterative training for the second sample training group is 2, and the target number of iterative training for the third sample training group is 4.
  • the number of images in the sample training group with a large number of images is not an integral multiple of the number of images in the sample training group with a small number of images
  • the corresponding targets of different sample original images in the sample training group with a small number of images can be respectively determined
  • the number of iterative training, that is, the target number of iterative training corresponding to different sample original images in the same sample training group may be the same or different.
  • the sample original image including the face area by acquiring the sample original image including the face area, determining the sample effect image corresponding to the sample original image, constructing a training sample set based on multiple sample original images and sample effect images corresponding to the sample original image, according to
  • the sample original image in the training sample set and the sample effect image corresponding to the sample original image are used to train the generative confrontation network, and the trained skin image processing generator is used as the target skin image processing model to solve the problem of beautifying the user's face through the model.
  • the beautification effect is poor and the naturalness of the beautified image is poor, and the effect of improving the beautification effect and the naturalness of the beautified image through the generative confrontation network is realized.
  • Fig. 3 is a schematic flowchart of a training method for a target skin image processing model provided by Embodiment 3 of the present disclosure. Augmentation, training of skin image processing generator and skin image processing discriminator For the process, refer to the technical solution of this embodiment. Wherein, explanations of terms that are the same as or corresponding to those in the foregoing embodiments will not be repeated here.
  • the method of this embodiment may include:
  • the target original image can be all or part of the sample original image, which is used for subsequent expansion of the training sample set.
  • a target original image is selected from multiple sample original images, and a sample effect image corresponding to each target original image is determined for subsequent processing and matching.
  • the preset lighting condition may be a lighting condition for changing the lighting condition of the target original image, and may include lighting location, lighting intensity, lighting color, light source form, and the like.
  • the lighting simulation processing may be a processing manner of superimposing light under preset lighting conditions on the target original image.
  • the sample augmented image may be an image processed by illumination simulation, and is used to perform sample augmentation on the sample original image.
  • One or more preset lighting conditions can be determined, and according to one or more preset lighting conditions, lighting simulation processing is performed on the target original image, and the processed image is used as a sample expanded image.
  • lighting simulation processing is performed on the sample effect image corresponding to the target original image according to one or more preset lighting conditions, and the processed image is used as a sample effect image corresponding to each sample expanded image.
  • the reason why the target original image and the sample effect image corresponding to the target original image are subjected to the same lighting simulation process is to ensure that the sample augmented image and the sample effect image corresponding to the sample augmented image are only subjected to skin image processing to avoid the influence of lighting conditions .
  • the sample input image is the image before the skin image processing in the training sample set
  • the expected effect image is the image obtained after the skin image processing in the training sample set.
  • the sample original image and the sample effect image corresponding to the sample original image are the original part of the training sample set, and the sample expansion image and the sample effect image corresponding to the sample expansion image are the training sample set extension of the .
  • the sample original image and the sample expanded image are used as the sample input image, that is, the image before skin image processing, and the sample effect image corresponding to the sample original image and the sample effect image corresponding to the sample expanded image are used as the desired effect image, that is, the skin image processing after the image.
  • a training sample set is constructed corresponding to the sample input image and the desired effect image.
  • the sample input images in the training sample set are input into the skin image processing generator in the generative confrontation network, and the sample generated images corresponding to each sample input image can be obtained through the processing of the skin image processing generator.
  • the difference between the sample generated image and the sample input image the difference between the images before and after the skin image processing generator can be judged, and according to the difference between the sample generated image and the expected effect image corresponding to the sample input image, the skin can be judged
  • the difference between the image processed by the image processing generator and the expected image the network parameters of the skin image processing generator are adjusted through the two determined differences, so that the sample generated image output by the subsequent skin image processing generator has the effect of skin processing relative to the sample input image, and compared to the expected effect image There are indistinguishable effects.
  • the network parameters of the skin image processing generator can be adjusted in the following manner:
  • Step 1 Calculate a first loss value between the sample generated image and the sample input image according to a preset first loss function.
  • the preset first loss function may be a loss function used to measure the difference between the sample generated image and the sample input image.
  • the first loss value may be an output value calculated by a preset first loss function, which represents a difference between the sample generated image and the sample input image.
  • the unprocessed part is as close as possible to the sample input image. Therefore, the eye area, nasolabial folds area, cheek area and other areas of interest that are more obviously processed can be excluded, and the first loss function is constructed based on the remaining areas except the area of interest, so that the difference in the remaining areas is not obvious.
  • Step 2 Calculate a second loss value between the sample generated image and the desired effect image corresponding to the sample input image according to a preset second loss function.
  • the preset second loss function may be a loss function used to measure the difference between the sample generated image and the expected effect image corresponding to the sample input image.
  • the second loss value can be the output value calculated by the preset second loss function, which represents the difference between the sample generated image and the expected effect image corresponding to the sample input image different.
  • the second loss function can be constructed according to the eye area, nasolabial folds area, cheek area, etc., which are relatively obvious attention areas, so as to reduce the difference between the sample generated image and the expected effect image corresponding to the sample input image.
  • Step 3 Adjust the network parameters of the skin image processing generator according to the first loss value and the second loss value.
  • the network parameters of the skin image processing generator can be adjusted according to the first loss value and the second loss value, so that the adjusted first loss value and the second loss value The combined loss value of the second loss value is decreased.
  • the comprehensive loss value may be calculated and processed according to the first loss value and the second loss value, and the calculation processing method may be sum calculation, weighted sum calculation, and the like.
  • S370 Train the skin image processing discriminator according to the sample generated image and the desired effect image corresponding to the sample generated image, and determine whether to end the skin image processing and generation based on the discrimination result of the sample generated image by the trained skin image processing discriminator device adjustment.
  • the skin image processing discriminator is trained according to the sample generated image and the expected effect image corresponding to the sample generated image, so that the skin image processing discriminator can effectively distinguish the sample generated image from the expected effect image corresponding to the sample generated image . Based on the discriminative result of the sample generated image by the trained skin image processing discriminator, it can be determined whether the sample generated image and the corresponding expected effect image corresponding to the sample generated image are distinguishable, and if so, it indicates that the sample generated image has poor effect , the parameters of the skin image processing generator need to be readjusted, and the adjustment of the skin image processing generator cannot be completed; if it is difficult to distinguish, it indicates that the image generated by the sample has a better effect and is close to the desired effect image, and the generation of the skin image processing can be terminated device adjustment.
  • the adjusted skin image processing generator can be used as the target skin image processing model.
  • the original image of the target is subjected to lighting simulation processing to obtain a sample expansion image, and the target original image is compared with the target original image according to the preset lighting conditions.
  • the corresponding sample effect image is subjected to illumination simulation processing to obtain a sample effect image corresponding to the sample expansion image, and the sample original image and the sample expansion image are used as sample input images, and the sample effect image corresponding to the sample original image and the sample expansion image
  • the corresponding sample effect image is used as the expected effect image
  • the training sample set is constructed according to the sample input image and the expected effect image, so as to reliably expand the training sample set, increase the number of model training samples, and effectively improve the subsequent model training effect.
  • the sample input image in the training sample set is input into the skin image processing generator in the generative adversarial network to obtain the sample generated image, and the skin image is paired according to the sample generated image, the sample input image and the desired effect image corresponding to the sample input image
  • the network parameters of the processing generator are adjusted so that the training process of the skin image processing generator achieves the purpose of balancing the beautification effect and the natural effect.
  • the skin image processing discriminator according to the sample generated image and the desired effect image corresponding to the sample generated image, and determine whether to end the skin image processing generator based on the discrimination result of the sample generated image by the trained skin image processing discriminator Adjustment, if yes, use the adjusted skin image processing generator as the target skin image processing model, which solves the problem of poor model training effect caused by the small number of model training samples, and it is difficult to balance beautification effect and natural effect at the same time
  • the training sample set is expanded, and the beautification effect and natural effect are considered comprehensively to improve the effect of user experience.
  • FIG. 4 is a schematic diagram of an image processing and model training method provided by Embodiment 4 of the present disclosure. As shown in FIG. 4 , the method of this embodiment may include: a model training part and an image processing part.
  • the model training part mainly includes:
  • a certain amount of high-definition facial data can be obtained by manual collection, for example: 500-2000 high-definition facial images, the number can be selected according to actual needs.
  • the resolution of the face image can be required to be no less than 1024*1024.
  • the distribution of face data should cover men and women as much as possible, and multiple skin age groups from 20 to 80 years old, covering a variety of faces angle etc.
  • high-definition facial images can be corrected for facial features.
  • the corrected content is mainly related to face lifting and firming, such as: removing dark circles, removing wrinkles, reducing nasolabial folds, filling facial depressions, etc.
  • a corrected face image (sample effect image) corresponding to each high-definition face image can be obtained.
  • the high-definition face image can be recorded as A, and the corrected face corresponding to each high-definition face image
  • the first image is denoted as B.
  • the lifting and compacting GAN model In the training process of the lifting and compacting GAN model, lighting condition simulation (lighting simulation processing), age distribution adaptation (according to the skin age corresponding to the sample original image in the training sample set and the sample original image corresponding to each skin age)
  • the number of images of the sample determines the target iterative training times of the sample original image, according to the sample original image, the sample effect image corresponding to the sample original image and the target iterative training times to train the generative confrontation network), skin color correction (skin color correction processing), Face high-dimensional semantic feature correction (according to the preset first loss function to calculate the first loss value between the sample generated image and the sample input image), face low-dimensional texture feature correction (according to the preset second loss function to calculate the sample generation Image algorithm strategies such as the second loss value between the image and the expected effect image corresponding to the sample input image) ensure that the lifting and compacting GAN network can not only retain the original skin quality information during the training process, but also only process it in a targeted manner Defect areas such as wrinkle
  • the image processing part mainly includes:
  • a user image (image to be processed) determined by the user through shooting or uploading is received.
  • the facial key point information is used to crop the face.
  • the method of face cropping is to identify and crop the face area in the user image uploaded by the user to obtain the cropped face image.
  • the face image can be adjusted according to the model image, and the range of the face image can be stretched or compressed to the same range as the face image in the model image, so as to improve the recognition of the subsequent skin lifting and firming treatment area, and improve image processing effects.
  • the technical solution of this embodiment through facial data collection, facial feature correction and supervised training Lift and tighten the GAN model to obtain the face beautification generator.
  • the technical solution of this embodiment through facial data collection, facial feature correction and supervised training Lift and tighten the GAN model to obtain the face beautification generator.
  • by receiving the user's image performing face cropping, processing based on the face beautification generator, and outputting a lifting and tightening face effect map, it solves the problem of serious homogeneity, serious loss of details, and low resolution when the user's face is beautified. Problems, to achieve the effects of simplifying the image processing process, improving the adaptability of image processing and the naturalness of image processing results.
  • Fig. 5 is a schematic structural diagram of an image processing device provided in Embodiment 5 of the present disclosure.
  • the image processing device provided in this embodiment can be implemented by software and/or hardware, and can be configured in a terminal and/or server to realize the present invention.
  • the image processing method in the embodiment is disclosed. As shown in Figure 5, the device may include:
  • the image acquisition module 510 is configured to acquire an image to be processed, wherein the image to be processed includes a facial region, and the skin state of the facial region of the image to be processed is at a first skin age; the skin processing module 520 is configured to convert the The image to be processed is input into the pre-trained target skin image processing model to obtain the target effect image corresponding to the image to be processed, wherein the target skin image processing model is based on the original image of the sample and the image corresponding to the original image of the sample
  • the sample effect image is obtained by training the initial skin image processing model, the skin state of the face area in the target effect image is at a second skin age, and the second skin age is less than or equal to the first skin age.
  • the initial skin image processing model includes a generation confrontation network
  • the generation confrontation network includes a skin image processing generator and a skin image processing discriminator
  • the device further includes:
  • the model training module is configured to acquire a sample original image including a face area, and determine a sample effect image corresponding to the sample original image, wherein the skin state of the face area of the sample original image is at a first skin age, and the sample The skin state of the face area of the effect image is at a second skin age, and the second skin age is less than or equal to the first skin age;
  • the training is constructed based on multiple sample original images and sample effect images corresponding to the sample original images
  • a sample set, the generative confrontation network is trained according to the sample original image in the training sample set and the sample effect image corresponding to the sample original image, and the trained skin image processing generator is used as the target skin image processing model.
  • the model training module is further configured to obtain a preliminary effect image obtained by performing skin age conversion processing on the facial area in the sample original image, and according to the preliminary effect The image determines the sample effect image, wherein the skin age conversion processing includes at least one of wrinkle lightening processing, dark circle lightening processing and sunken filling processing.
  • the model training module is further configured to perform skin color correction processing on the face area in the preliminary effect image according to the sample original image to obtain a sample effect image.
  • the model training module is further configured to calculate the first skin color mean value of the face area in the sample original image, and the first skin color mean value of the face area in the preliminary effect image
  • the second skin color mean value according to the first skin color mean value, the second skin color mean value, and the original color values corresponding to a plurality of pixels in the face area in the preliminary effect image, respectively determine the preliminary effect image Target color values corresponding to multiple pixel points of the face area, and generate a sample effect image according to the target color values.
  • the model training module is further configured to determine the face area in the sample original image, determine the skin area in the face area, and calculate the skin area The average value of the first skin color of multiple pixels in .
  • the model training module is also configured to acquire a target original image in multiple sample original images and a sample effect image corresponding to the target original image; according to preset Performing illumination simulation processing on the target original image under illumination conditions to obtain a sample expanded image, and performing illumination simulation processing on a sample effect image corresponding to the target original image according to the preset illumination conditions to obtain an expanded sample image Corresponding sample effect image; using the sample original image and the sample expanded image as a sample input image, using the sample effect image corresponding to the sample original image and the sample effect image corresponding to the sample expanded image as the desired effect image, constructing a training sample set according to the sample input image and the desired effect image.
  • the model training module is further configured to input the sample input image in the training sample set into the skin image processing generator in the generative confrontation network to obtain Generate an image from a sample; adjust the network parameters of the skin image processing generator according to the image generated by the sample, the input image of the sample, and the desired effect image corresponding to the input image of the sample; generate an image according to the sample and
  • the skin image processing discriminator is trained on the expected effect image corresponding to the sample generated image, and it is determined whether to end the processing and generation of the skin image based on the discriminant result of the sample generated image obtained by the trained skin image processing discriminator. Adjustment of the generator; if it is determined to end the adjustment of the skin image processing generator, the adjusted skin image processing generator is used as the target skin image processing model.
  • the model training module is further configured to calculate a first loss value between the sample generated image and the sample input image according to a preset first loss function; Calculate the second loss value between the sample generated image and the expected effect image corresponding to the sample input image according to the preset second loss function; calculate the skin according to the first loss value and the second loss value
  • the network parameters of the image processing generator are tuned.
  • the model training module is also set to be based on the skin age corresponding to the sample original image in the training sample set and the image number of the sample original image corresponding to each skin age Determine the target iterative training times of the original image of the sample, according to The original image of the sample, the effect image of the sample corresponding to the original image of the sample, and the target iterative training times train the generative confrontation network.
  • the model training module is further configured to conduct multiple sample original images in the training sample set according to the skin age corresponding to each sample original image in the training sample set Group processing to obtain sample training groups of at least two age groups, and determine the target iterative training times corresponding to each sample training group according to the image quantity of the sample original image corresponding to each sample training group; wherein, the sample training with a small number of images
  • the target iterative training number of the group is not lower than the target iterative training number of the sample training group with a large number of images.
  • the image acquisition module 510 is further configured to display at least one image acquisition control when receiving a special effect trigger operation for enabling a preset skin processing special effect;
  • the control trigger operation of at least one image acquisition control uses the image acquisition method corresponding to the triggered image acquisition control to acquire the image to be processed.
  • the above-mentioned device can execute the method provided by any embodiment of the present disclosure, and has corresponding functional modules and effects for executing the method.
  • the technical solution of the embodiment of the present disclosure by acquiring the image to be processed, inputting the image to be processed into the pre-trained target skin image processing model, and obtaining the target effect image corresponding to the image to be processed, it solves the problem of homogeneity when the user's face is beautified.
  • the problem of severe blurring, serious loss of details, and low resolution can be solved, and the image processing process can be simplified, the adaptability of image processing can be improved, and the naturalness of image processing results can be improved.
  • the multiple units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the names of multiple functional units are only for the convenience of distinguishing each other , and are not intended to limit the protection scope of the embodiments of the present disclosure.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by Embodiment 6 of the present disclosure.
  • the terminal equipment in the embodiments of the present disclosure may include but not limited to mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA), tablet computers (Portable Android Device, PAD), portable multimedia players (Portable Media Player, PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals) and other mobile terminals, and fixed terminals such as digital televisions (Television, TV), desktop computers and so on.
  • the electronic device 600 shown in FIG. 6 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.
  • an electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) Various appropriate actions and processes are performed by a program loaded into a random access memory (Random Access Memory, RAM) 603 by 608. In the RAM 603, various programs and data necessary for the operation of the electronic device 600 are also stored.
  • the processing device 601, ROM 602, and RAM 603 are connected to each other through a bus 605.
  • An edit/output (Input/Output, I/O) interface 604 is also connected to the bus 605 .
  • an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; including, for example, a liquid crystal display (Liquid Crystal Display, LCD) , an output device 607 such as a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609.
  • the communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data.
  • FIG. 6 shows electronic device 600 having various means, it is not a requirement to implement or possess all of the means shown. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602.
  • the processing device 601 When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.
  • the electronic device provided by the embodiment of the present disclosure belongs to the same concept as the image processing method provided by the above embodiment, and the technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same effect as the above embodiment .
  • An embodiment of the present disclosure provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the image processing method provided in the foregoing embodiments is implemented.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or Any combination of the above.
  • Examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), or flash memory), optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device .
  • the program code contained on the computer readable medium may be transmitted by any suitable medium, including but not limited to: electric wire, optical cable, radio frequency (Radio Frequency, RF), etc., or any suitable combination of the above.
  • the client and the server can communicate using any currently known or future network protocols such as Hypertext Transfer Protocol (HyperText Transfer Protocol, HTTP), and can communicate with digital data in any form or medium
  • the communication eg, communication network
  • Examples of communication networks include local area networks (Local Area Network, LAN), wide area networks (Wide Area Network, WAN), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently existing networks that are known or developed in the future.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device:
  • Acquiring an image to be processed wherein the image to be processed includes a facial area, and the skin state of the facial area of the image to be processed is at the first skin age; inputting the image to be processed to a pre-trained target skin image processing model , obtain the target effect image corresponding to the image to be processed, wherein the target skin image processing model is obtained by training the initial skin image processing model according to the sample original image and the sample effect image corresponding to the sample original image, the The skin state of the facial area in the target effect image is at a second skin age, and the second skin age is less than or equal to the first skin age.
  • Computer program code for carrying out the operations of the present disclosure may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional procedural programming languages - such as the "C" language or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user computer through any kind of network, including a LAN or WAN, or it can be connected to an external computer (eg via the Internet using an Internet Service Provider).
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the unit does not constitute a limitation of the unit itself in one case.
  • exemplary types of hardware logic components include: Field Programmable Gate Arrays (Field Programmable Gate Arrays, FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (Application Specific Standard Parts, ASSP), System on Chip (System on Chip, SOC), Complex Programmable Logic Device (Complex Programming Logic Device, CPLD) and so on.
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard drives, RAM, ROM, EPROM or flash memory, optical fibers, CD-ROMs, optical storage devices, magnetic storage devices, or Any suitable combination of the above.
  • Example 1 provides an image processing method, the method including:
  • the image to be processed includes a facial area, and the skin state of the facial area of the image to be processed is at a first skin age;
  • the target skin image processing model is based on the sample original image and the sample image
  • the sample effect image corresponding to the original image is obtained by training the initial skin image processing model, the skin state of the facial area in the target effect image is at a second skin age, and the second skin age is less than or equal to the first skin age.
  • Example 2 provides an image processing method, and the method further includes:
  • the initial skin image processing model includes a generation confrontation network, the generation confrontation network includes a skin image processing generator and a skin image processing discriminator, and the target skin image processing model is trained based on the following method:
  • Example 3 provides an image processing method, and the method further includes:
  • Example 4 provides an image processing method, the method further includes:
  • Example 5 provides an image processing method, and the method further includes:
  • Said performing skin color correction processing on the facial area in said preliminary effect image according to said sample original image to obtain a sample effect image including:
  • the first skin color average value, the second skin color average value, and the original color values corresponding to the multiple pixel points of the facial area in the preliminary effect image respectively determine multiple values of the facial area in the preliminary effect image.
  • a target color value corresponding to the pixel point, and a sample effect image is generated according to the target color value.
  • Example 6 provides an image processing method, and the method further includes:
  • the calculation of the first skin color mean value of the face area in the sample original image includes:
  • Example 7 provides an image processing method, and the method further includes:
  • sample original image and the sample expanded image as a sample input image
  • sample effect image corresponding to the sample original image and the sample effect image corresponding to the sample expanded image as a desired effect image, according to the sample
  • the input image and the desired effect image construct a training sample set.
  • Example 8 provides an image processing method, and the method further includes:
  • the generating confrontation network is trained according to the sample original image in the training sample set and the sample effect image corresponding to the sample original image, and the trained skin image is processed to generate
  • the synthesizer is used as the target skin image processing model, including:
  • the skin image processing discriminator is trained according to the sample generated image and the desired effect image corresponding to the sample generated image, and based on the skin image processing discriminator obtained through training, it is determined whether to End the adjustment to the skin image processing generator;
  • the adjusted skin image processing generator is used as a target skin image processing model.
  • Example 9 provides an image processing method, and the method further includes:
  • the adjusting the network parameters of the skin image processing generator according to the sample generated image, the sample input image and the desired effect image corresponding to the sample input image includes:
  • the network parameters of the skin image processing generator are adjusted according to the first loss value and the second loss value.
  • Example 10 provides an image processing method, and the method further includes:
  • the target iterative training times of the sample original image is determined, according to the sample original image, and the sample The sample effect image corresponding to the original image and the target iteration training times are used to train the generation confrontation network.
  • Example Eleven provides an image processing method, and the method further includes:
  • the target iterative training times of the sample original image is determined. number, including:
  • a plurality of sample original images in the training sample set are grouped to obtain at least two sample training groups of age groups, and according to the samples corresponding to each sample training group The image number of the original image determines the target iteration training times corresponding to each sample training group;
  • the target iterative training times of the sample training group with a relatively small number of images is not lower than the target iterative training times of the sample training group with a relatively large number of images.
  • Example 12 provides an image processing method, and the method further includes:
  • the acquisition of images to be processed includes:
  • a control trigger operation for at least one image acquisition control is received, and an image to be processed is acquired by using an image acquisition method corresponding to the triggered image acquisition control.
  • Example 13 provides an image processing device, which includes:
  • An image acquisition module configured to acquire an image to be processed, wherein the image to be processed includes a facial area, and the skin state of the facial area of the image to be processed is at a first skin age;
  • a skin processing module configured to input the image to be processed into a pre-trained target skin image processing model to obtain a target effect image corresponding to the image to be processed, wherein the target skin image processing model is based on the original sample
  • the image and the sample effect image corresponding to the sample original image are obtained by training the initial skin image processing model, and the skin state of the facial area in the target effect image is at a second skin age, and the second skin age is less than or equal to the specified skin age. Describe the first skin age.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本公开提供了一种图像处理方法、装置、电子设备及存储介质。该图像处理方法包括：获取待处理图像，其中，待处理图像包括面部区域，待处理图像的面部区域的皮肤状态处于第一皮肤年龄；将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。

Description

图像处理方法、装置、电子设备及存储介质
本申请要求在2022年01月30日提交中国专利局、申请号为202210114150.X的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，例如涉及一种图像处理方法、装置、电子设备及存储介质。
背景技术
基于人像美化相关技术人为改动皮肤的状态，可以达成皮肤更加紧致、饱满的目的。
可以通过对用户的面部多个区域进行逐一处理，以达到改变皮肤状态的效果，但是，该方式操作复杂、交互繁琐，存在用户体验度较差的问题。还可以通过类似于“一键美颜”的方式对用户的皮肤状态进行调整，但是，由于该方式的参数固定，会导致处理结果同质化严重问题，并且，细节信息损失较多，会造成图像质量较差的问题，影响用户的体验。
发明内容
本公开提供了一种图像处理方法、装置、电子设备及存储介质，以实现简化图像处理流程，提高图像处理的自适应性以及图像处理结果的自然度的效果。
第一方面，本公开提供了一种图像处理方法，该方法包括：
获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；
将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
第二方面，本公开还提供了一种图像处理装置，该装置包括：
图像获取模块，设置为获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；
皮肤处理模块，设置为将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
第三方面，本公开还提供了一种电子设备，该电子设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本公开所提供的图像处理方法。
第四方面，本公开还提供了一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现本公开所提供的图像处理方法。
第五方面，本公开还提供了一种计算机程序产品，包括承载在非暂态计算机可读介质上的计算机程序，所述计算机程序包含用于执行本公开所提供的图像处理方法的程序代码。
附图说明
图1为本公开实施例一所提供的一种图像处理方法的流程示意图；
图2为本公开实施例二所提供的一种目标皮肤图像处理模型的训练方法的流程示意图；
图3为本公开实施例三所提供的一种目标皮肤图像处理模型的训练方法的流程示意图；
图4为本公开实施例四所提供的一种图像处理和模型训练方法的示意图；
图5为本公开实施例五所提供的一种图像处理装置的结构示意图；
图6为本公开实施例六所提供的一种电子设备的结构示意图。
具体实施方式
下面将参照附图描述本公开的实施例。虽然附图中显示了本公开的一些实施例，然而本公开可以通过多种形式来实现，提供这些实施例是为了理解本公开。本公开的附图及实施例仅用于示例性作用。
本公开的方法实施方式中记载的多个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。 本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有指出，否则应该理解为“一个或多个”。
本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具备”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
实施例一
图1为本公开实施例一所提供的一种图像处理方法的流程示意图，本实施例可适用于对用户的面部区域进行皮肤状态调整的情况，该方法可以由图像处理装置来执行，该装置可以通过软件和/或硬件来实现，可配置于终端和/或服务器中来实现本公开实施例中的图像处理方法。
如图1所示，本实施例的方法可包括：
S110、获取待处理图像。
待处理图像包括面部区域，待处理图像的面部区域的皮肤状态处于第一皮肤年龄。待处理图像可以是待进行后续皮肤年轻化处理的图像，待处理图像可以是基于拍摄装置拍摄的包含面部区域的图像，也可以是上传的包含面部区域的图像等。
皮肤年龄可以是肌肤的年龄，例如可以根据皮肤的皱纹、毛孔、色斑、弹性、光滑度以及水合度等多个方面的数值进行确定，即，根据面部皮肤状态确定出的年龄。皮肤年龄和真实年龄可以相同也可以不同。一般地，每个真实年龄对应的皮肤年龄为真实年龄所处的年龄段中普遍人群对应的平均肤质。但， 也存在皮肤年龄与实际年龄不符的情况。真实年龄为根据出生日期计算出的年龄。
在本公开实施例中，第一皮肤年龄可以是用于描述待处理图像中面部区域的皮肤状态的信息。
可以基于设备的拍摄功能，拍摄包含面部区域的图像，将拍摄得到的图像作为待处理图像。还可以基于图像上传功能，上传包含面部区域的图像，并将上传的图像作为待处理图像。
在本公开实施例技术方案的基础上，可以通过下述方式获取待处理图像：
步骤一、当接收用于启用预设皮肤处理特效的特效触发操作时，展示至少一种图像获取控件。
预设皮肤处理特效可以是对面部区域进行与皮肤状态调整相关的特效，例如：降低肤龄的特效，也可以是多种皮肤处理的特效，只要在这些特效中包含对面部区域进行皮肤年轻化处理的部分就可以将这些特效确定为预设皮肤处理特效。特效触发操作可以是点击与特效触发相关联的控件，也可以是语音触发、动作触发、手势触发等触发操作。图像获取控件可以是与不同图像获取方式相对应的控件，图像获取控件可以是设置于应用软件界面上的虚拟标识，图像获取控件的表现形式可以有多种，例如：按钮或图标等。
当用户想要进行特效处理的时候，可以执行触发预设皮肤处理特效的触发操作。当接收用于启用预设皮肤处理特效的特效触发操作时，可以展示至少一种图像获取控件，以使用户可以通过触发图像获取控件确定后续进行预设皮肤处理特效的图像。
步骤二、接收针对至少一种图像获取控件的控件触发操作，采用所触发的图像获取控件对应的图像获取方式获取待处理图像。
在接收针对至少一种图像获取控件的控件触发操作时，确定控件触发操作所对应的图像获取方式，并根据确定出的图像获取方式获取待处理图像。
示例性的，若触发的图像获取控件为“拍照上传”，则可以确定图像获取方式为通过拍摄功能拍摄获取图像，因此，可以启动拍摄功能以获取待处理图像。若触发的图像获取控件为“本地上传”，则可以确定图像获取方式为将本地图片上传，因此，可以打开相册，以使用户选择相册中的图片上传。
S120、将待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与待处理图像对应的目标效果图像。
目标皮肤图像处理模型可以是用于对待处理图像进行皮肤图像处理的模型， 目标皮肤图像处理模型根据样本原始图像以及与样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，第二皮肤年龄小于或等于第一皮肤年龄。第二皮肤年龄可以是用于描述目标效果图像中面部区域的皮肤状态的信息。
样本原始图像可以是未经处理的面部图像，样本效果图可以是经过皮肤图像处理后的面部图像，例如：经过皮肤提拉处理后的面部图像等。初始皮肤图像处理模型可以是模型参数为默认参数的模型，作为后续模型训练的基础模型。目标效果图像为待处理图像经过目标皮肤图像处理模型处理后得到的图像，即待处理图像所对应的进行皮肤图像处理后的图像。
样本效果图像可以是通过图像处理软件(如：PhotoShop等)，对样本原始图像进行多种处理后的图像，例如：处理可以是去黑眼圈、去法令纹、去皱纹、凹陷填充等皮肤年轻化处理。样本原始图像和样本效果图像也可以是皮肤图像处理前后的图像集中的图像，将皮肤提拉前的图像确定为样本原始图像，将皮肤图像处理后的图像确定为样本效果图像。初始皮肤图像处理模型可以是神经网络模型、生成对抗网络模型等。
将待处理图像输入至预先训练得到的目标皮肤图像处理模型中，通过目标皮肤图像处理模型对待处理图像进行处理，将处理得到的图像作为目标效果图像，即得到皮肤图像处理后的待处理图像。
在本公开实施例技术方案的基础上，可以在预设显示区域显示目标效果图像。目标效果图像的展示方式可以有多种，在此并不做限制。例如，目标效果图像本身可以是静态展示也可以是动态展示，还可以是在目标效果图像上叠加其他展示效果进行展示。
预设显示区域可以是全部显示区域或全部显示区域中的部分显示区域。当预设显示区域为部分显示区域时，目标效果图像的展示方式可以是在显示区域中分区域展示待处理图像和目标效果图像，以便于查看待处理图像和目标效果图像之间的区别，直观了解待处理图像进行处理后的效果。
在得到目标效果图像后，可以将目标效果图像推送给用户，以使用户直观看到皮肤图像处理后的图像。为了使用户能够比对看出皮肤图像处理前后的区别，可以在待处理图像显示的同时，在预设显示区域显示目标效果图像。
目标效果图所对应的皮肤年龄小于或等于待处理图像所对应的皮肤年龄。由于用户的皮肤年龄可能存在与真实年龄不匹配的情况，此种情况下，可能存在目标效果图所对应的皮肤年龄仍然大于真实年龄，或者与真实年龄相差不大的情况。示例性的，用户A的真实年龄为20岁，用户A的待处理图像的皮肤 年龄为40岁，那么，目标效果图像的皮肤年龄小于或等于40岁，但是可能仍然大于20岁，也可能小于或等于20岁；用户B的真实年龄为60岁，用户B的待处理图像的皮肤年龄为40岁，那么，目标效果图像的皮肤年龄小于或等于40岁，可能远小于60岁，目标效果图像对应的皮肤年龄还是要根据图像处理情况来确定。
本公开实施例的技术方案，通过获取待处理图像，将待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与待处理图像对应的目标效果图像，解决了用户面部美化时同质化严重、细节损失严重、分辨率低的问题，实现简化图像处理流程，提高图像处理的自适应性以及图像处理结果的自然度的效果。
实施例二
图2为本公开实施例二所提供的一种目标皮肤图像处理模型的训练方法的流程示意图，本实施例在本公开实施例中任一技术方案的基础上进行说明，针对目标皮肤图像处理模型的训练方式可参见本实施例的技术方案。其中，与上述实施例相同或相应的术语的解释在此不再赘述。
如图2所示，本实施例的方法可包括：
S210、获取包括面部区域的样本原始图像，确定与样本原始图像对应的样本效果图像。
样本原始图像可以是未经处理的面部图像，样本效果图可以是经过皮肤图像处理后的面部图像。样本原始图像的面部区域的皮肤状态处于第一皮肤年龄，样本效果图像的面部区域的皮肤状态处于第二皮肤年龄，第二皮肤年龄小于或等于第一皮肤年龄。
可以通过拍摄、下载或上传等方式获取一定数量的包括面部区域的样本原始图像。进而，针对每一张样本原始图像可以通过逐步处理，处理主要为皮肤年轻化处理。将处理完成的图像，确定为与样本原始图像对应的样本效果图像，以使样本效果图像相对于样本原始图像更加年轻化。
在本公开实施例技术方案的基础上，可以通过下述方式确定与样本原始图像对应的样本效果图像：
获取对样本原始图像中的面部区域进行肤龄转换处理得到的初步效果图像，根据初步效果图像确定样本效果图像。
肤龄转换处理包括皱纹淡化处理、黑眼圈淡化处理及凹陷填充处理中的至 少一种。肤龄转化处理与后续训练得到的目标皮肤图像处理模型进行的皮肤图像处理相对应。初步效果图像可以是样本原始图像进行肤龄转换处理后的图像，即皮肤年轻化处理后的图像。
对每一张样本原始图像中的面部区域进行肤龄转换处理，以使样本原始图像中的面部区域更加年轻化，肤龄更小。将肤龄转换处理后的图像确定为初步效果图像。进而，可以将初步效果图像作为样本效果图像，也可以对初步效果图像进行调节处理，得到样本效果图像，其中，调节处理可以是亮度调节、饱和度调节、色彩调节、清晰度调节等。
在本公开实施例技术方案的基础上，可以通过下述方式根据初步效果图像确定样本效果图像：
根据样本原始图像对初步效果图像中的面部区域进行皮肤颜色校正处理，得到样本效果图像。
皮肤颜色校正处理可以是对初步效果图像中的皮肤区域的颜色进行调节，以使皮肤区域颜色贴近样本原始图像的处理方式。
以样本原始图像中面部区域的皮肤颜色为基准，对初步效果图像中的面部区域的皮肤颜色进行调节，使得初步效果图像中的面部区域的皮肤颜色更贴近基准，以提高图像处理的真实性。
在本公开实施例技术方案的基础上，可以通过下述步骤根据样本原始图像对初步效果图像中的面部区域进行皮肤颜色校正处理，得到样本效果图像：
步骤一、计算样本原始图像中面部区域的第一皮肤颜色均值，以及初步效果图像中的面部区域的第二皮肤颜色均值。
第一皮肤颜色均值可以是样本原始图像中面部区域所对应的多个像素点的颜色值的均值。第二皮肤颜色均值可以是初步效果图像中面部区域的所对应的多个像素点的颜色值的均值。
根据样本原始图像中面部区域所对应的多个像素点的颜色值，计算多个颜色值的均值得到第一皮肤颜色均值。根据初步效果图像中面部区域所对应的多个像素点的颜色值，计算多个颜色值的均值得到第二皮肤颜色均值。
可以通过下述方式计算样本原始图像中面部区域的第一皮肤颜色均值：
确定样本原始图像中的面部区域，并确定面部区域中的皮肤区域，计算皮肤区域中多个像素点的第一皮肤颜色均值。
皮肤区域可以是面部区域中除去眉毛区域、眼睛区域、嘴唇区域等非皮肤区域的剩余区域。
在样本原始图像中识别出面部区域，并在面部区域中的确定出皮肤区域，进而，将皮肤区域中多个像素点的颜色值的叠加求平均，得到第一皮肤颜色均值。
针对第二皮肤颜色均值的确定方式与上述方式类似，在此不再赘述。
步骤二、根据第一皮肤颜色均值、第二皮肤颜色均值以及初步效果图像中的面部区域的多个像素点对应的原始颜色值分别确定初步效果图像中的面部区域的多个像素点对应的目标颜色值，根据目标颜色值生成样本效果图像。
原始颜色值可以是初步效果图像中的面部区域的每个像素点对应的颜色值。目标颜色值可以是初步效果图像中的面部区域的每个像素点待调整为的颜色值。
根据第一皮肤颜色均值和第二皮肤颜色均值，可以确定样本原始图像和初步效果图像之间的颜色差异。进而，根据样本原始图像和初步效果图像之间的颜色差异对初步效果图像中的面部区域的每个像素点对应的原始颜色值进行处理，确定与每个像素点对应的目标颜色值。根据每个像素点对应的目标颜色值生成样本效果图像，以达到将初步效果图调整为皮肤颜色与样本原始图像相匹配的样本效果图。
示例性的，可以将初步效果图像中的面部区域的每个像素点对应的原始颜色值与第二皮肤颜色均值相减，得到每个像素点对应的去均值颜色值。进而，将每个像素点对应的去均值颜色值与第一皮肤颜色均值相加，得到每个像素点对应的目标颜色值。通过上述方式，可以进行皮肤颜色均值的调节，以使图像更真实。
S220、基于多张样本原始图像以及与样本原始图像对应的样本效果图像构建训练样本集，根据训练样本集中的样本原始图像以及与样本原始图像对应的样本效果图像对生成对抗网络进行训练，将训练完成的皮肤图像处理生成器作为目标皮肤图像处理模型。
初始皮肤图像处理模型包括生成对抗网络，生成对抗网络包括皮肤图像处理生成器和皮肤图像处理判别器。皮肤图像处理生成器可以是全连接神经网络、反卷积网络等，皮肤图像处理判别器可以是任意的判别器模型，比如全连接网络、包含卷积的网络等。训练样本集可以是由多张样本原始图像以及与每一张样本原始图像对应的样本效果图像组成的图像集，用于后续训练得到目标皮肤图像处理模型。
将多张样本原始图像以及与样本原始图像对应的样本效果图像组合得到训练样本集。根据训练样本集中的样本原始图像以及与样本原始图像对应的样本效果图像对生成对抗网络进行训练，可以是分别训练皮肤图像处理生成器和皮 肤图像处理判别器，进而，将训练完成的皮肤图像处理生成器作为目标皮肤图像处理模型。
由于训练样本集中的样本原始图像的年龄不同，后续处理得到的样本效果图像的显著程度不同。若多个年龄的样本数量不同，则存在样本不均衡的问题，会导致生成对抗网络的训练效果差的问题。因此，可以对不同年龄的样本数量进行均衡处理。在本公开实施例技术方案的基础上，通过下述方式对生成对抗网络进行训练：
根据训练样本集中的样本原始图像对应的皮肤年龄以及每个皮肤年龄对应的样本原始图像的图像数量确定样本原始图像的目标迭代训练次数，根据样本原始图像、与样本原始图像对应的样本效果图像以及目标迭代训练次数对生成对抗网络进行训练。
皮肤年龄可以是样本原始图像中面部区域所对应的人物的面部区域的皮肤状态所对应的年龄，并非是真实年龄。示例性的，若根据图像处理分析难以确定皮肤年龄的数值，只能确定到一定的数值范围，则可以将该数值范围作为皮肤年龄。目标迭代训练次数可以是根据样本原始图像对生成对抗网络进行训练的次数。
确定训练样本集中的每张样本原始图像对应的皮肤年龄，并确定每个皮肤年龄对应的样本原始图像的图像数量。为了使得多个皮肤年龄对应的样本原始图像的图像数量均衡，可以确定每个皮肤年龄对应的每张样本原始图像所对应的目标迭代训练次数。进而，将每个样本原始图像依据与样本原始图像相对应的目标迭代训练次数输入至生成对抗网络中进行训练。
示例性的，皮肤年龄A所对应的样本原始图像的图像数量为300，皮肤年龄B所对应的样本原始图像的图像数量为600。那么，可以将皮肤年龄A所对应的每一张样本原始图像的目标迭代训练次数确定为2，并保持皮肤年龄B所对应的每一张样本原始图像的目标迭代训练次数仍为1；还可以将皮肤年龄A所对应的样本原始图像中的随机150张的目标迭代训练次数确定为3，将剩余150张样本原始图像的目标迭代训练次数确定为1，并保持皮肤年龄B所对应的每一张样本原始图像的目标迭代训练次数仍为1；也可以将皮肤年龄A所对应的样本原始图像中的一张的目标迭代训练次数确定为301，将剩余299张样本原始图像的目标迭代训练次数确定为1，并保持皮肤年龄B所对应的每一张样本原始图像的目标迭代训练次数仍为1。
在本公开实施例技术方案的基础上，可以依据皮肤年龄进行分组处理，以提高样本均衡的效率。可以是：
根据训练样本集中每张样本原始图像对应的皮肤年龄将训练样本集中多张样本原始图像进行分组处理，得到至少两个年龄段的样本训练组，根据每个样本训练组对应的样本原始图像的图像数量确定每个样本训练组对应的目标迭代训练次数。
图像数量偏少的样本训练组的目标迭代训练次数不低于图像数量偏多的样本训练组的目标迭代训练次数。
确定训练样本集中每张样本原始图像对应的皮肤年龄，根据预先设定的分组需求，将多张样本原始图像进行分组处理，以分为至少两个年龄段对应的样本训练组。进而，根据每个样本训练组对应的样本原始图像的图像数量，增大图像数量偏少的样本训练组的目标迭代训练次数，以使多个样本训练组均衡。
示例性的，将皮肤年龄21-40作为第一年龄段，并将第一年龄段所对应的样本训练组确定为第一样本训练组；将皮肤年龄41-60作为第二年龄段，并将第二年龄段所对应的样本训练组确定为第二样本训练组；将皮肤年龄61-80作为第三年龄段，并将第三年龄段所对应的样本训练组确定为第三样本训练组。第一样本训练组对应的样本原始图像的图像数量为400，第二样本训练组对应的样本原始图像的图像数量为200，第三样本训练组对应的样本原始图像的图像数量为100。据此，可以确定第二样本训练组的目标迭代训练次数为2，第三样本训练组的目标迭代训练次数为4。若图像数量偏多的样本训练组的图像数量并不是图像数量偏少的样本训练组的图像数量的整数倍，则可以分别确定图像数量偏少的样本训练组的不同样本原始图像所对应的目标迭代训练次数，即同一样本训练组中不同的样本原始图像所对应的目标迭代训练次数可以相同也可以不同。
本实施例的技术方案，通过获取包括面部区域的样本原始图像，确定与样本原始图像对应的样本效果图像，基于多张样本原始图像以及与样本原始图像对应的样本效果图像构建训练样本集，根据训练样本集中的样本原始图像以及与样本原始图像对应的样本效果图像对生成对抗网络进行训练，将训练完成的皮肤图像处理生成器作为目标皮肤图像处理模型，解决了通过模型对用户面部进行美化时，美化效果较差以及美化后图像自然度较差的问题，实现了通过生成对抗网络在提高美化效果的同时，提高美化后图像自然度的效果。
实施例三
图3为本公开实施例三所提供的一种目标皮肤图像处理模型的训练方法的流程示意图，本实施例在本公开实施例中任一技术方案的基础上进行说明，针对训练样本集的样本扩充、皮肤图像处理生成器和皮肤图像处理判别器的训练 过程可参见本实施例的技术方案。其中，与上述实施例相同或相应的术语的解释在此不再赘述。
如图3所示，本实施例的方法可包括：
S310、获取包括面部区域的样本原始图像，确定与样本原始图像对应的样本效果图像。
S320、获取多张样本原始图像中的目标原始图像以及与目标原始图像对应的样本效果图像。
目标原始图像可以是样本原始图像中的全部或部分图像，用于后续进行训练样本集扩充。
从多张样本原始图像中选择出目标原始图像，并将与每张目标原始图像相对应的样本效果图像确定出来，以便后续处理和匹配。
S330、根据预设光照条件对目标原始图像进行光照模拟处理，得到样本扩充图像，并根据预设光照条件对与目标原始图像对应的样本效果图像进行光照模拟处理，得到与样本扩充图像对应的样本效果图像。
预设光照条件可以是用于改变目标原始图像光照情况的光照条件，可以包括光照部位、光照强度、光照颜色、光源形式等。光照模拟处理可以是将预设光照条件的光叠加在目标原始图像上的处理方式。样本扩充图像可以是光照模拟处理后的图像，用于对样本原始图像进行样本扩充。
可以确定一种或多种预设光照条件，并根据一种或多种预设光照条件对目标原始图像进行光照模拟处理，将处理后的图像作为样本扩充图像。相应的，根据一种或多种预设光照条件对与目标原始图像对应的样本效果图像进行光照模拟处理，将处理后的图像作为与每张样本扩充图像对应的样本效果图像。
将目标原始图像和与目标原始图像对应的样本效果图像进行相同的光照模拟处理的原因在于，保证样本扩充图像和与样本扩充图像对应的样本效果图像只是进行了皮肤图像处理，避免光照情况的影响。
S340、将样本原始图像和样本扩充图像作为样本输入图像，将与样本原始图像对应的样本效果图像和与样本扩充图像对应的样本效果图像作为期望效果图像，根据样本输入图像和期望效果图像构建训练样本集。
样本输入图像是训练样本集中皮肤图像处理之前的图像，期望效果图像是训练样本集中皮肤图像处理之后得到的图像。
样本原始图像和与样本原始图像对应的样本效果图像为训练样本集中的原始部分，样本扩充图像和与样本扩充图像对应的样本效果图像为训练样本集中 的扩充部分。将样本原始图像和样本扩充图像作为样本输入图像，即皮肤图像处理前的图像，将与样本原始图像对应的样本效果图像和与样本扩充图像对应的样本效果图像作为期望效果图像，即皮肤图像处理后的图像。进而，将样本输入图像和期望效果图像对应构建训练样本集。
S350、将训练样本集中的样本输入图像输入至生成对抗网络中的皮肤图像处理生成器中，得到样本生成图像。
将训练样本集中的样本输入图像输入至生成对抗网络中的皮肤图像处理生成器中，通过皮肤图像处理生成器的处理，可以得到与每张样本输入图像相对应的样本生成图像。
S360、根据样本生成图像、样本输入图像以及与样本输入图像对应的期望效果图像对皮肤图像处理生成器的网络参数进行调整。
根据样本生成图像和样本输入图像之间的差异，可以判断皮肤图像处理生成器处理前后图像之间的差异，根据样本生成图像和与样本输入图像对应的期望效果图像之间的差异，可以判断皮肤图像处理生成器处理后的图像与期望得到的图像的差异。进而，通过确定出的两种差异对皮肤图像处理生成器的网络参数进行调整，以使后续皮肤图像处理生成器输出的样本生成图像相对于样本输入图像存在皮肤处理的效果，相对于期望效果图像存在难以分辨的效果。
在本公开实施例技术方案的基础上，可以依据下述方式对皮肤图像处理生成器的网络参数进行调整：
步骤一、根据预设第一损失函数计算样本生成图像与样本输入图像之间的第一损失值。
预设第一损失函数可以是用于衡量样本生成图像与样本输入图像之间差异的损失函数。第一损失值可以是预设第一损失函数计算得到的输出值，表示样本生成图像与样本输入图像之间的差异。
为了保证皮肤图像处理生成器的效果的真实性，则要保证未经处理的部分尽可能的接近样本输入图像。因此，可以将眼部区域、法令纹区域、腮部区域等处理较为明显的关注区域排除，根据除关注区域外的剩余区域构建第一损失函数，以使剩余区域的差异不明显。
步骤二、根据预设第二损失函数计算样本生成图像与样本输入图像对应的期望效果图像之间的第二损失值。
预设第二损失函数可以是用于衡量样本生成图像与样本输入图像对应的期望效果图像之间差异的损失函数。第二损失值可以是预设第二损失函数计算得到的输出值，表示样本生成图像与样本输入图像对应的期望效果图像之间的差 异。
为了保证皮肤图像处理生成器的效果的有效性，则要保证已经处理的部分尽可能的接近期望效果图像。因此，可以根据眼部区域、法令纹区域、腮部区域等处理较为明显的关注区域构建第二损失函数，以降低样本生成图像和与样本输入图像对应的期望效果图像之间的差异。
步骤三、根据第一损失值与第二损失值对皮肤图像处理生成器的网络参数进行调整。
在得到第一损失值和第二损失值后，可以根据第一损失值和第二损失值分别对皮肤图像处理生成器的网络参数进行调整，以使调整后计算得到的第一损失值和第二损失值的综合损失值下降。其中，综合损失值可以是根据第一损失值和第二损失值计算处理得到的，计算处理方式可以是求和计算、加权求和计算等。
S370、根据样本生成图像以及与样本生成图像对应的期望效果图像对皮肤图像处理判别器进行训练，并基于训练得到的皮肤图像处理判别器对样本生成图像的判别结果确定是否结束对皮肤图像处理生成器的调整。
根据样本生成图像以及与样本生成图像对应的期望效果图像对皮肤图像处理判别器进行训练，以使皮肤图像处理判别器能够较为有效的区分样本生成图像和与该样本生成图像相对应的期望效果图像。基于训练得到的皮肤图像处理判别器对样本生成图像的判别结果可以确定样本生成图像和与该样本生成图像相对应的期望效果图像是否可区分，如果可区分，则表明样本生成图像的效果不佳，需要重新调整皮肤图像处理生成器的参数，不能结束对皮肤图像处理生成器的调整；如果已经难以区分，则表明样本生成图像的效果较好，贴近期望效果图像，可以结束对皮肤图像处理生成器的调整。
S380、如果确定结束对皮肤图像处理生成器的调整，则将调整得到的皮肤图像处理生成器作为目标皮肤图像处理模型。
如果基于训练得到的皮肤图像处理判别器对样本生成图像的判别结果确定可以结束对皮肤图像处理生成器的调整，则表明调整得到的皮肤图像处理生成器的效果稳定、自然、能够有效实现皮肤年轻化，可以将调整得到的皮肤图像处理生成器作为目标皮肤图像处理模型。
本实施例的技术方案，通过获取包括面部区域的样本原始图像，确定与样本原始图像对应的样本效果图像，获取多张样本原始图像中的目标原始图像以及与目标原始图像对应的样本效果图像，根据预设光照条件对目标原始图像进行光照模拟处理，得到样本扩充图像，并根据预设光照条件对与目标原始图像 对应的样本效果图像进行光照模拟处理，得到与样本扩充图像对应的样本效果图像，并将样本原始图像和样本扩充图像作为样本输入图像，将与样本原始图像对应的样本效果图像和与样本扩充图像对应的样本效果图像作为期望效果图像，根据样本输入图像和期望效果图像构建训练样本集，以对训练样本集进行可靠的扩充，使模型训练样本的增加，便于有效提高后续的模型训练效果。进而，将训练样本集中的样本输入图像输入至生成对抗网络中的皮肤图像处理生成器中，得到样本生成图像，根据样本生成图像、样本输入图像以及与样本输入图像对应的期望效果图像对皮肤图像处理生成器的网络参数进行调整，以使皮肤图像处理生成器的训练过程达到均衡美化效果和自然效果的目的。根据样本生成图像以及与样本生成图像对应的期望效果图像对皮肤图像处理判别器进行训练，并基于训练得到的皮肤图像处理判别器对样本生成图像的判别结果确定是否结束对皮肤图像处理生成器的调整，如果是，则将调整得到的皮肤图像处理生成器作为目标皮肤图像处理模型，解决了模型训练样本数量较少而导致的模型训练效果不佳的问题，以及难以同时兼顾美化效果和自然效果的问题，实现了对训练样本集的扩充，并综合考虑了美化效果和自然效果，来提高用户体验度的效果。
实施例四
图4为本公开实施例四所提供的一种图像处理和模型训练方法的示意图，如图4所示，本实施例的方法可包括：模型训练部分和图像处理部分。
模型训练部分主要包括：
1、脸部数据采集。
可以通过人工采集的方式获取一定数量的高清脸部数据(样本原始图像)，例如：500-2000张高清脸部图像，数量可以根据实际需求选择。为了保证模型训练的效果，可以要求脸部图像的分辨率不低于1024*1024，脸部数据分布要求尽可能覆盖男、女，20-80岁多个皮肤年龄段，涵盖脸部的多种角度等。
2、脸部特征矫正。
通过人工处理的方式可以将高清脸部图像进行脸部特征矫正，矫正的内容主要与脸部提拉紧致相关，例如：去除黑眼圈、去除皱纹、降低法令纹、填充脸部凹陷等。经过脸部特征矫正可以得到与每张高清脸部图像相对应的矫正脸部图像(样本效果图像)，可以将高清脸部图像记为A，将与每张高清脸部图像相对应的矫正脸部图像记为B。
3、有监督训练提拉紧致生成式对抗网络(Generative Adversarial Networks， GAN)模型，获取人脸美化生成器(目标皮肤图像处理模型)。
使用上述一定数量的高清脸部图像A，和与每张高清脸部图像相对应的矫正脸部图像B训练提拉紧致模型(生成对抗网络)。为了保证训练过程中，人脸美化生成器能自适应学习脸部提拉紧致相关的参数，需要指定监督策略。在提拉紧致GAN模型的训练过程中，加入了光照条件模拟(光照模拟处理)、年龄分布适配(根据训练样本集中的样本原始图像对应的皮肤年龄以及每个皮肤年龄对应的样本原始图像的图像数量确定样本原始图像的目标迭代训练次数，根据样本原始图像、与样本原始图像对应的样本效果图像以及目标迭代训练次数对生成对抗网络进行训练)、皮肤颜色矫正(皮肤颜色校正处理)、人脸高维语义特征矫正(根据预设第一损失函数计算样本生成图像与样本输入图像之间的第一损失值)、人脸低维纹理特征矫正(根据预设第二损失函数计算样本生成图像与样本输入图像对应的期望效果图像之间的第二损失值)等图像算法策略，保证提拉紧致GAN网络在训练过程中，不但能保留原始的肤质信息，而且只针对性地处理皱纹、黑眼圈、皮肤凹陷等缺陷区域，生成完整的人脸美化生成器，记为G(A，B)。
图像处理部分主要包括：
1、接收用户图像。
示例性的，在线上应用的环境中，接收用户通过拍摄或上传等方式确定的用户图像(待处理图像)。
2、脸部裁剪。
在用户上传用户图像后，使用脸部关键点信息进行脸部裁剪。人脸裁剪的方式是将用户上传的用户图像中的脸部区域进行识别和裁剪，得到裁剪后的脸部图像。并且，可以将脸部图像依据模特图像进行调整，将脸部图像的范围拉伸或压缩至与模特图像中的脸部图像相同的范围，以便提升后续皮肤提拉紧致处理区域的识别，提高图像处理效果。
3、基于人脸美化生成器进行处理。
将裁剪后的脸部图像输入至模型获取阶段得到的人脸美化生成器G(A，B)中，通过人脸美化生成器G(A，B)对脸部图像进行脸部信息提取，并对对脸部语义、纹理等信息等进行处理。
4、输出提拉紧致脸部效果图。
在预设显示区域自适应的输出具有提拉紧致效果的脸部效果图。
本实施例的技术方案，通过脸部数据采集、脸部特征矫正以及有监督训练 提拉紧致GAN模型，来获取人脸美化生成器。并且，通过接收用户图像，进行脸部裁剪，基于人脸美化生成器进行处理，输出提拉紧致脸部效果图，解决了用户面部美化时同质化严重、细节损失严重、分辨率低的问题，实现简化图像处理流程，提高图像处理的自适应性以及图像处理结果的自然度的效果。
实施例五
图5为本公开实施例五所提供的一种图像处理装置的结构示意图，本实施例所提供的图像处理装置可以通过软件和/或硬件来实现，可配置于终端和/或服务器中来实现本公开实施例中的图像处理方法。如图5所示，该装置可包括：
图像获取模块510，设置为获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；皮肤处理模块520，设置为将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
在本公开实施例中任一技术方案的基础上，所述初始皮肤图像处理模型包括生成对抗网络，所述生成对抗网络包括皮肤图像处理生成器和皮肤图像处理判别器，所述装置还包括：模型训练模块，设置为获取包括面部区域的样本原始图像，确定与所述样本原始图像对应的样本效果图像，其中，所述样本原始图像的面部区域的皮肤状态处于第一皮肤年龄，所述样本效果图像的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄；基于多张样本原始图像以及与所述样本原始图像对应的样本效果图像构建训练样本集，根据所述训练样本集中的样本原始图像以及与所述样本原始图像对应的样本效果图像对所述生成对抗网络进行训练，将训练完成的皮肤图像处理生成器作为目标皮肤图像处理模型。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为获取对所述样本原始图像中的面部区域进行肤龄转换处理得到的初步效果图像，根据所述初步效果图像确定样本效果图像，其中，所述肤龄转换处理包括皱纹淡化处理、黑眼圈淡化处理及凹陷填充处理中的至少一种。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为根据所述样本原始图像对所述初步效果图像中的面部区域进行皮肤颜色校正处理，得到样本效果图像。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为计算所述样本原始图像中面部区域的第一皮肤颜色均值，以及所述初步效果图像中的面部区域的第二皮肤颜色均值；根据所述第一皮肤颜色均值、所述第二皮肤颜色均值以及所述初步效果图像中的面部区域的多个像素点对应的原始颜色值分别确定所述初步效果图像中的面部区域的多个像素点对应的目标颜色值，根据所述目标颜色值生成样本效果图像。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为确定所述样本原始图像中的面部区域，并确定所述面部区域中的皮肤区域，计算所述皮肤区域中多个像素点的第一皮肤颜色均值。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为获取多张样本原始图像中的目标原始图像以及与所述目标原始图像对应的样本效果图像；根据预设光照条件对所述目标原始图像进行光照模拟处理，得到样本扩充图像，并根据所述预设光照条件对与所述目标原始图像对应的样本效果图像进行光照模拟处理，得到与所述样本扩充图像对应的样本效果图像；将所述样本原始图像和所述样本扩充图像作为样本输入图像，将与所述样本原始图像对应的样本效果图像和与所述样本扩充图像对应的样本效果图像作为期望效果图像，根据所述样本输入图像和所述期望效果图像构建训练样本集。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为将所述训练样本集中的样本输入图像输入至所述生成对抗网络中的皮肤图像处理生成器中，得到样本生成图像；根据所述样本生成图像、所述样本输入图像以及与所述样本输入图像对应的期望效果图像对所述皮肤图像处理生成器的网络参数进行调整；根据所述样本生成图像以及与所述样本生成图像对应的期望效果图像对皮肤图像处理判别器进行训练，并基于训练得到的所述皮肤图像处理判别器对所述样本生成图像的判别结果确定是否结束对所述皮肤图像处理生成器的调整；如果确定结束对所述皮肤图像处理生成器的调整，则将调整得到的皮肤图像处理生成器作为目标皮肤图像处理模型。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为根据预设第一损失函数计算所述样本生成图像与所述样本输入图像之间的第一损失值；根据预设第二损失函数计算所述样本生成图像与所述样本输入图像对应的期望效果图像之间的第二损失值；根据所述第一损失值与所述第二损失值对所述皮肤图像处理生成器的网络参数进行调整。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为根据所述训练样本集中的样本原始图像对应的皮肤年龄以及每个皮肤年龄对应的样本原始图像的图像数量确定所述样本原始图像的目标迭代训练次数，根据 所述样本原始图像、与所述样本原始图像对应的样本效果图像以及所述目标迭代训练次数对所述生成对抗网络进行训练。
在本公开实施例中任一技术方案的基础上，所述模型训练模块，还设置为根据所述训练样本集中每张样本原始图像对应的皮肤年龄将所述训练样本集中多张样本原始图像进行分组处理，得到至少两个年龄段的样本训练组，根据每个样本训练组对应的样本原始图像的图像数量确定每个样本训练组对应的目标迭代训练次数；其中，图像数量偏少的样本训练组的目标迭代训练次数不低于图像数量偏多的样本训练组的目标迭代训练次数。
在本公开实施例中任一技术方案的基础上，所述图像获取模块510，还设置为当接收用于启用预设皮肤处理特效的特效触发操作时，展示至少一种图像获取控件；接收针对至少一种图像获取控件的控件触发操作，采用所触发的图像获取控件对应的图像获取方式获取待处理图像。
上述装置可执行本公开任意实施例所提供的方法，具备执行方法相应的功能模块和效果。
本公开实施例的技术方案，通过获取待处理图像，将待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与待处理图像对应的目标效果图像，解决了用户面部美化时同质化严重、细节损失严重、分辨率低的问题，实现简化图像处理流程，提高图像处理的自适应性以及图像处理结果的自然度的效果。
上述装置所包括的多个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，多个功能单元的名称也只是为了便于相互区分，并不用于限制本公开实施例的保护范围。
实施例六
图6为本公开实施例六所提供的一种电子设备的结构示意图。下面参考图6，其示出了适于用来实现本公开实施例的电子设备(例如图6中的终端设备或服务器)600的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，PDA)、平板电脑(Portable Android Device，PAD)、便携式多媒体播放器(Portable Media Player，PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字电视(Television，TV)、台式计算机等等的固定终端。图6示出的电子设备600仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图6所示，电子设备600可以包括处理装置(例如中央处理器、图形处理器等)601，其可以根据存储在只读存储器(Read-Only Memory，ROM)602中的程序或者从存储装置608加载到随机访问存储器(Random Access Memory，RAM)603中的程序而执行多种适当的动作和处理。在RAM 603中，还存储有电子设备600操作所需的多种程序和数据。处理装置601、ROM 602以及RAM 603通过总线605彼此相连。编辑/输出(Input/Output，I/O)接口604也连接至总线605。
通常，以下装置可以连接至I/O接口604：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置606；包括例如液晶显示器(Liquid Crystal Display，LCD)、扬声器、振动器等的输出装置607；包括例如磁带、硬盘等的存储装置608；以及通信装置609。通信装置609可以允许电子设备600与其他设备进行无线或有线通信以交换数据。虽然图6示出了具有多种装置的电子设备600，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置609从网络上被下载和安装，或者从存储装置608被安装，或者从ROM 602被安装。在该计算机程序被处理装置601执行时，执行本公开实施例的方法中限定的上述功能。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
本公开实施例提供的电子设备与上述实施例提供的图像处理方法属于同一构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的效果。
实施例七
本公开实施例提供了一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述实施例所提供的图像处理方法。
本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者 任意以上的组合。计算机可读存储介质的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、RAM、ROM、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency，RF)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如超文本传输协议(HyperText Transfer Protocol，HTTP)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(Local Area Network，LAN)，广域网(Wide Area Network，WAN)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：
获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言— 诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括LAN或WAN—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开多种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在一种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(Field Programmable Gate Array，FPGA)、专用集成电路(Application Specific Integrated Circuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、片上系统(System on Chip，SOC)、复杂可编程逻辑设备(Complex Programming Logic Device，CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、RAM、ROM、EPROM或快闪存储器、光纤、CD-ROM、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，【示例一】提供了一种图像处理方法，该方法包括：
获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；
将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
根据本公开的一个或多个实施例，【示例二】提供了一种图像处理方法，该方法，还包括：
所述初始皮肤图像处理模型包括生成对抗网络，所述生成对抗网络包括皮肤图像处理生成器和皮肤图像处理判别器，所述目标皮肤图像处理模型基于如下方式训练得到：
获取包括面部区域的样本原始图像，确定与所述样本原始图像对应的样本效果图像，其中，所述样本原始图像的面部区域的皮肤状态处于第一皮肤年龄，所述样本效果图像的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄；
基于多张样本原始图像以及与所述样本原始图像对应的样本效果图像构建训练样本集，根据所述训练样本集中的样本原始图像以及与所述样本原始图像对应的样本效果图像对所述生成对抗网络进行训练，将训练完成的皮肤图像处理生成器作为目标皮肤图像处理模型。
根据本公开的一个或多个实施例，【示例三】提供了一种图像处理方法，该方法，还包括：
所述确定与所述样本原始图像对应的样本效果图像，包括：
获取对所述样本原始图像中的面部区域进行肤龄转换处理得到的初步效果图像，根据所述初步效果图像确定样本效果图像，其中，所述肤龄转换处理包括皱纹淡化处理、黑眼圈淡化处理及凹陷填充处理中的至少一种。
根据本公开的一个或多个实施例，【示例四】提供了一种图像处理方法，该方法还包括：
所述根据所述初步效果图像确定样本效果图像，包括：
根据所述样本原始图像对所述初步效果图像中的面部区域进行皮肤颜色校 正处理，得到样本效果图像。
根据本公开的一个或多个实施例，【示例五】提供了一种图像处理方法，该方法，还包括：
所述根据所述样本原始图像对所述初步效果图像中的面部区域进行皮肤颜色校正处理，得到样本效果图像，包括：
计算所述样本原始图像中面部区域的第一皮肤颜色均值，以及所述初步效果图像中的面部区域的第二皮肤颜色均值；
根据所述第一皮肤颜色均值、所述第二皮肤颜色均值以及所述初步效果图像中的面部区域的多个像素点对应的原始颜色值分别确定所述初步效果图像中的面部区域的多个像素点对应的目标颜色值，根据所述目标颜色值生成样本效果图像。
根据本公开的一个或多个实施例，【示例六】提供了一种图像处理方法，该方法，还包括：
所述计算所述样本原始图像中面部区域的第一皮肤颜色均值，包括：
确定所述样本原始图像中的面部区域，并确定所述面部区域中的皮肤区域，计算所述皮肤区域中多个像素点的第一皮肤颜色均值。
根据本公开的一个或多个实施例，【示例七】提供了一种图像处理方法，该方法，还包括：
所述基于多张样本原始图像以及与所述样本原始图像对应的样本效果图像构建训练样本集，包括：
获取多张样本原始图像中的目标原始图像以及与所述目标原始图像对应的样本效果图像；
根据预设光照条件对所述目标原始图像进行光照模拟处理，得到样本扩充图像，并根据所述预设光照条件对与所述目标原始图像对应的样本效果图像进行光照模拟处理，得到与所述样本扩充图像对应的样本效果图像；
将所述样本原始图像和所述样本扩充图像作为样本输入图像，将与所述样本原始图像对应的样本效果图像和与所述样本扩充图像对应的样本效果图像作为期望效果图像，根据所述样本输入图像和所述期望效果图像构建训练样本集。
根据本公开的一个或多个实施例，【示例八】提供了一种图像处理方法，该方法，还包括：
所述根据所述训练样本集中的样本原始图像以及与所述样本原始图像对应的样本效果图像对所述生成对抗网络进行训练，将训练完成的皮肤图像处理生 成器作为目标皮肤图像处理模型，包括：
将所述训练样本集中的样本输入图像输入至所述生成对抗网络中的皮肤图像处理生成器中，得到样本生成图像；
根据所述样本生成图像、所述样本输入图像以及与所述样本输入图像对应的期望效果图像对所述皮肤图像处理生成器的网络参数进行调整；
根据所述样本生成图像以及与所述样本生成图像对应的期望效果图像对皮肤图像处理判别器进行训练，并基于训练得到的所述皮肤图像处理判别器对所述样本生成图像的判别结果确定是否结束对所述皮肤图像处理生成器的调整；
如果确定结束对所述皮肤图像处理生成器的调整，则将调整得到的皮肤图像处理生成器作为目标皮肤图像处理模型。
根据本公开的一个或多个实施例，【示例九】提供了一种图像处理方法，该方法，还包括：
所述根据所述样本生成图像、所述样本输入图像以及与所述样本输入图像对应的期望效果图像对所述皮肤图像处理生成器的网络参数进行调整，包括：
根据预设第一损失函数计算所述样本生成图像与所述样本输入图像之间的第一损失值；
根据预设第二损失函数计算所述样本生成图像与所述样本输入图像对应的期望效果图像之间的第二损失值；
根据所述第一损失值与所述第二损失值对所述皮肤图像处理生成器的网络参数进行调整。
根据本公开的一个或多个实施例，【示例十】提供了一种图像处理方法，该方法，还包括：
所述根据所述训练样本集中的样本原始图像以及与所述样本原始图像对应的样本效果图像对所述生成对抗网络进行训练，包括：
根据所述训练样本集中的样本原始图像对应的皮肤年龄以及每个皮肤年龄对应的样本原始图像的图像数量确定所述样本原始图像的目标迭代训练次数，根据所述样本原始图像、与所述样本原始图像对应的样本效果图像以及所述目标迭代训练次数对所述生成对抗网络进行训练。
根据本公开的一个或多个实施例，【示例十一】提供了一种图像处理方法，该方法，还包括：
所述根据所述训练样本集中的样本原始图像对应的皮肤年龄以及每个皮肤年龄对应的样本原始图像的图像数量确定所述样本原始图像的目标迭代训练次 数，包括：
根据所述训练样本集中每张样本原始图像对应的皮肤年龄将所述训练样本集中多张样本原始图像进行分组处理，得到至少两个年龄段的样本训练组，根据每个样本训练组对应的样本原始图像的图像数量确定每个样本训练组对应的目标迭代训练次数；
其中，图像数量偏少的样本训练组的目标迭代训练次数不低于图像数量偏多的样本训练组的目标迭代训练次数。
根据本公开的一个或多个实施例，【示例十二】提供了一种图像处理方法，该方法，还包括：
所述获取待处理图像，包括：
当接收用于启用预设皮肤处理特效的特效触发操作时，展示至少一种图像获取控件；
接收针对至少一种图像获取控件的控件触发操作，采用所触发的图像获取控件对应的图像获取方式获取待处理图像。
根据本公开的一个或多个实施例，【示例十三】提供了一种图像处理装置，该装置包括：
图像获取模块，设置为获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；
皮肤处理模块，设置为将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
此外，虽然采用特定次序描绘了多个操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了多个实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的一些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的多种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。

Claims (16)

  1. 一种图像处理方法，包括：
    获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；
    将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
  2. 根据权利要求1所述的方法，其中，所述初始皮肤图像处理模型包括生成对抗网络，所述生成对抗网络包括皮肤图像处理生成器和皮肤图像处理判别器，所述目标皮肤图像处理模型基于如下方式训练得到：
    获取包括面部区域的样本原始图像，确定与所述样本原始图像对应的样本效果图像，其中，所述样本原始图像的面部区域的皮肤状态处于所述第一皮肤年龄，所述样本效果图像的面部区域的皮肤状态处于所述第二皮肤年龄；
    基于多张样本原始图像以及与所述样本原始图像对应的样本效果图像构建训练样本集，根据所述训练样本集中的样本原始图像以及与所述样本原始图像对应的样本效果图像对所述生成对抗网络进行训练，将训练完成的皮肤图像处理生成器作为所述目标皮肤图像处理模型。
  3. 根据权利要求2所述的方法，其中，所述确定与所述样本原始图像对应的样本效果图像，包括：
    获取对所述样本原始图像中的面部区域进行肤龄转换处理得到的初步效果图像，根据所述初步效果图像确定所述样本效果图像，其中，所述肤龄转换处理包括皱纹淡化处理、黑眼圈淡化处理及凹陷填充处理中的至少一种。
  4. 根据权利要求3所述的方法，其中，所述根据所述初步效果图像确定所述样本效果图像，包括：
    根据所述样本原始图像对所述初步效果图像中的面部区域进行皮肤颜色校正处理，得到所述样本效果图像。
  5. 根据权利要求4所述的方法，其中，所述根据所述样本原始图像对所述初步效果图像中的面部区域进行皮肤颜色校正处理，得到所述样本效果图像，包括：
    计算所述样本原始图像中面部区域的第一皮肤颜色均值，以及所述初步效果图像中的面部区域的第二皮肤颜色均值；
    根据所述第一皮肤颜色均值、所述第二皮肤颜色均值以及所述初步效果图像中的面部区域的多个像素点对应的原始颜色值分别确定所述初步效果图像中的面部区域的多个像素点对应的目标颜色值，根据所述目标颜色值生成所述样本效果图像。
  6. 根据权利要求5所述的方法，其中，所述计算所述样本原始图像中面部区域的第一皮肤颜色均值，包括：
    确定所述样本原始图像中的面部区域，并确定所述面部区域中的皮肤区域，计算所述皮肤区域中多个像素点的第一皮肤颜色均值。
  7. 根据权利要求2所述的方法，其中，所述基于多张样本原始图像以及与所述样本原始图像对应的样本效果图像构建训练样本集，包括：
    获取所述多张样本原始图像中的目标原始图像以及与所述目标原始图像对应的样本效果图像；
    根据预设光照条件对所述目标原始图像进行光照模拟处理，得到样本扩充图像，并根据所述预设光照条件对与所述目标原始图像对应的样本效果图像进行光照模拟处理，得到与所述样本扩充图像对应的样本效果图像；
    将所述样本原始图像和所述样本扩充图像作为样本输入图像，将与所述样本原始图像对应的样本效果图像和与所述样本扩充图像对应的样本效果图像作为期望效果图像，根据所述样本输入图像和所述期望效果图像构建所述训练样本集。
  8. 根据权利要求7所述的方法，其中，所述根据所述训练样本集中的样本原始图像以及与所述样本原始图像对应的样本效果图像对所述生成对抗网络进行训练，将训练完成的皮肤图像处理生成器作为所述目标皮肤图像处理模型，包括：
    将所述训练样本集中的样本输入图像输入至所述生成对抗网络中的皮肤图像处理生成器中，得到样本生成图像；
    根据所述样本生成图像、所述样本输入图像以及与所述样本输入图像对应的期望效果图像对所述皮肤图像处理生成器的网络参数进行调整；
    根据所述样本生成图像以及与所述样本生成图像对应的期望效果图像对所述皮肤图像处理判别器进行训练，并基于训练得到的所述皮肤图像处理判别器对所述样本生成图像的判别结果确定是否结束对所述皮肤图像处理生成器的调整；
    响应于确定结束对所述皮肤图像处理生成器的调整，将调整得到的皮肤图 像处理生成器作为所述目标皮肤图像处理模型。
  9. 根据权利要求8所述的方法，其中，所述根据所述样本生成图像、所述样本输入图像以及与所述样本输入图像对应的期望效果图像对所述皮肤图像处理生成器的网络参数进行调整，包括：
    根据预设第一损失函数计算所述样本生成图像与所述样本输入图像之间的第一损失值；
    根据预设第二损失函数计算所述样本生成图像与所述样本输入图像对应的期望效果图像之间的第二损失值；
    根据所述第一损失值与所述第二损失值对所述皮肤图像处理生成器的网络参数进行调整。
  10. 根据权利要求2所述的方法，其中，所述根据所述训练样本集中的样本原始图像以及与所述样本原始图像对应的样本效果图像对所述生成对抗网络进行训练，包括：
    根据所述训练样本集中的样本原始图像对应的皮肤年龄以及每个皮肤年龄对应的样本原始图像的图像数量确定所述样本原始图像的目标迭代训练次数，根据所述样本原始图像、与所述样本原始图像对应的样本效果图像以及所述目标迭代训练次数对所述生成对抗网络进行训练。
  11. 根据权利要求10所述的方法，其中，所述根据所述训练样本集中的样本原始图像对应的皮肤年龄以及每个皮肤年龄对应的样本原始图像的图像数量确定所述样本原始图像的目标迭代训练次数，包括：
    根据所述训练样本集中每张样本原始图像对应的皮肤年龄将所述训练样本集中多张样本原始图像进行分组处理，得到至少两个年龄段的样本训练组，根据每个样本训练组对应的样本原始图像的图像数量确定每个样本训练组对应的目标迭代训练次数；
    其中，图像数量偏少的样本训练组的目标迭代训练次数不低于图像数量偏多的样本训练组的目标迭代训练次数。
  12. 根据权利要求1所述的方法，其中，所述获取待处理图像，包括：
    当接收用于启用预设皮肤处理特效的特效触发操作时，展示至少一种图像获取控件；
    接收针对至少一种图像获取控件的控件触发操作，采用所触发的图像获取控件对应的图像获取方式获取待处理图像。
  13. 一种图像处理装置，包括：
    图像获取模块，设置为获取待处理图像，其中，所述待处理图像包括面部区域，所述待处理图像的面部区域的皮肤状态处于第一皮肤年龄；
    皮肤处理模块，设置为将所述待处理图像输入至预先训练得到的目标皮肤图像处理模型中，得到与所述待处理图像对应的目标效果图像，其中，所述目标皮肤图像处理模型根据样本原始图像以及与所述样本原始图像对应的样本效果图像对初始皮肤图像处理模型训练得到，所述目标效果图像中的面部区域的皮肤状态处于第二皮肤年龄，所述第二皮肤年龄小于或等于所述第一皮肤年龄。
  14. 一种电子设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序；
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-12中任一所述的图像处理方法。
  15. 一种计算机可读存储介质，存储有计算机程序，所述程序被处理器执行时实现如权利要求1-12中任一所述的图像处理方法。
  16. 一种计算机程序产品，包括承载在非暂态计算机可读介质上的计算机程序，所述计算机程序包含用于执行如权利要求1-12中任一所述的图像处理方法的程序代码。
PCT/CN2023/072098 2022-01-30 2023-01-13 图像处理方法、装置、电子设备及存储介质 WO2023143129A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210114150.X 2022-01-30
CN202210114150.XA CN114445302A (zh) 2022-01-30 2022-01-30 图像处理方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023143129A1 true WO2023143129A1 (zh) 2023-08-03

Family

ID=81371504

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/072098 WO2023143129A1 (zh) 2022-01-30 2023-01-13 图像处理方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN114445302A (zh)
WO (1) WO2023143129A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117094966A (zh) * 2023-08-21 2023-11-21 青岛美迪康数字工程有限公司 基于图像扩增的舌图像识别方法、装置和计算机设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114445302A (zh) * 2022-01-30 2022-05-06 北京字跳网络技术有限公司 图像处理方法、装置、电子设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107909095A (zh) * 2017-11-07 2018-04-13 江苏大学 一种基于深度学习的图像识别方法
CN111429536A (zh) * 2020-03-13 2020-07-17 深圳市雄帝科技股份有限公司 一种对图像中肤色进行调色的方法、系统及其存储介质
CN111553854A (zh) * 2020-04-21 2020-08-18 维沃移动通信有限公司 一种图像处理方法及电子设备
US11055514B1 (en) * 2018-12-14 2021-07-06 Snap Inc. Image face manipulation
CN113177891A (zh) * 2021-04-28 2021-07-27 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113255396A (zh) * 2020-02-07 2021-08-13 北京达佳互联信息技术有限公司 图像处理模型的训练方法及装置、图像处理方法及装置
CN114445302A (zh) * 2022-01-30 2022-05-06 北京字跳网络技术有限公司 图像处理方法、装置、电子设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107909095A (zh) * 2017-11-07 2018-04-13 江苏大学 一种基于深度学习的图像识别方法
US11055514B1 (en) * 2018-12-14 2021-07-06 Snap Inc. Image face manipulation
CN113255396A (zh) * 2020-02-07 2021-08-13 北京达佳互联信息技术有限公司 图像处理模型的训练方法及装置、图像处理方法及装置
CN111429536A (zh) * 2020-03-13 2020-07-17 深圳市雄帝科技股份有限公司 一种对图像中肤色进行调色的方法、系统及其存储介质
CN111553854A (zh) * 2020-04-21 2020-08-18 维沃移动通信有限公司 一种图像处理方法及电子设备
CN113177891A (zh) * 2021-04-28 2021-07-27 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN114445302A (zh) * 2022-01-30 2022-05-06 北京字跳网络技术有限公司 图像处理方法、装置、电子设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117094966A (zh) * 2023-08-21 2023-11-21 青岛美迪康数字工程有限公司 基于图像扩增的舌图像识别方法、装置和计算机设备
CN117094966B (zh) * 2023-08-21 2024-04-05 青岛美迪康数字工程有限公司 基于图像扩增的舌图像识别方法、装置和计算机设备

Also Published As

Publication number Publication date
CN114445302A (zh) 2022-05-06

Similar Documents

Publication Publication Date Title
WO2023143129A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN110766777B (zh) 虚拟形象的生成方法、装置、电子设备及存储介质
WO2022068487A1 (zh) 风格图像生成方法、模型训练方法、装置、设备和介质
CN106682632B (zh) 用于处理人脸图像的方法和装置
CN112419151B (zh) 图像退化处理方法、装置、存储介质及电子设备
WO2023125374A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN110782515A (zh) 虚拟形象的生成方法、装置、电子设备及存储介质
CN112562019A (zh) 图像色彩调整方法及装置、计算机可读介质和电子设备
US20230386001A1 (en) Image display method and apparatus, and device and medium
CN111866483B (zh) 颜色还原方法及装置、计算机可读介质和电子设备
WO2022132032A1 (zh) 人像图像处理方法及装置
WO2022037602A1 (zh) 表情变换方法、装置、电子设备和计算机可读介质
WO2023109829A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2023040749A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2023143126A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN114419300A (zh) 风格化图像生成方法、装置、电子设备及存储介质
CN113449851A (zh) 数据处理方法及设备
CN110689546A (zh) 个性化头像的生成方法、装置、设备及存储介质
CN116934577A (zh) 一种风格图像生成方法、装置、设备及介质
CN113658065A (zh) 图像降噪方法及装置、计算机可读介质和电子设备
WO2023143118A1 (zh) 图像处理方法、装置、设备及介质
WO2023193613A1 (zh) 高光渲染方法、装置、介质及电子设备
CN110059739B (zh) 图像合成方法、装置、电子设备和计算机可读存储介质
WO2023239299A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN115953597B (zh) 图像处理方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23746011

Country of ref document: EP

Kind code of ref document: A1