CN111815504A - Image generation method and device - Google Patents

Image generation method and device Download PDF

Info

Publication number
CN111815504A
CN111815504A CN202010618887.6A CN202010618887A CN111815504A CN 111815504 A CN111815504 A CN 111815504A CN 202010618887 A CN202010618887 A CN 202010618887A CN 111815504 A CN111815504 A CN 111815504A
Authority
CN
China
Prior art keywords
image
target
initial
area
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010618887.6A
Other languages
Chinese (zh)
Inventor
张文杰
李果
樊鸿飞
蔡媛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Kingsoft Cloud Network Technology Co Ltd
Original Assignee
Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Kingsoft Cloud Network Technology Co Ltd filed Critical Beijing Kingsoft Cloud Network Technology Co Ltd
Priority to CN202010618887.6A priority Critical patent/CN111815504A/en
Publication of CN111815504A publication Critical patent/CN111815504A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G06T3/04
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • G06T5/70
    • G06T5/77
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30168Image quality inspection

Abstract

The application relates to a method and a device for generating an image, wherein the method comprises the following steps: detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image; determining a target image area from the image area corresponding to each object part, wherein a first quality parameter corresponding to a first object part shown in the target image area is lower than a first threshold value; generating a target portion image for the first target portion, wherein a second quality parameter of a second target portion shown in the target portion image is higher than or equal to the first threshold, and the second target portion and the first target portion belong to the same portion type; and acquiring a target image corresponding to the initial image according to the target position image. The method and the device solve the technical problem that the beautifying effect is poor when the object displayed in the image is beautified.

Description

Image generation method and device
Technical Field
The present application relates to the field of computers, and in particular, to a method and an apparatus for generating an image.
Background
At present, beautification processing is performed on an object displayed in an image, parameters are adjusted on the basis of an original object, but for a part of objects, because some features on the original object are too strong (for example, a face is taken as an example, a scar is formed on the face, a part of deformity is formed, eyes are too small, and the like), the current processing mode can only perform parameter adjustment on a part of the original object with stronger features, and the adjusted result still keeps stronger features, and cannot achieve better beautification effect.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The application provides an image generation method and device, which are used for at least solving the technical problem of poor beautifying effect when an object displayed in an image is beautified in the related technology.
According to an aspect of an embodiment of the present application, there is provided an image generating method, including:
detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image;
determining a target image area from the image area corresponding to each object part, wherein a first quality parameter corresponding to a first object part shown in the target image area is lower than a first threshold value;
generating a target part image for the first target part, wherein a second quality parameter of a second target part shown in the target part image is higher than or equal to the first threshold, and the second target part and the first target part belong to the same part type;
and acquiring a target image corresponding to the initial image according to the target position image.
Optionally, generating a target portion image for the first target portion comprises:
carrying out fuzzy processing on the target image area to obtain a target fuzzy image;
and generating the target part image corresponding to the target blurred image through a target image generation model, wherein the target image generation model is obtained by training an initial image generation model through an image area sample marked with a part image sample, the quality parameter sample of the target part shown in the part image sample is higher than or equal to a second threshold value, and the image area sample is obtained by blurring the part image sample.
Optionally, the generating, by a target image generation model, the target portion image corresponding to the target blurred image includes:
inputting the target blurred image into a target generation layer of the target image generation model, wherein the target image generation model includes the target generation layer and a target countermeasure layer;
and acquiring the target part image output by the target generation layer.
Optionally, before inputting the target blurred image into the generation layer of the target image generation model, the method further comprises:
fixing first model parameters of an initial generation layer, and training the initial countermeasure layer by using image area samples marked with position image samples to obtain the target countermeasure layer, wherein the initial image generation model comprises the initial generation layer and the initial countermeasure layer;
and fixing the second model parameters of the target confrontation layer, and training the initial generation layer by using the image area sample marked with the position image sample to obtain the target generation layer.
Optionally, the blurring processing is performed on the target image region, and obtaining a target blurred image includes one of:
blurring the target image area on the initial image to obtain a target blurred image;
extracting the target image area from the initial image to obtain an area image; and carrying out fuzzy processing on the area image to obtain the target fuzzy image.
Optionally, the obtaining a target image corresponding to the initial image according to the target position image includes:
determining the target image as the target image under the condition that the target blurred image is obtained by blurring the target image area on the initial image;
and replacing the target image area on the initial image with the target part image to obtain the target image under the condition that the area image is subjected to blurring processing to obtain the target blurred image.
Optionally, the determining a target image region from the image region corresponding to each target region includes:
acquiring a quality parameter corresponding to each object part through a target object detection model, wherein the target object detection model is obtained by training an initial object detection model by using an object part sample marked with a quality parameter sample;
acquiring a target part with a quality parameter lower than the first threshold value from the one or more target parts as the first target part;
and determining the area of the first object part on the initial image as the target image area.
According to another aspect of the embodiments of the present application, there is also provided an image generating apparatus, including:
the detection module is used for detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image;
a determining module, configured to determine a target image area from image areas corresponding to each of the object portions, where a first quality parameter corresponding to a first object portion shown in the target image area is lower than a first threshold;
a generation module configured to generate an object region image for the first object region, wherein a second quality parameter of a second object region shown in the object region image is higher than or equal to the first threshold, and the second object region and the first object region belong to the same region type;
and the acquisition module is used for acquiring a target image corresponding to the initial image according to the target position image.
According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program which, when executed, performs the above-described method.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above method through the computer program.
In the embodiment of the application, an image area corresponding to each object part in one or more object parts of a target object is detected from an initial image; determining a target image area from the image area corresponding to each object part, wherein a first quality parameter corresponding to a first object part shown in the target image area is lower than a first threshold value; generating a target portion image for the first target portion, wherein a second quality parameter of a second target portion shown in the target portion image is higher than or equal to the first threshold, and the second target portion and the first target portion belong to the same portion type; the method comprises the steps of detecting an image area corresponding to each of one or more target parts of a target object on an initial image according to a target part image, dividing the initial image into image areas corresponding to the target parts, determining a first target part which does not meet the requirement of a first threshold value, namely a first target part needing to be beautified, determining an image area corresponding to the first target part as a target image area, generating a target part image for showing the target part which belongs to the same type as the first target part, enabling a second quality parameter of a second target part shown in the target part image to meet the requirement of the first threshold value, beautifying the initial image by using the target part image to obtain an beautified target image, and adjusting parameters on the basis of the first target part on the premise that the first target part which does not meet the requirement in the beautifying process is not simply adjusted on the basis of the first target part The method achieves the purpose of beautifying, generates a new object position image meeting the requirements for the object position image, and uses the newly generated object position image to beautify the original image so as to achieve the purpose of beautifying, thereby realizing the technical effect of improving the beautifying effect when beautifying the object displayed in the image, and further solving the technical problem of poor beautifying effect when beautifying the object displayed in the image.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic diagram of a hardware environment of a method of generating an image according to an embodiment of the present application;
FIG. 2 is a flow chart of an alternative method of image generation according to an embodiment of the present application;
FIG. 3 is a first diagram illustrating division of a subject region according to an embodiment of the present application;
FIG. 4 is a second diagram illustrating object segmentation according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a method for beautifying a face image according to an alternative embodiment of the present application;
FIG. 6 is a schematic diagram of an alternative image generation apparatus according to an embodiment of the present application;
fig. 7 is a block diagram of a terminal according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of embodiments of the present application, there is provided an embodiment of a method of generation of an image.
Alternatively, in the present embodiment, the image generation method described above may be applied to a hardware environment constituted by the terminal 101 and the server 103 as shown in fig. 1. As shown in fig. 1, a server 103 is connected to a terminal 101 through a network, which may be used to provide services (such as game services, application services, etc.) for the terminal or a client installed on the terminal, and a database may be provided on the server or separately from the server for providing data storage services for the server 103, and the network includes but is not limited to: the terminal 101 is not limited to a PC, a mobile phone, a tablet computer, and the like. The image generation method according to the embodiment of the present application may be executed by the server 103, the terminal 101, or both the server 103 and the terminal 101. The terminal 101 executing the image generation method according to the embodiment of the present application may be executed by a client installed thereon.
Fig. 2 is a flowchart of an alternative image generation method according to an embodiment of the present application, and as shown in fig. 2, the method may include the following steps:
step S202, detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image;
step S204, determining a target image area from the image area corresponding to each target part, wherein a first quality parameter corresponding to a first target part shown in the target image area is lower than a first threshold value;
step S206, generating an object region image for the first object region, wherein a second quality parameter of a second object region shown in the object region image is higher than or equal to the first threshold, and the second object region and the first object region belong to the same region type;
step S208, a target image corresponding to the initial image is obtained according to the target position image.
Through the steps S202 to S208, an image area corresponding to each of one or more object parts of the target object is detected on the initial image, the initial image is divided into image areas corresponding to the object parts, a first object part which does not meet the requirement of a first threshold value, that is, a first object part which needs to be beautified, is determined from the image areas corresponding to the object parts, an object part image for showing the object part belonging to the same type as the first object part is generated for the first object part, so that a second quality parameter of a second object part shown in the object part image can meet the requirement of the first threshold value, the initial image is beautified by using the object part image to obtain an beautified target image, and it can be seen that parameter adjustment is simply performed on the basis of the first object part for the first object part which does not meet the requirement in the beautification process to achieve the aim of beautification, but a new object position image meeting the requirements is generated for the object position image, and the newly generated object position image is used for beautifying the original image to achieve the purpose of beautifying, thereby realizing the technical effect of improving the beautifying effect when the object displayed in the image is beautified, and further solving the technical problem of poor beautifying effect when the object displayed in the image is beautified.
In the technical solution provided in step S202, the target object may include, but is not limited to: human faces, animal faces, etc.
Optionally, in this embodiment, the one or more target parts are respective parts divided on the target object. Fig. 3 is a first schematic diagram of object region division according to an embodiment of the present application, and as shown in fig. 3, taking a face as an example, one or more object regions may include, but are not limited to: eyes, eyebrows, nose, mouth, teeth, cheeks, etc. Fig. 4 is a second schematic diagram of object region division according to an embodiment of the present application, and as shown in fig. 4, a target region having two symmetrical portions may also be subdivided, for example: the eyes are divided into left and right eyes, the eyebrows are divided into left and right eyebrows, and the mouth is divided into upper and lower lips.
Optionally, in this embodiment, the manner of detecting the image region corresponding to each of the one or more target portions of the target object from the initial image may be, but is not limited to, a manner of semantic segmentation or a conventional method. Semantic segmentation is a subclass of image segmentation, which is used to semantically segment each pixel in an image and label each pixel with a category. The conventional method refers to a non-deep learning method.
Optionally, in the present embodiment, the semantic segmentation may be implemented by, but is not limited to, a Deep Neural Network (DNN). When the hidden layer in the single-layer neural network is extended to multiple layers, the hidden layer is called a deep neural network. The input-to-output mapping task can be accomplished by a training process DNN. The input and output of the DNN are different for different tasks. For the image segmentation task, an image is input, and pixel regions contained in each object in the image are output. For image generation or image enhancement tasks, the input is a low-quality image and the output is a high-quality image.
Optionally, in this embodiment, the semantic segmentation process may use, but is not limited to, the depeplab v3+ network model of DNN, which is currently a better semantic segmentation network. The pre-training model of the network cannot segment each face part of the human face, and based on the network model, a data set (input is a human face image, and output is labels and corresponding pixel regions of each face part) is configured, so that a network capable of achieving segmentation of the face part of the human face is trained.
Optionally, in this embodiment, the acquired initial image may be, but is not limited to, an image with a resolution meeting a certain standard, such as: setting a resolution threshold, if the resolution of the image uploaded by a user is lower than the resolution threshold, performing up-sampling processing on the image, adjusting the resolution of the image to the resolution threshold, and taking the adjusted image as an initial image. And if the resolution of the image uploaded by the user is higher than or equal to the resolution threshold, directly taking the image uploaded by the user as an initial image.
Alternatively, in this embodiment, the upsampling manner may be, but is not limited to, a Bicubic (Bicubic) method, a Bilinear interpolation method, or the like.
In the technical solution provided in step S204, the first quality parameter is a quality parameter corresponding to a first object portion displayed in the target image region. The quality parameter can be used to measure the quality of the target region. The determination of the quality parameter may also be realized by a deep neural network. The quality parameter can be, but is not limited to, considered as a score for the target object, and the score is high or low to indicate the quality of the target object, such as: taking the target object as a face as an example, the first quality parameter may be, but is not limited to, a color value indicating the face, and a higher score indicates a higher color value. If the first quality parameter is lower than the first threshold, it indicates that the color value of the target object does not reach the preset first threshold, and the target object with a lower color value may be considered.
In the solution provided in step S206, the target site image may be generated for the first target site using, but not limited to, a network structure for generating a high quality image. Such as: a StyleGAN network structure or other similar image enhancement or image generation network is employed.
Alternatively, in the present embodiment, the second quality parameter of the second object region shown in the object region image is higher than or equal to the first threshold, and therefore, it can be seen that the object region image is an image with a higher quality than the target image area.
Optionally, in this embodiment, the second object region and the first object region belong to the same region type, such as: if the first object portion is a mouth, then the second object portion is also a mouth. If the first object portion is the upper lip, then the second object portion is also the upper lip.
In the technical solution provided in step S208, the object position image may be added to the initial image by means of, but not limited to, mapping, matting, replacing, and the like, so as to obtain a beautified target image.
Optionally, in this embodiment, if the initial image is obtained by up-sampling an image uploaded by a user, after the target image is obtained, the target image may also be correspondingly down-sampled, so as to obtain a final image, and the final image is returned to the user.
Optionally, in this embodiment, the down-sampling mode and the up-sampling mode are corresponding, but the method may also be, but is not limited to, Bicubic interpolation method, Bilinear interpolation method, or the like. If the Bicubic interpolation method is adopted for the upsampling, the Bicubic interpolation method is also adopted for the downsampling. If the Bilinear interpolation method is adopted for the upsampling, the Bilinear interpolation method is also adopted for the downsampling.
As an alternative embodiment, generating an object region image for the first object region includes:
s11, carrying out fuzzy processing on the target image area to obtain a target fuzzy image;
and S12, generating the target part image corresponding to the target blurred image through a target image generation model, wherein the target image generation model is obtained by training an initial image generation model through an image area sample labeled with a part image sample, the quality parameter sample of the target part shown in the part image sample is higher than or equal to a second threshold value, and the image area sample is obtained by blurring the part image sample.
Optionally, in this embodiment, the target image generation model is obtained by training the initial image generation model using an image area sample labeled with a part image sample, the quality parameter sample of the target part shown in the part image sample is higher than or equal to a second threshold, and the image area sample is obtained by performing a blurring process on the part image sample. Since the target image generation model is obtained by training the initial image generation model using the blurred image, the target image region is blurred to obtain a target blurred image before the image is input to the target image generation model, and the target blurred image is input to the target image generation model.
Optionally, in the present embodiment, the manner of gaussian processing may include, but is not limited to, gaussian blurring, and other blurring methods.
As an alternative embodiment, generating the target portion image corresponding to the target blurred image through the target image generation model includes:
s21, inputting the target blurred image into a target generation layer of the target image generation model, wherein the target image generation model includes the target generation layer and a target countermeasure layer;
and S22, acquiring the target part image output by the target generation layer.
Optionally, in this embodiment, the target image generation model may include, but is not limited to, a target generation layer for generating an image and a target countermeasure layer for performing countermeasure training. Such as: the target image generation model may be, but is not limited to, a network structure using GANs (Generative adaptive Networks). The target image generation model may be, but is not limited to, a network structure using StyleGAN, a network structure of ProGAN, and the like.
As an alternative embodiment, before inputting the target blurred image into the generation layer of the target image generation model, the method further includes:
s31, fixing first model parameters of an initial generation layer, and training the initial countermeasure layer by using image area samples marked with position image samples to obtain the target countermeasure layer, wherein the initial image generation model comprises the initial generation layer and the initial countermeasure layer;
and S32, fixing the second model parameters of the target confrontation layer, and training the initial generation layer by using the image area sample marked with the position image sample to obtain the target generation layer.
Alternatively, in this embodiment, before inputting the target blurred image into the generation layer of the target image generation model, the target generation layer and the target confrontation layer may be trained separately, so as to obtain the target image generation model. The first model parameter of the initial generation layer may be first fixed, the initial generation layer may be trained using a training set to obtain the target generation layer, and then the second model parameter of the target generation layer may be fixed, and the initial generation layer may be trained using the training set to obtain the target generation layer.
Optionally, in this embodiment, the StyleGAN network itself is a pure face generation network, the input is a set of latent codes (a string of random numbers), and the output is a high-definition face image. In this embodiment, the network of the StyleGAN is modified, and the input latent code is changed into a blurred image, so that the trained network of the StyleGAN can well complete the function of producing a high-definition image from a blurred image area in the image. In order to achieve the global beautification effect of the target object, when a training data set is generated, high-definition and high-quality image samples with high scores are selected as output images in a targeted mode, and then target parts on the image samples are subjected to fuzzy processing randomly to serve as input images in the training data set.
As an alternative embodiment, the blurring processing on the target image area to obtain the target blurred image includes one of the following steps:
s41, blurring the target image area on the initial image to obtain the target blurred image;
s42, extracting the target image area from the initial image to obtain an area image; and carrying out fuzzy processing on the area image to obtain the target fuzzy image.
Optionally, in this embodiment, the entire initial image after the target image area is blurred may be generated and beautified, or the target image area may be separately extracted for image generation and beautification, and then merged into the initial image.
As an alternative embodiment, acquiring the target image corresponding to the initial image according to the target region image includes:
s51, determining the target image as the target image when the target blurred image is obtained by blurring the target image area on the initial image;
s52, when the region image is blurred to obtain the target blurred image, replacing the target image region in the initial image with the target region image to obtain the target image.
Optionally, in this embodiment, if the target image area on the initial image is blurred to obtain the target blurred image, the target portion image output by the target image generation model is the final result of generating and beautifying the initial image, and if the area image is blurred to obtain the target blurred image, the target image area on the initial image is replaced with the target portion image to obtain the target image, where the target portion image only showing the second target portion is output by the target image generation model.
Optionally, in this embodiment, replacing the target image region on the initial image with the target part image may be, but is not limited to, a manner of pasting and fusing the target part image in the target image region after performing a blurring process on the target image region on the initial image, or a manner of matting and stitching may also be used, for example: and (4) scratching the target image region from the initial image, and splicing the image of the object part to the scratched image.
As an alternative embodiment, the determining the target image area from the image area corresponding to each target portion includes:
s61, acquiring quality parameters corresponding to each object part through a target object detection model, wherein the target object detection model is obtained by training an initial object detection model through object part samples marked with quality parameter samples;
s62, acquiring a target region with a quality parameter lower than the first threshold value from the one or more target regions as the first target region;
and S63, determining the area of the first object part on the initial image as the target image area.
Optionally, in this embodiment, each detected target region is evaluated by a deep learning algorithm, so as to obtain a quality parameter (a value range interval may be 0 to 100 minutes) of each target region. A higher quality parameter indicates a higher quality. And comparing the quality parameter of each target part with a first threshold value to obtain a target part with the quality parameter lower than the first threshold value as a first target part, and determining the area of the first target part on the initial image as a target image area.
The present application further provides an alternative embodiment, which provides a face image beautification method based on deep learning, fig. 5 is a schematic diagram of a face image beautification method according to an alternative embodiment of the present application, as shown in fig. 5, the method includes the following steps:
step S502, the face image is sampled, the face image (equivalent to the initial image) uploaded by the user is received, and certain requirements are made on the resolution of the input image in order to better complete the subsequent functions of face component (equivalent to the object part) detection and face generation and beautification. Thus, for images with a resolution less than 128 x 128, the up-sampling is first performed to a resolution of 128 x 128, while for face images with a resolution greater than 128 x 128, no processing is performed. If the input image is not a square image, it may be up-sampled to a minimum of 128 pixels. The upsampling method may use Bicubic (Bicubic).
Step S504, detecting the face components, detecting and segmenting each face component in the face image, including eyebrows (left and right), eyes (left and right), nose, lips (up and down), teeth, cheeks, etc. The detection method is a deep learning method, depeplab v3+ of DNN is used, the pre-training model of the network itself cannot segment the components of the face, in this optional embodiment, based on the network model, the prepared data set input parameters for model training are face images, the output parameters are the label of each face component and the corresponding pixel region, and a network capable of achieving face component segmentation is used in the training.
It should be noted that the face segmentation network segments the hair region at the same time, and in this optional embodiment, the hair region may be processed or not processed.
Step S506, evaluating the face components, and evaluating each detected face component by using a deep learning algorithm to obtain the score (the value domain interval is 0-100) of each component.
Step S508, performing fuzzy processing on the component areas with the scores lower than the threshold, setting the threshold to be 50 in all the areas at present, and marking the components with the scores lower than the threshold. For three paired components, eyebrow, eye, lip, if the score of either left/right or up/down component is below a threshold, then both pairs of components are marked (e.g., if the left eye score is below a threshold, then both eyes are marked). And blurring all marked components in the original face image. If all component scores are above the threshold, the artwork is not processed.
In this alternative embodiment, the blurring method may be, but is not limited to, gaussian blurring, where the blurring strength is determined according to the blurring degree of the original input face image, and the more blurred the original image, the lower the strength used in blurring the marking component. Because the original blurred image can not be blurred with high intensity, otherwise, the image information is lost too much, which is not beneficial to the generation and beautification of the later image, and the blurring intensity can be increased properly for the original clearer image.
And S510, generating and beautifying a face image, and performing detail generation and face beautification on the blurred image by using the modified StyleGAN network model for the whole image blurred in the S508. The StyleGAN itself is a pure face generation network, the input is a set of latent codes (a string of random numbers), and the output is a high-definition face image.
In the optional embodiment, the network of the StyleGAN is modified, and the input is changed from the latent code to an image, so that the function of producing a high-definition component from a fuzzy face component in the image can be well completed. Meanwhile, in order to achieve the global beautification effect of the human face, when a training data set is generated, high-definition high-quality human face images with high scores are selected as output images in a targeted mode, and then face components on the human face images are subjected to fuzzy processing randomly to serve as input images in the training data set.
In step S512, the image resolution is restored, and for the image with the resolution less than 128 × 128, since the up-sampling operation is performed at the time of the processing just started to change the resolution, the down-sampling operation may be performed before the output so that the resolution of the output image matches the input image. The specific down-sampling method may be Bicubic (Bicubic) as in the up-sampling.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
According to another aspect of the embodiment of the present application, there is also provided an image generation apparatus for implementing the image generation method. Fig. 6 is a schematic diagram of an alternative image generation apparatus according to an embodiment of the present application, and as shown in fig. 6, the apparatus may include:
a detection module 62, configured to detect, from the initial image, an image region corresponding to each of one or more object portions of the target object;
a determining module 64, configured to determine a target image area from the image areas corresponding to each of the object portions, where a first quality parameter corresponding to a first object portion shown in the target image area is lower than a first threshold;
a generating module 66, configured to generate an object region image for the first object region, wherein a second quality parameter of a second object region shown in the object region image is higher than or equal to the first threshold, and the second object region and the first object region belong to the same region type;
an obtaining module 68, configured to obtain a target image corresponding to the initial image according to the target position image.
It should be noted that the detecting module 62 in this embodiment may be configured to execute the step S202 in this embodiment, the determining module 64 in this embodiment may be configured to execute the step S204 in this embodiment, the generating module 66 in this embodiment may be configured to execute the step S206 in this embodiment, and the obtaining module 68 in this embodiment may be configured to execute the step S208 in this embodiment.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may operate in a hardware environment as shown in fig. 1, and may be implemented by software or hardware.
Through the module, the initial image is divided into the image areas corresponding to the object parts by detecting the image area corresponding to each object part in one or more object parts of the target object on the initial image, the first object part which does not meet the requirement of a first threshold value, namely the first object part which needs to be beautified, is determined from the image areas corresponding to the object parts, the object part image used for showing the object part belonging to the same type as the first object part is generated for the first object part, so that the second quality parameter of the second object part shown in the object part image can meet the requirement of the first threshold value, the beautification is carried out by using the initial image of the object part to obtain the beautified target image, and the aim of beautifying is achieved by simply adjusting the parameter on the basis of the first object part not for the first object part which does not meet the requirement in the beautification process, but a new object position image meeting the requirements is generated for the object position image, and the newly generated object position image is used for beautifying the original image to achieve the purpose of beautifying, thereby realizing the technical effect of improving the beautifying effect when the object displayed in the image is beautified, and further solving the technical problem of poor beautifying effect when the object displayed in the image is beautified.
As an alternative embodiment, the generating module includes:
the processing unit is used for carrying out fuzzy processing on the target image area to obtain a target fuzzy image;
and the generating unit is used for generating the target part image corresponding to the target blurred image through a target image generation model, wherein the target image generation model is obtained by training an initial image generation model through an image area sample marked with a part image sample, the quality parameter sample of the target part shown in the part image sample is higher than or equal to a second threshold value, and the image area sample is obtained by blurring the part image sample.
As an alternative embodiment, the generating unit is configured to:
inputting the target blurred image into a target generation layer of the target image generation model, wherein the target image generation model includes the target generation layer and a target countermeasure layer;
and acquiring the target part image output by the target generation layer.
As an alternative embodiment, the apparatus further comprises:
a first training module, configured to fix first model parameters of an initial generation layer before the target blurred image is input into a target generation layer of the target image generation model, and train the initial countermeasure layer using an image area sample to which a position image sample is labeled, so as to obtain the target countermeasure layer, where the initial image generation model includes the initial generation layer and the initial countermeasure layer;
and the second training module is used for fixing second model parameters of the target confrontation layer and training the initial generation layer by using the image area sample marked with the position image sample to obtain the target generation layer.
As an alternative embodiment, the processing unit is adapted to one of:
blurring the target image area on the initial image to obtain a target blurred image;
extracting the target image area from the initial image to obtain an area image; and carrying out fuzzy processing on the area image to obtain the target fuzzy image.
As an alternative embodiment, the obtaining module includes:
a first determining unit, configured to determine the target area image as the target image when the target blurred image is obtained by performing blurring processing on the target image area on the initial image;
and the replacing unit is used for replacing the target image area on the initial image with the target part image to obtain the target image under the condition that the area image is subjected to blurring processing to obtain the target blurred image.
As an alternative embodiment, the determining module includes:
a first obtaining unit, configured to obtain a quality parameter corresponding to each target portion through a target object detection model, where the target object detection model is obtained by training an initial object detection model using a target portion sample labeled with a quality parameter sample;
a second acquisition unit configured to acquire, as the first object region, an object region whose quality parameter is lower than the first threshold value from among the one or more object regions;
and the second determining unit is used for determining the area of the first object part on the initial image as the target image area.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be operated in a hardware environment as shown in fig. 1, and may be implemented by software, or may be implemented by hardware, where the hardware environment includes a network environment.
According to another aspect of the embodiments of the present application, there is also provided an electronic apparatus, such as a server or a terminal, for implementing the image generation method.
Fig. 7 is a block diagram of a terminal according to an embodiment of the present application, and as shown in fig. 7, the terminal may include: one or more processors 701 (only one of which is shown), a memory 703, and a transmission means 705. as shown in fig. 7, the terminal may further include an input-output device 207.
The memory 703 may be used to store software programs and modules, such as program instructions/modules corresponding to the image generation method and apparatus in the embodiment of the present application, and the processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 203, that is, implements the image generation method described above. The memory 703 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, the memory 703 may further include memory located remotely from the processor 701, which may be connected to the terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 705 is used for receiving or transmitting data via a network, and may also be used for data transmission between a processor and a memory. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 705 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 705 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
Among other things, the memory 703 is used to store application programs.
The processor 701 may call the application program stored in the memory 703 through the transmission means 705 to perform the following steps:
detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image;
determining a target image area from the image area corresponding to each object part, wherein a first quality parameter corresponding to a first object part shown in the target image area is lower than a first threshold value;
generating a target part image for the first target part, wherein a second quality parameter of a second target part shown in the target part image is higher than or equal to the first threshold, and the second target part and the first target part belong to the same part type;
and acquiring a target image corresponding to the initial image according to the target position image.
By adopting the embodiment of the application, a scheme for generating the image is provided. The method comprises the steps of detecting an image area corresponding to each object part in one or more object parts of a target object on an initial image, dividing the initial image into image areas corresponding to the object parts, determining a first object part which does not meet the requirement of a first threshold value, namely a first object part which needs to be beautified, determining the corresponding image area as a target image area, generating an object part image for showing the object part belonging to the same type as the first object part for the first object part, enabling a second quality parameter of a second object part shown in the object part image to meet the requirement of the first threshold value, beautifying the initial image by using the object part image to obtain an beautified target image, and achieving the aim of simply adjusting the parameter on the basis of the first object part when the first object part which does not meet the requirement in the beautifying process, but a new object position image meeting the requirements is generated for the object position image, and the newly generated object position image is used for beautifying the original image to achieve the purpose of beautifying, thereby realizing the technical effect of improving the beautifying effect when the object displayed in the image is beautified, and further solving the technical problem of poor beautifying effect when the object displayed in the image is beautified.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.
It can be understood by those skilled in the art that the structure shown in fig. 7 is only an illustration, and the terminal may be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. Fig. 7 is a diagram illustrating a structure of the electronic device. For example, the terminal may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 7, or have a different configuration than shown in FIG. 7.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
Embodiments of the present application also provide a storage medium. Alternatively, in the present embodiment, the storage medium described above may be used for a program code for executing the image generating method.
Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:
detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image;
determining a target image area from the image area corresponding to each object part, wherein a first quality parameter corresponding to a first object part shown in the target image area is lower than a first threshold value;
generating a target part image for the first target part, wherein a second quality parameter of a second target part shown in the target part image is higher than or equal to the first threshold, and the second target part and the first target part belong to the same part type;
and acquiring a target image corresponding to the initial image according to the target position image.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.
Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (10)

1. A method of generating an image, comprising:
detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image;
determining a target image area from the image area corresponding to each object part, wherein a first quality parameter corresponding to a first object part shown in the target image area is lower than a first threshold value;
generating a target part image for the first target part, wherein a second quality parameter of a second target part shown in the target part image is higher than or equal to the first threshold, and the second target part and the first target part belong to the same part type;
and acquiring a target image corresponding to the initial image according to the target position image.
2. The method of claim 1, wherein generating an object site image for the first object site comprises:
carrying out fuzzy processing on the target image area to obtain a target fuzzy image;
and generating the target part image corresponding to the target blurred image through a target image generation model, wherein the target image generation model is obtained by training an initial image generation model through an image area sample marked with a part image sample, the quality parameter sample of the target part shown in the part image sample is higher than or equal to a second threshold value, and the image area sample is obtained by blurring the part image sample.
3. The method of claim 2, wherein generating the object region image corresponding to the target blurred image through a target image generation model comprises:
inputting the target blurred image into a target generation layer of the target image generation model, wherein the target image generation model includes the target generation layer and a target countermeasure layer;
and acquiring the target part image output by the target generation layer.
4. The method of claim 3, wherein prior to inputting the target blurred image into the generation layer of the target image generation model, the method further comprises:
fixing first model parameters of an initial generation layer, and training the initial countermeasure layer by using image area samples marked with position image samples to obtain the target countermeasure layer, wherein the initial image generation model comprises the initial generation layer and the initial countermeasure layer;
and fixing the second model parameters of the target confrontation layer, and training the initial generation layer by using the image area sample marked with the position image sample to obtain the target generation layer.
5. The method of claim 2, wherein blurring the target image region to obtain a target blurred image comprises one of:
blurring the target image area on the initial image to obtain a target blurred image;
extracting the target image area from the initial image to obtain an area image; and carrying out fuzzy processing on the area image to obtain the target fuzzy image.
6. The method of claim 5, wherein obtaining the target image corresponding to the initial image according to the target portion image comprises:
determining the target image as the target image under the condition that the target blurred image is obtained by blurring the target image area on the initial image;
and replacing the target image area on the initial image with the target part image to obtain the target image under the condition that the area image is subjected to blurring processing to obtain the target blurred image.
7. The method of claim 1, wherein determining a target image region from the image region corresponding to each object region comprises:
acquiring a quality parameter corresponding to each object part through a target object detection model, wherein the target object detection model is obtained by training an initial object detection model by using an object part sample marked with a quality parameter sample;
acquiring a target part with a quality parameter lower than the first threshold value from the one or more target parts as the first target part;
and determining the area of the first object part on the initial image as the target image area.
8. An image generation apparatus, comprising:
the detection module is used for detecting an image area corresponding to each object part in one or more object parts of the target object from the initial image;
a determining module, configured to determine a target image area from image areas corresponding to each of the object portions, where a first quality parameter corresponding to a first object portion shown in the target image area is lower than a first threshold;
a generation module configured to generate an object region image for the first object region, wherein a second quality parameter of a second object region shown in the object region image is higher than or equal to the first threshold, and the second object region and the first object region belong to the same region type;
and the acquisition module is used for acquiring a target image corresponding to the initial image according to the target position image.
9. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method of any of the preceding claims 1 to 7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any of the preceding claims 1 to 7 by means of the computer program.
CN202010618887.6A 2020-06-30 2020-06-30 Image generation method and device Pending CN111815504A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010618887.6A CN111815504A (en) 2020-06-30 2020-06-30 Image generation method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010618887.6A CN111815504A (en) 2020-06-30 2020-06-30 Image generation method and device

Publications (1)

Publication Number Publication Date
CN111815504A true CN111815504A (en) 2020-10-23

Family

ID=72855675

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010618887.6A Pending CN111815504A (en) 2020-06-30 2020-06-30 Image generation method and device

Country Status (1)

Country Link
CN (1) CN111815504A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344812A (en) * 2021-05-31 2021-09-03 维沃移动通信(杭州)有限公司 Image processing method and device and electronic equipment

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109218695A (en) * 2017-06-30 2019-01-15 中国电信股份有限公司 Video image enhancing method, device, analysis system and storage medium
CN110008817A (en) * 2019-01-29 2019-07-12 北京奇艺世纪科技有限公司 Model training, image processing method, device, electronic equipment and computer readable storage medium
CN110298785A (en) * 2019-06-29 2019-10-01 北京字节跳动网络技术有限公司 Image beautification method, device and electronic equipment
CN110782413A (en) * 2019-10-30 2020-02-11 北京金山云网络技术有限公司 Image processing method, device, equipment and storage medium
CN110796593A (en) * 2019-10-15 2020-02-14 腾讯科技(深圳)有限公司 Image processing method, device, medium and electronic equipment based on artificial intelligence
CN111105375A (en) * 2019-12-17 2020-05-05 北京金山云网络技术有限公司 Image generation method, model training method and device thereof, and electronic equipment
WO2020114421A1 (en) * 2018-12-05 2020-06-11 华中科技大学 Object detection model training method and target object detection method
CN111340722A (en) * 2020-02-20 2020-06-26 Oppo广东移动通信有限公司 Image processing method, processing device, terminal device and readable storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109218695A (en) * 2017-06-30 2019-01-15 中国电信股份有限公司 Video image enhancing method, device, analysis system and storage medium
WO2020114421A1 (en) * 2018-12-05 2020-06-11 华中科技大学 Object detection model training method and target object detection method
CN110008817A (en) * 2019-01-29 2019-07-12 北京奇艺世纪科技有限公司 Model training, image processing method, device, electronic equipment and computer readable storage medium
CN110298785A (en) * 2019-06-29 2019-10-01 北京字节跳动网络技术有限公司 Image beautification method, device and electronic equipment
CN110796593A (en) * 2019-10-15 2020-02-14 腾讯科技(深圳)有限公司 Image processing method, device, medium and electronic equipment based on artificial intelligence
CN110782413A (en) * 2019-10-30 2020-02-11 北京金山云网络技术有限公司 Image processing method, device, equipment and storage medium
CN111105375A (en) * 2019-12-17 2020-05-05 北京金山云网络技术有限公司 Image generation method, model training method and device thereof, and electronic equipment
CN111340722A (en) * 2020-02-20 2020-06-26 Oppo广东移动通信有限公司 Image processing method, processing device, terminal device and readable storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
张俊升;徐晶晶;余伟;: "面部美化图像质量无参考评价方法", 计算机应用, no. 04, 10 April 2020 (2020-04-10) *
许若波;卢涛;王宇;张彦铎;: "基于组合学习的人脸超分辨率算法", 计算机应用, no. 03, 10 March 2020 (2020-03-10) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344812A (en) * 2021-05-31 2021-09-03 维沃移动通信(杭州)有限公司 Image processing method and device and electronic equipment

Similar Documents

Publication Publication Date Title
Ren et al. Low-light image enhancement via a deep hybrid network
JP7413400B2 (en) Skin quality measurement method, skin quality classification method, skin quality measurement device, electronic equipment and storage medium
CN106682632B (en) Method and device for processing face image
US11657479B2 (en) Deep feature generative adversarial neural networks
US10198801B2 (en) Image enhancement using self-examples and external examples
CN109191414A (en) A kind of image processing method, device, electronic equipment and storage medium
Panetta et al. Tmo-net: A parameter-free tone mapping operator using generative adversarial network, and performance benchmarking on large scale hdr dataset
Xiao et al. Single image dehazing based on learning of haze layers
CN111008935B (en) Face image enhancement method, device, system and storage medium
US11521299B2 (en) Retouching digital images utilizing separate deep-learning neural networks
CN111145311B (en) Multi-attribute editing method, system and device for high-resolution face image
CN108288064B (en) Method and device for generating pictures
Guo et al. Image completion using structure and texture GAN network
CN107609487B (en) User head portrait generation method and device
US11670031B2 (en) System and method for automatically generating an avatar with pronounced features
CN109241930B (en) Method and apparatus for processing eyebrow image
CN111815504A (en) Image generation method and device
CN111488887B (en) Image processing method and device based on artificial intelligence
KR20230028481A (en) High Resolution Controllable Face Aging Using Spatial Aware Conditional GANs
CN117095019A (en) Image segmentation method and related device
CN115100334B (en) Image edge tracing and image animation method, device and storage medium
CN110766631A (en) Face image modification method and device, electronic equipment and computer readable medium
CN112712571B (en) Object plane mapping method, device and equipment based on video
CN117197855A (en) Face key point labeling method and device, electronic equipment and storage medium
CN113223128B (en) Method and apparatus for generating image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination