CN111553854A

CN111553854A - Image processing method and electronic equipment

Info

Publication number: CN111553854A
Application number: CN202010315907.2A
Authority: CN
Inventors: 李红
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2020-04-21
Filing date: 2020-04-21
Publication date: 2020-08-18

Abstract

The embodiment of the invention discloses an image processing method and electronic equipment, wherein the method comprises the following steps: receiving a first input of a user to the image processing interface under the condition that a first image is displayed on the image processing interface; responding to the first input, and processing the first image by generating a confrontation network model to obtain a second image; the generation of the confrontation network model comprises a generator and an arbiter. The problems of large image distortion and lack of reality after image processing are solved.

Description

Image processing method and electronic equipment

Technical Field

The embodiment of the invention relates to the technical field of image processing, in particular to an image processing method and electronic equipment.

Background

With the development of electronic devices and mobile internet, users have become an indispensable part of daily life to use electronic devices to take portraits.

At present, the correction effect of the image processing function applied to the electronic equipment is not good, for example, the distortion is large, the repair result is not natural, and the skin texture is not real. Users often want the image after the beauty treatment to be still natural and real, and people can hardly see the beauty treatment trace. The current technology can not meet the requirements of users on the real natural feeling of the images after the beauty correction. Therefore, how to maintain the reality of the image after being beautified becomes a problem to be solved.

Disclosure of Invention

The embodiment of the invention provides an image processing method and electronic equipment, and aims to solve the problems that in the related art, an image processed by an image is large in distortion and lacks of sense of reality.

In order to solve the technical problem, the invention is realized as follows:

in a first aspect, an embodiment of the present invention provides an image processing method, which may specifically include:

receiving a first input of a user under the condition that a first image is displayed on an image processing interface;

responding to the first input, and processing the first image by generating a confrontation network model to obtain a second image; the generation of the confrontation network model comprises a generator and an arbiter.

In a second aspect, an embodiment of the present invention provides an electronic device, which may specifically include:

the receiving module is used for receiving a first input of a user under the condition that a first image is displayed on the image processing interface;

the processing module is used for responding to the first input, processing the first image by generating a confrontation network model to obtain a second image; the generation of the confrontation network model comprises a generator and an arbiter.

In a third aspect, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and when executed by the processor, the computer program implements the image processing method according to the first aspect.

In a fourth aspect, there is provided a computer-readable storage medium having stored thereon a computer program for causing a computer to execute the image processing method according to the first aspect if the computer program is executed in the computer.

In the embodiment of the invention, in response to the image processing operation of the user on the first image, the image processing is carried out on the first image by utilizing the generation countermeasure network model to obtain the processed second image. The trained generator for generating the countermeasure network can generate an almost 'false-to-false' picture, so that the discriminator for generating the countermeasure network is difficult to judge whether the picture generated by the generator is real or not, the reality of the image after the beautifying processing can be greatly improved, and therefore, the image processing model is used for processing the first image, the problems that the difference between the processed image and the reality is large and the reality lacks when the user processes the image are solved, the natural and real image (for example, the texture of a portrait is reserved) can be obtained, and the use experience of the user is improved.

Drawings

The present invention will be better understood from the following description of specific embodiments thereof taken in conjunction with the accompanying drawings, in which like or similar reference characters designate like or similar features.

Fig. 1 is a schematic diagram of training for generating a countermeasure network according to an embodiment of the present invention;

FIG. 2 is a training flow diagram for generating a confrontation network model according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of an image processing method according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating another image processing method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of an image processing interface according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating another image processing method according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention;

fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Embodiments of the present invention provide an image processing method, an image processing apparatus, an electronic device, and a storage medium, to solve the problems of large distortion and lack of reality of an image processed in the related art.

In recent years, with the increasing popularity of electronic devices, more and more people spend more time taking photos and sharing, and a large part of the photos belong to portrait photos. Many people have blemishes on their faces due to the different skin qualities of each person, and it is becoming increasingly urgent for this group of users to remove these blemishes while preserving the realism of the image, which has also led to the development of many algorithms for implementing image processing. Many electronic devices have their own image beautifying function, and there are also many application programs installed in the electronic devices to implement the image beautifying function.

In the current image beautifying processing process, on one hand, the online real-time beautifying is related, the image processing is to grind skins and the like by using the traditional filtering technology, and as the real-time processing speed needs to be reached, the processing of an image processing algorithm is mainly based on global operation, so that the processed portrait photo basically has no texture characteristics, has larger distortion, and can be judged as two people before and after the photo is taken. On the other hand, regarding the post-shooting beauty treatment, such image processing uses various application programs to perform beauty treatment on the post-shooting photo, and generally, a search method is adopted for repairing the skin based on the traditional image filtering algorithm, so that the repairing result is not real, and the skin texture and the surrounding area have great difference.

That is, the current image processing methods all cause the processed image to be distorted greatly. Moreover, the current image processing method cannot enable a user to perform customized modification. For some users, they want to retain facial features, such as nevi, at specific locations, and current methods cannot retain such facial features for the user.

At present, the distortion of the processed image is large, mainly because the identity recognition is less considered in the algorithm for image processing, so that after the user uses the beautifying function to process the image with the portrait, the processed image and the image target before processing are difficult to be identified as the same person, namely, the processed photo has insufficient reality and poor naturalness, and the user experience is poor. To solve the problem, the invention proposes training to generate a confrontation network model based on the identity preservation of multi-dimensional features, and considers several aspects influencing the identity of the person to limit the identity of the person in the image before and after beauty so that the person looks like the same person before and after treatment.

Several dimensions are considered that are related to identity recognition: skin color, pores, texture, facial form, proportion of five sense organs, etc. The following are presented in order:

firstly, skin color: skin tones cannot be too far apart after image processing. For example, a yellow race cannot be beautified as a white race.

Second, pores: the pores can not completely disappear after the image processing, and the current image processing algorithm can not keep the pores basically, so that the human face looks very smooth and very unreal.

Thirdly, texture: the texture occupies a very important proportion in the authenticity of the face, different users have different facial textures, and the textures before and after beautifying should be kept as much as possible

Fourth, facial form: the shape of the face is also important for identifying people, for example, most people cannot accept changing a Chinese face into a melon seed face. Therefore, the face shape of the user is kept as much as possible for the user who is more concerned about the authenticity.

Fifth, proportion of five sense organs: the human face is composed of a plurality of parts, and when the face is beautified, the harmony between each part and the whole human face is kept as much as possible.

With the rapid development of artificial intelligence technology in recent years and the improvement of computer performance, processing by artificial intelligence in the field of image processing is becoming a popular trend. The applicant finds that by using a model trained by a Generated Antagonistic Network (GAN) to perform image processing, the generated image is more real, the texture of the human facial skin is more real, and the display effect of the human facial skin is closer to the display effect of a real human face.

The generation of the countermeasure network includes a generator and an arbiter. In the training process of generating the countermeasure network, the aim of the generator is to generate a real picture as much as possible to deceive the discriminator. The goal of the discriminator is to separate the picture generated by the generator from the actual picture as much as possible. Under the optimal state of the trained generation countermeasure network, the generator can generate enough pictures which are 'false-like', and for the discriminator, the discriminator is difficult to judge whether the pictures generated by the generator are true or not, so that the sense of reality of the beautified images can be greatly improved.

The applicant finds that the generation of the confrontation network model based on the GAN training can basically ensure that the processed image keeps the reality in the aspects of skin color, pores, textures, facial shapes and five-sense organ proportion in the process of being applied to image processing.

By taking fig. 1 as an example, the GAN is briefly introduced, and it is assumed that each set of training samples includes a portrait photo before beauty, a portrait photo after speckle reduction, and a portrait photo after skin grinding. In the training process of generating the confrontation network, inputting the portrait photos before beauty into the generator, generating a generated image based on the portrait photos before beauty by the generator, determining confrontation loss by the discriminator based on the generated image generated by the generator and the portrait pictures after speckle removal and acne removal or the portrait pictures after buffing, and performing confrontation training between the generator and the discriminator by the GAN according to the confrontation loss until a preset training stop condition is met to obtain a trained generated confrontation network model. The trained sense of reality of the generated image for generating the confrontation network model basically enables people not to see processing traces, and the requirements of users on the sense of reality of the image can be met.

The generation of the countermeasure network model involved in the image processing method provided by the embodiment of the present invention is described below with reference to fig. 2.

Fig. 2 is a training flowchart for generating a confrontation network model according to an embodiment of the present invention.

Generating the confrontation network model comprises a generator model and a discriminator model, and the training method can comprise S210-S260, which is specifically as follows:

s210, acquiring a training sample set; the training sample set includes a plurality of training samples, each training sample including an original image and a target label image.

Wherein the training sample comprises an unlabeled original image and an already labeled (e.g., manually labeled) target label image.

The target label image may be an image with a defect area removed from the original image.

Specifically, in order to make the target label image more natural and closer to the skin of the real person, the original image may be manually subjected to image processing (e.g., using PS to remove a defective region of the original image) to obtain the target label image. For example, the above mentioned target label image may include: the portrait picture after removing the speckles and the pox, the portrait picture after grinding the skin, the image after generating the beautiful nevus, and the like.

In one embodiment, the training sample set contains 3 x M sets of image pairs, each set containing an original image, a first type of artifact corrected image, and a second type of artifact corrected image, where M represents M different users. And meanwhile, carrying out data preprocessing on the original image in the training sample set by the training sample set to obtain a variola region binary mask image, namely, the final model input comprises the original image, the first type flaw region binary mask image, the first type flaw corrected image and the first type flaw corrected image.

In the above step related to performing data preprocessing on the original image in the training sample set to obtain the two-value mask map of the speckle-pox region, the method specifically includes:

carrying out graying processing on the original image to obtain a grayed original image; carrying out binarization processing on the original image subjected to the graying processing to obtain an original image subjected to binarization processing; extracting and calculating the original image after binarization processing, screening out a preset type defect area, and performing pixel filling on the screened out preset type defect area to enable the pixel value of the preset type defect area to be 1 and the pixel values of the rest background areas to be 0, so that a preset type defect area binary mask image can be obtained. Next, training to generate an antagonistic network model can be performed. That is, in order to improve the training effect, the training sample may include an original image and a preset type defect region binary mask map.

For each training sample, the following steps are respectively performed:

and S220, inputting the original image into the generator model to obtain a corrected image.

The generator is used for generating a new image, namely a corrected image according to the original image. In the training process of generating the countermeasure network, the generator in the countermeasure network aims to generate a real picture as much as possible to pass the detection of the discriminator.

Illustratively, the pre-beauty photo (original image) is input to the generator, which generates a generated image (corrected image) based on the pre-beauty photo.

And S230, inputting the corrected image and the target label image into a discriminator model, and determining a countermeasure loss value between the corrected image and the target label image.

Wherein the discriminator determines the countermeasure loss value based on the generated image (corrected image) generated by the generator and the image (target label image) from which the defective region in the original image is eliminated. In the training process of generating the countermeasure network, the discriminator aims to distinguish the picture generated by the generator from the real picture as much as possible, and to determine the difference (i.e., the countermeasure loss value) between the picture generated by the generator and the real picture (target label image).

The step of obtaining the countermeasure loss value between the corrected image and the target label image may specifically include: based on the generated image (modified image) generated by the generator and the target label image in the training sample, calculating a loss value between the modified image and the target label image, namely a confrontation loss value, and then adjusting parameters for generating the confrontation network model according to the confrontation loss value.

And S240, determining a loss function value for generating the confrontation network model according to the prediction result of each training sample.

Specifically, the prediction result of each training sample is used to represent the degree of difference between the modified image and the target label image, that is, the loss function value for generating the confrontation network model is determined according to the confrontation loss value of each training sample.

And S250, judging whether the loss function value of the generated confrontation network model meets a preset training stop condition or not.

Wherein the training stopping condition may be that the loss function value converges or is less than a preset threshold.

And S260, if the model parameters of the generated confrontation network model are not met, adjusting the model parameters of the generated confrontation network model, and training the adjusted generated confrontation network model by using the training sample set until a preset training stop condition is met to obtain the trained generated confrontation network model.

After updating the model parameters, repeating the steps of S210-S250, and so on, and continuously iterating to enable the loss function value to meet the preset condition. Wherein, when the loss function converges or is less than the preset threshold, the achievement of the loss function value can be considered to satisfy the preset condition. And determining the optimal parameters, namely the training of the generation confrontation network model is completed, so that the trained generation confrontation network model is obtained.

Or, in the process of generating the confrontation network model training, the learning rate adopts an attenuation strategy that every 5 epochs is reduced by 30%, the training can be stopped when 50 epochs are trained totally, and the generated confrontation network model after training is obtained, wherein 1 epoch is equal to one time of training by using all samples in the training set.

Wherein, an Adam optimizer is adopted in the process of generating the training of the confrontation network model. The loss function (loss) employed in the process includes: reconstruct L1 Loss, perceive Loss, GANloss, and Style Loss; and a discriminator loss. loss is used to measure the degree of discrepancy between the predicted and true values of the model, where the WGAN-GP training method is used.

In order to solve the problem that the reality of the processed image cannot be maintained at present, the generated confrontation network model introduced above can be used for processing the original image to obtain the image after the beautifying processing, and the advantages of the generated confrontation network model can be fully exerted, so that the processed image is more real and natural, and the user experience is improved. Based on this, the embodiment of the invention provides an image processing method.

The following describes an image processing method provided by an embodiment of the present invention.

Fig. 3 is a flowchart illustrating an image processing method according to an embodiment of the present invention.

As shown in fig. 3, the image processing method may include S310-S320, and the method is applied to an electronic device, and specifically as follows:

s310, receiving a first input of a user when a first image is displayed on the image processing interface.

And S320, responding to the first input, and processing the first image by generating a confrontation network model to obtain a second image.

The generation of the confrontation network model comprises a generator and a discriminator; the generator is obtained by training according to the original image and the corrected image, and the discriminator is obtained by training according to the corrected image and the target label image.

According to the image processing method, the first image is processed through the trained generation confrontation network model, and the processed image, namely the second image, is obtained, so that flaws in the image can be repaired, and the reality of the image after beautifying processing can be kept.

The contents of S310-S320 are described below:

first, referring to S310, in a possible embodiment, in a case that a first image is displayed on an image processing interface, receiving a first input, where the receiving of the first input in this step may specifically include: a first input of a user to a first preset control of the image processing interface is received.

Under the condition that a first image is displayed on the image processing interface, a user selects a preset control with functional description according to a function desired by the user, namely, a first input of the user to the first preset control of the image processing interface is received. For example, the preset controls may include "remove mottle" or "scrub" such as a preset control with a functional description.

Referring next to S320, in one embodiment, in response to the first input, the first image is processed through a generative confrontation network model associated with the first preset control to obtain a second image. For example, after receiving the input of the user to the 'remove speckle pox' button, the generation of the confrontation network model can perform speckle pox removing treatment on the first image to obtain a second image after speckle pox removing. Or after receiving the input of the user to the 'peeling' button, the generation of the confrontation network model can perform peeling processing on the first image to obtain a peeled second image.

The beauty overall is a relatively subjective process, different users have different aesthetics, and a user-editable beauty function is provided, namely, when the user is not satisfied with the treatment effect of a certain area, a second preset control, such as a local correction button, can be clicked.

As an implementation manner of the present application, in order to improve the accuracy of image processing, generate an image more conforming to the aesthetic sense of a user, and improve the satisfaction of the user, the above step of processing the first image by generating the confrontation network model may specifically include:

processing the target modification region in the first image by generating a countering network model.

The target correction area is a local area to be corrected, and can also be a local defect area.

The local area to be corrected can be a local area to be corrected selected by the user from the first image, namely an unsatisfactory area selected by the user from the first image; or the local area to be corrected, which is automatically identified by the electronic device from the first image, that is, the area in which the defect still exists in the first image identified by the electronic device.

Optionally, in a possible embodiment, an input of a user to select a target modification area from the first image is received; and responding to the input of selecting the target correction area from the first image by the user, and processing the target correction area by generating a confrontation network model to obtain a second image.

When the user is not satisfied with the processing effect of a certain area, firstly, a local correction button can be clicked, and the electronic equipment receives the input of the local correction button of the image processing interface from the user; in response to the user's input of the "local correction" button, the first image is controlled to be in a local area processable state. Then, the user selects an unsatisfactory area (namely a target correction area), then clicks a 'generation' button, and the algorithm processes the target correction area in the first image again by generating a confrontation network model according to the target correction area selected by the user to obtain a new beauty effect, namely a second image.

Therefore, the image processing effect can be more accordant with the aesthetic sense of the user, and the satisfaction degree of the user is improved.

As shown in fig. 4, as another implementation manner of the present application, in order to provide more image processing effects to the user to enable the user to find the image processing result of the most mental apparatus, after S320, the following steps may be further included:

s330, displaying the first image and at least one second image on an image processing interface; receiving a second input; and responding to the second input, and saving a target image corresponding to the second input in the at least one second image.

The step of displaying the first image and the at least one second image on the image processing interface may specifically include the following steps:

the first image (i.e. the original image to be processed) and the at least one second image are displayed on the image processing interface. The second image may be a second image obtained by processing a different target correction area in the first image. For example, the first and second images displayed on the image processing interface are corrected images of the forehead region, and the first and second images displayed on the image processing interface are corrected images of the cheek region.

That is, when at least two second images are displayed on the image display interface, the local correction areas of the at least two second images are different, that is, the display effects of the at least two second images are different.

Wherein the foregoing involves receiving a second input; in response to the second input, the step of saving the target image corresponding to the second input in the at least one second image may be specifically configured to receive the second input that the user selects the target image from the at least one second image, and save the target image in response to the second input.

And displaying a second image obtained after processing different target correction areas in the first image on an image display interface, so that a user can select an image processing result of the centrometer, clicking 'generation' if the user is satisfied with one second image, determining the second image selected by the user as a target image, and storing the target image. Here, more image processing effects can be provided for the user, so that the user finds the most mental image processing result, and the user experience is improved.

In addition, the step of displaying the first image and the at least one second image on the image processing interface may specifically include the following steps: and generating at least one second image according to the generated countermeasure network model, and displaying the first image (namely the original image to be processed) and the at least one second image on the image processing interface.

When at least two second images are displayed on the image display interface, the second images can be generated after the first images are processed by using image processing models with different processing functions (namely, generation confrontation network models generated by training with different training samples). For example, the first second image displayed on the image processing interface is an image obtained by adding a nevus americanus to the first image, and the second image displayed on the image processing interface is an image obtained by adding a tear nevus to the first image. That is, the display effects of the at least two second images with respect to each other are also different.

For example, if the user is not satisfied with the facial appearance effect of the user, the user may click the "recommend" button to generate the confrontation network model to call up at least two images obtained by processing the first image for display, or may click the "generate" button to display a plurality of facial appearance effects output by the confrontation network model stored in advance in S320, if the user likes this recommend effect, the user may apply the facial appearance effect to the current portrait, and if the user is not satisfied with this recommend effect, the user may continue to click the "recommend" button to display the next recommend effect. More image processing effects can be provided for the user, so that the user can find the most mental image processing result, and the user experience is improved.

For some users, the users are more interested in the evaluation of others, and for the users, the aesthetic value of the users can change correspondingly along with the change of time. Based on these two points, the above-mentioned "recommendation" function may also provide some popular aesthetics for the face information (face shape, gender, etc.) in the current processing as labels of different processing effects to recommend to the user, such as a nevus of beauty, a nevus with meaning, etc., and the user determines whether to keep the nevus. Meanwhile, a list can be maintained for the recommendations and updated regularly, so that the aesthetic recommendations can keep up with the social development steps.

According to the embodiment of the invention, in response to the image processing operation of the user on the first image, the first image is subjected to image processing by using the trained generation confrontation network model, and a processed second image is obtained. The trained generator for generating the countermeasure network can generate an almost 'false-to-false' picture, so that the discriminator for generating the countermeasure network is difficult to judge whether the picture generated by the generator is real or not, the reality of the image after the beautifying processing can be greatly improved, and therefore, the image processing model is used for processing the first image, the problems that the difference between the processed image and the reality is large and the reality lacks when the user processes the image are solved, the natural and real image (for example, the texture of a portrait is reserved) can be obtained, and the use experience of the user is improved.

Based on the image processing interface shown in fig. 5, correspondingly, the present application also provides a specific implementation manner of yet another image processing method. Still another image processing method provided by the embodiment of the invention is described below with reference to fig. 6.

S610, receiving a first input of 'removing speckle and pox' from the image processing interface by the user under the condition that the first image is displayed on the image processing interface.

S620, responding to the first input, and performing speckle-pox removing treatment on the first image through a first image processing model associated with speckle-pox removing to obtain a second image after speckle-pox removing.

Wherein the first image processing model comprises a generator and a discriminator; the generator is obtained by training according to the original image and the corrected image, and the discriminator is obtained by training according to the corrected image and the target label image; the corrected image is obtained by correcting the original image by the preset type of defects, and the preset type of defects comprise first type of defects or second type of defects.

And S630, displaying the second image after the speckle and pox is removed in an image preview area of the image processing interface.

Displaying the second image in the image preview area can facilitate the user to check the image processing effect, and also facilitate the user to perform subsequent local correction operation again in the image preview area.

S640, receiving a second input of local correction of the image processing interface by the user; controlling the second image to be in a local area processable state in response to a second input; receiving a third input of selecting a target correction area from the second image by the user; and responding to a third input, processing the target correction area through a first image processing model associated with the speckle removal effect, and obtaining a third image.

S650, receiving a fourth input of 'recommendation' of the user to the image processing interface; in response to a fourth input, canceling the display of the second image, and sequentially displaying fourth images in the image preview area according to a preset sequence; the fourth image is at least two images obtained by processing the first image according to the first image processing model, and the correction effect of the fourth image is different from that of the second image;

in a case where a fifth input of "generation" of the image processing interface by the user is received, the second image is replaced with a fourth image currently displayed on the image preview interface.

S660, when receiving a fifth input of "generation" of the image processing interface by the user, saving the correction effect information of the fourth image currently displayed on the image preview interface.

When a user selects a certain recommended effect, the currently selected effect is recorded and stored in a recommendation list for the next use. The user can conveniently and quickly find the common and favorite image processing effect, and the user experience is improved.

S670, receiving a sixth input of the user to the skin grinding; and in response to a sixth input, performing dermabrasion processing on the second image through a second image processing model associated with the "dermabrasion", so as to obtain a fifth dermabraded image.

The second image processing model is an image processing model, and the image processing model comprises a generator and a discriminator; the generator is obtained by training according to the original image and the corrected image, and the discriminator is obtained by training according to the corrected image and the target label image; the corrected image is obtained by correcting the original image by the preset type of defects, and the preset type of defects comprise first type of defects or second type of defects.

The preset type corresponding to the modified image for training the first image processing model is different from the preset type corresponding to the modified image for training the second image processing model.

The second image processing model generates a countermeasure network model, and the generation of the countermeasure network model comprises a generator and a discriminator; the generator is obtained by training according to the original image and the corrected image, and the discriminator is obtained by training according to the corrected image and the target label image; the corrected image is an image obtained by correcting the original image by using a preset type flaw, wherein the preset type flaw comprises a first type flaw or a second type flaw; the preset type corresponding to the modified image for training the first image processing model is different from the preset type corresponding to the modified image for training the second image processing model.

For users with flawed faces, some users only want to remove speckles on the faces and do not want to grind the skins because of good skin conditions of the users, so that the real facial textures of the users can be kept; for some users, the skin conditions of the users are not ideal, so that the face with the speckle-removing pox is required to be subjected to skin grinding again.

To address this need, the present invention supports different processing types and degrees of beauty, separating the image processing functions of the different processing types, i.e., a first image processing model for processing images of a first type of blemish and a second image processing model for processing images of a second type of blemish. For example, the speckle removing pox and the skin grinding pox are separated, the speckle removing pox result and the skin grinding pox result are respectively given, meanwhile, the degree of the beautifying effect is graded, the higher the degree is, the stronger the beautifying effect is, and the beautifying effect which is most suitable for the user is selected by the user.

Therefore, the user can treat different types of flaws respectively to achieve the optimal treatment effect.

S680, receiving target input of a user to a target preset control, wherein the target preset control comprises at least one of 'speckle removing', 'local correction', 'recommendation', 'generation' and 'peeling'; and responding to the target input, and withdrawing the function operation corresponding to the target preset control. The user can conveniently and timely correct misoperation and improve the convenience of use of the user.

During the actual operation of the user, the user may operate by mistake or want to return to the previous processing effect. At this time, the target input of the user to the target preset control can be received, the functional operation corresponding to the target preset control is withdrawn in response to the target input, and the requirement that the user returns to the previous processing effect is met.

In the description of removing speckle and pox and grinding skin of S630-S680, the sequences of removing speckle and pox and grinding skin can be interchanged, or can be replaced by other preset spaces representing image processing functions, such as 'face thinning', 'big eye' or 'blush'.

In summary, in the embodiments of the present invention, in response to an image processing operation performed on a first image by a user, an image processing model based on the generated countermeasure network training is used to perform image processing on the first image, so as to obtain a processed second image. Therefore, the problem that the difference between the processed image and reality is large and the processed image is too false when the user processes the image is solved, so that a natural and real image (for example, the texture of a portrait is kept) can be obtained, and the use experience of the user is improved.

Based on the image processing method provided by the above embodiment, correspondingly, the application also provides a specific implementation manner of the image processing device. An image processing apparatus provided by an embodiment of the present invention is described below with reference to fig. 7. Fig. 7 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.

As shown in fig. 7, the apparatus 700 may specifically include:

the receiving module 710 is configured to receive a first input of a user when a first image is displayed on the image processing interface.

A processing module 720, configured to respond to the first input, process the first image by generating a confrontation network model to obtain a second image; the generation of the confrontation network model comprises a generator and an arbiter.

In a possible embodiment, the processing module 720 is specifically configured to process the target modification area in the first image by generating a countering network model.

In a possible embodiment, the processing module 720 may be further configured to display the first image and the at least one second image on the image processing interface; receiving a second input; and responding to the second input, and saving a target image corresponding to the second input in the at least one second image.

In addition, the apparatus 700 may further include a training module 730, where the training module 730 is configured to obtain a training sample set, where the training sample set includes a plurality of training samples, and each training sample includes an original image and a target label image; inputting the original image into a generator to obtain a corrected image; inputting the corrected image and the target label image into a discriminator, and determining a resistance loss value between the corrected image and the target label image; determining a first loss function value of a generated countermeasure network model according to the countermeasure loss value of each training sample; judging whether the loss function value of the first generation countermeasure network model meets a preset training stop condition or not; and if the loss function value does not meet the preset training stopping condition, adjusting model parameters of the first generation antagonistic network model, training the adjusted first generation antagonistic network model by using the training sample set until the first generation antagonistic network model meets the preset training stopping condition, and obtaining the trained first generation antagonistic network model.

In summary, in the embodiments of the present invention, in response to an image processing operation performed on a first image by a user, an image processing is performed on the first image by using a generative confrontation network model based on generative confrontation network training, so as to obtain a processed second image. Therefore, the problem that the difference between the processed image and reality is large and the processed image is too false when the user processes the image is solved, so that a natural and real image (for example, the texture of a portrait is kept) can be obtained, and the use experience of the user is improved.

The electronic device 800 includes, but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, a processor 810, and a power supply 811. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 8 does not constitute a limitation of the electronic device, and that the electronic device may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.

It should be understood that, in the embodiment of the present invention, the radio frequency unit 801 may be used for receiving and sending signals during a message sending and receiving process or a call process, and specifically, receives downlink resources from a base station and then processes the downlink resources to the processor 810; in addition, the uplink resource is transmitted to the base station. In general, radio frequency unit 801 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. Further, the radio frequency unit 801 can also communicate with a network and other devices through a wireless communication system.

The electronic device provides wireless broadband internet access to the user via the network module 802, such as to assist the user in sending and receiving e-mails, browsing web pages, and accessing streaming media.

The audio output unit 803 may convert an audio resource received by the radio frequency unit 801 or the network module 802 or stored in the memory 809 into an audio signal and output as sound. Also, the audio output unit 803 may also provide audio output related to a specific function performed by the electronic apparatus 800 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 803 includes a speaker, a buzzer, a receiver, and the like.

The input unit 804 is used for receiving an audio or video signal. The input Unit 804 may include a Graphics Processing Unit (GPU) 8041 and a microphone 8042, and the Graphics processor 8041 processes image resources of still images or videos obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 808. The image frames processed by the graphics processor 8041 may be stored in the memory 809 (or other storage medium) or transmitted via the radio frequency unit 801 or the network module 802. The microphone 8042 can receive sound and can process such sound into an audio asset. The processed audio resources may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 801 in case of a phone call mode.

The electronic device 800 also includes at least one sensor 805, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the brightness of the display panel 8081 according to the brightness of ambient light and a proximity sensor that can turn off the display panel 8081 and/or the backlight when the electronic device 800 is moved to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of an electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 805 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.

The display unit 806 is used to display information input by the user or information provided to the user. The Display unit 806 may include a Display panel 8061, and the Display panel 8061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 807 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus. Specifically, the user input unit 807 includes a touch panel 8071 and other input devices 8072. The touch panel 8071, also referred to as a touch screen, may collect touch operations by a user on or near the touch panel 8071 (e.g., operations by a user on or near the touch panel 8071 using a finger, a stylus, or any other suitable object or accessory). The touch panel 8071 may include two portions of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 810, receives a command from the processor 810, and executes the command. In addition, the touch panel 8071 can be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 8071, the user input unit 807 can include other input devices 8072. In particular, other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.

Further, the touch panel 8071 can be overlaid on the display panel 8081, and when the touch panel 8071 detects a touch operation on or near the touch panel 8071, the touch operation can be transmitted to the processor 810 to determine the type of the touch event, and then the processor 810 can provide a corresponding visual output on the display panel 8081 according to the type of the touch event. Although the touch panel 8071 and the display panel 8081 are shown in fig. 8 as two separate components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 8071 and the display panel 8081 may be integrated to implement the input and output functions of the electronic device, and the implementation is not limited herein.

The interface unit 808 is an interface for connecting an external device to the electronic apparatus 800. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless resource port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 808 may be used to receive input from an external device (e.g., resource information, power, etc.) and transmit the received input to one or more elements within the electronic apparatus 800 or may be used to transmit resources between the electronic apparatus 800 and the external device.

Memory 809 may be used to store software programs and various resources. The memory 809 may mainly include a storage program area and a storage resource area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage resource area may store resources (such as audio resources, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 809 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 810 is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, and performs various functions and processing resources of the electronic device by running or executing software programs and/or modules stored in the memory 809 and calling resources stored in the memory 809, thereby performing overall monitoring of the electronic device. Processor 810 may include one or more processing units; preferably, the processor 810 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 810.

The electronic device 800 may further include a power supply 811 (e.g., a battery) for powering the various components, and preferably, the power supply 811 may be logically coupled to the processor 810 via a power management system to manage charging, discharging, and power consumption management functions via the power management system.

In addition, the electronic device 800 includes some functional modules that are not shown, and are not described in detail herein.

Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, which, when executed in a computer, causes the computer to perform the steps of the image processing method of an embodiment of the present invention.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling an electronic device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. An image processing method applied to an electronic device, comprising:

receiving a first input under the condition that a first image is displayed on an image processing interface;

responding to the first input, and processing the first image by generating a confrontation network model to obtain a second image;

wherein the generating a countermeasure network model includes a generator and a discriminator.

2. The method according to claim 1, characterized in that said processing of said first image by generating a countering network model is in particular:

and processing the target correction area in the first image through the generation countermeasure network model.

3. The method of claim 2, further comprising, after the processing the first image by generating a countering network model:

displaying the first image and at least one second image on the image processing interface;

receiving a second input;

and responding to the second input, and saving a target image corresponding to the second input in the at least one second image.

4. The method of claim 1, wherein the generating the antagonistic network model is obtained by a goal training process comprising:

acquiring a training sample set, wherein the training sample set comprises a plurality of training samples, and each training sample comprises an original image and a target label image;

inputting the original image into the generator to obtain a corrected image;

inputting the corrected image and the target label image into the discriminator, and determining a countermeasure loss value between the corrected image and the target label image;

determining a loss function value of the generated confrontation network model according to the confrontation loss value of each training sample;

judging whether the loss function value of the generated confrontation network model meets a preset training stop condition or not;

and if the loss function value does not meet the preset training stop condition, adjusting the model parameters of the generated confrontation network model, and training the adjusted generated confrontation network model by using the training sample set until the generated confrontation network model meets the preset training stop condition to obtain the trained generated confrontation network model.

5. An electronic device, comprising:

the processing module is used for responding to the first input, processing the first image by generating a confrontation network model to obtain a second image; wherein the generating a countermeasure network model includes a generator and a discriminator.

6. The electronic device according to claim 5, wherein the processing module is specifically configured to process the target correction area in the first image by generating an antagonistic network model.

7. The electronic device of claim 6, wherein the processing module is further configured to display the first image and the at least one second image on the image processing interface;

receiving a second input;

8. The electronic device of claim 5, further comprising a training module to obtain a training sample set, the training sample set comprising a plurality of training samples, each training sample comprising an original image and a target label image;

inputting the original image into the generator to obtain a corrected image;

9. An electronic device, comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the image processing method of claims 1-4.

10. A computer-readable storage medium, having stored thereon a computer program which, if executed in a computer, causes the computer to execute the image processing method according to claims 1-4.