WO2022267327A1

WO2022267327A1 - Pigmentation prediction method and apparatus, and device and storage medium

Info

Publication number: WO2022267327A1
Application number: PCT/CN2021/132553
Authority: WO
Inventors: 齐子铭; 刘兴云; 罗家祯; 陈福兴; 李志阳
Original assignee: 厦门美图宜肤科技有限公司
Priority date: 2021-06-24
Filing date: 2021-11-23
Publication date: 2022-12-29
Also published as: CN113379716A; JP2023534328A; KR20230001005A; CN113379716B; JP7385046B2

Abstract

The present application belongs to the technical field of image recognition processing. Provided are a pigmentation prediction method and apparatus, and a device and a storage medium. The method comprises: acquiring an image to be subjected to prediction; inputting said image into a pigmentation prediction model, obtained by means of pre-training, for prediction processing, wherein the pigmentation prediction model is a fully convolutional generative adversarial network model; and obtaining a pigmentation prediction result map by using the pigmentation prediction model. By means of the present application, a pigmentation change situation of facial skin of a user can be predicted.

Description

A stain prediction method, device, equipment and storage medium

Cross References to Related Applications

This application claims the priority of the Chinese patent application with the application number 2021107071008 and titled "A Method, Device, Equipment, and Storage Medium for Stain Prediction" submitted to the China Patent Office on June 24, 2021, the entire contents of which are incorporated by reference incorporated in this application.

technical field

The present application relates to the technical field of image recognition and generation, and in particular, to a stain prediction method, device, equipment, and storage medium.

Background technique

With the increase of age, human facial skin will have problems such as aging and diseases. In order to prevent the occurrence of these problems, it is usually necessary to predict the future changes of the human face, so as to carry out early preventive intervention.

The prediction of skin changes in the prior art usually predicts the degree of skin laxity and wrinkle, and cannot realize the prediction of future changes in pigmentation. Therefore, it is urgent to provide a method that can realize future changes in pigmentation Prediction of changes, so that users can timely understand the changes of skin pigmentation in the future.

Contents of the invention

The present application provides a color spot prediction method, device, equipment and storage medium, which can realize the prediction of the change of color spots on the user's facial skin.

Some embodiments of the present application provide a method for predicting stains, which may include:

Obtain the image to be predicted;

Input the image to be predicted into the pre-trained stain prediction model for prediction processing, and the stain prediction model is a fully convolutional generation confrontation network model;

The color spot prediction result map is obtained through the color spot prediction model.

Optionally, before inputting the image to be predicted into the pre-trained color spot prediction model for prediction processing, it may also include:

Determine the color spot information in the image to be predicted, the color spot information may include: the position and category of the color spot;

According to the color speckle information in the image to be predicted, the image to be predicted is preprocessed to obtain preprocessed multi-frame images, and the multi-frame images may respectively include: an image without color speckle and an image marked with a color speckle category;

Inputting the image to be predicted into the pre-trained color spot prediction model for prediction processing may include:

The preprocessed multi-frame images are input into the pre-trained stain prediction model for prediction processing.

Obtain the target image to be processed, which may include color spot information;

A plurality of target channel images are respectively determined according to the target image to be processed, and the target channel images may include: a multi-channel image with color spot information removed and a color spot category channel image;

Combine multiple target channel images and target noise images, and input them together into the trained stain prediction model in the neural network structure to be trained; the target noise image is a randomly generated noise image.

Optionally, determining a plurality of target channel images respectively according to the target image to be processed may include:

Determine the color spot information in the target image to be processed;

According to the speckle information in the target image to be processed, the speckle removal process is performed on the target image to be processed, and the target image to be processed with the speckle information removed is obtained.

The speckle detection process is performed on the target image to be processed, and the channel image of the speckle category is obtained.

Optionally, performing color spot detection processing on the target image to be processed to obtain a color spot category channel image may include:

Determine the position of each type of color spot information in the target image to be processed respectively;

Set grayscale information of the type corresponding to the color spot information at the position of the color spot information to obtain a color spot category channel image.

Optionally, before a plurality of target channel images and target noise images are merged and input into the neural network structure to be trained before the trained stain prediction model, the method may include:

Normalize the target channel image and the target noise image respectively to obtain the target input image;

Combine multiple target channel images and target noise images, and input them together into the trained color spot prediction model in the neural network structure to be trained, which can include:

The target input image is input into the neural network structure to be trained to obtain the trained stain prediction model.

Another embodiment of the present application provides a stain prediction device, which may include: an acquisition module, a prediction module, and an output module;

An acquisition module that can be configured to acquire an image to be predicted;

The prediction module can be configured to input the image to be predicted into a pre-trained stain prediction model for prediction processing, and the stain prediction model is a fully convolutional generation confrontation network model;

The output module can be configured to obtain a stain prediction result map through the stain prediction model.

Optionally, the device may further include: a preprocessing module; the preprocessing module may be configured to determine color spot information in the image to be predicted, and the color spot information may include: the position and category of the color spot; The speckle information in the image to be predicted is preprocessed to obtain multi-frame images after preprocessing. The multi-frame images can include: images without speckles and images with speckle categories; another prediction module can be configured It is used to input the preprocessed multi-frame images into the pre-trained stain prediction model for prediction processing.

Optionally, the pre-processing module may also be configured to acquire the target image to be processed, which may include color spot information; respectively determine a plurality of target channel images according to the target image to be processed, the target channel image may include: Remove the multi-channel image of the color spot information and the channel image of the color spot category; merge multiple target channel images, and input them together into the trained color spot prediction model in the neural network structure to be trained; the target noise image is a randomly generated noise image.

Optionally, the preprocessing module can also be configured to determine color speckle information in the target to-be-processed image; perform color speckle removal processing on the target to-be-processed image according to the color speckle information in the target to-be-processed image, to obtain removed color speckles Target image to be processed with speckle information.

Optionally, the preprocessing module may also be configured to perform color spot detection processing on the target image to be processed to obtain a color spot category channel image.

Optionally, the preprocessing module can be configured to determine the position of each type of color spot information in the target image to be processed respectively; set the grayscale information of the type corresponding to the color spot information at the position of the color spot information to obtain the color spot information Speckle category channel image.

Optionally, the preprocessing module can also be configured to perform normalization processing on the target channel image and the target noise image respectively to obtain the target input image; input the target input image into the training neural network structure to obtain the trained color Spot prediction model.

Some other embodiments of the present application provide a computer device, the computer device may include: a memory, a processor, a computer program that can run on the processor is stored in the memory, and when the processor executes the computer program, the above-mentioned color spots are realized The steps of the prediction method.

Still other embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method for predicting color spots are realized.

The beneficial effects of the embodiments of the present application at least include:

In a stain prediction method, device, device, and storage medium provided in the embodiments of the present application, the image to be predicted can be obtained; the image to be predicted can be input into the pre-trained stain prediction model for prediction processing, and the stain prediction model Generate an adversarial network model for the full convolution; get the color spot prediction result map through the color spot prediction model. Among them, the fully convolutional generation confrontation network structure is used as the spot prediction module, which can realize the prediction and processing of the future changes of the skin spots, and then enable the user to know the change trend of the skin spots in time.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, so It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

FIG. 1 is a schematic flow diagram of a stain prediction method provided in an embodiment of the present application;

Fig. 2 is the second schematic flow diagram of the stain prediction method provided by the embodiment of the present application;

Fig. 3 is a schematic flow chart three of the color spot prediction method provided by the embodiment of the present application;

Fig. 4 is a schematic flow diagram 4 of the stain prediction method provided in the embodiment of the present application;

Fig. 5 is a schematic flow diagram five of the stain prediction method provided in the embodiment of the present application;

FIG. 6 is the sixth schematic flow chart of the stain prediction method provided by the embodiment of the present application;

FIG. 7 is a schematic structural diagram of a stain prediction device provided in an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a computer device provided by an embodiment of the present application.

detailed description

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. The components of the embodiments of the application generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations.

Accordingly, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In the description of the present application, it should be noted that the terms "first", "second", "third" and so on are only used to distinguish descriptions, and should not be understood as indicating or implying relative importance.

It should be noted that there is no prediction method for the change of human facial skin pigmentation in the prior art. In the embodiment of the present application, the change of human facial skin pigmentation in a certain period of time can be realized, so that users can know in time to the pigmentation changes on the face.

The specific implementation process of the stain prediction method provided in the embodiments of the present application will be explained in detail below.

Fig. 1 is a schematic flow chart of the stain prediction method provided in the embodiment of the present application. Please refer to Fig. 1, the method may include:

S110: Acquire an image to be predicted.

Optionally, the image to be predicted may be a photo of the user, an image of a human face, or any image corresponding to the skin that needs to be predicted for color spots, and the image may be a cropped image with a preset size.

Optionally, the execution subject of this method may be a related program on the computer device, for example: a preset program of the skin predictor, a certain function of the electronic facial cleanser, etc., which are not specifically limited here and can be set according to actual needs.

Optionally, the image to be predicted may be sent to the computer device by other devices, or may also be captured by the computer device through a shooting device, etc., and no specific limitation is set here.

S120: Input the image to be predicted into the pre-trained stain prediction model to perform prediction processing.

Among them, the stain prediction model is a fully convolutional generation confrontation network model.

Optionally, after the image to be predicted is determined, the image to be predicted can be input to a pre-trained mottle prediction model for prediction processing, wherein the mottle prediction model can be a fully convolutional generation confrontation network model obtained after pre-training .

Wherein, the stain prediction model may be obtained by pre-training of the computer device, or may be sent to the computer device by other electronic devices, which is not limited herein.

Optionally, the fully convolutional generation confrontation network model can be a generation confrontation network composed of multiple convolutional neural networks, wherein the generation confrontation network can be a mutual The game learns network models that produce reasonably good outputs.

S130: Obtain a color spot prediction result map through the color spot prediction model.

Optionally, after the prediction process is carried out by the color spot prediction model, a color spot prediction result map can be obtained, wherein the color spot prediction result map can show the change of the color spot in the future period of time in the image to be predicted, the specific time It can be set according to actual needs, and there is no limitation here.

Optionally, the color speckle prediction result graph may include multiple graphs, which are respectively used to represent the changes of the color spots on the facial skin in the image to be predicted after a certain period of time in the future.

In a stain prediction method provided in the embodiment of the present application, the image to be predicted can be obtained; the image to be predicted can be input into the pre-trained stain prediction model for prediction processing, and the stain prediction model is a fully convolutional generative confrontation network Model; the color spot prediction result map is obtained through the color spot prediction model. Among them, the fully convolutional generation confrontation network structure is used as the spot prediction module, which can realize the prediction and processing of the future changes of the skin spots, and then enable the user to know the change trend of the skin spots in time.

Another specific implementation process of the stain prediction method provided in the embodiment of the present application will be explained in detail below.

Fig. 2 is a schematic flow diagram 2 of the mottle prediction method provided by the embodiment of the present application. Please refer to Fig. 2, before inputting the image to be predicted into the pre-trained mottle prediction model for prediction processing, it also includes:

S210: Determine color speckle information in the image to be predicted.

Wherein, the color spot information may include: the position and type of the color spot.

Optionally, the color spot information may be the color spot information on the skin in the image to be predicted, for example, it may include: the position, category, etc. of each color spot on the skin in the image to be predicted, each color spot The position of the color spot can be recorded in the form of the range of coordinates, and the category of the color spot can be recorded in the form of identification.

Optionally, a preset color speckle recognition algorithm may be used to acquire and determine the color speckle information in the image to be predicted.

S220: Perform preprocessing on the image to be predicted according to the color speckle information in the image to be predicted to obtain multi-frame images after preprocessing.

Wherein, the multiple frames of images may respectively include: an image without color speckle and an image marked with a color speckle category.

Optionally, the preprocessing of the image to be predicted may include color speckle removal processing and color speckle determination processing, wherein the above-mentioned image without color speckle can be obtained through the color speckle removal process, that is, the image to be predicted without color speckle information ; Through the color speckle determination process, an image marked with a color speckle category can be obtained, and different grayscale values in the image can represent different color speckle categories.

S230: Input the preprocessed multi-frame images into the color speckle prediction model obtained in advance to perform prediction processing.

Optionally, after the above-mentioned plurality of preprocessed multi-frame images are respectively determined, these images may be combined and input into a pre-trained stain prediction model for prediction, so as to obtain corresponding prediction results.

Fig. 3 is a schematic flow chart of the mottle prediction method provided in the embodiment of the present application III. Please refer to Fig. 3, before inputting the image to be predicted into the pre-trained mottle prediction model for prediction processing, it may also include:

S310: Acquire a target image to be processed.

Wherein, the target image to be processed may include speckle information.

Optionally, the target image to be processed may be a sample image used for training the color spot prediction model, the sample image has skin, and the skin includes color spot information.

Optionally, the target image to be processed may be a large number of pre-collected sample images, for example, images of facial spots downloaded through the network, etc., which are not specifically limited here.

S320: Determine a plurality of target channel images respectively according to the target image to be processed.

Wherein, the target channel image may include: a multi-channel image with color spot information removed and a color spot category channel image.

Optionally, the target image to be processed can be processed separately to obtain a plurality of target channel images, wherein the multi-channel image from which color speckle information is removed can be obtained after the target image to be processed is subjected to color speckle removal processing. Images of blotches in the same way. The color speckle category channel image may be obtained by identifying the color speckle on the target image to be processed, which is the same as the aforementioned method of acquiring an image marked with a color speckle category.

S330: Combine multiple target channel images and target noise images, and input them together into the trained color speckle prediction model in the neural network structure to be trained.

Among them, the target noise image is a randomly generated noise image.

Optionally, after acquiring the above-mentioned multiple target channel images, these target channel images and pre-generated target noise images can be combined and input together into the neural network structure to be trained for training. After the training is completed, the above-mentioned stain prediction model can be obtained .

Fig. 4 is a schematic flow diagram 4 of the speckle prediction method provided by the embodiment of the present application. Please refer to Fig. 4, and determine a plurality of target channel images according to the target image to be processed, which may include:

S410: Determine color speckle information in the target image to be processed.

Optionally, after the target image to be processed is determined, the color speckle information may be determined, specifically, the color speckle information may be determined by means of the aforementioned color speckle identification.

S420: Perform color speckle removal processing on the target image to be processed according to color speckle information in the target image to be processed, to obtain a target image to be processed with color speckle information removed.

Optionally, after the above stain information is obtained, stain removal processing may be performed according to the stain information. All color speckle information in the target image to be processed is removed to obtain a target image to be processed, wherein the target image to be processed does not have color speckle information.

Optionally, after the target image to be processed is obtained, channel processing can be performed to obtain the color channels of red, green and blue respectively, and the red channel image with color spot information removed, the green channel image with color spot information removed, and the color spot removed image can be obtained. Informational blue channel image.

Optionally, after the target image to be processed is obtained, the image can be subjected to color speckle detection processing, and the image of the color speckle category channel can be obtained further. The specific process is as follows:

Fig. 5 is a schematic flow chart of the mottle prediction method provided in the embodiment of the present application fifth, please refer to Fig. 5, perform the mottle detection process on the target image to be processed, and obtain the mottle category channel image, including:

S510: Determine the position of each type of color speckle information in the target image to be processed respectively.

Optionally, the position of each type of color spot in the target image to be processed may be determined by means of color spot identification.

S520: Grayscale information of a type corresponding to the color spot information may be set at the position of the color spot information to obtain a color spot category channel image.

Optionally, after determining the position of each type of stain, the corresponding gray value can be set at the corresponding position. Different gray values can be used to represent different types of stains. The specific position and range of the gray value It can represent the position and size of the color spot, determine the grayscale information corresponding to each color spot information in the image and set it up, you can get the above color spot category channel image, which can be represented by different grayscales Channel images of different speckle types.

Fig. 6 is a schematic flow diagram of the mottle prediction method provided by the embodiment of the present application 6. Please refer to Fig. 6. Multiple target channel images and target noise images are combined and input together into the neural network structure to be trained to obtain the trained mottle prediction Before the model, the method can include:

S610: Perform normalization processing on the target channel image and the target noise image respectively to obtain a target input image.

Optionally, the above-mentioned multiple target channel images and target noise images are merged and input together into the trained color speckle prediction model in the neural network structure to be trained. The target channel image includes the aforementioned red channel image with color speckle information removed, The green channel image with color spot information removed, the blue channel image with color spot information removed, and the color spot category channel image, these four types of images can be combined with the target noise image to obtain a five-channel image, which is input to the neural network structure to be trained together Obtain the trained speckle prediction model.

Optionally, during the normalization process, specifically, the red channel image with color speckle information removed, the green channel image with color speckle information removed, the blue channel image with color speckle information removed, and the above-mentioned target noise in the target channel image can be specifically The image is normalized to the (-1,1) interval, and the stain category channel image is normalized to the (0,1) interval. The specific calculation formula is as follows:

Im g _(-1,1) = (Im g*2/255)-1;

Wherein, Img is a single-channel image in the interval 0-255, and Im g _{(-1, 1)} is a map in the interval (-1, 1) after normalization.

ClsMask _(0,1) = ClsMask/255;

Among them, ClsMask is a single-channel image in the interval of 0 to 255, and ClsMask _{(0, 1)} is a normalized image in the interval (0, 1).

S620: Input the target input image into the training neural network structure to obtain the trained stain prediction model.

Optionally, after the above target input image is obtained, the target input image may be input into the above neural network structure to obtain a trained stain prediction model.

The specific structure of the color spot prediction model adopted in the embodiment of the present application is explained in detail below:

The model adopts an encoding-decoding structure, and the upsampling of the decoding part adopts the combination of the nearest neighbor upsampling + convolution layer, and the activation function of the output layer is Tanh, and the specific structural relationship can be shown in Table 1.

Table 1

Among them, Leakyrelu is a kind of conventional activation function in deep learning, negativeslope is a configuration parameter in the activation function, kh is the height of the convolution kernel, kw is the width of the convolution kernel, and padding is the feature map for the convolution operation Extended pixel value, stride is the step size of convolution, group is the number of convolution kernel groups, scale_factor, Mode are the parameters of the upsampling layer. Among them, scale_factor means upsampling to 2 times the size, and Mode=nearest means upsampling in the nearest neighbor mode.

Optionally, the model also includes a discriminant network part to distinguish real and fake images with different resolutions. In the embodiment of the present application, discriminators of three scales may be used to distinguish images with resolutions of 512x512, 256x256, and 128x128, respectively. For images of different resolutions, it can be obtained by downsampling.

Optionally, the model can be set to 20,000 samples during the training process, and various gains can be performed on the image of each sample, such as flip, rotation, translation, affine transformation, exposure, contrast adjustment, blur, etc. to achieve Improvements in robustness.

Optionally, the optimization algorithm for network training uses the Adam algorithm, the learning rate of the generating network is 0.0002, and the learning rate of the discriminant network is 0.0001.

Optionally, the specific calculation of the loss function of the model is as follows:

L=L ₁ +L ₂ +L _vgg +L _adv ;

L ₁ =|generate-GT|;

L ₂ =||generate-GT||;

Lvgg＝∑ _i ||L _perceptual (generate)-L _perceptual (GT)||;

Among them, Generate represents the output of the network; GT is the image generated by the target. Both L ₁ and L ₂ are loss functions, L _vgg is a perceptual loss function, and L _adv is a generative adversarial loss function. Lperceptual means perceptual loss. Perceptual loss refers to inputting the output (generated graph) generate and GT of the network into another network, extracting the feature tensor of the corresponding layer, and calculating the difference between the feature tensors. i is i-th sample.

The following describes the devices, devices, and storage media corresponding to the color speckle prediction method provided by the present application. For the specific implementation process and technical effects, refer to the above, and will not be repeated below.

Fig. 7 is a schematic structural diagram of a speckle prediction device provided in the embodiment of the present application, please refer to Fig. 7, the device includes: an acquisition module 100, a prediction module 200, and an output module 300;

The obtaining module 100 may be configured to obtain an image to be predicted;

The prediction module 200 may be configured to input the image to be predicted into a pre-trained stain prediction model for prediction processing, and the stain prediction model is a fully convolutional generation confrontation network model;

The output module 300 may be configured to obtain a color speckle prediction result map through the color speckle prediction model.

Optionally, the device further includes: a preprocessing module 400; the preprocessing module 400 can be configured to determine color speckle information in the image to be predicted, and the color speckle information includes: the position and category of the color speckle; according to the image to be predicted The speckle information in the image to be predicted is preprocessed to obtain preprocessed multi-frame images, and the multi-frame images respectively include: images without speckles and images marked with speckle categories; another prediction module 200 can be configured It is used to input the preprocessed multi-frame images into the pre-trained stain prediction model for prediction processing.

Optionally, the preprocessing module 400 may also be configured to acquire the target image to be processed, the target image to be processed includes color spot information; respectively determine a plurality of target channel images according to the target image to be processed, the target channel image includes: Multi-channel images of color spot information and channel images of color spot categories; multiple target channel images and target noise images are combined, and input into the trained color spot prediction model in the neural network structure to be trained together; target noise images are randomly generated noisy image.

Optionally, the pre-processing module 400 may also be configured to determine color speckle information in the target image to be processed; perform color speckle removal processing on the target image to be processed according to the color speckle information in the target image to be processed to obtain The target image to be processed with color speckle information; channel processing is performed on the target image to be processed with color speckle information removed to obtain a multi-channel image with color speckle information removed.

Optionally, the preprocessing module 400 may also be configured to perform color spot detection processing on the target image to be processed to obtain a color spot category channel image.

Optionally, the preprocessing module 400 may be configured to determine the position of each type of color speckle information in the target image to be processed respectively; set grayscale information of the type corresponding to the color speckle information at the position of the color speckle information, to obtain The blob category channel image.

Optionally, the preprocessing module 400 can also be configured to perform normalization processing on the target channel image and the target noise image respectively to obtain the target input image; input the target input image into the training neural network structure to obtain the trained Stain prediction model.

The above-mentioned apparatus is used to execute the methods provided in the foregoing embodiments, and its implementation principles and technical effects are similar, and details are not repeated here.

The above modules may be one or more integrated circuits configured to implement the above method, for example: one or more specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), or, one or more microprocessors, or, One or more Field Programmable Gate Arrays (Field Programmable Gate Array, FPGA for short), etc. For another example, when one of the above modules is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, referred to as CPU) or other processors that can call program codes. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC for short).

FIG. 8 is a schematic structural diagram of a computer device provided by an embodiment of the present application. Please refer to FIG. 8 . The computer device includes: a memory 500 and a processor 600. The memory 500 stores a computer program that can run on the processor 600. The processor 600 When the computer program is executed, the steps of the above stain prediction method are realized.

Some embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method for predicting color spots are realized.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

The above-mentioned integrated units implemented in the form of software functional units may be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or a processor (English: processor) to execute the methods of the various embodiments of the present application. partial steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), magnetic disk or optical disc, etc. Various media that can store program code.

The above is only a specific embodiment of the application, but the scope of protection of the application is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the application, and should be covered in Within the protection scope of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application. For those skilled in the art, there may be various modifications and changes in the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the protection scope of this application.

Industrial Applicability

The present application provides a stain prediction method, device, equipment and storage medium. The method includes: obtaining an image to be predicted; inputting the image to be predicted into a color spot prediction model obtained in advance for prediction processing, and the color spot prediction model is a fully convolutional generation confrontation network model; obtaining a color spot prediction through the color spot prediction model Result graph. This application can realize the prediction of the color spot change of the user's facial skin.

In addition, it can be understood that the stain prediction method, device, device and storage medium of the present application are reproducible and can be used in various industrial applications. For example, the stain prediction method, device, device and storage medium of the present application can be used in fields requiring image recognition processing.

Claims

A method for predicting stains, characterized in that said method comprises:

Obtain the image to be predicted;

Inputting the image to be predicted into a pre-trained mottle prediction model for prediction processing, the mottle prediction model is a fully convolutional generation confrontation network model;

A stain prediction result map is obtained through the stain prediction model.
The method according to claim 1, wherein before inputting the image to be predicted into the color spot prediction model obtained through pre-training for prediction processing, further comprising:

Determining color spot information in the image to be predicted, where the color spot information includes: the position and category of the color spot;

According to the color spot information in the image to be predicted, the image to be predicted is preprocessed to obtain preprocessed multi-frame images, and the multi-frame images respectively include: an image without color spots and a category marked with color spots Image;

Said inputting the image to be predicted into the pre-trained color spot prediction model for prediction processing includes:

The preprocessed multi-frame images are input into the pre-trained stain prediction model for prediction processing.
The method according to claim 1 or 2, wherein before inputting the image to be predicted into the color spot prediction model obtained through pre-training for prediction processing, further comprising:

Acquiring the target image to be processed, the target image to be processed includes color spot information;

A plurality of target channel images are respectively determined according to the target image to be processed, and the target channel images include: a multi-channel image with color spot information removed and a color spot category channel image;

Merge multiple target channel images and target noise images, and input them together into the trained stain prediction model in the neural network structure to be trained; the target noise images are randomly generated noise images.
The method according to claim 3, wherein said determining a plurality of target channel images respectively according to said target image to be processed comprises:

Determine the color speckle information in the target image to be processed;

Perform color speckle removal processing on the target image to be processed according to color speckle information in the target image to be processed, to obtain a target image to be processed with color speckle information removed.
The method according to claim 3 or 4, wherein said determining a plurality of target channel images respectively according to said target image to be processed comprises:

The stain detection process is performed on the target image to be processed to obtain the stain category channel image.
The method according to claim 5, wherein the step of performing stain detection processing on the target image to be processed to obtain the stain category channel image includes:

Respectively determine the position of each type of color spot information in the target image to be processed;

Gray information of a type corresponding to the color spot information is set at the position of the color spot information to obtain the color spot category channel image.
The method according to any one of claims 3 to 6, wherein the multiple target channel images and target noise images are combined and input together into the trained color spots in the neural network structure to be trained Before predicting the model, the method includes:

respectively performing normalization processing on the target channel image and the target noise image to obtain a target input image;

Said inputting a plurality of said target channel images and target noise images respectively into the neural network structure to be trained to obtain the trained stain prediction model includes:

The target input image is input into the neural network structure to be trained to obtain the trained stain prediction model.
A stain prediction device, characterized in that the device comprises: an acquisition module, a prediction module, and an output module;

The acquiring module is configured to acquire an image to be predicted;

The prediction module is configured to input the image to be predicted into a color speckle prediction model obtained in advance for prediction processing, and the color speckle prediction model is a fully convolutional generative adversarial network model;

The output module is configured to obtain a stain prediction result map through the stain prediction model.
The speckle prediction device according to claim 8, wherein the device further comprises a preprocessing module and another prediction module,

Wherein, the preprocessing module is configured to:

determining color speckle information in the image to be predicted, wherein the color speckle information includes: the position and category of the color speckle;

According to the color speckle information in the image to be predicted, the image to be predicted is preprocessed to obtain preprocessed multi-frame images, wherein the multi-frame images respectively include: an image without color speckle and an image marked with colored images of the Spot category, and

Wherein, the additional prediction module is configured for:

The multi-frame images after the preprocessing are input into the pre-trained stain prediction model for prediction processing.
The speckle prediction device according to claim 9, wherein the preprocessing module is further configured to:

Acquiring the target image to be processed, wherein the target image to be processed includes the color spot information;

A plurality of target channel images are respectively determined according to the target image to be processed, wherein the target channel images include: a multi-channel image and a color spot category channel image from which the stain information is removed;

Merge multiple target channel images and target noise images, and input them together into the trained stain prediction model in the neural network structure to be trained, wherein the target noise images are randomly generated noise images.
The speckle prediction device according to claim 10, wherein the preprocessing module is also configured to:

determining the color spot information in the target image to be processed;

performing color spot removal processing on the target image to be processed according to the color spot information in the target image to be processed, to obtain a target image to be processed with color spot information removed;
The speckle prediction device according to claim 10 or 11, wherein the preprocessing module is further configured to:

The stain detection process is performed on the target image to be processed to obtain a stain category channel image.
The speckle prediction device according to any one of claims 10 to 12, wherein the preprocessing module is further configured to:

Respectively determine the position of each type of color spot information in the target image to be processed;

Gray information of a type corresponding to the color spot information is set at the position of the color spot information to obtain the color spot category channel image.
The stain prediction device according to any one of claims 10 to 13, wherein the preprocessing module is further configured to:

respectively performing normalization processing on the target channel image and the target noise image to obtain a target input image;

The target input image is input into the neural network structure to be trained to obtain the trained stain prediction model.
A computer device, characterized by comprising: a memory and a processor, wherein a computer program that can run on the processor is stored in the memory, and when the processor executes the computer program, the above claim 1 is realized to the step of the method described in any one of 7.
A computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.