JP2023534328A - Color spot prediction method, device, equipment and storage medium - Google Patents

Abstract

The present invention provides a color spot prediction method, apparatus, equipment, and storage medium, and belongs to the technical field of image recognition processing. The method includes the step of acquiring a predicted image, inputting the predicted image to a pre-trained color speckle prediction model to perform prediction processing, wherein the color speckle prediction model is a fully convolutional adversarial generative network model. and obtaining a color speckle prediction result graph by the color speckle prediction model. The present invention is capable of predicting the mottling change of the user's facial skin.
[Selection drawing] Fig. 1

Description

[Related Application]
The present invention is based on the patent application in China filed with the Intellectual Property Office of China on June 24, 2021, with application number 2021107071008 and titled "Method, Apparatus, Equipment and Storage Medium for Color Spot Prediction". Priority is claimed, the entire contents of which are incorporated herein.

The present invention belongs to the technical field of image recognition and generation, and more particularly relates to a color spot prediction method, apparatus, equipment and storage medium.

With aging, the skin of the face suffers from problems such as aging and disease. In order to prevent the occurrence of problems such as aging and diseases and achieve early prevention, it is usually necessary to predict future changes in the face.
Regarding the prediction of skin changes in the prior art, the degree of sagging of the skin, the degree of wrinkles, etc. are generally predicted, but the prediction of color mottling change has not yet been realized. Therefore, it is necessary to develop a technology that can predict changes in skin color mottling in order for users to capture changes in skin color mottling over a certain period of time in the future.

The present invention provides a color blemish prediction method, apparatus, equipment, and storage medium that can realize prediction of color blemish changes on the skin of a user's face.
Embodiments of the present invention provide a color spot prediction method. The method includes a step of obtaining a predicted image, and a step of inputting the predicted image to a color spot prediction model obtained by training in advance to perform prediction processing, wherein the color spot prediction model is completely It is a convolutional adversarial generative network model; and obtaining a chroma prediction result graph by the chroma prediction model.
Preferably, the step of determining color speckle information in the predicted image before inputting the predicted image to a pre-trained color speckle prediction model to perform prediction processing, wherein the color speckle information is A step of preprocessing the predicted image based on the color spot information in the predicted image to obtain a preprocessed multi-frame image, comprising: the multi-frame image includes an image of achromatic speckles and an image labeled with color speckles. The step of inputting the predicted image into a pre-trained and acquired color spot prediction model and performing prediction processing includes inputting the preprocessed multi-frame image into a pre-trained and acquired color spot prediction model, It includes the step of performing a prediction process.
Preferably, the step of inputting the predicted image into a pre-trained and obtained color speckle prediction model and obtaining a target image before performing the prediction process, wherein the target image contains color speckle information. determining a plurality of target channel images based on the target processed image, the target channel images comprising a multi-channel image with speckle information removed and a speckle classified channel image; combining a plurality of said target channel images and target noise images and inputting them together into a neural network structure to be trained to obtain a trained chroma prediction model, wherein the target noise images are randomly selected from is the generated noise image.
Preferably, based on the target processed image, determining the plurality of target channel images comprises determining speckle information in the target processed image; and based on the speckle information in the target processed image, a step of performing a color spot removal process on the target image to be processed, and obtaining a target image from which the color spot information has been removed.
Preferably, the step of determining a plurality of target channel images based on the target processed image includes performing a speckle detection process on the target processed image to obtain a speckle classified channel image.
Preferably, the step of performing the color speckle detection process on the target processed image to obtain the color speckle classified channel image includes the step of determining the position of each category of color speckle information in the target processed image; setting the gray-scale information of the gradation corresponding to the chroma information at the position of the information to obtain the chroma-classified channel image.
Preferably, before the step of combining a plurality of target channel images and target noise images and jointly inputting them into a neural network structure to be trained to obtain a trained chroma prediction model, the method comprises: obtaining a target input image by performing a normalization process on each of the channel images and the target noise image, inputting a plurality of target channel images and target noise images into a neural network structure to be trained; Obtaining a trained chroma prediction model includes inputting a target input image into a neural network structure to be trained to obtain a trained chroma prediction model.
Another embodiment of the present invention provides a chroma predictor. The apparatus comprises an acquisition module, a prediction module, and an output module, wherein the acquisition module is used to acquire a predicted image, and the prediction module applies the predicted image to the pre-trained color speckle prediction. input to the model and used to perform prediction processing, the color speckle prediction model is a fully convolutional adversarial generative network model, and the output module obtains a color speckle prediction result graph according to the color speckle prediction model is used as
Preferably, the apparatus comprises a preprocessing module and another prediction module, wherein the preprocessing module is color speckle information in a predicted image, the color speckle information includes color speckle location and classification. color speckle information is determined, preprocessing is performed on the predicted image based on the color speckle information in the predicted image, and a multi-frame image after preprocessing, wherein the multi-frame image is Another prediction module is used to obtain an image of mottling and an image labeled with mottling classification, and another prediction module inputs the pre-processed multi-frame image into a pre-trained and acquired mottling prediction model. , is used to perform the prediction process.
Preferably, the preprocessing module further obtains a target processed image, respectively determines a plurality of target channel images based on the target processed image, combines the plurality of target channel images and the target noise image, and Second, it is used to obtain a trained chroma prediction model by inputting a neural network structure to be trained, the target processed image contains the chroma information, and the target channel image is the The target noise image is a randomly generated noise image, including the multi-channel image and the color-speckled channel image from which color speckle information has been removed.
Preferably, the preprocessing module determines the color speckle information in the target processed image, and performs color speckle removal processing on the target processed image based on the color speckle information in the target processed image. , are used to obtain the target processed image with mottle information removed.
Preferably, the preprocessing module is used to perform speckle detection processing on the target processed image to obtain a speckle classified channel image.
Preferably, the preprocessing module determines the position of each type of color speckle information in the target processed image, sets the type of grayscale information corresponding to the color speckle information at the position of the color speckle information, and sets the color It is used to acquire a mottled channel image.
Preferably, the preprocessing module performs a normalization process on each of the target channel image and the target noise image to obtain a target input image, inputs the target input image to a neural network structure to be trained, It is used to obtain a trained chroma prediction model.
Another embodiment of the present invention provides a computing device. The computer device includes a memory and a processor, the memory stores a computer program executed by the processor, and the processor executes the computer program to perform the steps of the color spot prediction method.
Another embodiment of the invention provides a computer-readable storage medium. A computer readable storage medium stores a computer program, and when the computer program is executed by a processor, it performs the steps of the color spot prediction method.

Embodiments of the present invention have at least the following effects.
In the color spot prediction method, device, equipment and storage medium provided by the embodiments of the present invention, a predicted image is obtained, and the predicted image is input to a color spot prediction model obtained by pre-training to perform prediction processing. , where the color blemish prediction model is a fully convolutional generative adversarial network model, and further, the color blemish prediction model can obtain a color blemish prediction result graph. In addition, by using the fully convolutional adversarial generation network structure as the blemish prediction module, it is possible to perform prediction processing on the change of skin blemish and inform the user of the change trend of skin blemish.

To describe the technical solutions in the embodiments of the present invention more clearly, the details are described below with reference to the drawings of the embodiments. It should be noted that the following drawings illustrate some embodiments of the invention by way of example and are not intended to limit the invention. The configuration and control can be changed as appropriate without departing from the gist of the present invention, and can be easily obtained by those skilled in the art from the drawings presented in this specification.
FIG. 1 is a flowchart of a chroma prediction method provided by an embodiment of the present invention; Fig. 2 is a flow chart of a color spot prediction method provided by an embodiment of the present invention; Fig. 3 is a flow chart of a color spot prediction method provided by an embodiment of the present invention; FIG. 4 is a flow chart of a color spot prediction method provided by an embodiment of the present invention; FIG. 5 is a flow chart of a color spot prediction method provided by an embodiment of the present invention; FIG. 6 is a flow chart of a chroma prediction method provided by an embodiment of the present invention; 1 is a structural conceptual diagram of a color spot prediction device provided by an embodiment of the present invention; FIG. 1 is a structural conceptual diagram of a computer device provided by an embodiment of the present invention; FIG.

To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the details of the present invention are hereinafter described with reference to the drawings showing the embodiments of the present invention. In addition, the embodiment shown below is a part of embodiment regarding this invention, and all the embodiments are not described in the following contents. Also, the components of the embodiments of the invention described and illustrated in the drawings herein can be arranged and changed in a variety of different configurations.
Accordingly, the description of the embodiments of the present invention provided in the accompanying drawings are not intended to limit the scope of the claims herein, but rather to describe the embodiments by way of example. In addition, all embodiments improved or modified by persons skilled in the art based on the embodiments of the present application are included in the scope of the claims of the present application.
In the accompanying drawings, similar elements are given similar symbols and descriptions, and for example, symbols and descriptions defined in one drawing refer to similar elements in other drawings. Please note. Also, unless otherwise specified, details regarding definitions and interpretations of the elements may be omitted.
Also, in the context of this specification, terms such as "first", "second", "third" are used only to distinguish descriptions and are not intended to indicate or imply relative importance. do not have.
In addition, in the prior art, there is no method for predicting changes in facial skin color mottling, and in an embodiment of the present invention, it is possible to predict changes in facial skin color mottling after a certain period of time in the future. It is possible to predict changes in color spots on one's face.
The details of the execution process of the color spot prediction method provided by the embodiments of the present invention are described below.

FIG. 1 is a flowchart diagram 1 of a color spot prediction method provided by an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps.

S110: Acquire a predicted image.

The predicted image is any image corresponding to the skin for which color mottle prediction needs to be performed, such as a user's photograph, face image, for example. The image may be an image corresponding to a preset size after trimming.

The execution subject of the method is a related program in a computer device, and specifically includes, for example, a preset program of a skin prediction device, a function of an electronic facial cleansing device, etc., but is not particularly limited here. can be set accordingly.
Also, the predicted image may be an image transmitted from another device to the computer device, or an image captured by the computer device using an imaging device or the like, and is not particularly limited here.
S120: The image to be predicted is input to a pre-trained and acquired color spot prediction model, and prediction processing is performed.

The chroma prediction model is a fully convolutional generative adversarial network model.

In another embodiment, after determining the predicted image, the predicted image can be input to a pre-trained mottle prediction model for prediction processing. The color spot prediction model is a pre-trained and acquired fully convolutional generative adversarial network model.

The color spot prediction model may be obtained by training in advance by the computer device, or may be transmitted to the computer device by another electronic device. In addition, it is not specifically limited here.

A fully convolutional generative adversarial network model is a generative adversarial network composed of multiple convolutional neural networks. A generative adversarial network is a network model that produces good output through interaction (learning by game theory) of a generative model and a discriminative model.

S130: Acquire a color blemish prediction result graph based on the color blemish prediction model.

In another embodiment, a color blemish prediction result graph can be obtained after performing prediction processing using a color blemish prediction model. The color blemish prediction result graph can display the changes in color blemish of the facial skin in the predicted image after a certain period of time, and the specific period value can be set according to actual needs. However, it is not particularly limited here.

Preferably, the color blemish prediction result graph includes a plurality of color blemish change states, and the color blemish change state is a color blemish state in which the color blemish of the facial skin in the predicted image changes after different periods of time. be.

One chroma prediction method provided by an embodiment of the present invention is to obtain a predicted image, and apply the predicted image to a pre-trained chroma prediction model (here, the chroma prediction model is fully It is a convolutional adversarial generation network model), prediction processing is performed, and a color speckle prediction result graph can be obtained by the color speckle prediction model. By using the fully convolutional adversarial generation network structure as the blemish prediction module, the future changes of skin blemishes can be predicted and processed, so that the user can predict the changing trend of skin blemishes.

Another execution process of the color spot prediction method provided by the embodiments of the present invention is described in detail below.

FIG. 2 is a flowchart diagram 2 of a color spot prediction method provided by an embodiment of the present invention. As shown in FIG. 2, the to-be-predicted image may be input to a pre-trained and acquired color-spot prediction model, and the following steps may be performed prior to the prediction process.

S210: Determine color spot information in the predicted image.

The speckle information includes the location and classification of the speckle.

The color spot information is color spot information on the skin in the predicted image, and specifically includes, for example, the position and classification of each color spot on the skin in the predicted image. The location of each speck can be recorded in the form of a coordinate range, and the speck classification can be recorded in the form of a label.

Preferably, a preset chroma identification algorithm can be used to obtain and determine the chroma information in the predicted image.

S220: Based on the color spot information in the predicted image, preprocess the predicted image to obtain a preprocessed multi-frame image.

The multi-frame image includes an image of achromatic speckles and an image labeled with color speckles.

In another embodiment, the preprocessing of the predicted image includes color speckle removal processing and color speckle determination processing. By the color spot removal process, it is possible to acquire the colorless image, that is, the predicted image without color spot information. The mottling determination process can obtain an image labeled with mottling classification. Note that the color mottling classification of the image can be represented by different grayscale values.

The step of inputting the predicted image to the pre-trained and acquired color spot prediction model and performing the prediction process may further include the following steps.

S230: The pre-processed multi-frame image is input to a pre-trained and acquired color spot prediction model for prediction processing.

In another embodiment, after determining each of the above multiple pre-processed multi-frame images, these images are combined and input together into a pre-trained and acquired color spot prediction model for prediction. , corresponding prediction results can be obtained.

Hereinafter, the execution process of the color spot prediction method provided by the embodiments of the present invention will be described in detail.

FIG. 3 is a flowchart diagram 3 of a color spot prediction method provided by an embodiment of the present invention. As shown in FIG. 3, the following steps may be further included before inputting the predicted image to the pre-trained color spot prediction model and performing the prediction process.

S310: Obtain a target processed image.

The target image to be processed may include mottle information.

The target processed image is a sample image used to train the mottle prediction model, the sample image comprising an image of the skin, the skin image containing the mottle information. .

The target image to be processed is a large number of pre-collected sample images (eg, images of complexion blemishes downloaded via a network, etc.), but the present invention is not limited thereto.

S320: Determining a plurality of target channel images respectively based on the target processed image.

The target channel image includes a multi-channel image with speckle information removed and a speckle-classified channel image.
In another embodiment, for each target processed image, processing is performed to obtain multiple target channel images. The multi-channel image from which color speckle information has been removed is obtained by performing color speckle removal processing on the target image to be processed, and the method is the same as that for obtaining the colorless speckle image. The color speckle-classified channel image is obtained by performing color speckle identification on the target image to be processed, and is the same as the method of obtaining the image labeled with the color speckle class.
S330: Combine multiple target channel images and target noise images and input them together into a trained neural network structure to obtain a trained chroma prediction model.
The target noise image is a randomly generated noise image.
In another embodiment, after acquiring a plurality of target channel images as described above, these target channel images and pre-generated target noise images are combined together to input and train a neural network structure to be trained. do. By this, after training, the color spot prediction model described above can be obtained.
A specific execution process of the color spot prediction method provided by the embodiments of the present invention will be described below.

FIG. 4 is a flowchart diagram 4 of a color spot prediction method provided by an embodiment of the present invention. Determining each of the plurality of target channel images based on the target processed image, as shown in FIG. 4, may further include the following steps.

S410: Determining color spot information in the target processed image.

Preferably, the mottle information can be determined after determining the target processed image. Specifically, determination of color spot information can be realized by the method of color spot identification described above.

S420: Based on the color speckle information in the target image to be processed, color speckle removal processing is performed on the target image to be processed to obtain the target image from which the color speckle information is removed.

Preferably, after obtaining the color spot information, color spot removal processing is performed based on the color spot information. All color mottle information in the target processed image is removed to obtain the target processed image. The target processed image does not contain speckle information.

In another embodiment, after obtaining the target processed image, channel processing may be performed to obtain each of the three color channels of red, green, and blue. Specifically, it is possible to acquire a red-channel image from which color-spot information is removed, a green-channel image from which color-spot information is removed, and a blue-channel image from which color-spot information is removed.

Determining the plurality of target channel images based on the target processed image may further include the following steps.

A color speckle detection process is performed on the target image to be processed to obtain a color speckle classified channel image.

Preferably, after obtaining the target image to be processed, the image may be subjected to color speckle detection processing to further obtain a color speckle classified channel image. This process is as follows.

FIG. 5 is a flow chart diagram 5 of a chroma prediction method provided by an embodiment of the present invention. As shown in FIG. 5, the step of performing color speckle detection processing on the target processed image to obtain a color speckle classified channel image further includes the following steps.

S510: Determine the location of each category of mottling information in the target processed image.

Preferably, the method of color speckle identification can determine the location of each class of color specks in the target processed image.

S520: Set the gray level information of the classification corresponding to the color speckle information at the position of the color speckle information, and acquire the color speckle classified channel image.

Preferably, after obtaining each class of color spot location, the corresponding location can be set with the corresponding gray scale value. Different classes of mottling are characterized by different grayscale values. A particular location and range with that grayscale value can indicate the location and size of the speckle. After determining and setting the gray-level information corresponding to each category of mottle information in the image, the above mottle-categorized channel image can be obtained. This is specifically a channel image with different color mottles represented in different shades of gray.

Another execution process of the color spot prediction method provided by the embodiments of the present invention is described in detail below.

FIG. 6 is a flow chart diagram 6 of a chroma prediction method provided by an embodiment of the present invention. As shown in FIG. 6, before combining a plurality of target channel images and target noise images and inputting them together into a neural network structure to be trained to obtain a trained chroma prediction model, the method is , may further include the following steps.

S610: Normalize each of the target channel image and the target noise image to obtain a target input image.

Preferably, in the above-described trained chroma prediction model obtained by combining a plurality of target channel images and target noise images and inputting them together into a neural network structure to be trained, the target channel image is including a red channel image with speckle information removed, a green channel image with speckle information removed, a blue channel image with speckle information removed, and a color speckle classified channel image; In combination, a 5-channel image can be acquired. These can be input together into a trained neural network structure to obtain a trained chroma prediction model.

Preferably, in the normalization process, a red channel image with color speckle information removed in the target channel image, a green channel image with color speckle information removed, a blue channel image with color speckle information removed, and the above-described target noise image. It can be normalized to the interval (-1,1). Normalizing the speckle-classified channel image to the (0,1) interval is

In this formula, Img is a three-channel image from 0 to 255,

Also, the step of combining a plurality of target channel images and target noise images and inputting them together into a neural network structure to be trained to obtain a trained chroma prediction model may further include the following steps: can.

S620: Input the target input image into the neural network structure to be trained to obtain a trained color spot prediction model.

Preferably, after obtaining the above target input image, the target input image can be input into the above neural network structure to obtain a trained mottle prediction model.

Details of the structure of the color spot prediction model used in the embodiment of the present invention will be described below.

The model uses an encoding-decoding structure, the upsampling of the decoding unit uses a combination of nearest neighbor upsampling + convolutional layers, the activation function of the output layer is Tanh, Specific structural relationships are shown in Table 1

Leakyrelu is a type of conventional activation function in deep learning, negative slope is one configuration parameter of the activation function, kh is the height of the convolution kernel, and kw is the convolution kernel padding is the pixel value of the feature map extension during the convolution operation, stride is the convolution step size, group is the number of groups in the convolution kernel, scale_factor, Mode is Both are parameters of the upsampling layer. scale_factor represents the size when upsampling is doubled, and Mode=nearest represents using nearest neighbor upsampling.

Preferably, the model includes a discriminant network portion, and in an embodiment of the present invention that can discriminate between real and false images of different resolutions, respectively, three scales of discriminators are used, 512x512, A 256x256, 128x128 resolution image is discriminated. Images of different resolutions can be obtained by the following downsampling.

Preferably, the model can be set to 20000 samples during training, and for each sample image, multiple gains (e.g. flips, rotations, translations, affine transformations, exposure, contrast adjustment, blurring, etc.).

Preferably, the optimization algorithm for network training uses the Adam algorithm, with a learning rate of 0.0002 for the generator network and a learning rate of 0.0001 for the discriminant network.

Preferably, the formula for the loss function of the model is
Generate is the output of the network and GT is the target generated image. L ₁ , L ₂ are loss functions, L _vgg is the Perceptual Loss, and L _adv is the generative adversarial loss function. Lperceptual is the perceptual loss, which refers to inputting the network's output (generative diagram) generate and GT into another network, extracting the feature tensors of the corresponding layers, and computing the difference between the feature tensors. means. where i is the i-th sample.

Hereinafter, the apparatus, equipment and storage medium for executing the color spot prediction method provided by the present invention will be described in detail. The specific implementation process and technical effects can be referred to the above content, and the details are omitted here.

FIG. 7 is a structural conceptual diagram of a color spot prediction device provided by an embodiment of the present invention. As shown in FIG. 7, the device comprises an acquisition module 100 , a prediction module 200 and an output module 300 .

Acquisition module 100 is configured to acquire a predicted image.

The prediction module 200 is configured to input the predicted image to a pre-trained and acquired color speckle prediction model for prediction processing, and the color speckle prediction model is a fully convolutional generative adversarial network model.

The output module 300 is configured to obtain a color speckle prediction result graph by the color speckle prediction model.

Preferably, the device further comprises a pretreatment module 400 . The pre-processing module 400 is configured to determine mottle information in the predicted image. The speckle information includes the location and classification of the speckle. Based on the color spot information in the predicted image, the predicted image is preprocessed to obtain a preprocessed multi-frame image. The multi-frame image includes an image of achromatic speckles and an image labeled with color speckles. Also, the prediction module 200 may be configured to perform prediction processing by inputting the pre-processed multi-frame image into a color spot prediction model obtained by training in advance.

Preferably, the preprocessing module 400 may be configured to obtain the target processed image. The target processed image includes color mottle information. A plurality of target channel images are determined based on the target processed image. The target channel image includes a multi-channel image with speckle information removed and a speckle-classified channel image. A plurality of target channel images and target noise images are combined and input together into a trained neural network structure to obtain a trained chroma prediction model. The target noise image is a randomly generated noise image.

Preferably, the pre-processing module 400 may be used to determine mottle information in the target processed image. Based on the color spot information in the target image to be processed, color spot removal processing is performed on the target image to be processed, and a target image to be processed from which the color spot information is removed is obtained. Channel processing can be performed on the target image from which the color spot information has been removed, and a multi-channel image from which the color spot information has been removed can be obtained.

Preferably, the pre-processing module 400 may be used to perform speckle detection processing on the target processed image to obtain a speckle classified channel image.

Preferably, the pre-processing module 400 may be used to determine the location of each type of speckle information in the target processed image. By setting gray gradation information of a type corresponding to the color speckle information at the position of the color speckle information, a channel image classified by color speckle type can be acquired.

Preferably, the pre-processing module 400 may be used to perform a normalization process on each of the target channel image and the target noise image to obtain a target input image. Also, the target input image is input into the neural network structure to be trained to obtain a trained color spot prediction model.

The above-described apparatus implements the method provided by the above-described embodiments, and its implementation principle and technical effects are similar, and the details are omitted here.
The above modules may be configured to be implemented in one or more integrated circuits for performing the above methods. For example, one or more specific integrated circuits (Application Specific Integrated Circuit, ASIC), or one or more microprocessors, or one or more field programmable gate arrays (FPGA), etc. are mentioned. Also, when certain modules described above are implemented in the form of processing element scheduler code, the processing element may invoke a general-purpose processor (e.g., Central Processing Unit (CPU)) or other program code. available processors). Also, these modules can be integrated and implemented in the form of a system-on-a-chip (SOC).
FIG. 8 is a structural conceptual diagram of a computer device provided by an embodiment of the present invention. As shown in FIG. 8, the computing device comprises memory 500 and processor 600 . Memory 500 stores computer programs that are executed by processor 600 . When the processor 600 executes the computer program, it performs the steps of the color mottling prediction method described above.
An embodiment of the present invention provides a computer-readable storage medium. A computer program is stored in the storage medium, and when the computer program is executed by the processor, the above steps of the color spot prediction method are executed.
In some embodiments provided by the present invention, the disclosed apparatus and methods may also be implemented in other ways. The above device embodiments are only examples, for example, the division of units is only the division of logical functions, and other division methods may be used in actual implementations. Also, multiple units or elements may be combined or integrated into another system. Some configurations may be omitted or not perform their function. Also, a mutual or direct coupling or communicative connection may be electrical, mechanical or otherwise through an indirect coupling or communicative connection of some interface, device or unit.
Units described as separate elements may or may not be physically separate, and elements depicted as units may or may not be physical units. Well, it can also be co-located or distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution in this embodiment.
Furthermore, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit physically exists alone, or two or more units are integrated into one unit. can be Also, the integration unit described above can be implemented in the form of hardware or in the form of a functional unit of hardware and software.
The above integrated units implemented in the form of software functional units can be stored in a computer readable storage medium. Also, the software functional unit described above is stored on a storage medium and includes several commands for making a computer device (personal computer, server, network device, etc.) or processor functional. This allows the methods of various embodiments of the invention to be carried out. The storage medium includes USB memory, external hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, optical disk, and various other storage media capable of storing program code. including.
What has been described above are only specific embodiments of the present application, and the claims of the present application are not limited thereto. Also, modifications or improvements can be easily conceived by those skilled in the art who are familiar with the technical scope disclosed in the present application. All such modifications and improvements are included within the scope of the claims of this application. The scope of protection claimed in this application shall follow the scope of the claims.
The above descriptions are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, this application is susceptible to various modifications and changes. Any equivalent change, replacement or improvement without departing from the technical idea of the present invention is included in the claims of the present application.

The present invention provides a color spot prediction method, apparatus, equipment and storage medium. The method includes the steps of acquiring a predicted image, inputting the predicted image to a pre-trained color speckle prediction model to perform prediction processing, and the color speckle prediction model is a fully convolutional generative adversarial network model. and obtaining a color spot prediction result graph by the color spot prediction model. The present invention is capable of predicting the mottling change of the user's facial skin.
Moreover, the color spot prediction method, apparatus, equipment, and storage medium of the present invention have reproducibility and can be used for various industrial applications. For example, the color spot prediction method, apparatus, equipment, and storage medium of the present invention can be used in fields requiring image recognition processing.

Claims (16)

obtaining a predicted image;
inputting the predicted image to a pre-trained color speckle prediction model to perform prediction processing, wherein the color speckle prediction model is a fully convolutional generative adversarial network model;
and obtaining a color spot prediction result graph from the color spot prediction model. Before inputting the predicted image to a color spot prediction model obtained by training in advance and performing prediction processing,
determining color speckle information in the predicted image, wherein the color speckle information includes color speckle location and classification;
A step of performing preprocessing on the predicted image based on color spot information in the predicted image to obtain a preprocessed multi-frame image, wherein the multi-frame image has no color spots. including the image and the image labeled with the color mottling classification;
The step of performing prediction processing by inputting the predicted image to a color spot prediction model obtained by training in advance,
2. The method of claim 1, further comprising inputting the pre-processed multi-frame image into a pre-trained color spot prediction model for prediction processing. Before performing prediction processing by inputting the predicted image to a color spot prediction model obtained by training in advance,
obtaining a target processed image, said target processed image including color mottle information;
determining a plurality of target channel images based on the target processed image, the target channel images comprising a multi-channel image with speckle information removed and a speckle classified channel image;
combining a plurality of the target channel images and the target noise images and inputting them together into a trained neural network structure to obtain a trained chroma prediction model, wherein the target noise images are random 3. A method according to claim 1 or 2, further comprising the step of being a noise image generated in . determining a plurality of target channel images based on the target processed image,
determining mottling information in the target processed image;
and obtaining a target processed image from which the color spot information is removed by performing a color spot removal process on the target image to be processed based on the color spot information in the target image to be processed. 4. A method according to claim 3. determining a plurality of target channel images based on the target processed image,
5. A method according to claim 3 or 4, comprising performing a speckle detection process on the target processed image to obtain the speckle classified channel image. The step of performing color speckle detection processing on the target image to be processed and acquiring the color speckle-classified channel image includes:
determining the location of each category of mottling information in the target processed image;
6. The method according to claim 5, further comprising the step of: setting grayscale information corresponding to the color speckle information at the position of the color speckle information, and acquiring the color speckle classified channel image. Method. Before combining and jointly inputting the plurality of target channel images and target noise images into a neural network structure to be trained to obtain a trained chroma prediction model, the method comprises:
obtaining a target input image by performing a normalization process on each of the target channel image and the target noise image;
inputting the plurality of target channel images and target noise images into a neural network structure to be trained to obtain a trained chroma prediction model;
A method according to any one of claims 3 to 6, characterized in that it comprises inputting the target input image into a neural network structure to be trained to obtain a trained color spot prediction model. . A color spot prediction device comprising an acquisition module, a prediction module, and an output module,
the acquisition module is used to acquire a predicted image;
The prediction module is used to input the predicted image to a pre-trained and acquired color speckle prediction model to perform prediction processing, and the color speckle prediction model is a fully convolutional adversarial generative network model. can be,
The color spot prediction device, wherein the output module is used to obtain a color spot prediction result graph by the color spot prediction model. The color spot prediction device includes a preprocessing module and another prediction module,
The pretreatment module is
color speckle information in the predicted image, wherein the color speckle information determines the color speckle information including the position and classification of the color speckle;
Preprocessing is performed on the predicted image based on color spot information in the predicted image, and a preprocessed multi-frame image is obtained, wherein the multi-frame image includes an image of colorless spots and an image of color spots. used to obtain an image labeled classification,
The other prediction module is
9. The color spot prediction device according to claim 8, wherein the multi-frame image after the preprocessing is input to a color spot prediction model obtained by training in advance and used to perform prediction processing. . The pretreatment module further comprises:
obtain a target processed image;
determining each of a plurality of target channel images based on the target processed image;
a plurality of said target channel images and target noise images are combined and used to jointly input into a trained neural network structure to obtain a trained chroma prediction model;
the target image to be processed includes the color spot information;
the target channel image includes a multi-channel image from which the color speckle information has been removed and a color speckle classified channel image;
10. The color spot prediction device according to claim 9, wherein the target noise image is a randomly generated noise image. The pretreatment module further comprises:
determining the color mottling information in the target processed image;
Based on the color speckle information in the target image, the target image is subjected to color speckle removal processing to obtain a target image from which the color speckle information is removed. The color spot prediction device according to claim 10, characterized by: The pretreatment module further comprises:
12. The color speckle prediction apparatus according to claim 10, wherein the color speckle prediction apparatus is used to perform color speckle detection processing on the target image to be processed to obtain a color speckle classified channel image. The pretreatment module further comprises:
determining the location of each type of color mottling information in the target processed image;
10. Claims 10 to 10, characterized in that gray gradation information of a type corresponding to the color speckle information is set at the position of the color speckle information, and used to acquire the color speckle classified channel image. 13. The color spot prediction device according to any one of 12. The pretreatment module further comprises:
obtaining a target input image by performing normalization processing on each of the target channel image and the target noise image;
14. The method according to any one of claims 10 to 13, wherein the target input image is used to input a neural network structure to be trained to obtain a trained color spot prediction model. color spot prediction device. A computer device comprising a memory and a processor, wherein the memory stores a computer program executed by the processor, and when the processor executes the computer program, the A computing device for performing the steps of the color spot prediction method according to any one. A computer readable storage medium, said computer readable storage medium storing a computer program, said computer program being executed by a processor, said color according to any one of claims 1 to 7. A computer-readable storage medium for performing steps of a speckle prediction method.