WO2019144855A1 - 图像处理方法、存储介质和计算机设备 - Google Patents

图像处理方法、存储介质和计算机设备 Download PDF

Info

Publication number
WO2019144855A1
WO2019144855A1 PCT/CN2019/072491 CN2019072491W WO2019144855A1 WO 2019144855 A1 WO2019144855 A1 WO 2019144855A1 CN 2019072491 W CN2019072491 W CN 2019072491W WO 2019144855 A1 WO2019144855 A1 WO 2019144855A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sample
stage
model
order
Prior art date
Application number
PCT/CN2019/072491
Other languages
English (en)
French (fr)
Inventor
李旻骏
黄浩智
马林
刘威
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP19743676.9A priority Critical patent/EP3745349A4/en
Publication of WO2019144855A1 publication Critical patent/WO2019144855A1/zh
Priority to US16/880,883 priority patent/US11276207B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present application relates to the field of computer technology, and in particular, to an image processing method, a storage medium, and a computer device.
  • image-based processing has become more diverse.
  • image processing techniques such as image feature conversion processing, such as image color feature conversion, image light and shadow feature conversion or image style feature conversion.
  • the image feature conversion processing is realized mainly by diffusing the image texture of the target feature to the image region of the image to be processed based on the texture synthesis.
  • mismatching is likely to occur, resulting in distortion of the resulting image.
  • an image processing method a storage medium, and a computer device are provided.
  • An image processing method is performed by a computer device, the method comprising:
  • a non-volatile storage medium storing computer readable instructions, when executed by one or more processors, causes one or more processors to perform the following steps:
  • a computer device comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor such that the processor performs the following steps:
  • FIG. 1 is a schematic flow chart of an image processing method in an embodiment
  • FIG. 2 is a schematic diagram of a model of a first stage image conversion model in one embodiment
  • FIG. 3 is a logic diagram of an image processing process in an embodiment
  • FIG. 4 is a logic diagram of a first stage image transformation model in an embodiment
  • FIG. 5 is a logic diagram of training a second stage image conversion model in one embodiment
  • FIG. 6 is a schematic diagram of an image before and after image conversion in an embodiment
  • Figure 7 is a block diagram showing the structure of an image processing apparatus in an embodiment
  • Figure 8 is a block diagram showing the structure of an image processing apparatus in another embodiment
  • Figure 9 is a diagram showing the internal structure of a computer device in an embodiment.
  • Figure 10 is a diagram showing the internal structure of a computer device in another embodiment.
  • FIG. 1 is a schematic flow chart of an image processing method in an embodiment. This embodiment is mainly illustrated by the method applied to a computer device. Referring to FIG. 1, the image processing method specifically includes the following steps:
  • the image category is the category to which the image feature reflected by the image belongs.
  • the image feature may specifically be a color feature, a style feature, or a content feature.
  • image categories classified according to color features such as black and white image categories or color image categories, etc.
  • image categories classified according to style features such as sketch image categories or oil painting image categories
  • image categories classified according to content features such as apple images Category or orange image category, etc.
  • the image to be processed is an image to be subjected to image category conversion.
  • the computer device may be a user terminal, and the user terminal may directly obtain a to-be-processed image that belongs to a certain image category uploaded by the user, and may also accept a to-be-processed image that belongs to a certain image category and transmitted by other user terminals.
  • the computer device may also be a server, and the server may receive a to-be-processed image belonging to a certain image category uploaded by the user terminal.
  • image conversion refers to converting an image of one image category into an image of another image category.
  • the image transformation model is a machine learning model with image transformation ability after training. Machine learning English is called Machine Learning, referred to as ML.
  • the machine learning model can adopt a neural network model, a support vector machine (SVM) or a logistic regression model.
  • the first stage image transformation model is used to initially convert an image of the first image category into an image of the second image category.
  • the image transformed by the first stage image transformation model allows for some degree of error with the image of the second image category that is intended to be converted. It can be understood that the image detail feature conversion capability of the first stage image transformation model is not limited here, and the transformation can be completed on the overall feature of the image.
  • the computer device may train the first stage image conversion model according to the first sample belonging to the first image category and the second sample belonging to the second image category.
  • the first stage image conversion model may specifically employ a convolutional neural network model.
  • the structure of the convolutional neural network model may specifically be an encoder-decoder structure.
  • the encoder can be composed of multiple layers of convolutional layers, and the decoder can be composed of multiple layers of deconvolution layers.
  • the encoder part converts the input image into a feature map with low resolution but a large number of channels, and the feature map can be regarded as an encoding of the original image.
  • the decoder portion then decodes the feature map into an image of the second image category.
  • FIG. 2 shows a schematic diagram of a model of a first stage image transformation model in one embodiment.
  • the first stage image conversion model is an encoder-decoder structure.
  • the encoder structure 210 includes a three-layer convolution layer 211
  • the decoder structure 220 includes a three-layer deconvolution layer 221.
  • the first stage image conversion model when the first stage image conversion model is trained, when the first sample belonging to the first image category and the second sample belonging to the second image category are in one-to-one correspondence, the first sample is corresponding to the first sample.
  • the second sample is used as the training label of the first sample, so that the first stage image conversion model is obtained by supervised training according to the first sample and the corresponding training label.
  • the construction when training the first stage image conversion model, when there is no first sample and the corresponding second sample, and there is no first sample corresponding to the second sample, then the construction will be
  • the image of one image category is converted to a first stage image transformation model of the image of the second image category, and the first stage image inverse transformation model of the image of the second image category is converted to an image of the first image category.
  • the first sample is sequentially subjected to a first-stage image transformation model and a first-stage image inverse transformation model to form a closed loop
  • the second sample is sequentially subjected to a first-stage image inverse transformation model and a first-stage image transformation model to form a closed loop, thereby utilizing the loop
  • first sample and the second sample herein means that the first sample and the second sample are identical except for the image features used to distinguish the categories.
  • first image category is a black and white image category
  • second image category is a color image category.
  • apples in the first sample and the second sample are only different in color, and the size or position is the same.
  • Figure 3 shows a logic diagram of an image processing process in one embodiment.
  • the computer device may input the image to be processed x belonging to the first image category A into the first-stage image conversion model G1 to obtain the first intermediate image y1 output by the first-stage image conversion model G1.
  • the image conversion may be a conversion of non-image content, such as image style conversion, image color conversion, or image depth conversion.
  • Image conversion can also be the transformation of image content, such as modifying objects in an image. Specifically, the apple image of the apple image category is modified to the orange image of the orange image category.
  • the second stage image transformation model is used to further convert the image output by the first stage image conversion model into an image of the second image category. It can be understood that the second stage image transformation model is required to complete the transformation on the image detail features.
  • the computer device may train the second intermediate image conversion model according to the first intermediate image output by the first stage image conversion model and the second sample belonging to the second image category.
  • the second-stage image conversion model may specifically employ a convolutional neural network model.
  • the structure of the convolutional neural network model may specifically be an encoder-decoder structure.
  • the encoder may be composed of a plurality of convolutional layers
  • the decoder may be composed of a plurality of deconvolution layers.
  • the first sample when training the second-stage image conversion model, when the first sample belonging to the first image category and the second sample belonging to the second image category are in one-to-one correspondence, the first sample is corresponding to The second sample is used as a training label of the first intermediate image corresponding to the first sample, so that the second stage image conversion model is obtained by supervised training according to the first intermediate image and the corresponding training label.
  • the construction when training the first stage image conversion model, when there is no first sample and the corresponding second sample, and there is no first sample corresponding to the second sample, then the construction will be An intermediate image is converted to a second stage image transformation model of the image of the second image category, and a second stage image inverse transformation model is used to convert the image of the second image category to the first intermediate image.
  • the first sample is sequentially subjected to a first-stage image transformation model, a second-stage image transformation model, a first-stage image inverse transformation model, and a second-stage image inverse transformation model to form a closed loop
  • the second sample is sequentially subjected to the first-stage image reverse direction.
  • the transformation model, the second-stage image inverse transformation model, the first-stage image transformation model and the second-stage image transformation model form a closed loop, so that the second-stage image transformation model is obtained by unsupervised training using the loop consistency constraint.
  • the first intermediate image y1 may be continuously input to the second-stage image conversion model G2 to obtain the second-stage image conversion model G2 output.
  • weight is a relative concept for an object.
  • the weight of an object reflects the relative importance of the object in the overall evaluation. Since computer equipment is usually processed in the form of a matrix of digital image data when processing images, the weight of the image is also in the form of a matrix.
  • the computer device may preset a weight mapping function to obtain a weight matrix corresponding to the second intermediate image by the weight mapping function mapping.
  • a weight mapping function may be preset by the computer device.
  • the computer device can use the image to be processed and the second intermediate image as independent variables of the weight mapping function to obtain a weight matrix corresponding to the intermediate image as the independent variable.
  • the computer device may use the to-be-processed image and the first intermediate image as arguments of the weight mapping function to obtain a weight matrix corresponding to the intermediate image as the independent variable.
  • the computer device may also use the to-be-processed image, the first intermediate image, and the second intermediate image as the independent variables of the weight mapping function to obtain a weight matrix corresponding to one of the intermediate images, and further obtain another weight according to the weight matrix.
  • the computer device may further use the first intermediate image and the second intermediate image as the independent variables of the weight mapping function to obtain a weight matrix corresponding to one of the intermediate images, and further obtain a weight matrix corresponding to the other intermediate image according to the weight matrix.
  • One of the intermediate images herein may be the first intermediate image or the second intermediate image. It can be understood that the order of execution of S108 and S110 is not limited herein. S108 may be performed before S110, may be performed after S110, and may also be executed in synchronization with S110. The order of execution of S108 and S110 depends on the weight mapping function previously set by the computer device.
  • S108 includes: inputting the image to be processed, the first intermediate image and the second intermediate image into the first weight prediction model, to obtain a second weight matrix corresponding to the second intermediate image.
  • S110 includes: obtaining, according to the second weight matrix, a first weight matrix corresponding to the first intermediate image; and a sum of the first weight matrix and the second weight matrix is a preset matrix.
  • the weight prediction model is a machine learning model with weight prediction ability after training.
  • the weight prediction model may specifically adopt a shallow convolutional neural network model.
  • the shallow convolutional neural network model may specifically be composed of three convolutional layers.
  • the structure of the first two layers of convolution layer is Convolution-InstanceNorm-ReLU (activation function) structure, and the structure of the last layer of convolution layer is Convolution-Sigmoid (activation function) )structure.
  • the first weight prediction model is a machine learning model that performs weight prediction when converting an image of a first image category into an image of a second image category.
  • the second weight prediction model mentioned hereinafter is a machine learning model for performing weight prediction when converting an image of the second image category into an image of the first image category.
  • the input of the first weight prediction model is a to-be-processed image, a first intermediate image, and a second intermediate image, and the image sizes of the three frames of images are the same.
  • the output of the first weight prediction model is a weight matrix that is consistent with the size of the input image, and each element of the weight matrix represents the weight of the pixel value of the second intermediate image at the corresponding pixel location.
  • the first weight matrix and the second weight matrix are matrices of the same size and dimension, and the sum of the first weight matrix and the second weight matrix is a preset matrix, and the preset matrix may be both a size and a dimension and a second The all-one matrix with the same weight matrix.
  • the matrix elements of the all-one matrix are all 1.
  • the computer device may input the image to be processed, the first intermediate image and the second intermediate image into the first weight prediction model, and obtain a second weight matrix corresponding to the second intermediate image output by the first weight prediction model.
  • the sum of the first weight matrix corresponding to the first intermediate image and the second weight matrix corresponding to the second intermediate image is a preset matrix.
  • the computer device may subtract the second weight matrix corresponding to the second intermediate image by using the preset matrix to obtain a first weight matrix corresponding to the first intermediate image.
  • the computer device inputs the first intermediate image y1 output by the first-stage image conversion model G1 and the second intermediate image y2 output by the second-stage image conversion model G2 into the first weight prediction model Gf to obtain a first weight.
  • the weight matrix corresponding to the second intermediate image y2 outputted by the prediction model Gf is obtained, thereby obtaining a weight matrix corresponding to the first intermediate image y1.
  • the weighted prediction learning is performed by using the powerful learning and presentation capabilities of the machine learning model, and the trained machine learning model predicts the weight, which is better than the traditional method for predicting the weight process.
  • the first intermediate image and the second intermediate image are merged according to the corresponding first weight matrix and the second weight matrix to obtain a target image corresponding to the image to be processed and belonging to the second image category.
  • the computer device may weight the first intermediate image and the second intermediate image according to respective weight matrixes to obtain a target image corresponding to the image to be processed and belonging to the second image category.
  • the first intermediate image and the second intermediate image are merged according to respective weight matrixes by the adaptive fusion layer to obtain a target image corresponding to the image to be processed and belonging to the second image category.
  • S112 includes: multiplying each pixel value of the first intermediate image by each matrix element of the first weight matrix to obtain a first target image belonging to the second image category; Each pixel value is multiplied by each matrix element of the second weight matrix to obtain a second target image belonging to the second image category; according to the first target image and the second target image, corresponding to the image to be processed and belonging to the second image The target image of the image category.
  • each pixel element in the pixel matrix has a one-to-one weight element in the weight matrix, that is, each weight element of the weight matrix represents the weight of the pixel value of the corresponding pixel element in the corresponding intermediate image.
  • the computer device multiplies each pixel value of the first intermediate image by each matrix element of the first weight matrix to obtain a first target image belonging to the second image category, and then sets each pixel value of the second intermediate image. And multiplying each matrix element of the second weight matrix by bit to obtain a second target image belonging to the second image category, and then adding each pixel value of the first target image and the second target image bit by bit to obtain an image to be processed Corresponding and belonging to the target image of the second image category.
  • the computer device may calculate the fused target image according to the first intermediate image y1, the weight matrix, the second intermediate image y2, and the weight matrix.
  • the finally obtained target image combines the outputs of the plurality of phase models, which can largely overcome the problem of image distortion and improve the image conversion effect.
  • the image to be processed when the image to be processed belonging to the first image category is converted into an image belonging to the second image category, the image to be processed is automatically subjected to the first-stage image conversion model and the second-stage image conversion model, and After separately determining the weight matrix corresponding to the outputs of the two-stage models, the target images are obtained by adaptively merging the outputs of the two models according to the corresponding weight matrix.
  • the image to be processed is processed by a plurality of stages of the model, and the resulting target image is combined with the output of the plurality of stage models, which can greatly overcome the problem of image distortion and improve the target image converted from the image to be processed. Conversion effect.
  • S104 includes: down-sampling the image to be processed to obtain a compressed image with reduced image size; inputting the compressed image into the first-stage image conversion model, and outputting the first image having the same image size as the compressed image image.
  • S106 includes: up-sampling the first intermediate image to obtain an enlarged image having the same image size as the image size of the image to be processed; inputting the enlarged image into the second-stage image conversion model, and outputting the image size to be the same as the image size of the enlarged image Two intermediate images.
  • downsampling or downsampling is a way of processing image compression.
  • the image size of the image after the downsampling operation is reduced, and the degree of reduction is related to the sampling period of the downsampling.
  • Upsampling, or image interpolation is a way of magnifying an image.
  • the image size of the image after the upsampling operation is amplified, and the degree of amplification is related to the sampling period of the upsampling.
  • the computer device can perform a downsampling operation on the image to be processed.
  • the method of downsampling may specifically be mean sampling or extreme sampling.
  • the resolution of the image obtained after the downsampling operation is reduced to 1/4 of the resolution of the image to be processed.
  • the computer device further inputs the compressed image with the reduced image size after the downsampling into the first stage image conversion model, and obtains the first intermediate image whose image size output by the first stage image conversion model is the same as the image size of the compressed image. Since the image size of the first intermediate image is smaller than the size of the image to be processed at this time, the computer device needs to perform the previous down sampling operation on the first intermediate image before inputting the first intermediate image into the second-stage image conversion model.
  • the upsampling operation increases the resolution of the feature map obtained after the upsampling operation to 4 times the resolution of the feature map before the upsampling operation to ensure the second intermediate image and the output of the second stage image conversion model output The resolution of the processed image is the same.
  • the image when the image is converted in the first stage, the image is compressed and converted, so that the amount of image data processed by the model is small, so that when the image conversion accuracy is not high, the image conversion effect can be ensured, and Improve image conversion time.
  • the image processing method further includes the step of training the model.
  • the step of training the first stage image transformation model is specifically as follows: acquiring a first sample belonging to the first image category and a second sample belonging to the second image category; and sequentially passing the first sample through the first stage image transformation model and The first stage image reverse transformation model obtains the first sample single-stage recovery image; the second sample is sequentially subjected to the first stage image reverse transformation model and the first stage image transformation model to obtain a second sample single-stage recovery image; The difference between the first sample and the first sample single-stage restored image, and the difference between the second sample and the second sample single-stage restored image, and the first-stage image transformation model and the first-stage image inverse transformation model are adjusted until the training is satisfied. End training when the condition is stopped.
  • the first sample is an image belonging to the first image category, and is used for training the first-stage image transformation model and the first-stage image inverse transformation model.
  • the second sample is an image belonging to the second image category, and is also used to train the first stage image transformation model and the first stage image inverse transformation model.
  • the first stage image conversion model is a machine learning model that converts an image of the first image category into an image of the second image category, and the first stage image inverse transformation model converts the image of the second image category into an image of the first image category. Machine learning model.
  • the first sample single-stage recovery image is an image belonging to the first image category, and the first sample is converted into the image of the second image category after the first-stage image conversion model, and then recovered by the first-stage image reverse transformation model.
  • the second sample single-stage recovery image is an image belonging to the second image category, which is obtained after the second sample is converted into the image of the first image category by the first-stage image reverse transformation model, and then restored by the first-stage image transformation model. An image of the second image category.
  • the scenario applied in this embodiment is that there is no scenario with the first sample and the corresponding second sample, and there is no first sample corresponding to the second sample, that is, the actual is not available as the first scenario.
  • the computer device may acquire the first sample belonging to the first image category, and sequentially pass the first sample to the first stage image transformation model and the first stage image inverse transformation model to obtain a first sample single-stage recovery image.
  • the first sample single-stage recovery image is an image intended to be restored to the first sample, then it can be understood that the first-stage image transformation model and the first-stage image inverse transformation model
  • the purpose of training is to minimize the difference between the first sample and the first sample single-stage recovery image.
  • the computer device may acquire the second sample belonging to the second image category, and sequentially pass the second sample through the first sample.
  • the one-stage image reverse transformation model and the first-stage image transformation model obtain a single-stage single-stage restoration image.
  • the single-sample recovery image of the second sample is an image intended to be restored to the second sample, so it can be understood that the training purpose of the first-stage image transformation model and the first-stage image inverse transformation model is minimized.
  • the difference between the first sample and the first sample single-stage restored image, and the difference between the second sample and the second sample single-stage restored image may specifically be a norm of the image pixel difference value or 2 Norm and so on.
  • FIG. 4 shows a logic diagram of a training first stage image transformation model in one embodiment.
  • the computer device may acquire the first sample x' belonging to the first image category A, and input x' sequentially into the first-stage image conversion model G1 and the first-stage image inverse transformation model F1 to obtain the first sample list.
  • the stage restores the image F 1 (G 1 (x')).
  • F 1 (G 1 (x')) is the image intended to return to x′, then it can be understood that the training purpose of the model is to minimize the difference between F 1 (G 1 (x′)) and x′. .
  • the computer device can further acquire the second sample y′ belonging to the second image category B, and input y′ into the first-stage image inverse transformation model F1 and the first-stage image transformation model G1 in sequence to obtain the second sample single-stage restoration image G 1 . (F 1 (y')).
  • G 1 (F 1 (y')) is the image intended to return to y′, then it can be understood that the training purpose of the model is to minimize the difference between G 1 (F 1 (y′)) and y′. .
  • L cyc1 is a loss function based on loop consistency.
  • the process of model training may include adjusting the model parameters of the first stage image transformation model and the first stage image inverse transformation model to minimize the process of L cyc1 .
  • the machine learning model is unsupervisedly trained by using the loop consistency constraint, thereby realizing a machine learning model for training between images of any two image categories, and is no longer limited to traditional supervised training of machine learning.
  • the model's dependence on sample tags extends the image processing application scenario.
  • the training end condition may be that the number of trainings for the model reaches a preset number of trainings.
  • the computer device can count the number of trainings when training the model. When the count reaches the preset number of trainings, the model is judged to satisfy the training end condition, and the training of the model is ended.
  • the training end condition may also be that the discriminative performance index of the adjusted authentication model reaches a preset index, and the image conversion performance index of the adjusted image conversion model reaches a preset index.
  • the image processing method further includes the step of optimizing the first stage image transformation model.
  • the step of optimizing the first stage image transformation model is as follows: after obtaining the first sample through the first stage image transformation model, the first sample single stage transformation image output by the first stage image transformation model; obtaining the second sample After the first stage image reverse transformation model, the second sample single-stage transformation image output by the first stage image reverse transformation model is input; the first sample and the second sample single-stage transformation image are respectively input into the first stage image reverse transformation identification.
  • the model obtains the identification confidence of the first sample and the identification confidence of the single-stage transformed image of the second sample respectively; and inputs the second sample and the single-stage transformed image of the first sample into the first-stage image transformation identification model, respectively.
  • the discrimination confidence of the second sample and the discrimination confidence of the single sample single-stage transformed image are obtained. According to the difference between the first sample and the first sample single-stage restored image, and the difference between the second sample and the second sample single-stage restored image, the first-stage image transformation model and the first-stage image inverse transformation model are adjusted until the content is satisfied.
  • Ending the training when the training is stopped including: maximizing the identification confidence of the first sample and the direction of the confidence of the second sample, minimizing the confidence of the second sample single-stage transformed image, the first sample list
  • the discrimination confidence of the phase-converted image, the difference between the first sample and the single-sample restored image of the first sample, and the direction of the difference between the second-sample and the second-sample single-stage restored image, and the first-stage image transformation identification model is adjusted, The first stage image reverse transformation identification model, the first stage image transformation model and the first stage image inverse transformation model are terminated until the training stop condition is satisfied.
  • the identification model is a machine learning model that has the ability to identify after training.
  • the first stage image conversion authentication model is used to identify whether the input image is an image originally belonging to the second image category, and output the authentication confidence of the authentication result, that is, the first stage image conversion discrimination confidence.
  • the first stage image inverse transformation identification model is used to identify whether the input image is an image belonging to the first image category, and output the authentication confidence of the discrimination result, that is, the first stage image reverse conversion discrimination confidence.
  • the first stage image conversion discrimination confidence corresponds to the input image one by one, and is a degree of credibility indicating that the input image is an image originally belonging to the second image category.
  • the higher the confidence level the higher the probability that the input image is the original image belonging to the second image category.
  • the first stage image reverse conversion discrimination confidence corresponds to the input image one by one, and is a degree of credibility indicating that the input image is an image originally belonging to the first image category.
  • the higher the confidence level the higher the probability that the input image is the original image belonging to the first image category.
  • the first-stage image conversion model hopes to learn how to convert the input image of the first image category into the image of the second image category, and makes the generated image deceive the first stage.
  • the image transformation authentication model is such that the first stage image transformation authentication model assumes that the input is an image originally belonging to the second image category.
  • the first stage image inverse transformation model hopes to learn how to convert the input image of the second image category into the image of the first image category, and make the generated image deceive the first stage image reverse transformation identification model, so that the first stage image
  • the inverse transformation identification model assumes that the image originally belonging to the second image category is input.
  • the computer device can be x 'and F 1 (y') of the input image inverse conversion model D x1 identify a first stage, to obtain x), F 1 (y ' ) The discrimination confidence D x1 (F 1 (y')), which inputs y' and G 1 (x') into the first-stage image inverse transformation discrimination model D y1 to obtain the discrimination confidence D y1 (y') of y' , G 1 (x ') of the differential confidence D y1 (G 1 (x' )).
  • L adv1 log(D y1 (y'))+log(1-D y1 (G 1 (x')))+log(D x1 (x'))+log(1-D x1 (F 1 (y '))) (2)
  • L adv1 is a loss function based on the anti-learning constraint.
  • the process of model training may include adjusting the model parameters of the first stage image transformation discrimination model and the first stage image inverse transformation discrimination model to maximize the process of L adv1 .
  • Maximizing L adv1 is the process of maximizing D x1 (x') and D y1 (y'), minimizing D x1 (F 1 (y')) and D y1 (G 1 (x')).
  • the antagonistic optimization image transformation model and the identification model are expressed as:
  • ⁇ 1 is the cyclic consistency constraint weight.
  • the process of maximizing L adv1 and minimizing L cyc1 can be rotated. That is to maximize L adv1 and then minimize L cyc1 in one sample training, and minimize L cyc1 and then maximize L adv1 in the next sample training.
  • training of two models of an image conversion model and an authentication model is included.
  • the process of training the image transformation model is to learn how to convert one type of image into another type of image.
  • the process of training the identification model is to learn whether the input image is an original image or an image generated by an image transformation model.
  • the image transformation model learns to generate an image that is more similar to the original image, so as to interfere with the judgment of the authentication model, and the identification model learns more accurately the judgment of the original image and the generated image.
  • the two models compete against each other and promote each other, so that the trained model is obtained. The performance is better, so that when the image conversion model obtained by training is used for image conversion, the problem of image distortion can be overcome to a great extent, and the image conversion effect is improved.
  • first-stage image conversion model and the first-stage image inverse transformation model are supervisedly trained
  • the first-stage image transformation identification model and the first-stage image reverse transformation can also be combined.
  • the identification model is used for confrontational training.
  • the step of training the second-stage image conversion model is as follows: the first sample is sequentially passed through the first-stage image conversion model and the second-stage image conversion model to obtain the first output of the first-stage image conversion model. a first-order transformed image of the sample, and a second-order transformed image of the first sample output by the second-stage image conversion model; and the first sample is obtained according to the first-order transformed image of the first sample and the second-order transformed image of the first sample Corresponding and belonging to the first image conversion image of the second image category; the first sample conversion image is sequentially subjected to the first stage image inverse transformation model and the second phase image inverse transformation model to obtain the first stage image inverse transformation model output a first sample first-order restored image, and a second-stage image reverse-transformed model output first-sample second-order restored image; according to the first sample first-order restored image and the first sample second-order restored image, obtained and The first sample corresponding to the first sample and belonging to the first image category recovers the image; the second sample
  • the first stage image transformation model has been trained and can be used directly.
  • Figure 5 illustrates a logic diagram of training a second stage image transformation model in one embodiment.
  • Computer equipment Input the first stage image inverse transformation model F1 to obtain the first sample first order restoration image followed by Input the second stage image inverse transformation model F2 to obtain the first sample second order restoration image According to with Obtaining a first sample recovery image corresponding to the first sample and belonging to the first image category
  • x′′ is an image intended to be restored to x
  • L cyc2 is a loss function based on loop consistency.
  • the process of model training may include adjusting the model parameters of the second stage image transformation model and the second stage image inverse transformation model to minimize the process of L cyc2 .
  • the machine learning model is unsupervisedly trained by using the loop consistency constraint, thereby realizing a machine learning model for training between images of any two image categories, and is no longer limited to traditional supervised training of machine learning.
  • the model's dependence on sample tags extends the image processing application scenario.
  • the image processing method further includes the step of optimizing the second stage image transformation model.
  • the step of optimizing the second-stage image transformation model is specifically as follows: the first sample and the second sample transformed image are respectively input into the second-stage image reverse transformation discrimination model, and the first sample is obtained with the identification confidence and the second sample respectively. And determining the confidence of the conversion image; and inputting the second sample and the first sample transformed image into the second-stage image transformation and discriminating model respectively, respectively obtaining the identification confidence of the second sample and the authentication confidence of the first sample transformed image.
  • the second stage image conversion model and the second stage image inverse transformation model are adjusted until the training stop condition is satisfied.
  • Ending the training includes: maximizing the confidence of the first sample and the direction of the confidence of the second sample, minimizing the confidence of the second sample transformed image, the confidence of the first sample transformed image, The difference between the first sample and the first sample restored image, and the direction of the difference between the second sample and the second sample restored image, the second stage image transformation identification model, the second stage image reverse transformation identification model, the second stage The image transformation model and the second-stage image inverse transformation model end the training until the training stop condition is met.
  • the computer device can Input the second stage image inverse transformation identification model D x2 to obtain the discrimination confidence D x2 (x') of x', Identification confidence Put y' with Entering the second stage image transformation discrimination model D y2 to obtain the identification confidence D y2 (y') of y', Identification confidence
  • L adv2 is a loss function based on the anti-learning constraint.
  • the process of model training may include adjusting the model parameters of the second stage image transformation discrimination model and the second stage image inverse transformation discrimination model to maximize L adv2 .
  • Maximizing L adv2 is maximizing D x2 (x') and D y2 (y'), minimizing versus the process of.
  • the antagonistic optimization image transformation model and the identification model are expressed as:
  • ⁇ 2 is the cyclic consistency constraint weight.
  • the process of maximizing L adv2 and minimizing L cyc2 can be rotated. That is, first maximize L adv2 and then minimize L cyc2 in one sample training, and minimize L cyc2 and then maximize L adv2 in the next sample training.
  • training of two models of an image conversion model and an authentication model is included.
  • the process of training the image transformation model is to learn how to convert one type of image into another type of image.
  • the process of training the identification model is to learn whether the input image is an original image or an image generated by an image transformation model.
  • the image transformation model learns to generate an image that is more similar to the original image, so as to interfere with the judgment of the authentication model, and the identification model learns more accurately the judgment of the original image and the generated image.
  • the two models compete against each other and promote each other, so that the trained model is obtained. The performance is better, so that when the image conversion model obtained by training is used for image conversion, the problem of image distortion can be overcome to a great extent, and the image conversion effect is improved.
  • the second-stage image transformation model and the second-stage image inverse transformation model are supervisedly trained
  • the second-stage image transformation identification model and the second-stage image reverse transformation can also be combined.
  • the identification model is used for confrontational training.
  • a first sample transformed image corresponding to the first sample and belonging to the second image category is obtained, including: The first sample, the first sample first-order transformed image and the first sample second-order transformed image are jointly input into the first weight prediction model, and a weight matrix corresponding to the second-sample transformed image of the first sample is obtained; according to the weight matrix, a weight matrix corresponding to the first-order transformed image of the first sample; and merging the first-sample first-order transformed image and the first-sample second-order transformed image according to respective weight matrixes to obtain a correspondence with the first sample, and A first sample transformed image belonging to the second image category.
  • the first sample of the category restores the image.
  • obtaining, according to the second sample first-order transformed image and the second sample second-order transformed image, a second sample transformed image corresponding to the second sample and belonging to the first image category including: converting the second sample and the second sample into a first order
  • the image and the second sample second-order transformed image are input into the second weight prediction model to obtain a weight matrix corresponding to the second-sample transformed image of the second sample; and the weight matrix corresponding to the first-order transformed image of the second sample is obtained according to the weight matrix;
  • the second sample first-order transformed image and the second sample second-order transformed image are fused according to respective weight matrixes to obtain a second sample transformed image corresponding to the second sample and belonging to the first image category.
  • the order restored image and the second sample second-order restored image are jointly input into the first weight prediction model to obtain a weight matrix corresponding to the second sample second-order restored image; and according to the weight matrix, a weight matrix corresponding to the second sample first-order restored image is obtained
  • merging the second sample first-order restored image and the second sample second-order restored image according to respective corresponding weight matrices to obtain a second sample restored image corresponding to the second sample and belonging to the second image category.
  • the second stage image conversion model and the second stage image inverse transformation model are adjusted until the training stop condition is satisfied.
  • Ending the training comprising: adjusting the first weight prediction model, the second weight prediction model, and the second stage image according to the difference between the first sample and the first sample restored image, and the difference between the second sample and the second sample restored image
  • the transformation model and the second stage image inverse transformation model are terminated until the training stop condition is met.
  • the computer device can input x', y1, and y2 together into the first weight prediction model Gf, obtain a weight matrix ⁇ x' of y2 output by Gf, and then obtain a weight matrix (1- ⁇ x' ) of y1, Then the first sample conversion image is Computer equipment with Co-enter the second weight prediction model Ff to obtain the Ff output Weight matrix Then get Weight matrix Then the first sample recovery image is
  • the computer device can input y', x1 and x2 together into the second weight prediction model Ff, obtain the weight matrix ⁇ y' of the x2 output of the Ff, and then obtain the weight matrix (1- ⁇ y' ) of x1, then the second sample transformation
  • the image is Computer equipment with Co-enter the first weight prediction model Gf to obtain the Gf output Weight matrix Then get Weight matrix Then the second sample recovery image is
  • the machine learning model is unsupervisedly trained by using the loop consistency constraint, thereby realizing a machine learning model for training between images of any two image categories, and is no longer limited to traditional supervised training of machine learning.
  • the model's dependence on sample tags extends the image processing application scenario.
  • Figure 6 is a diagram showing an image before and after image conversion in one embodiment.
  • the input image is an image of the first image category
  • the horse in the input image is a single color horse.
  • the output image is an image of the second image category, and the horse in the output image is a horse of a plurality of colors.
  • an image processing apparatus 700 is provided.
  • the image processing apparatus 700 includes an acquisition module 701, a first stage conversion module 702, a second stage conversion module 703, a determination module 704, and a fusion module 705.
  • the various modules included in the image processing apparatus 700 may be implemented in whole or in part by software, hardware, or a combination thereof.
  • the obtaining module 701 is configured to acquire a to-be-processed image belonging to the first image category.
  • the first stage conversion module 702 is configured to input the image to be processed into the first stage image conversion model to obtain a first intermediate image.
  • the second stage conversion module 703 is configured to convert the first intermediate image into the second intermediate image by using the second stage image transformation model.
  • the determining module 704 is configured to determine a second weight matrix corresponding to the second intermediate image, and determine a first weight matrix corresponding to the first intermediate image.
  • the fusion module 705 is configured to fuse the first intermediate image and the second intermediate image according to the corresponding first weight matrix and the second weight matrix to obtain a target image corresponding to the image to be processed and belonging to the second image category.
  • the first stage conversion module 702 is further configured to perform downsampling on the image to be processed to obtain a compressed image with reduced image size; and input the compressed image into the first stage image conversion model, and output the image size and the compressed image.
  • the second stage conversion module 703 is further configured to perform upsampling on the first intermediate image to obtain an enlarged image having the same image size as the image size of the image to be processed; and input the enlarged image into the second stage image conversion model, and output image size and magnification.
  • the determining module 704 is further configured to input the image to be processed, the first intermediate image and the second intermediate image into the first weight prediction model, to obtain a second weight matrix corresponding to the second intermediate image;
  • the two weight matrix obtains a first weight matrix corresponding to the first intermediate image; the sum of the first weight matrix and the second weight matrix is a preset matrix.
  • the fusion module 705 is further configured to multiply each pixel value of the first intermediate image by each matrix element of the first weight matrix to obtain a first target image belonging to the second image category; Each pixel value of the intermediate image is multiplied by each matrix element of the second weight matrix to obtain a second target image belonging to the second image category; and corresponding to the image to be processed according to the first target image and the second target image, And belongs to the target image of the second image category.
  • the image processing apparatus 700 further includes: a first stage model training module 706, configured to acquire a first sample belonging to the first image category and a second sample belonging to the second image category; The first stage image transformation model and the first stage image reverse transformation model are sequentially obtained, and the first sample single-stage recovery image is obtained; the second sample is sequentially subjected to the first stage image inverse transformation model and the first stage image transformation model to obtain the first The two-stage single-stage recovery image; and the difference between the first sample and the first sample single-stage recovery image, and the difference between the second sample and the second sample single-stage recovery image, adjusting the first-stage image transformation model and the first The stage image is inversely transformed into a model until the training stop condition is met.
  • a first stage model training module 706 configured to acquire a first sample belonging to the first image category and a second sample belonging to the second image category
  • the first stage image transformation model and the first stage image reverse transformation model are sequentially obtained, and the first sample single-stage recovery image is obtained
  • the second sample is sequentially subject
  • the first stage model training module 706 is further configured to obtain a first sample single-stage transformed image output by the first stage image conversion model after the first sample passes the first stage image conversion model; After the first sample is subjected to the inverse transformation model of the first stage image, the second sample is converted into a single stage by the first stage image reverse transformation model; the first sample and the second sample single stage transformation image are respectively input into the first stage image reverse direction Transforming the identification model, respectively obtaining the identification confidence of the first sample and the identification confidence of the single-stage transformed image of the second sample; respectively inputting the second sample and the single-stage transformed image of the first sample into the first-stage image transformation and identification model, Obtaining a confidence of confidence of the second sample and a degree of confidence of the first sample of the single-stage transformed image; and minimizing the second in accordance with the direction of maximizing the confidence of the first sample and the degree of confidence of the second sample The confidence of the identification of the sample single-stage transformed image, the confidence of the first sample of the single-stage transformed image, the first sample passes the
  • the image processing apparatus 700 further includes a first stage model training module 706 and a second stage model training module 707.
  • the second stage model training module 707 is configured to sequentially pass the first sample to the first stage image transformation model and the second stage image transformation model to obtain a first sample first order transformation image output by the first stage image transformation model, and a first-stage second-order transformed image outputted by the second-stage image conversion model; and corresponding to the first sample and belonging to the second image category according to the first-sample first-order transformed image and the first-sample second-order transformed image
  • the first sample is transformed into an image; the first sample transformed image is sequentially subjected to the first stage image inverse transformation model and the second phase image inverse transformation model, and the first sample first order restoration of the first stage image inverse transformation model output is obtained.
  • the second stage model training module 707 is further configured to input the first sample and the second sample transformed image into the second stage image inverse transform identification model, respectively, to obtain the first sample's authentication confidence and the first sample.
  • the discrimination confidence of the two sample transformed images; the second sample and the first sample transformed image are respectively input into the second stage image transformation and discrimination model, respectively, and the discrimination confidence of the second sample and the discrimination confidence of the first sample transformed image are respectively obtained.
  • the second stage image transformation identification model the second stage image reverse transformation identification model, and the second stage image transformation model
  • the second stage image transformation model is inversely transformed to the model until the training stop condition is met.
  • the second stage model training module 707 is further configured to input the first sample, the first sample first-order transformed image, and the first sample second-order transformed image into the first weight prediction model, and obtain the same a weight matrix corresponding to the second-order transformed image of the sample; according to the weight matrix, a weight matrix corresponding to the first-order transformed image of the first sample is obtained; and the first-order transformed image of the first sample and the second-order transformed image of the first sample are followed The respective weight matrixes are fused to obtain a first sample transformed image corresponding to the first sample and belonging to the second image category.
  • the second sample transformed image, the second sample first-order restored image, and the second sample second-order restored image are jointly input into the first weight prediction model to obtain a weight matrix corresponding to the second sample second-order restored image; and according to the weight matrix, a second sample first-order restored image corresponding weight matrix; the second sample first-order restored image and the second sample second-order restored image are fused according to respective corresponding weight matrices, to obtain a second sample corresponding to the second image category
  • the second sample recovers the image; and adjusts the first weight prediction model, the second weight prediction model, and the second according to the difference between the first sample and the first sample restored image, and the difference between the second sample and the second sample restored image
  • the stage image transformation model and the second stage image inverse transformation model end the training until the training stop condition is met.
  • Figure 9 is a diagram showing the internal structure of a computer device in one embodiment.
  • the computer device may specifically be a user terminal.
  • the computer device includes a processor, a memory, a network interface, an input device, and a display screen connected by a system bus.
  • the memory comprises a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium of the computer device stores an operating system and can also store computer program computer readable instructions that, when executed by the processor, cause the processor to implement an image processing method.
  • Also stored in the internal memory is computer program computer readable instructions that, when executed by the processor, cause the processor to perform an image processing method.
  • the display screen of the computer device may be a liquid crystal display or an electronic ink display screen.
  • the input device may be a touch layer covered on the display screen, or may be a button, a trackball or a touchpad provided on a computer device casing, or may be An external keyboard, trackpad, or mouse.
  • FIG. 9 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • FIG. 10 is a diagram showing the internal structure of a computer device in one embodiment.
  • the computer device may specifically be a user terminal or a server.
  • the computer device includes a processor, memory, and network interface connected by a system bus.
  • the memory comprises a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium of the computer device stores an operating system and can also store computer readable instructions that, when executed by the processor, cause the processor to implement an image processing method.
  • the internal memory can also store computer readable instructions that, when executed by the processor, cause the processor to perform an image processing method. It will be understood by those skilled in the art that the structure shown in FIG.
  • FIG. 10 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • the image processing apparatus provided by the present application may be implemented in the form of a computer readable instruction, which may be run on a computer device as shown in FIG. 9 or FIG.
  • the dysfunctional storage medium can store various instruction modules constituting the image processing apparatus, for example, the acquisition module 701, the first-stage conversion module 702, the second-stage conversion module 703, the determination module 704, the fusion module 705, and the like shown in FIG.
  • the computer readable instructions comprising the various instruction modules cause the processor to perform the steps in the image processing methods of the various embodiments of the present application described in this specification.
  • the computer device shown in Fig. 9 or Fig. 10 can acquire a to-be-processed image belonging to the first image category by the acquisition module 701 in the image processing apparatus 700 as shown in Fig. 7.
  • the image to be processed is input to the first stage image conversion model by the first stage conversion module 702 to obtain a first intermediate image.
  • the first intermediate image is converted to the second intermediate image by the second stage conversion module 703 by the second stage image conversion model.
  • a weight matrix corresponding to each of the first intermediate image and the second intermediate image is determined by the determining module 704, respectively.
  • the first intermediate image and the second intermediate image are merged according to respective weight matrixes by the fusion module 705 to obtain a target image corresponding to the image to be processed and belonging to the second image category.
  • a computer readable storage medium having stored thereon computer readable instructions that, when executed by a processor, cause a processor to perform the image processing method described above step.
  • the steps of the image processing method herein may be the steps in the image processing method of each of the above embodiments.
  • a computer apparatus comprising a memory and a processor having stored therein computer readable instructions that, when executed by a processor, cause the processor to perform the steps of the image processing method described above.
  • the steps of the image processing method herein may be the steps in the image processing method of each of the above embodiments.
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

一种图像处理方法，包括：获取属于第一图像类别的待处理图像；将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像；通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像；确定所述第二中间图像对应的第二权重矩阵；确定所述第一中间图像对应的第一权重矩阵；将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像。

Description

图像处理方法、存储介质和计算机设备
本申请要求于2018年01月26日提交中国专利局，申请号为2018100785448，申请名称为“图像处理方法、装置、存储介质和计算机设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别是涉及一种图像处理方法、存储介质和计算机设备。
背景技术
随着计算机技术的发展和图像处理技术的进步，基于图像的处理方式变得越来越多样。目前常用的图像处理技术如图像的特征转化处理，比如图像颜色特征转化、图像光影特征转化或者图像风格特征转化等。
然而，传统的图像处理过程中，主要是通过基于纹理合成的方式，将目标特征的图像纹理扩散到待处理图像的图像区域，来实现图像特征转化处理。但在采用该方式处理时容易出现误匹配的情况，从而导致得到的图像产生失真。
发明内容
根据本申请提供的各种实施例，提供一种图像处理方法、存储介质和计算机设备。
一种图像处理方法，由计算机设备执行，所述方法包括：
获取属于第一图像类别的待处理图像；
将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像；
通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像；
确定所述第二中间图像对应的第二权重矩阵；
确定所述第一中间图像对应的第一权重矩阵；及
将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像。
一种存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个 处理器执行时，使得一个或多个处理器执行以下步骤：
获取属于第一图像类别的待处理图像；
将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像；
通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像；
确定所述第二中间图像对应的第二权重矩阵；
确定所述第一中间图像对应的第一权重矩阵；及
将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像。
一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行以下步骤：
获取属于第一图像类别的待处理图像；
将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像；
通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像；
确定所述第二中间图像对应的第二权重矩阵；
确定所述第一中间图像对应的第一权重矩阵；及
将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为一个实施例中图像处理方法的流程示意图；
图2为一个实施例中第一阶段图像转化模型的模型示意图；
图3为一个实施例中图像处理过程的逻辑示意图；
图4为一个实施例中训练第一阶段图像转化模型的逻辑示意图；
图5为一个实施例中训练第二阶段图像转化模型的逻辑示意图；
图6为一个实施例中图像转化前后的图像示意图；
图7为一个实施例中图像处理装置的模块结构图；
图8为另一个实施例中图像处理装置的模块结构图；
图9为一个实施例中计算机设备的内部结构图；及
图10为另一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
图1为一个实施例中图像处理方法的流程示意图。本实施例主要以该方法应用于计算机设备来举例说明。参照图1，该图像处理方法具体包括如下步骤：
S102，获取属于第一图像类别的待处理图像。
其中，图像类别是图像所反映的图像特征所属的类别。图像特征具体可以是颜色特征、风格特征或者内容特征等。相应地，根据颜色特征分类得到的图像类别比如黑白图像类别或者彩色图像类别等；根据风格特征分类得到的图像类别比如素描图像类别或者油画图像类别等；根据内容特征分类得到的图像类别比如苹果图像类别或者橘子图像类别等。
具体地，待处理图像是待进行图像类别转化的图像。其中，计算机设备可以是用户终端，用户终端可直接获取用户上传的属于某一图像类别的待处理图像，也可接受其他用户终端传递的属于某一图像类别的待处理图像。计算机设备也可以是服务器，服务器可接收用户终端上传的属于某一图像类别的待处理图像。
S104，将待处理图像输入第一阶段图像转化模型，得到第一中间图像。
其中，图像转化是指将一种图像类别的图像转化为另一种图像类别的图像。图像转化模型是经过训练后具有图像转化能力的机器学习模型。机器学习英文全称为Machine Learning，简称ML。该机器学习模型可采用神经网络模型，支持向量机(Support Vector Machine，SVM)或者逻辑回归模型等。
第一阶段图像转化模型用于将第一图像类别的图像初步转化为第二图像类别的图像。通过第一阶段图像转化模型转化得到的图像允许与意图转化为的第二图像类别的图像存 在一定程度上的误差。可以理解，这里不对第一阶段图像转化模型的图像细节特征转化能力作限定，在图像整体特征上完成转化即可。
具体地，计算机设备可事先根据属于第一图像类别的第一样本和属于第二图像类别的第二样本训练得到第一阶段图像转化模型。在本实施例中，第一阶段图像转化模型具体地可采用卷积神经网络模型。该卷积神经网络模型的结构具体可以是编码器-解码器结构。编码器可由多层卷积层构成，解码器则可由多层反卷积层构成。其中，编码器部分将输入图像转化为分辨率低但通道数目更多的特征图，该特征图可以看作是对原图像的一种编码。解码器部分则将特征图解码为第二图像类别的图像。
举例说明，图2示出了一个实施例中第一阶段图像转化模型的模型示意图。参考图2，该第一阶段图像转化模型为编码器-解码器结构。编码器结构210包括三层卷积层211，解码器结构220包括三层反卷积层221。
在一个实施例中，在训练第一阶段图像转化模型时，当属于第一图像类别的第一样本和属于第二图像类别的第二样本一一对应时，则将与第一样本对应的第二样本作为该第一样本的训练标签，从而根据第一样本和相应的训练标签有监督地训练得到第一阶段图像转化模型。
在一个实施例中，在训练第一阶段图像转化模型时，当不存在与第一样本和对应的第二样本，也不存在与第二样本对应的第一样本时，则构建将第一图像类别的图像转换为第二图像类别的图像的第一阶段图像转化模型，和将第二图像类别的图像转换为第一图像类别的图像的第一阶段图像逆向转化模型。将第一样本依次经过第一阶段图像转化模型和第一阶段图像逆向转化模型形成闭环，将第二样本依次经过第一阶段图像逆向转化模型和第一阶段图像转化模型形成闭环，从而利用循环一致性约束无监督地训练得到第一阶段图像转化模型。
可以理解，这里的第一样本和第二样本对应是指第一样本与第二样本除用以区分类别的图像特征外，其他图像特征均相同。举例说明，比如第一图像类别为黑白图像类别，第二图像类别为彩色图像类别，对于苹果图像，第一样本与第二样本中的苹果仅颜色不同，大小形状或者位置均相同。
图3示出了一个实施例中图像处理过程的逻辑示意图。参考图3，计算机设备可将属于第一图像类别A的待处理图像x输入第一阶段图像转化模型G1，得到第一阶段图像转化模型G1输出的第一中间图像y1。
在本实施例中，图像转换可以是非图像内容的转化，比如图像风格转化、图像颜色转化或者图像景深转化等。图像转换也可以是图像内容的转化，比如修改图像中的物体等。具体地，将苹果图像类别的苹果图像修改为橘子图像类别的橘子图像。
S106，通过第二阶段图像转化模型将第一中间图像转化为第二中间图像。
其中，第二阶段图像转化模型用于将第一阶段图像转化模型输出的图像进一步转化为第二图像类别的图像。可以理解，这里要求第二阶段图像转化模型在图像细节特征上完成转化。
具体地，计算机设备可事先根据第一阶段图像转化模型输出的第一中间图像和属于第二图像类别的第二样本训练得到第二阶段图像转化模型。在本实施例中，第二阶段图像转化模型具体地可采用卷积神经网络模型。卷积神经网络模型的结构具体可以是编码器-解码器结构。编码器可由多层卷积层构成，解码器可由多层反卷积层构成。
在一个实施例中，在训练第二阶段图像转化模型时，当属于第一图像类别的第一样本和属于第二图像类别的第二样本一一对应时，则将与第一样本对应的第二样本作为该第一样本所对应的第一中间图像的训练标签，从而根据第一中间图像和相应的训练标签有监督地训练得到第二阶段图像转化模型。
在一个实施例中，在训练第一阶段图像转化模型时，当不存在与第一样本和对应的第二样本，也不存在与第二样本对应的第一样本时，则构建将第一中间图像转换为第二图像类别的图像的第二阶段图像转化模型，和将第二图像类别的图像转换为第一中间图像的第二阶段图像逆向转化模型。将第一样本依次经过第一阶段图像转化模型、第二阶段图像转化模型、第一阶段图像逆向转化模型和第二阶段图像逆向转化模型形成闭环，将第二样本依次经过第一阶段图像逆向转化模型、第二阶段图像逆向转化模型、第一阶段图像转化模型和第二阶段图像转化模型形成闭环，从而利用循环一致性约束无监督地训练得到第二阶段图像转化模型。
继续参考图3，计算机设备得到第一阶段图像转化模型G1输出的第一中间图像y1后，可继续将第一中间图像y1输入第二阶段图像转化模型G2，得到第二阶段图像转化模型G2输出的第二中间图像y2。
S108，确定第二中间图像对应的第二权重矩阵。
其中，权重是针对某一对象而言相对的概念。某一对象的权重反映的是该对象在整体评价中的相对重要程度。由于计算机设备在处理图像时，通常是以矩阵形式的数字图像数 据进行处理，故图像的权重则也是矩阵的形式。
在一个实施例中，计算机设备可预先设置权重映射函数，从而通过该权重映射函数映射得到第二中间图像对应的权重矩阵。具体地，计算机设备可预先设置的权重映射函数可以有多种。计算机设备可将待处理图像和第二中间图像作为权重映射函数的自变量，得到该作为自变量的中间图像所对应的权重矩阵。
S110，确定第一中间图像对应的第一权重矩阵。
具体地，计算机设备可将待处理图像和第一中间图像作为权重映射函数的自变量，得到该作为自变量的中间图像所对应的权重矩阵。
在一个实施例中，计算机设备也可将待处理图像、第一中间图像和第二中间图像作为权重映射函数的自变量，得到其中一个中间图像所对应的权重矩阵，进而根据该权重矩阵得到另一个中间图像所对应的权重矩阵。计算机设备还可将第一中间图像与第二中间图像作为权重映射函数的自变量，得到其中一个中间图像所对应的权重矩阵，进而根据该权重矩阵得到另一个中间图像所对应的权重矩阵。这里的其中一个中间图像可以是第一中间图像，也可以是第二中间图像。可以理解，这里对S108与S110的执行先后顺序不作限定。S108可在S110之前执行，也可在S110之后执行，还可与S110同步执行。S108与S110的执行先后顺序依赖于计算机设备事先设置的权重映射函数。
在一个实施例中，S108包括：将待处理图像、第一中间图像与第二中间图像共同输入第一权重预测模型，得到与第二中间图像对应的第二权重矩阵。S110包括：根据第二权重矩阵，得到与第一中间图像对应的第一权重矩阵；第一权重矩阵与第二权重矩阵之和为预设矩阵。
其中，权重预测模型是经过训练后具有权重预测能力的机器学习模型。在本实施例中，权重预测模型具体可采用浅层卷积神经网络模型。比如，该浅层卷积神经网络模型具体可以由三层卷积层构成。前两层卷积层的结构均为Convolution(卷积)-InstanceNorm(归一化)-ReLU(激活函数)结构，最后一层卷积层的结构则为Convolution(卷积)-Sigmoid(激活函数)结构。
第一权重预测模型是将第一图像类别的图像转化为第二图像类别的图像时进行权重预测的机器学习模型。下文中提到的第二权重预测模型则是将第二图像类别的图像转化为第一图像类别的图像时进行权重预测的机器学习模型。
第一权重预测模型的输入是待处理图像、第一中间图像与第二中间图像，这三帧图像 的图像尺寸相同。第一权重预测模型的输出是与输入图像大小一致的权重矩阵，权重矩阵的每个元素代表第二中间图像在相应像素位置的像素值的权重。第一权重矩阵与第二权重矩阵是大小和维数均相同的矩阵，而且第一权重矩阵与第二权重矩阵之和为预设矩阵，预设矩阵具体可以是大小和维数均与第二权重矩阵相同的全一矩阵。全一矩阵的各矩阵元素均为1。
具体地，计算机设备可将待处理图像、第一中间图像与第二中间图像共同输入第一权重预测模型，得到第一权重预测模型输出的与第二中间图像对应的第二权重矩阵。这里，第一中间图像对应的第一权重矩阵与第二中间图像对应的第二权重矩阵之和为预设矩阵。计算机设备可通过该预设矩阵减去第二中间图像对应的第二权重矩阵，得到与第一中间图像对应的第一权重矩阵。
再参考图3，计算机设备将第一阶段图像转化模型G1输出的第一中间图像y1与第二阶段图像转化模型G2输出的第二中间图像y2共同输入第一权重预测模型Gf，得到第一权重预测模型Gf输出的与第二中间图像y2对应的权重矩阵，进而得到第一中间图像y1对应的权重矩阵。
在本实施例中，利用机器学习模型强大的学习和表示能力进行权重预测学习，所训练得到的机器学习模型对权重进行预测，较传统方法对权重进程预测的效果更好。
S112，将第一中间图像、第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与待处理图像对应、且属于第二图像类别的目标图像。
具体地，计算机设备可将第一中间图像和第二中间图像按照各自对应的权重矩阵加权求和得到与待处理图像对应、且属于第二图像类别的目标图像。
在一个实施例中通过自适应融合层将第一中间图像和第二中间图像按照各自对应的权重矩阵融合，得到目标与待处理图像对应、且属于第二图像类别的目标图像。
在一个实施例中，S112包括：将第一中间图像的各像素值与第一权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别的第一目标图像；将第二中间图像的各像素值与第二权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别第二目标图像；根据第一目标图像与第二目标图像，得到与待处理图像对应、且属于第二图像类别的目标图像。
其中，各中间图像的像素点所形成的像素点矩阵的大小与该中间图像所对应的权重矩阵的大小相同。那么，像素点矩阵中各像素点元素均在权重矩阵存在一一对应的权重元素，也就是说，权重矩阵的每个权重元素代表相应中间图像中相应像素点元素的像素值的权 重。
具体地，计算机设备将第一中间图像的各像素值与第一权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别的第一目标图像，然后将第二中间图像的各像素值与第二权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别第二目标图像，再将第一目标图像与第二目标图像的各像素值按位相加，得到与待处理图像对应、且属于第二图像类别的目标图像。
参考图3，计算机设备可根据第一中间图像y1、权重矩阵、第二中间图像y2和权重矩阵计算得到融合后的目标图像。
在本实施例中，最终得到的目标图像融合了多个阶段模型的输出，能够极大程度上克服图像失真的问题，提高图像转化效果。
上述图像处理方法，在意图将属于第一图像类别的待处理图像转化为属于第二图像类别的图像时，自动将待处理图像依次经过第一阶段图像转化模型和第二阶段图像转化模型，并在分别确定两个阶段的模型的输出各自所对应的权重矩阵后，即根据相应的权重矩阵自适应融合两个模型的输出得到目标图像。这样待处理图像经过了多个阶段的模型处理，而且最终得到的目标图像融合了多个阶段模型的输出，能够极大程度上克服图像失真的问题，提高由待处理图像转化得到的目标图像的转化效果。
在一个实施例中，S104包括：对待处理图像进行下采样，得到图像尺寸缩小后的压缩图像；将压缩图像输入第一阶段图像转化模型，输出图像尺寸与压缩图像的图像尺寸相同的第一中间图像。S106包括：对第一中间图像进行上采样，得到图像尺寸与待处理图像的图像尺寸相同的放大图像；将放大图像输入第二阶段图像转化模型，输出图像尺寸与放大图像的图像尺寸相同的第二中间图像。
其中，下采样或称为降采样，是进行图像压缩的一种处理方式。经过下采样操作后的图像的图像尺寸会缩小，缩小的程度与下采样的采样周期相关。上采样或称为图像插值，是将图像放大的一种处理方式。经过上采样操作后的图像的图像尺寸会放大，放大的程度与上采样的采样周期相关。
具体地，计算机设备可对待处理图像进行下采样操作。下采样的方式具体可以是均值采样或者极值采样。比如，下采样的方式为对2*2像素区域进行均值，那么其中一个2*2像素区域对应的像素值矩阵为[1,2,3,4]，那么下采样得到的像素值为：(1+2+3+4)/4＝2.5。下采样操作后得到的图像的分辨率减小为待处理图像分辨率的1/4。
计算机设备进而将下采样后图像尺寸缩小的压缩图像输入第一阶段图像转化模型，得到第一阶段图像转化模型输出的图像尺寸与压缩图像的图像尺寸相同的第一中间图像。由于此时第一中间图像的图像尺寸小于待处理图像的尺寸，故计算机设备需在将第一中间图像输入第二阶段图像转化模型之前，对第一中间图像执行与在前的下采样操作相应的上采样操作，使得上采样操作后得到的特征图的分辨率增大为上采样操作前的特征图的分辨率的4倍，以保证第二阶段图像转化模型输出的第二中间图像与待处理图像的分辨率一致。
在本实施例中，在第一阶段的图像转化时，将图像压缩后进行转化，使得模型处理的图像数据量小，这样在图像转换精度要求不高时，既可以保证图像转化效果，又能提高图像转化时间。
在一个实施例中，该图像处理方法还包括训练模型的步骤。其中，训练第一阶段图像转化模型的步骤具体如下：获取属于第一图像类别的第一样本和属于第二图像类别的第二样本；将第一样本依次经过第一阶段图像转化模型和第一阶段图像逆向转化模型，得到第一样本单阶段恢复图像；将第二样本依次经过第一阶段图像逆向转化模型和第一阶段图像转化模型，得到第二样本单阶段恢复图像；及按照第一样本与第一样本单阶段恢复图像的差异，及第二样本与第二样本单阶段恢复图像的差异，调整第一阶段图像转化模型和第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
其中，第一样本是属于第一图像类别的图像，用于训练第一阶段图像转化模型和第一阶段图像逆向转化模型。第二样本是属于第二图像类别的图像，也用于训练第一阶段图像转化模型和第一阶段图像逆向转化模型。第一阶段图像转化模型是将第一图像类别的图像转化为第二图像类别的图像的机器学习模型，第一阶段图像逆向转化模型是将第二图像类别的图像转化为第一图像类别的图像的机器学习模型。
第一样本单阶段恢复图像是属于第一图像类别的图像，是第一样本经过第一阶段图像转化模型转化为第二图像类别的图像后，又经过第一阶段图像逆向转化模型恢复得到的属于第一图像类别的图像。第二样本单阶段恢复图像是属于第二图像类别的图像，是第二样本经过第一阶段图像逆向转化模型转化为第一图像类别的图像后，又经过第一阶段图像转化模型恢复得到的属于第二图像类别的图像。
可以理解，本实施例所应用的场景是不存在与第一样本和对应的第二样本，也不存在与第二样本对应的第一样本的场景，也就是实际并没有可以作为第一样本的训练标签的属于第二图像类别的图像，及没有可以作为第二样本的训练标签的属于第一图像类别的图像 的模型训练场景。所以在本实施例中利用循环一致性，将第一样本作为其经过图像转化与图像逆向转化后恢复的图像的优化目标，将第二样本作为其经过图像逆向转化与图像转化后恢复的图像的优化目标。
具体地，计算机设备可获取属于第一图像类别的第一样本，将第一样本依次经过第一阶段图像转化模型和第一阶段图像逆向转化模型，得到第一样本单阶段恢复图像。此时，依据循环一致性约束，第一样本单阶段恢复图像即为意图恢复至第一样本的图像，那么可以理解的是，第一阶段图像转化模型和第一阶段图像逆向转化模型的训练目的即为最小化第一样本与第一样本单阶段恢复图像之间的差异的过程。
进一步地，为了防止多个第一样本在经过第一阶段图像转化模型的转化时均转化为相同的图像，计算机设备可获取属于第二图像类别的第二样本，将第二样本依次经过第一阶段图像逆向转化模型和第一阶段图像转化模型，得到第二样本单阶段恢复图像。此时第二样本单阶段恢复图像即为意图恢复至第二样本的图像，那么可以理解的是，第一阶段图像转化模型和第一阶段图像逆向转化模型的训练目的即为最小化第一样本与第一样本单阶段恢复图像之间的差异，以及第二样本与第二样本单阶段恢复图像之间的差异的过程。
其中，第一样本与第一样本单阶段恢复图像之间的差异，以及第二样本与第二样本单阶段恢复图像之间的差异具体可以是图像像素点差值的1范数或者2范数等。
举例说明，图4示出了一个实施例中训练第一阶段图像转化模型的逻辑示意图。参考图4，计算机设备可获取属于第一图像类别A的第一样本x′，将x′依次输入第一阶段图像转化模型G1和第一阶段图像逆向转化模型F1后得到第一样本单阶段恢复图像F 1(G 1(x′))。此时F 1(G 1(x′))即为意图恢复至x′的图像，那么可以理解的是模型的训练目的即为最小化F 1(G 1(x′))与x′的差异。计算机设备可再获取属于第二图像类别B的第二样本y′，将y′依次输入第一阶段图像逆向转化模型F1和第一阶段图像转化模型G1后得到第二样本单阶段恢复图像G 1(F 1(y′))。此时G 1(F 1(y′))即为意图恢复至y′的图像，那么可以理解的是模型的训练目的即为最小化G 1(F 1(y′))与y′的差异。
那么第一阶段的模型训练中，循环一致性约束表达为：
L cyc1＝||F 1(G 1(x′))-x′|| 1+||G 1(F 1(y′))-y′|| 1          (1)
其中，L cyc1为基于循环一致性的损失函数。模型训练的过程可以包括调整第一阶段图像转化模型与第一阶段图像逆向转化模型的模型参数以最小化L cyc1的过程。
在本实施例中，利用了循环一致性约束非监督地训练机器学习模型，从而实现训练得 到任意两种图像类别的图像之间转化的机器学习模型，不再受限于传统监督地训练机器学习模型时对样本标签的依赖，扩展了图像处理应用场景。
在一个实施例中，训练结束条件可以是对模型的训练次数达到预设训练次数。计算机设备可在对模型进行训练时，对训练次数进行计数，当计数达到预设训练次数时，判定模型满足训练结束条件，并结束对模型的训练。
在一个实施例中，训练结束条件也可以是调整后的鉴别模型的鉴别性能指标达到预设指标，调整后的图像转化模型的图像转化性能指标达到预设指标。
在一个实施例中，该图像处理方法还包括优化第一阶段图像转化模型的步骤。其中，优化第一阶段图像转化模型的步骤具体如下：获取第一样本经过第一阶段图像转化模型后，由第一阶段图像转化模型输出的第一样本单阶段转化图像；获取第二样本经过第一阶段图像逆向转化模型后，由第一阶段图像逆向转化模型输出的第二样本单阶段转化图像；将第一样本和第二样本单阶段转化图像分别输入第一阶段图像逆向转化鉴别模型，分别得到第一样本的鉴别置信度和第二样本单阶段转化图像的鉴别置信度；及将第二样本和第一样本单阶段转化图像分别输入第一阶段图像转化鉴别模型，分别得到第二样本的鉴别置信度和第一样本单阶段转化图像的鉴别置信度。按照第一样本与第一样本单阶段恢复图像的差异，及第二样本与第二样本单阶段恢复图像的差异，调整第一阶段图像转化模型和第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：按照最大化第一样本的鉴别置信度和第二样本的鉴别置信度的方向、最小化第二样本单阶段转化图像的鉴别置信度、第一样本单阶段转化图像的鉴别置信度、第一样本与第一样本单阶段恢复图像的差异，及第二样本与第二样本单阶段恢复图像的差异的方向，调整第一阶段图像转化鉴别模型、第一阶段图像逆向转化鉴别模型、第一阶段图像转化模型和第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
其中，鉴别模型是经过训练后具有鉴别能力的机器学习模型。在本实施例中，第一阶段图像转化鉴别模型，用于鉴别输入的图像是否为原始属于第二图像类别的图像，并输出鉴别结果的鉴别置信度，即第一阶段图像转化鉴别置信度。第一阶段图像逆向转化鉴别模型，用于鉴别输入的图像是否为原始属于第一图像类别的图像，并输出鉴别结果的鉴别置信度，即第一阶段图像逆向转化鉴别置信度。
第一阶段图像转化鉴别置信度与输入图像一一对应，是表示输入图像是原始属于第二图像类别的图像的可信程度。鉴别置信度越高，表示输入图像是原始属于第二图像类别的 图像的可能性越高。第一阶段图像逆向转化鉴别置信度与输入图像一一对应，是表示输入图像是原始属于第一图像类别的图像的可信程度。鉴别置信度越高，表示输入图像是原始属于第一图像类别的图像的可能性越高。
可以理解，本实施例中采用对抗学习的方式，第一阶段图像转化模型希望学习如何把输入的第一图像类别的图像转化为第二图像类别的图像，并使得生成的图像能够欺骗第一阶段图像转化鉴别模型，使得第一阶段图像转化鉴别模型以为输入的是原始属于第二图像类别的图像。第一阶段图像逆向转化模型则希望学习如何把输入的第二图像类别的图像转化为第一图像类别的图像，并使得生成的图像能够欺骗第一阶段图像逆向转化鉴别模型，使得第一阶段图像逆向转化鉴别模型以为输入的是原始属于第二图像类别的图像。
继续参考图4，计算机设备可将x′与F 1(y′)输入第一阶段图像逆向转化鉴别模型D x1，得到x′的鉴别置信度D x1(x′)，F 1(y′)的鉴别置信度D x1(F 1(y′))，将y′与G 1(x′)输入第一阶段图像逆向转化鉴别模型D y1，得到y′的鉴别置信度D y1(y′)，G 1(x′)的鉴别置信度D y1(G 1(x′))。
那么第一阶段的模型训练中，对抗学习约束表达为：
L adv1＝log(D y1(y′))+log(1-D y1(G 1(x′)))+log(D x1(x′))+log(1-D x1(F 1(y′)))   (2)
其中，L adv1为基于对抗学习约束的损失函数。模型训练的过程可以包括调整第一阶段图像转化鉴别模型与第一阶段图像逆向转化鉴别模型的模型参数以最大化L adv1的过程。最大化L adv1也就是最大化D x1(x′)与D y1(y′)，最小化D x1(F 1(y′))与D y1(G 1(x′))的过程。
那么在训练过程中，对抗地优化图像转化模型和鉴别模型表达为：
Figure PCTCN2019072491-appb-000001
其中，λ 1为循环一致性约束权重。最大化L adv1与最小化L cyc1的过程可以轮换进行。即在一个样本训练时先最大化L adv1再最小化L cyc1，在下一个样本训练时先最小化L cyc1再最大化L adv1。
在本实施例中，包括图像转化模型和鉴别模型两个模型的训练。其中，训练图像转化模型的过程在于学习如何将一种类别的图像转化为另一种类别的图像，训练鉴别模型的过程在于学习判断输入的图像是原始图像还是通过图像转化模型生成的图像。这样图像转化模型学习生成与原始图像更相似的图像，以干扰鉴别模型的判断，鉴别模型学习更加精准地进行原始图像和生成图像的判断，两个模型相互对抗，相互促进，使得训练得到的模型 性能更优，从而在使用训练得到的图像转化模型进行图像转化时，能够极大程度上克服图像失真的问题，提高图像转化效果。
在一个实施例中，本领域技术人员可以理解，有监督地训练第一阶段图像转化模型和第一阶段图像逆向转化模型时，也可结合第一阶段图像转化鉴别模型和第一阶段图像逆向转化鉴别模型进行对抗式训练。
在一个实施例中，训练第二阶段图像转化模型的步骤具体如下：将第一样本依次经过第一阶段图像转化模型和第二阶段图像转化模型，得到第一阶段图像转化模型输出的第一样本一阶转化图像，和第二阶段图像转化模型输出的第一样本二阶转化图像；根据第一样本一阶转化图像和第一样本二阶转化图像，得到与第一样本对应、且属于第二图像类别的第一样本转化图像；将第一样本转化图像依次经过第一阶段图像逆向转化模型和第二阶段图像逆向转化模型，得到第一阶段图像逆向转化模型输出的第一样本一阶恢复图像，和第二阶段图像逆向转化模型输出的第一样本二阶恢复图像；根据第一样本一阶恢复图像和第一样本二阶恢复图像，得到与第一样本对应、且属于第一图像类别的第一样本恢复图像；将第二样本依次经过第一阶段图像逆向转化模型和第二阶段图像逆向转化模型，得到第一阶段图像逆向转化模型输出的第二样本一阶转化图像，和第二阶段图像逆向转化模型输出的第二样本二阶转化图像；根据第二样本一阶转化图像和第二样本二阶转化图像，得到与第二样本对应、且属于第一图像类别的第二样本转化图像；将第二样本转化图像依次经过第一阶段图像转化模型和第二阶段图像转化模型，得到第一阶段图像转化模型输出的第二样本一阶恢复图像，和第二阶段图像转化模型输出的第二样本二阶恢复图像；根据第二样本一阶恢复图像和第二样本二阶恢复图像，得到与第二样本对应、且属于第二图像类别的第二样本恢复图像；及按照第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异，调整第二阶段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
可以理解，在训练第二阶段图像转化模型时，第一阶段图像转化模型已经训练完成，可以直接使用。
举例说明，图5示出了一个实施例中训练第二阶段图像转化模型的逻辑示意图。参考图5，计算机设备可获取属于第一图像类别A的第一样本x′，将x′依次输入第一阶段图像转化模型G1得到第一样本一阶转化图像y 1＝G 1(x′)，然后将y1输入第二阶段图像转化模型G2得到第一样本二阶转化图像y 2＝G 2(y 1)，再根据y1和y2即可得到与第一样 本对应、且属于第二图像类别的第一样本转化图像
Figure PCTCN2019072491-appb-000002
计算机设备再将
Figure PCTCN2019072491-appb-000003
输入第一阶段图像逆向转化模型F1得到第一样本一阶恢复图像
Figure PCTCN2019072491-appb-000004
然后将
Figure PCTCN2019072491-appb-000005
输入第二阶段图像逆向转化模型F2得到第一样本二阶恢复图像
Figure PCTCN2019072491-appb-000006
再根据
Figure PCTCN2019072491-appb-000007
和
Figure PCTCN2019072491-appb-000008
即可得到与第一样本对应、且属于第一图像类别的第一样本恢复图像
Figure PCTCN2019072491-appb-000009
此时x″即为意图恢复至x′的图像，那么可以理解的是模型的训练目的即为最小化x″与x′的差异。
计算机设备可再获取属于第二图像类别B的第二样本y′，将y′依次输入第一阶段图像逆向转化模型F1得到第二样本一阶转化图像x 1＝F 1(y′)，然后将x1输入第二阶段图像逆向转化模型F2得到第二样本二阶转化图像x 2＝F 2(x 1)，再根据x1和x2即可得到与第二样本对应、且属于第一图像类别的第二样本转化图像
Figure PCTCN2019072491-appb-000010
计算机设备再将
Figure PCTCN2019072491-appb-000011
输入第一阶段图像转化模型G1得到第二样本一阶恢复图像
Figure PCTCN2019072491-appb-000012
然后将
Figure PCTCN2019072491-appb-000013
输入第二阶段图像转化模型G2得到第二样本二阶恢复图像
Figure PCTCN2019072491-appb-000014
再根据
Figure PCTCN2019072491-appb-000015
和
Figure PCTCN2019072491-appb-000016
即可得到与第二样本对应、且属于第二图像类别的第二样本恢复图像
Figure PCTCN2019072491-appb-000017
此时y″即为意图恢复至y′的图像，那么可以理解的是模型的训练目的即为最小化y″与y′的差异。
那么第二阶段的模型训练中，循环一致性约束表达为：
Figure PCTCN2019072491-appb-000018
其中，L cyc2为基于循环一致性的损失函数。模型训练的过程可以包括调整第二阶段图像转化模型与第二阶段图像逆向转化模型的模型参数以最小化L cyc2的过程。
在本实施例中，利用了循环一致性约束非监督地训练机器学习模型，从而实现训练得到任意两种图像类别的图像之间转化的机器学习模型，不再受限于传统监督地训练机器学习模型时对样本标签的依赖，扩展了图像处理应用场景。
在一个实施例中，该图像处理方法还包括优化第二阶段图像转化模型的步骤。其中，优化第二阶段图像转化模型的步骤具体如下：将第一样本和第二样本转化图像分别输入第二阶段图像逆向转化鉴别模型，分别得到第一样本的鉴别置信度和第二样本转化图像的鉴别置信度；及将第二样本和第一样本转化图像分别输入第二阶段图像转化鉴别模型，分别得到第二样本的鉴别置信度和第一样本转化图像的鉴别置信度。按照第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异，调整第二阶段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：按照最大化第一 样本的鉴别置信度和第二样本的鉴别置信度的方向、最小化第二样本转化图像的鉴别置信度、第一样本转化图像的鉴别置信度、第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异的方向，调整第二阶段图像转化鉴别模型、第二阶段图像逆向转化鉴别模型、第二阶段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
继续参考图5，计算机设备可将x′与
Figure PCTCN2019072491-appb-000019
输入第二阶段图像逆向转化鉴别模型D x2，得到x′的鉴别置信度D x2(x′)，
Figure PCTCN2019072491-appb-000020
的鉴别置信度
Figure PCTCN2019072491-appb-000021
将y′与
Figure PCTCN2019072491-appb-000022
输入第二阶段图像转化鉴别模型D y2，得到y′的鉴别置信度D y2(y′)，
Figure PCTCN2019072491-appb-000023
的鉴别置信度
Figure PCTCN2019072491-appb-000024
那么第二阶段的模型训练中，对抗学习约束表达为：
Figure PCTCN2019072491-appb-000025
其中，L adv2为基于对抗学习约束的损失函数。模型训练的过程可以包括调整第二阶段图像转化鉴别模型与第二阶段图像逆向转化鉴别模型的模型参数以最大化L adv2的过程。最大化L adv2也就是最大化D x2(x′)与D y2(y′)，最小化
Figure PCTCN2019072491-appb-000026
与
Figure PCTCN2019072491-appb-000027
的过程。
那么在第二阶段的训练过程中，对抗地优化图像转化模型和鉴别模型表达为：
Figure PCTCN2019072491-appb-000028
其中，λ 2为循环一致性约束权重。最大化L adv2与最小化L cyc2的过程可以轮换进行。即在一个样本训练时先最大化L adv2再最小化L cyc2，在下一个样本训练时先最小化L cyc2再最大化L adv2。
在本实施例中，包括图像转化模型和鉴别模型两个模型的训练。其中，训练图像转化模型的过程在于学习如何将一种类别的图像转化为另一种类别的图像，训练鉴别模型的过程在于学习判断输入的图像是原始图像还是通过图像转化模型生成的图像。这样图像转化模型学习生成与原始图像更相似的图像，以干扰鉴别模型的判断，鉴别模型学习更加精准地进行原始图像和生成图像的判断，两个模型相互对抗，相互促进，使得训练得到的模型性能更优，从而在使用训练得到的图像转化模型进行图像转化时，能够极大程度上克服图像失真的问题，提高图像转化效果。
在一个实施例中，本领域技术人员可以理解，有监督地训练第二阶段图像转化模型和第二阶段图像逆向转化模型时，也可结合第二阶段图像转化鉴别模型和第二阶段图像逆向转化鉴别模型进行对抗式训练。
在一个实施例中，根据第一样本一阶转化图像和第一样本二阶转化图像，得到与第一 样本对应、且属于第二图像类别的第一样本转化图像，包括：将第一样本、第一样本一阶转化图像和第一样本二阶转化图像共同输入第一权重预测模型，得到与第一样本二阶转化图像对应的权重矩阵；根据权重矩阵，得到与第一样本一阶转化图像对应的权重矩阵；及将第一样本一阶转化图像和第一样本二阶转化图像按照各自对应的权重矩阵融合，得到与第一样本对应、且属于第二图像类别的第一样本转化图像。
根据第一样本一阶恢复图像和第一样本二阶恢复图像，得到与第一样本对应、且属于第一图像类别的第一样本恢复图像，包括：将第一样本转化图像、第一样本一阶恢复图像和第一样本二阶恢复图像共同输入第二权重预测模型，得到与第一样本二阶恢复图像对应的权重矩阵；根据权重矩阵，得到与第一样本一阶恢复图像对应的权重矩阵；及将第一样本一阶恢复图像和第一样本二阶恢复图像按照各自对应的权重矩阵融合，得到与第一样本对应、且属于第一图像类别的第一样本恢复图像。
根据第二样本一阶转化图像和第二样本二阶转化图像，得到与第二样本对应、且属于第一图像类别的第二样本转化图像，包括：将第二样本、第二样本一阶转化图像和第二样本二阶转化图像共同输入第二权重预测模型，得到与第二样本二阶转化图像对应的权重矩阵；根据权重矩阵，得到与第二样本一阶转化图像对应的权重矩阵；及将第二样本一阶转化图像和第二样本二阶转化图像按照各自对应的权重矩阵融合，得到与第二样本对应、且属于第一图像类别的第二样本转化图像。
根据第二样本一阶恢复图像和第二样本二阶恢复图像，得到与第二样本对应、且属于第二图像类别的第二样本恢复图像，包括：将第二样本转化图像、第二样本一阶恢复图像和第二样本二阶恢复图像共同输入第一权重预测模型，得到与第二样本二阶恢复图像对应的权重矩阵；根据权重矩阵，得到与第二样本一阶恢复图像对应的权重矩阵；及将第二样本一阶恢复图像和第二样本二阶恢复图像按照各自对应的权重矩阵融合，得到与第二样本对应、且属于第二图像类别的第二样本恢复图像。
按照第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异，调整第二阶段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：按照第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异，调整第一权重预测模型、第二权重预测模型、第二阶段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
继续参考图5，计算机设备可将x′、y1和y2共同输入第一权重预测模型Gf，得到 Gf输出的y2的权重矩阵α x′，继而得到y1的权重矩阵(1-α x′)，那么第一样本转化图像即为
Figure PCTCN2019072491-appb-000029
计算机设备再可将
Figure PCTCN2019072491-appb-000030
和
Figure PCTCN2019072491-appb-000031
共同输入第二权重预测模型Ff，得到Ff输出的
Figure PCTCN2019072491-appb-000032
的权重矩阵
Figure PCTCN2019072491-appb-000033
继而得到
Figure PCTCN2019072491-appb-000034
的权重矩阵
Figure PCTCN2019072491-appb-000035
那么第一样本恢复图像即为
Figure PCTCN2019072491-appb-000036
计算机设备可将y′、x1和x2共同输入第二权重预测模型Ff，得到Ff输出的x2的权重矩阵α y′，继而得到x1的权重矩阵(1-α y′)，那么第二样本转化图像即为
Figure PCTCN2019072491-appb-000037
计算机设备再可将
Figure PCTCN2019072491-appb-000038
和
Figure PCTCN2019072491-appb-000039
共同输入第一权重预测模型Gf，得到Gf输出的
Figure PCTCN2019072491-appb-000040
的权重矩阵
Figure PCTCN2019072491-appb-000041
继而得到
Figure PCTCN2019072491-appb-000042
的权重矩阵
Figure PCTCN2019072491-appb-000043
那么第二样本恢复图像即为
Figure PCTCN2019072491-appb-000044
在本实施例中，利用了循环一致性约束非监督地训练机器学习模型，从而实现训练得到任意两种图像类别的图像之间转化的机器学习模型，不再受限于传统监督地训练机器学习模型时对样本标签的依赖，扩展了图像处理应用场景。
图6示出了一个实施例中图像转化前后的图像示意图。参考图6，可以看到输入图像为第一图像类别的图像，输入图像中的马为单一颜色的马。输出图像为第二图像类别的图像，输出图像中的马则为多种颜色的马。
应该理解的是，虽然上述各实施例的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，上述各实施例中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
如图7所示，在一个实施例中，提供了一种图像处理装置700。参照图7，该图像处理装置700包括：获取模块701、第一阶段转化模块702、第二阶段转化模块703、确定模块704和融合模块705。图像处理装置700中包括的各个模块可全部或部分通过软件、硬件或其组合来实现。
获取模块701，用于获取属于第一图像类别的待处理图像。
第一阶段转化模块702，用于将待处理图像输入第一阶段图像转化模型，得到第一中间图像。
第二阶段转化模块703，用于通过第二阶段图像转化模型将第一中间图像转化为第二中间图像。
确定模块704，用于确定第二中间图像对应的第二权重矩阵；确定第一中间图像对应的第一权重矩阵。
融合模块705，用于将第一中间图像、第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与待处理图像对应、且属于第二图像类别的目标图像。
在一个实施例中，第一阶段转化模块702还用于对待处理图像进行下采样，得到图像尺寸缩小后的压缩图像；及将压缩图像输入第一阶段图像转化模型，输出图像尺寸与压缩图像的图像尺寸相同的第一中间图像。第二阶段转化模块703还用于对第一中间图像进行上采样，得到图像尺寸与待处理图像的图像尺寸相同的放大图像；及将放大图像输入第二阶段图像转化模型，输出图像尺寸与放大图像的图像尺寸相同的第二中间图像。
在一个实施例中，确定模块704还用于将待处理图像、第一中间图像与第二中间图像共同输入第一权重预测模型，得到与第二中间图像对应的第二权重矩阵；及根据第二权重矩阵，得到与第一中间图像对应的第一权重矩阵；第一权重矩阵与第二权重矩阵之和为预设矩阵。
在一个实施例中，融合模块705还用于将第一中间图像的各像素值与第一权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别的第一目标图像；将第二中间图像的各像素值与第二权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别第二目标图像；及根据第一目标图像与第二目标图像，得到与待处理图像对应、且属于第二图像类别的目标图像。
在一个实施例中，图像处理装置700还包括：第一阶段模型训练模块706，用于获取属于第一图像类别的第一样本和属于第二图像类别的第二样本；将第一样本依次经过第一阶段图像转化模型和第一阶段图像逆向转化模型，得到第一样本单阶段恢复图像；将第二样本依次经过第一阶段图像逆向转化模型和第一阶段图像转化模型，得到第二样本单阶段恢复图像；及按照第一样本与第一样本单阶段恢复图像的差异，及第二样本与第二样本单阶段恢复图像的差异，调整第一阶段图像转化模型和第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
在一个实施例中，第一阶段模型训练模块706还用于获取第一样本经过第一阶段图像 转化模型后，由第一阶段图像转化模型输出的第一样本单阶段转化图像；获取第二样本经过第一阶段图像逆向转化模型后，由第一阶段图像逆向转化模型输出的第二样本单阶段转化图像；将第一样本和第二样本单阶段转化图像分别输入第一阶段图像逆向转化鉴别模型，分别得到第一样本的鉴别置信度和第二样本单阶段转化图像的鉴别置信度；将第二样本和第一样本单阶段转化图像分别输入第一阶段图像转化鉴别模型，分别得到第二样本的鉴别置信度和第一样本单阶段转化图像的鉴别置信度；及按照最大化第一样本的鉴别置信度和第二样本的鉴别置信度的方向、最小化第二样本单阶段转化图像的鉴别置信度、第一样本单阶段转化图像的鉴别置信度、第一样本与第一样本单阶段恢复图像的差异，及第二样本与第二样本单阶段恢复图像的差异的方向，调整第一阶段图像转化鉴别模型、第一阶段图像逆向转化鉴别模型、第一阶段图像转化模型和第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
如图8所示，在一个实施例中，图像处理装置700还包括：第一阶段模型训练模块706和第二阶段模型训练模块707。
第二阶段模型训练模块707，用于将第一样本依次经过第一阶段图像转化模型和第二阶段图像转化模型，得到第一阶段图像转化模型输出的第一样本一阶转化图像，和第二阶段图像转化模型输出的第一样本二阶转化图像；根据第一样本一阶转化图像和第一样本二阶转化图像，得到与第一样本对应、且属于第二图像类别的第一样本转化图像；将第一样本转化图像依次经过第一阶段图像逆向转化模型和第二阶段图像逆向转化模型，得到第一阶段图像逆向转化模型输出的第一样本一阶恢复图像，和第二阶段图像逆向转化模型输出的第一样本二阶恢复图像；根据第一样本一阶恢复图像和第一样本二阶恢复图像，得到与第一样本对应、且属于第一图像类别的第一样本恢复图像；将第二样本依次经过第一阶段图像逆向转化模型和第二阶段图像逆向转化模型，得到第一阶段图像逆向转化模型输出的第二样本一阶转化图像，和第二阶段图像逆向转化模型输出的第二样本二阶转化图像；根据第二样本一阶转化图像和第二样本二阶转化图像，得到与第二样本对应、且属于第一图像类别的第二样本转化图像；将第二样本转化图像依次经过第一阶段图像转化模型和第二阶段图像转化模型，得到第一阶段图像转化模型输出的第二样本一阶恢复图像，和第二阶段图像转化模型输出的第二样本二阶恢复图像；根据第二样本一阶恢复图像和第二样本二阶恢复图像，得到与第二样本对应、且属于第二图像类别的第二样本恢复图像；及按照第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异，调整第二阶 段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
在一个实施例中，第二阶段模型训练模块707还用于将第一样本和第二样本转化图像分别输入第二阶段图像逆向转化鉴别模型，分别得到第一样本的鉴别置信度和第二样本转化图像的鉴别置信度；将第二样本和第一样本转化图像分别输入第二阶段图像转化鉴别模型，分别得到第二样本的鉴别置信度和第一样本转化图像的鉴别置信度；及按照最大化第一样本的鉴别置信度和第二样本的鉴别置信度的方向、最小化第二样本转化图像的鉴别置信度、第一样本转化图像的鉴别置信度、第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异的方向，调整第二阶段图像转化鉴别模型、第二阶段图像逆向转化鉴别模型、第二阶段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
在一个实施例中，第二阶段模型训练模块707还用于将第一样本、第一样本一阶转化图像和第一样本二阶转化图像共同输入第一权重预测模型，得到与第一样本二阶转化图像对应的权重矩阵；根据权重矩阵，得到与第一样本一阶转化图像对应的权重矩阵；将第一样本一阶转化图像和第一样本二阶转化图像按照各自对应的权重矩阵融合，得到与第一样本对应、且属于第二图像类别的第一样本转化图像。将第一样本转化图像、第一样本一阶恢复图像和第一样本二阶恢复图像共同输入第二权重预测模型，得到与第一样本二阶恢复图像对应的权重矩阵；根据权重矩阵，得到与第一样本一阶恢复图像对应的权重矩阵；将第一样本一阶恢复图像和第一样本二阶恢复图像按照各自对应的权重矩阵融合，得到与第一样本对应、且属于第一图像类别的第一样本恢复图像。将第二样本、第二样本一阶转化图像和第二样本二阶转化图像共同输入第二权重预测模型，得到与第二样本二阶转化图像对应的权重矩阵；根据权重矩阵，得到与第二样本一阶转化图像对应的权重矩阵；将第二样本一阶转化图像和第二样本二阶转化图像按照各自对应的权重矩阵融合，得到与第二样本对应、且属于第一图像类别的第二样本转化图像。将第二样本转化图像、第二样本一阶恢复图像和第二样本二阶恢复图像共同输入第一权重预测模型，得到与第二样本二阶恢复图像对应的权重矩阵；根据权重矩阵，得到与第二样本一阶恢复图像对应的权重矩阵；将第二样本一阶恢复图像和第二样本二阶恢复图像按照各自对应的权重矩阵融合，得到与第二样本对应、且属于第二图像类别的第二样本恢复图像；及按照第一样本与第一样本恢复图像的差异，及第二样本与第二样本恢复图像的差异，调整第一权重预测模型、第二权重预测模型、第二阶段图像转化模型和第二阶段图像逆向转化模型，直至满足训练停止条件 时结束训练。
图9示出了一个实施例中计算机设备的内部结构图。该计算机设备具体可以是用户终端。如图9所示，该计算机设备包括通过系统总线连接的处理器、存储器、网络接口、输入设备和显示屏。其中，存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质存储有操作系统，还可存储有计算机程序计算机可读指令，该计算机程序计算机可读指令被处理器执行时，可使得处理器实现图像处理方法。该内存储器中也可储存有计算机程序计算机可读指令，该计算机程序计算机可读指令被处理器执行时，可使得处理器执行图像处理方法。计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏等，输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，也可以是外接的键盘、触控板或鼠标等。本领域技术人员可以理解，图9中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
图10示出了一个实施例中计算机设备的内部结构图。该计算机设备具体可以是用户终端或者服务器。如图10所示，该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中，存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质存储有操作系统，还可存储有计算机可读指令，该计算机可读指令被处理器执行时，可使得处理器实现图像处理方法。该内存储器中也可储存有计算机可读指令，该计算机可读指令被处理器执行时，可使得处理器执行图像处理方法。本领域技术人员可以理解，图10中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，本申请提供的图像处理装置可以实现为一种计算机可读指令的形式，计算机可读指令可在如图9或图10所示的计算机设备上运行，计算机设备的非易失性存储介质可存储组成该图像处理装置的各个指令模块，比如，图7所示的获取模块701、第一阶段转化模块702、第二阶段转化模块703、确定模块704和融合模块705等。各个指令模块组成的计算机可读指令使得处理器执行本说明书中描述的本申请各个实施例的图像处理方法中的步骤。
例如，图9或图10所示的计算机设备可以通过如图7所示的图像处理装置700中的 获取模块701获取属于第一图像类别的待处理图像。通过第一阶段转化模块702将待处理图像输入第一阶段图像转化模型，得到第一中间图像。通过第二阶段转化模块703通过第二阶段图像转化模型将第一中间图像转化为第二中间图像。通过确定模块704分别确定第一中间图像与第二中间图像各自对应的权重矩阵。通过融合模块705将第一中间图像和第二中间图像按照各自对应的权重矩阵融合，得到与待处理图像对应、且属于第二图像类别的目标图像。
在一个实施例中，提供了一种计算机可读存储介质，该计算机可读存储介质上存储有计算机可读指令，该计算机可读指令被处理器执行时，使得处理器执行上述图像处理方法的步骤。此处图像处理方法的步骤可以是上述各个实施例的图像处理方法中的步骤。
在一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器中储存有计算机可读指令，计算机可读指令被处理器执行时，使得处理器执行上述图像处理方法的步骤。此处图像处理方法的步骤可以是上述各个实施例的图像处理方法中的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。 因此，本申请专利的保护范围应以所附权利要求为准。

Claims (19)

  1. 一种图像处理方法，由计算机设备执行，所述方法包括：
    获取属于第一图像类别的待处理图像；
    将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像；
    通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像；
    确定所述第二中间图像对应的第二权重矩阵；
    确定所述第一中间图像对应的第一权重矩阵；及
    将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像。
  2. 根据权利要求1所述的方法，其特征在于，所述将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像，包括：
    对所述待处理图像进行下采样，得到图像尺寸缩小后的压缩图像；及
    将所述压缩图像输入第一阶段图像转化模型，输出图像尺寸与所述压缩图像的图像尺寸相同的第一中间图像；
    所述通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像，包括：
    对所述第一中间图像进行上采样，得到图像尺寸与所述待处理图像的图像尺寸相同的放大图像；及
    将所述放大图像输入第二阶段图像转化模型，输出图像尺寸与所述放大图像的图像尺寸相同的第二中间图像。
  3. 根据权利要求1所述的方法，其特征在于，所述确定所述第二中间图像对应的第二权重矩阵，包括：
    将所述待处理图像、所述第一中间图像与所述第二中间图像共同输入第一权重预测模型，得到与所述第二中间图像对应的第二权重矩阵；
    所述确定所述第一中间图像对应的第一权重矩阵，包括：
    根据所述第二权重矩阵，得到与所述第一中间图像对应的第一权重矩阵；所述第一权重矩阵与所述第二权重矩阵之和为预设矩阵。
  4. 根据权利要求3所述的方法，其特征在于，所述将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像，包括：
    将所述第一中间图像的各像素值与所述第一权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别的第一目标图像；
    将所述第二中间图像的各像素值与所述第二权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别第二目标图像；及
    根据所述第一目标图像与所述第二目标图像，得到与所述待处理图像对应、且属于第二图像类别的目标图像。
  5. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    获取属于第一图像类别的第一样本和属于第二图像类别的第二样本；
    将所述第一样本依次经过所述第一阶段图像转化模型和第一阶段图像逆向转化模型，得到第一样本单阶段恢复图像；
    将所述第二样本依次经过所述第一阶段图像逆向转化模型和所述第一阶段图像转化模型，得到第二样本单阶段恢复图像；及
    按照所述第一样本与所述第一样本单阶段恢复图像的差异，及所述第二样本与所述第二样本单阶段恢复图像的差异，调整所述第一阶段图像转化模型和所述第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  6. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    获取所述第一样本经过所述第一阶段图像转化模型后，由所述第一阶段图像转化模型输出的第一样本单阶段转化图像；
    获取所述第二样本经过所述第一阶段图像逆向转化模型后，由所述第一阶段图像逆向转化模型输出的第二样本单阶段转化图像；
    将所述第一样本和所述第二样本单阶段转化图像分别输入第一阶段图像逆向转化鉴别模型，分别得到所述第一样本的鉴别置信度和所述第二样本单阶段转化图像的鉴别置信度；及
    将所述第二样本和所述第一样本单阶段转化图像分别输入第一阶段图像转化鉴别模型，分别得到所述第二样本的鉴别置信度和所述第一样本单阶段转化图像的鉴别置信度；
    所述按照所述第一样本与所述第一样本单阶段恢复图像的差异，及所述第二样本与所述第二样本单阶段恢复图像的差异，调整所述第一阶段图像转化模型和所述第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：
    按照最大化所述第一样本的鉴别置信度和所述第二样本的鉴别置信度的方向、最小 化所述第二样本单阶段转化图像的鉴别置信度、所述第一样本单阶段转化图像的鉴别置信度、所述第一样本与所述第一样本单阶段恢复图像的差异，及所述第二样本与所述第二样本单阶段恢复图像的差异的方向，调整所述第一阶段图像转化鉴别模型、所述第一阶段图像逆向转化鉴别模型、所述第一阶段图像转化模型和所述第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  7. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    将所述第一样本依次经过所述第一阶段图像转化模型和所述第二阶段图像转化模型，得到所述第一阶段图像转化模型输出的第一样本一阶转化图像，和所述第二阶段图像转化模型输出的第一样本二阶转化图像；
    根据所述第一样本一阶转化图像和所述第一样本二阶转化图像，得到与所述第一样本对应、且属于所述第二图像类别的第一样本转化图像；
    将所述第一样本转化图像依次经过所述第一阶段图像逆向转化模型和第二阶段图像逆向转化模型，得到所述第一阶段图像逆向转化模型输出的第一样本一阶恢复图像，和所述第二阶段图像逆向转化模型输出的第一样本二阶恢复图像；
    根据所述第一样本一阶恢复图像和所述第一样本二阶恢复图像，得到与所述第一样本对应、且属于所述第一图像类别的第一样本恢复图像；
    将所述第二样本依次经过所述第一阶段图像逆向转化模型和所述第二阶段图像逆向转化模型，得到所述第一阶段图像逆向转化模型输出的第二样本一阶转化图像，和所述第二阶段图像逆向转化模型输出的第二样本二阶转化图像；
    根据所述第二样本一阶转化图像和所述第二样本二阶转化图像，得到与所述第二样本对应、且属于所述第一图像类别的第二样本转化图像；
    将所述第二样本转化图像依次经过所述第一阶段图像转化模型和第二阶段图像转化模型，得到所述第一阶段图像转化模型输出的第二样本一阶恢复图像，和所述第二阶段图像转化模型输出的第二样本二阶恢复图像；
    根据所述第二样本一阶恢复图像和所述第二样本二阶恢复图像，得到与所述第二样本对应、且属于所述第二图像类别的第二样本恢复图像；及
    按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  8. 根据权利要求7所述的方法，其特征在于，所述方法还包括：
    将所述第一样本和所述第二样本转化图像分别输入第二阶段图像逆向转化鉴别模型，分别得到所述第一样本的鉴别置信度和所述第二样本转化图像的鉴别置信度；及
    将所述第二样本和所述第一样本转化图像分别输入第二阶段图像转化鉴别模型，分别得到所述第二样本的鉴别置信度和所述第一样本转化图像的鉴别置信度；
    所述按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：
    按照最大化所述第一样本的鉴别置信度和所述第二样本的鉴别置信度的方向、最小化所述第二样本转化图像的鉴别置信度、所述第一样本转化图像的鉴别置信度、所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异的方向，调整所述第二阶段图像转化鉴别模型、所述第二阶段图像逆向转化鉴别模型、所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  9. 根据权利要求7所述的方法，其特征在于，所述根据所述第一样本一阶转化图像和所述第一样本二阶转化图像，得到与所述第一样本对应、且属于所述第二图像类别的第一样本转化图像，包括：
    将所述第一样本、所述第一样本一阶转化图像和所述第一样本二阶转化图像共同输入第一权重预测模型，得到与所述第一样本二阶转化图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第一样本一阶转化图像对应的权重矩阵；及
    将所述第一样本一阶转化图像和所述第一样本二阶转化图像按照各自对应的权重矩阵融合，得到与所述第一样本对应、且属于第二图像类别的第一样本转化图像；
    所述根据所述第一样本一阶恢复图像和所述第一样本二阶恢复图像，得到与所述第一样本对应、且属于所述第一图像类别的第一样本恢复图像，包括：
    将所述第一样本转化图像、所述第一样本一阶恢复图像和所述第一样本二阶恢复图像共同输入第二权重预测模型，得到与所述第一样本二阶恢复图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第一样本一阶恢复图像对应的权重矩阵；及
    将所述第一样本一阶恢复图像和所述第一样本二阶恢复图像按照各自对应的权重矩阵融合，得到与所述第一样本对应、且属于第一图像类别的第一样本恢复图像；
    所述根据所述第二样本一阶转化图像和所述第二样本二阶转化图像，得到与所述第二样本对应、且属于所述第一图像类别的第二样本转化图像，包括：
    将所述第二样本、所述第二样本一阶转化图像和所述第二样本二阶转化图像共同输入所述第二权重预测模型，得到与所述第二样本二阶转化图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第二样本一阶转化图像对应的权重矩阵；及
    将所述第二样本一阶转化图像和所述第二样本二阶转化图像按照各自对应的权重矩阵融合，得到与所述第二样本对应、且属于第一图像类别的第二样本转化图像；
    所述根据所述第二样本一阶恢复图像和所述第二样本二阶恢复图像，得到与所述第二样本对应、且属于所述第二图像类别的第二样本恢复图像，包括：
    将所述第二样本转化图像、所述第二样本一阶恢复图像和所述第二样本二阶恢复图像共同输入所述第一权重预测模型，得到与所述第二样本二阶恢复图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第二样本一阶恢复图像对应的权重矩阵；及
    将所述第二样本一阶恢复图像和所述第二样本二阶恢复图像按照各自对应的权重矩阵融合，得到与所述第二样本对应、且属于第二图像类别的第二样本恢复图像；
    所述按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：
    按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第一权重预测模型、所述第二权重预测模型、所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  10. 一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行以下步骤：
    获取属于第一图像类别的待处理图像；
    将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像；
    通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像；
    确定所述第二中间图像对应的第二权重矩阵；
    确定所述第一中间图像对应的第一权重矩阵；及
    将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像。
  11. 根据权利要求10所述的计算机设备，其特征在于，所述将所述待处理图像输入第一阶段图像转化模型，得到第一中间图像，包括：
    对所述待处理图像进行下采样，得到图像尺寸缩小后的压缩图像；及
    将所述压缩图像输入第一阶段图像转化模型，输出图像尺寸与所述压缩图像的图像尺寸相同的第一中间图像；
    所述通过第二阶段图像转化模型将所述第一中间图像转化为第二中间图像，包括：
    对所述第一中间图像进行上采样，得到图像尺寸与所述待处理图像的图像尺寸相同的放大图像；及
    将所述放大图像输入第二阶段图像转化模型，输出图像尺寸与所述放大图像的图像尺寸相同的第二中间图像。
  12. 根据权利要求10所述的计算机设备，其特征在于，所述确定所述第二中间图像对应的第二权重矩阵，包括：
    将所述待处理图像、所述第一中间图像与所述第二中间图像共同输入第一权重预测模型，得到与所述第二中间图像对应的第二权重矩阵；
    所述确定所述第一中间图像对应的第一权重矩阵，包括：
    根据所述第二权重矩阵，得到与所述第一中间图像对应的第一权重矩阵；所述第一权重矩阵与所述第二权重矩阵之和为预设矩阵。
  13. 根据权利要求12所述的计算机设备，其特征在于，所述将所述第一中间图像、所述第二中间图像按照对应的第一权重矩阵、第二权重矩阵进行融合，得到与所述待处理图像对应、且属于第二图像类别的目标图像，包括：
    将所述第一中间图像的各像素值与所述第一权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别的第一目标图像；
    将所述第二中间图像的各像素值与所述第二权重矩阵的各矩阵元素按位相乘，得到属于第二图像类别第二目标图像；及
    根据所述第一目标图像与所述第二目标图像，得到与所述待处理图像对应、且属于第二图像类别的目标图像。
  14. 根据权利要求10所述的计算机设备，其特征在于，所述计算机可读指令被所述处理器执行时，还使得所述处理器执行以下步骤：
    获取属于第一图像类别的第一样本和属于第二图像类别的第二样本；
    将所述第一样本依次经过所述第一阶段图像转化模型和第一阶段图像逆向转化模型，得到第一样本单阶段恢复图像；
    将所述第二样本依次经过所述第一阶段图像逆向转化模型和所述第一阶段图像转化模型，得到第二样本单阶段恢复图像；及
    按照所述第一样本与所述第一样本单阶段恢复图像的差异，及所述第二样本与所述第二样本单阶段恢复图像的差异，调整所述第一阶段图像转化模型和所述第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  15. 根据权利要求14所述的计算机设备，其特征在于，所述计算机可读指令被所述处理器执行时，还使得所述处理器执行以下步骤：
    获取所述第一样本经过所述第一阶段图像转化模型后，由所述第一阶段图像转化模型输出的第一样本单阶段转化图像；
    获取所述第二样本经过所述第一阶段图像逆向转化模型后，由所述第一阶段图像逆向转化模型输出的第二样本单阶段转化图像；
    将所述第一样本和所述第二样本单阶段转化图像分别输入第一阶段图像逆向转化鉴别模型，分别得到所述第一样本的鉴别置信度和所述第二样本单阶段转化图像的鉴别置信度；及
    将所述第二样本和所述第一样本单阶段转化图像分别输入第一阶段图像转化鉴别模型，分别得到所述第二样本的鉴别置信度和所述第一样本单阶段转化图像的鉴别置信度；
    所述按照所述第一样本与所述第一样本单阶段恢复图像的差异，及所述第二样本与所述第二样本单阶段恢复图像的差异，调整所述第一阶段图像转化模型和所述第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：
    按照最大化所述第一样本的鉴别置信度和所述第二样本的鉴别置信度的方向、最小化所述第二样本单阶段转化图像的鉴别置信度、所述第一样本单阶段转化图像的鉴别置信度、所述第一样本与所述第一样本单阶段恢复图像的差异，及所述第二样本与所述第二样本单阶段恢复图像的差异的方向，调整所述第一阶段图像转化鉴别模型、所述第一阶段图像逆向转化鉴别模型、所述第一阶段图像转化模型和所述第一阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  16. 根据权利要求14所述的计算机设备，其特征在于，所述计算机可读指令被所述处理器执行时，还使得所述处理器执行以下步骤：
    将所述第一样本依次经过所述第一阶段图像转化模型和所述第二阶段图像转化模型，得到所述第一阶段图像转化模型输出的第一样本一阶转化图像，和所述第二阶段图像转化模型输出的第一样本二阶转化图像；
    根据所述第一样本一阶转化图像和所述第一样本二阶转化图像，得到与所述第一样本对应、且属于所述第二图像类别的第一样本转化图像；
    将所述第一样本转化图像依次经过所述第一阶段图像逆向转化模型和第二阶段图像逆向转化模型，得到所述第一阶段图像逆向转化模型输出的第一样本一阶恢复图像，和所述第二阶段图像逆向转化模型输出的第一样本二阶恢复图像；
    根据所述第一样本一阶恢复图像和所述第一样本二阶恢复图像，得到与所述第一样本对应、且属于所述第一图像类别的第一样本恢复图像；
    将所述第二样本依次经过所述第一阶段图像逆向转化模型和所述第二阶段图像逆向转化模型，得到所述第一阶段图像逆向转化模型输出的第二样本一阶转化图像，和所述第二阶段图像逆向转化模型输出的第二样本二阶转化图像；
    根据所述第二样本一阶转化图像和所述第二样本二阶转化图像，得到与所述第二样本对应、且属于所述第一图像类别的第二样本转化图像；
    将所述第二样本转化图像依次经过所述第一阶段图像转化模型和第二阶段图像转化模型，得到所述第一阶段图像转化模型输出的第二样本一阶恢复图像，和所述第二阶段图像转化模型输出的第二样本二阶恢复图像；
    根据所述第二样本一阶恢复图像和所述第二样本二阶恢复图像，得到与所述第二样本对应、且属于所述第二图像类别的第二样本恢复图像；及
    按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  17. 根据权利要求16所述的计算机设备，其特征在于，所述计算机可读指令被所述处理器执行时，还使得所述处理器执行以下步骤：
    将所述第一样本和所述第二样本转化图像分别输入第二阶段图像逆向转化鉴别模型，分别得到所述第一样本的鉴别置信度和所述第二样本转化图像的鉴别置信度；及
    将所述第二样本和所述第一样本转化图像分别输入第二阶段图像转化鉴别模型，分别得到所述第二样本的鉴别置信度和所述第一样本转化图像的鉴别置信度；
    所述按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：
    按照最大化所述第一样本的鉴别置信度和所述第二样本的鉴别置信度的方向、最小化所述第二样本转化图像的鉴别置信度、所述第一样本转化图像的鉴别置信度、所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异的方向，调整所述第二阶段图像转化鉴别模型、所述第二阶段图像逆向转化鉴别模型、所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  18. 根据权利要求16所述的计算机设备，其特征在于，所述根据所述第一样本一阶转化图像和所述第一样本二阶转化图像，得到与所述第一样本对应、且属于所述第二图像类别的第一样本转化图像，包括：
    将所述第一样本、所述第一样本一阶转化图像和所述第一样本二阶转化图像共同输入第一权重预测模型，得到与所述第一样本二阶转化图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第一样本一阶转化图像对应的权重矩阵；及
    将所述第一样本一阶转化图像和所述第一样本二阶转化图像按照各自对应的权重矩阵融合，得到与所述第一样本对应、且属于第二图像类别的第一样本转化图像；
    所述根据所述第一样本一阶恢复图像和所述第一样本二阶恢复图像，得到与所述第一样本对应、且属于所述第一图像类别的第一样本恢复图像，包括：
    将所述第一样本转化图像、所述第一样本一阶恢复图像和所述第一样本二阶恢复图像共同输入第二权重预测模型，得到与所述第一样本二阶恢复图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第一样本一阶恢复图像对应的权重矩阵；及
    将所述第一样本一阶恢复图像和所述第一样本二阶恢复图像按照各自对应的权重矩阵融合，得到与所述第一样本对应、且属于第一图像类别的第一样本恢复图像；
    所述根据所述第二样本一阶转化图像和所述第二样本二阶转化图像，得到与所述第二样本对应、且属于所述第一图像类别的第二样本转化图像，包括：
    将所述第二样本、所述第二样本一阶转化图像和所述第二样本二阶转化图像共同输入所述第二权重预测模型，得到与所述第二样本二阶转化图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第二样本一阶转化图像对应的权重矩阵；及
    将所述第二样本一阶转化图像和所述第二样本二阶转化图像按照各自对应的权重矩阵融合，得到与所述第二样本对应、且属于第一图像类别的第二样本转化图像；
    所述根据所述第二样本一阶恢复图像和所述第二样本二阶恢复图像，得到与所述第二样本对应、且属于所述第二图像类别的第二样本恢复图像，包括：
    将所述第二样本转化图像、所述第二样本一阶恢复图像和所述第二样本二阶恢复图像共同输入所述第一权重预测模型，得到与所述第二样本二阶恢复图像对应的权重矩阵；
    根据所述权重矩阵，得到与所述第二样本一阶恢复图像对应的权重矩阵；及
    将所述第二样本一阶恢复图像和所述第二样本二阶恢复图像按照各自对应的权重矩阵融合，得到与所述第二样本对应、且属于第二图像类别的第二样本恢复图像；
    所述按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练，包括：
    按照所述第一样本与所述第一样本恢复图像的差异，及所述第二样本与所述第二样本恢复图像的差异，调整所述第一权重预测模型、所述第二权重预测模型、所述第二阶段图像转化模型和所述第二阶段图像逆向转化模型，直至满足训练停止条件时结束训练。
  19. 一种存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如权利要求1至9中任一项所述的方法的步骤。
PCT/CN2019/072491 2018-01-26 2019-01-21 图像处理方法、存储介质和计算机设备 WO2019144855A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19743676.9A EP3745349A4 (en) 2018-01-26 2019-01-21 IMAGE PROCESSING, COMPUTER STORAGE MEDIA AND COMPUTER DEVICE
US16/880,883 US11276207B2 (en) 2018-01-26 2020-05-21 Image processing method, storage medium, and computer device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810078544.8 2018-01-26
CN201810078544.8A CN108305238B (zh) 2018-01-26 2018-01-26 图像处理方法、装置、存储介质和计算机设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/880,883 Continuation US11276207B2 (en) 2018-01-26 2020-05-21 Image processing method, storage medium, and computer device

Publications (1)

Publication Number Publication Date
WO2019144855A1 true WO2019144855A1 (zh) 2019-08-01

Family

ID=62866445

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/072491 WO2019144855A1 (zh) 2018-01-26 2019-01-21 图像处理方法、存储介质和计算机设备

Country Status (4)

Country Link
US (1) US11276207B2 (zh)
EP (1) EP3745349A4 (zh)
CN (2) CN108305238B (zh)
WO (1) WO2019144855A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112561792A (zh) * 2020-12-25 2021-03-26 北京百度网讯科技有限公司 图像风格迁移方法、装置、电子设备及存储介质
KR20210150968A (ko) * 2020-06-04 2021-12-13 엔비직스 엘티디 디스플레이 장치 및 시스템

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108305238B (zh) 2018-01-26 2022-03-29 腾讯科技（深圳）有限公司 图像处理方法、装置、存储介质和计算机设备
US11164067B2 (en) * 2018-08-29 2021-11-02 Arizona Board Of Regents On Behalf Of Arizona State University Systems, methods, and apparatuses for implementing a multi-resolution neural network for use with imaging intensive applications including medical imaging
CN113902921B (zh) * 2018-11-30 2022-11-25 腾讯科技（深圳）有限公司 图像处理方法、装置、设备及存储介质
CN109859152B (zh) * 2018-12-25 2021-05-25 北京奇艺世纪科技有限公司 模型生成方法、图像增强方法、装置及计算机可读存储介质
CN109902723A (zh) * 2019-01-31 2019-06-18 北京市商汤科技开发有限公司 图像处理方法及装置
EP3857506A4 (en) * 2019-05-03 2022-01-05 Samsung Electronics Co., Ltd. IMAGE PROCESSING DEVICE AND IMAGE PROCESSING METHOD THEREOF
CN110473151B (zh) * 2019-07-04 2022-05-03 北京航空航天大学 基于分区卷积和关联损失的双阶段图像补全方法及系统
US11416998B2 (en) * 2019-07-30 2022-08-16 Microsoft Technology Licensing, Llc Pixel classification to reduce depth-estimation error
US11080833B2 (en) * 2019-11-22 2021-08-03 Adobe Inc. Image manipulation using deep learning techniques in a patch matching operation
CN113496234A (zh) * 2020-03-19 2021-10-12 中移(上海)信息通信科技有限公司 图像分类模型训练方法、图像分类方法、装置及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106651766A (zh) * 2016-12-30 2017-05-10 深圳市唯特视科技有限公司 一种基于深度卷积神经网络的图像风格迁移方法
AU2017101166A4 (en) * 2017-08-25 2017-11-02 Lai, Haodong MR A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks
CN107516290A (zh) * 2017-07-14 2017-12-26 北京奇虎科技有限公司 图像转换网络获取方法、装置、计算设备及存储介质
CN108305238A (zh) * 2018-01-26 2018-07-20 腾讯科技（深圳）有限公司 图像处理方法、装置、存储介质和计算机设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8885976B1 (en) * 2013-06-20 2014-11-11 Cyberlink Corp. Systems and methods for performing image fusion
US9053558B2 (en) * 2013-07-26 2015-06-09 Rui Shen Method and system for fusing multiple images
CN103617596A (zh) * 2013-10-12 2014-03-05 中山大学 一种基于流型转换的图像色彩风格变换方法
EP2950306A1 (en) * 2014-05-29 2015-12-02 Samsung Electronics Polska Spolka z organiczona odpowiedzialnoscia A method and system for building a language model
US9734143B2 (en) * 2015-12-17 2017-08-15 Facebook, Inc. Multi-media context language processing
CN106211196B (zh) * 2016-06-22 2019-05-10 西北工业大学 基于马尔科夫随机场的多无人航行体协同一致性方法
CN106373109B (zh) * 2016-08-31 2018-10-26 南方医科大学 一种医学图像模态合成的方法
CN107220929A (zh) * 2017-06-23 2017-09-29 深圳市唯特视科技有限公司 一种使用循环一致对抗网络的非成对图像转化方法
CN107577985B (zh) * 2017-07-18 2019-10-15 南京邮电大学 基于循环生成对抗网络的人脸头像卡通化的实现方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106651766A (zh) * 2016-12-30 2017-05-10 深圳市唯特视科技有限公司 一种基于深度卷积神经网络的图像风格迁移方法
CN107516290A (zh) * 2017-07-14 2017-12-26 北京奇虎科技有限公司 图像转换网络获取方法、装置、计算设备及存储介质
AU2017101166A4 (en) * 2017-08-25 2017-11-02 Lai, Haodong MR A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks
CN108305238A (zh) * 2018-01-26 2018-07-20 腾讯科技（深圳）有限公司 图像处理方法、装置、存储介质和计算机设备

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HUANG, HAOZHI ET AL.: "Real-Time Neural Style Transfer for Videos", 2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 21 July 2017 (2017-07-21), pages 7044 - 7052, XP033250071, ISSN: 1063-6919 *
See also references of EP3745349A4

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20210150968A (ko) * 2020-06-04 2021-12-13 엔비직스 엘티디 디스플레이 장치 및 시스템
KR102575670B1 (ko) 2020-06-04 2023-09-07 엔비직스 엘티디 디스플레이 장치 및 시스템
CN112561792A (zh) * 2020-12-25 2021-03-26 北京百度网讯科技有限公司 图像风格迁移方法、装置、电子设备及存储介质
CN112561792B (zh) * 2020-12-25 2023-10-03 北京百度网讯科技有限公司 图像风格迁移方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
EP3745349A1 (en) 2020-12-02
EP3745349A4 (en) 2021-10-20
CN108305238A (zh) 2018-07-20
CN108305238B (zh) 2022-03-29
CN110555821B (zh) 2022-04-05
CN110555821A (zh) 2019-12-10
US20200286263A1 (en) 2020-09-10
US11276207B2 (en) 2022-03-15

Similar Documents

Publication Publication Date Title
WO2019144855A1 (zh) 图像处理方法、存储介质和计算机设备
WO2021073493A1 (zh) 图像处理方法及装置、神经网络的训练方法、合并神经网络模型的图像处理方法、合并神经网络模型的构建方法、神经网络处理器及存储介质
US20210158096A1 (en) Systems and methods for performing direct conversion of image sensor data to image analytics
US20230021661A1 (en) Forgery detection of face image
CN113066017B (zh) 一种图像增强方法、模型训练方法及设备
US20200013205A1 (en) Colorizing Vector Graphic Objects
US11887270B2 (en) Multi-scale transformer for image analysis
CN112990219A (zh) 用于图像语义分割的方法和装置
US11669943B2 (en) Dual-stage system for computational photography, and technique for training same
WO2023169582A1 (zh) 图像增强方法、装置、设备及介质
CN113961736A (zh) 文本生成图像的方法、装置、计算机设备和存储介质
CN115565043A (zh) 结合多表征特征以及目标预测法进行目标检测的方法
CN116645598A (zh) 一种基于通道注意力特征融合的遥感图像语义分割方法
CN116453232A (zh) 人脸活体检测方法、人脸活体检测模型的训练方法和装置
US20220301106A1 (en) Training method and apparatus for image processing model, and image processing method and apparatus
WO2020187029A1 (zh) 图像处理方法及装置、神经网络的训练方法、存储介质
Xie et al. Dual camera snapshot high-resolution-hyperspectral imaging system with parallel joint optimization via physics-informed learning
US20230343137A1 (en) Method and apparatus for detecting key point of image, computer device and storage medium
CN115546577A (zh) 一种多模态数据集的数据增强方法及装置
CN115700771A (zh) 编解码方法及装置
RU2777354C2 (ru) Система распознавания изображения: beorg smart vision
US20240161487A1 (en) Adaptive mixed-resolution processing
US20230237628A1 (en) Modeling continuous kernels to generate an enhanced digital image from a burst of digital images
US20240233074A9 (en) Image processing method and device, training method of neural network, image processing method based on combined neural network model, constructing method of combined neural network model, neural network processor, and storage medium
CN115471896A (zh) 证件照制作方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19743676

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019743676

Country of ref document: EP

Effective date: 20200826