WO2021109876A1 - 图像处理方法、装置、设备及存储介质 - Google Patents

图像处理方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2021109876A1
WO2021109876A1 PCT/CN2020/130090 CN2020130090W WO2021109876A1 WO 2021109876 A1 WO2021109876 A1 WO 2021109876A1 CN 2020130090 W CN2020130090 W CN 2020130090W WO 2021109876 A1 WO2021109876 A1 WO 2021109876A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
style
transfer
transferred
model
Prior art date
Application number
PCT/CN2020/130090
Other languages
English (en)
French (fr)
Inventor
朱圣晨
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to EP20896430.4A priority Critical patent/EP4064189A1/en
Publication of WO2021109876A1 publication Critical patent/WO2021109876A1/zh
Priority to US17/824,043 priority patent/US20220284638A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the embodiments of the present application relate to the field of image processing technology, and in particular, to an image processing method, device, device, and storage medium.
  • the user can perform style transfer processing on the original image, thereby generating a new image.
  • the style transfer process refers to the process of using the original image as the content image, selecting another image as the style image, and using the style transfer algorithm to generate a new image whose content is similar to the content image and the style is similar to the style image.
  • the original image is subjected to style transfer processing by adding a layer of filter on the upper layer of the original image.
  • the embodiments of the present application provide an image processing method, device, equipment, and storage medium.
  • the technical solution is as follows:
  • an embodiment of the present application provides an image processing method, and the method includes:
  • the image to be transferred includes at least one of the following: the foreground image and the background image, and the style selection information includes n types of transfer styles, The n is a positive integer;
  • an output image is generated.
  • an embodiment of the present application provides an image processing device, and the device includes:
  • the image segmentation module is used to segment the original image to obtain the foreground image and the background image;
  • the image processing module is configured to perform style transfer processing on the image to be transferred based on the style selection information to obtain the transferred image; wherein the image to be transferred includes at least one of the following: the foreground image, the background image, and the style selection information Including n types of migration styles, where n is a positive integer;
  • the image generation module is used to generate an output image based on the migration image.
  • an embodiment of the present application provides a computer device, the computer device includes a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned aspects.
  • an embodiment of the present application provides a computer-readable storage medium in which a computer program is stored, and the computer program is loaded and executed by a processor to realize the image processing described in the above-mentioned aspect. method.
  • Fig. 1 is a flowchart of an image processing method provided by an embodiment of the present application
  • Fig. 2 is a schematic structural diagram of a style transfer model provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a convolutional layer provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a residual layer provided by an embodiment of the present application.
  • Fig. 5 is a flowchart of an image segmentation method provided by an embodiment of the present application.
  • Fig. 6 is a flowchart of an image processing method provided by another embodiment of the present application.
  • FIG. 7 is a block diagram of an image processing device provided by an embodiment of the present application.
  • FIG. 8 is a block diagram of an image processing device provided by another embodiment of the present application.
  • Fig. 9 is a structural block diagram of a computer device provided by an embodiment of the present application.
  • the execution subject of each step may be a computer device, and the computer device refers to an electronic device with computing and processing capabilities.
  • the computer device can be a terminal, for example, a terminal can be a mobile phone, a tablet computer, a multimedia player, or other portable electronic devices; the computer device can also be a server, and the server can be a server or a server cluster composed of multiple servers.
  • the embodiment of the present application provides an image processing method, the method includes:
  • the image to be transferred includes at least one of the following: the foreground image and the background image, and the style selection information includes n types of transfer styles, The n is a positive integer;
  • an output image is generated.
  • the performing style transfer processing on the image to be transferred based on the style selection information to obtain the transferred image includes:
  • the performing style transfer processing on the image to be transferred based on the style selection information through the style transfer model to obtain the transferred image includes:
  • the performing style transfer processing on the image to be transferred based on the m transfer styles by the style transfer model to obtain the transferred image includes:
  • the product of the weight of the i-th transfer style and the style parameter of the i-th transfer style is determined as the target of the i-th transfer style Style parameter, said i is a positive integer less than or equal to said m;
  • the image to be migrated includes the foreground image and the background image;
  • the step of performing style transfer processing on the image to be transferred based on the style selection information through the style transfer model, and before obtaining the transferred image further includes:
  • a model that supports style transfer processing using first style selection information and second style selection information is selected from a set of style transfer models as the style transfer model, and the set of style transfer models includes at least one model, and the first style
  • the selection information represents the migration style corresponding to the foreground image
  • the second style selection information represents the migration style corresponding to the background image.
  • the image to be migrated includes the foreground image and the background image;
  • the step of performing style transfer processing on the image to be transferred based on the style selection information through the style transfer model, and before obtaining the transferred image further includes:
  • a model that supports the use of first style selection information for style transfer processing is selected from a set of style transfer models as a first style transfer model, the set of style transfer models includes at least one model, and the first style selection information represents the foreground A transfer style corresponding to the image, where the first style transfer model is used to perform style transfer processing on the foreground image;
  • a model that supports the use of second style selection information for style transfer processing is selected from the set of style transfer models as the second style transfer model, where the second style selection information represents the transfer style corresponding to the background image, and the second style transfer model
  • the style transfer model is used to perform style transfer processing on the background image.
  • the performing style transfer processing on the image to be transferred based on the style selection information by the style transfer model, and before obtaining the transferred image further includes:
  • the training data includes at least one training sample
  • the training sample includes a training image and a style image
  • the style image refers to an image used as a reference style when performing style transfer processing
  • the value of the loss function is determined based on the content feature of the training image, the content feature of the training migration image, the style feature of the training migration image, and the style feature of the style image, include:
  • the value of the loss function is determined based on the value of the content loss function and the value of the style loss function.
  • the style transfer model includes a depth separable convolutional layer, an instance grouping layer, a nonlinear activation layer, a nearest neighbor upsampling layer, and an output layer;
  • the depth separable convolution layer is used to perform a depth convolution operation on the image to be transferred to obtain a first output feature; perform a point-by-point convolution operation on the first output feature to obtain a second output feature;
  • the examples are grouped into one layer for normalizing the second output feature based on the transfer style parameter to obtain a third output feature, where the transfer style parameter refers to the transfer style parameter corresponding to the image to be transferred ;
  • the non-linear activation layer is used to perform a non-linear operation on the third output feature to obtain a fourth output feature
  • the nearest neighbor upsampling layer is used to perform an interpolation operation on the fourth output feature to obtain a fifth output feature, the resolution of the fifth output feature is greater than the resolution of the fourth output feature;
  • the output layer is configured to output the migration image after performing a convolution operation on the fifth output feature.
  • the image to be migrated includes the foreground image and the background image;
  • the generating an output image based on the migration image includes:
  • the foreground style transfer image refers to an image obtained after performing style transfer processing on the foreground image
  • the background style transfer image refers to an image obtained after performing style transfer processing on the background image
  • the segmentation processing on the original image to obtain the foreground image and the background image includes:
  • the original image is respectively multiplied by the foreground grayscale matrix and the background grayscale matrix to obtain the foreground image and the background image.
  • the performing style transfer processing on the image to be transferred based on the style selection information, before obtaining the transferred image further includes:
  • the image confirmation instruction indicates that the image to be transferred includes the foreground image, acquiring first style selection information
  • the image confirmation instruction indicates that the image to be transferred includes the background image, acquiring second style selection information
  • the image confirmation instruction indicates that the image to be transferred includes the foreground image and the background image, acquiring the first style selection information and the second style selection information;
  • the first style selection information is used to perform style transfer processing on the foreground image
  • the second style selection information is used to perform style transfer processing on the background image
  • FIG. 1 shows a flowchart of an image processing method provided by an embodiment of the present application.
  • the method can include the following steps.
  • Step 101 Perform segmentation processing on the original image to obtain a foreground image and a background image.
  • the original image can be any image.
  • the foreground image refers to the image corresponding to the person or object close to the front in the original image
  • the background image refers to the image corresponding to the scene that sets off the subject in the original image.
  • Step 102 Perform style transfer processing on the image to be transferred based on the style selection information to obtain the transferred image.
  • the image to be migrated refers to an image that needs to undergo style transfer processing.
  • the image to be migrated includes at least one of the following: a foreground image and a background image. That is, the image to be migrated may include a foreground image, or The image may include a background image, or the image to be transferred may include a foreground image and a background image.
  • the computer device When the image to be transferred includes a foreground image, the computer device performs style transfer processing on the foreground image based on the style selection information to obtain a foreground style transfer image; when the image to be transferred includes a background image, the computer device performs style transfer on the background image based on the style selection information Processing to obtain a background style transfer image; when the image to be transferred includes a foreground image and a background image, the computer device performs style transfer processing on the foreground image and the background image to obtain the foreground style transfer image and the background style transfer image.
  • the style selection information includes n transfer styles, where n is a positive integer.
  • the style selection information may be a style selection vector, where the style selection vector includes n-dimensional elements, and each element corresponds to a migration style. When the value of the element is 0, it indicates that the migration style corresponding to the element is not It is regarded as the transfer style corresponding to the image to be transferred, and the transfer style corresponding to the image to be transferred refers to the transfer style used when the image to be transferred is subjected to style transfer processing.
  • the style selection vector can be expressed as [0,0,1], and [0,0,1] means Use the third transfer style to perform style transfer processing on the image to be transferred; the style selection vector can also be expressed as [0,1,0], [0,1,0] means that the second transfer style is used to perform style transfer processing on the image to be transferred; style The selection vector can also be expressed as [1,0,0], and [1,0,0] indicates that the first transfer style is used to perform style transfer processing on the image to be transferred.
  • Step 103 Generate an output image based on the migration image.
  • the output image refers to an image whose content is similar to the original image and whose style is similar to the transferred image.
  • the computer device When the image to be transferred includes a foreground image, the computer device generates an output image based on the foreground style transfer image and the original image; when the image to be transferred includes a background image, the computer device generates an output image based on the background style transfer image and the original image;
  • the transfer image includes a foreground image and a background image
  • the computer device generates an output image based on the foreground style transfer image and the background style transfer image.
  • the transferred image is obtained, and the output image is generated based on the transferred image.
  • the embodiment of the present application The foreground image or the background image or the foreground image and the background image can be individually subjected to style transfer processing. Compared with the superimposition of a layer of filter on the upper layer of the original image in the related technology, the effect diversity of the output image is improved.
  • the computer device performs style transfer processing on the image to be transferred based on the style selection information through the style transfer model to obtain the transferred image.
  • the style transfer model refers to a model for performing style transfer processing on the image to be transferred.
  • the style transfer model may perform style transfer processing through deep learning.
  • the style transfer model can realize single style transfer processing, or multi-style transfer processing.
  • the style transfer model can be run on a GPU (Graphics Processing Unit, graphics processor), and running the style transfer model on the GPU can improve the calculation speed of the style transfer model.
  • the foregoing steps may include the following sub-steps:
  • the user can select m transfer styles from n transfer styles as the transfer style corresponding to the image to be transferred.
  • the user can select any one or any two or three migration styles from the three migration styles as the migration style corresponding to the image to be migrated.
  • the style transfer process is performed on the image to be transferred based on the m transfer styles through the style transfer model to obtain the transferred image.
  • the migration image can be obtained in the following manner:
  • the product of the weight of the i-th migration style and the style parameter of the i-th migration style is determined as the target style parameter of the i-th migration style, and i is less than Or a positive integer equal to m;
  • the weight of the i-th transfer style is used to indicate the proportion of the i-th transfer style in the m transfer styles. Exemplarily, the greater the weight of the i-th transfer style, the more obvious the style of the i-th transfer style in the transferred image.
  • the weight of the i-th migration style may be set by the user, or may be preset by the computer device.
  • the style parameters of the migration style refer to the parameters that characterize the migration style.
  • the style parameters can include the mean value of the migration style and the standard deviation of the migration style. In other possible implementations, the style parameters can also include other parameters. For example, the style parameters can include The variance of the migration style.
  • the style selection information is [0,0,1] there are three transfer styles: the first transfer style, the second transfer style, and the third transfer style.
  • the weight of the first transfer style is 0, and the weight of the second transfer style. If it is 0, the weight of the third transfer style is 1, that is, the user selects the third transfer style as the transfer style corresponding to the image to be transferred.
  • the style selection information is [0.5,0.5,0], that is, the weight of the first transfer style is 0.5, the weight of the second transfer style is 0.5, and the weight of the third transfer style is 0, that is, the user selects the first transfer style
  • the second transfer style is used as the transfer style corresponding to the image to be transferred.
  • the mean value of the first transfer style is 0.2 and the standard deviation is 0.4; the mean value of the second transfer style is 0.3 and the standard deviation is 0.6.
  • the target style parameters include the target mean value and the target standard deviation
  • the sum of the target mean values of the various transfer styles in the m transfer styles is determined as the transfer mean value corresponding to the image to be transferred; the various transfer styles in the m transfer styles
  • the sum of the target standard deviation of the style is determined as the migration standard deviation corresponding to the image to be migrated.
  • the migration mean value corresponding to the image to be migrated is 0.5
  • the migration standard deviation is 0.7.
  • the image to be transferred includes a foreground image and a background image
  • the foreground image corresponds to the first style selection information
  • the background image corresponds to the second style selection information.
  • the calculation method of the transfer style parameter corresponding to the foreground image and the transfer style parameter corresponding to the background image is similar to the foregoing calculation method, and will not be repeated here.
  • the style transfer model is used to perform style transfer processing on the image to be transferred based on the style selection information.
  • the style transfer model can realize single style transfer processing or multiple style transfer processing, which improves The diversity of migration images is improved.
  • the sum of the target style parameters of the various transfer styles in the m transfer styles is determined as the transfer style parameter corresponding to the image to be transferred, so that multi-style transfer can be realized.
  • the transfer image can be generated based on the style selection information and the image to be transferred through the style transfer model, which is easy to operate.
  • the image to be migrated includes a foreground image and a background image.
  • the style transfer model needs to be determined first.
  • the style transfer model can be determined in the following ways:
  • a model that supports the use of the first style selection information and the second style selection information for style transfer processing is selected from the style transfer model set as the style transfer model.
  • the set of style transfer models includes at least one model, the first style selection information represents the transfer style corresponding to the foreground image, and the second style selection information represents the transfer style corresponding to the background image.
  • the first style selection information and the second style selection information may be the same or different, that is, the migration styles corresponding to the foreground image and the background image may be the same or different.
  • the number of transfer styles included in the first style selection information and the second style selection information is the same, and both are n.
  • the computer device By selecting a style transfer model that supports the first style selection information and the second style selection information to perform the style transfer processing on the image to be transferred, the computer device only needs to call one style transfer model to realize the style transfer processing of the foreground image and the background image, and reduce This reduces the storage pressure of computer equipment.
  • a model that supports the first style selection information for style transfer processing is selected from the style transfer model set as the first style transfer model, and the second style selection information is selected from the style transfer model set to support the use of the second style selection information for style transfer
  • the processed model serves as the second style transfer model.
  • the style transfer model set includes at least one model
  • the first style selection information represents the transfer style corresponding to the foreground image
  • the first style transfer model is used to perform style transfer processing on the foreground image.
  • the second style selection information represents the transfer style corresponding to the background image, and the second style transfer model is used to perform style transfer processing on the background image.
  • the calculation time of the first style transfer model is the shortest in the style transfer model set, or the calculation accuracy of the first style transfer model is the highest in the style transfer model set.
  • the calculation time of the second style transfer model is the shortest in the style transfer model set, or the calculation accuracy of the second style transfer model is the highest in the style transfer model set.
  • the calculation time represents the calculation efficiency
  • the model with the shortest calculation time is selected as the style transfer model to ensure the calculation efficiency of the style transfer model.
  • the calculation accuracy can be determined based on the style matching degree between the transferred image output by the style transfer model and the style image.
  • the higher the style matching degree the higher the calculation accuracy; conversely, the lower the style matching degree, the higher the calculation accuracy. low.
  • the model with the highest calculation accuracy is selected as the style transfer model, which can be applied to business scenarios that require higher calculation accuracy.
  • the selection of the first style transfer model and the second style transfer model can be determined based on different requirements of the foreground image and the background image.
  • the model that supports the first style information and the highest calculation accuracy can be selected from the collection of style transfer models as the first style transfer model; the background image requires higher calculation time, you can The model that supports the second style information and has the shortest computing time is selected from the set of style transfer models as the second style transfer model.
  • the model with the shortest calculation time as the style transfer model by selecting the model with the shortest calculation time as the style transfer model, the calculation efficiency of the style transfer model is ensured; by selecting the model with the highest calculation accuracy as the style transfer model, it can be applied It is used in business scenarios that require high calculation accuracy.
  • the style transfer model is determined by the above two different basis, and the selection of the style transfer model is more flexible.
  • the style transfer model needs to be trained first, and the training process is as follows:
  • the training data includes at least one training sample
  • the training sample includes a training image and a style image.
  • the style image refers to an image used as a reference style during style transfer processing.
  • the training image can be any image, for example, The training image may be a foreground image or a background image, which is not limited in the embodiment of the present application.
  • the training transfer image refers to the image obtained by performing style transfer processing on the training image.
  • the style transfer model performs style transfer processing on the training image based on a single style, that is, the style selection information is a one-hot type, such as [0,0, 1].
  • the content feature of the training image is used to characterize the image content contained in the training image
  • the content feature of the training transfer image is used to characterize the image content contained in the training transfer image
  • the style feature of the training transfer image is used to characterize the style of the training transfer image.
  • the style feature is used to characterize the style of the style image.
  • the computer device determines the value of the content loss function based on the content feature of the training image and the content feature of the training migration image; determines the value of the style loss function based on the style feature of the training migration image and the style feature of the style image; based on the content The value of the loss function and the value of the style loss function determine the value of the loss function.
  • the computer device can use VGG (Visual Geometry Group) -19 to extract the content features of the training image and the training migration image, for example, use the output feature of the relu4_1 (stimulus 4_1) layer in VGG-19 as the training The content characteristics of the image and the training transfer image.
  • VGG Visual Geometry Group
  • l represents the number of layers
  • the computer device can use VGG-19 to extract the style features of the training transfer image and the style image. For example, select the relu1_1 (incentive 1_1) layer, relu2_1 (incentive 2_1) layer, and relu3_1 (incentive 3_1) layer in VGG-19. , The output features of the relu4_1 (stimulus 4_1) layer are used as the style features of the training transfer image and the style image.
  • C j represents the number of channels of the j-th layer
  • H j represents the length of the j-th layer
  • W j represents the width of the j-th layer
  • ⁇ j (x) h, w, c represent the characteristic values of the j-th layer of VGG-19
  • ⁇ j (x) h, w, c′ represents the transpose of ⁇ j (x) h, w, c.
  • the sum of the value of the content loss function and the value of the style loss function is determined as the value of the loss function.
  • the calculation formula of the loss function L( ⁇ ) is as follows:
  • ⁇ represents the weight of the content loss function in the loss function
  • ⁇ represents the weight of the style loss function in the loss function
  • the style transfer model is trained based on the value of the loss function, and the trained style transfer model is obtained.
  • the training of the style transfer model is stopped, and the trained style transfer model is obtained.
  • the computer device trains the style parameters included in the style transfer model based on the value of the loss function.
  • the style transfer model is trained based on the content features of the training images, the content features of the training transfer images, the style features of the training transfer images, and the style features of the style images. This makes the style transfer model obtained by the final training more accurate.
  • FIG. 2 shows a schematic structural diagram of a style transfer model provided by an embodiment of the present application.
  • the image to be transferred will pass through the first convolutional layer 21, the second convolutional layer 22, and the third convolutional layer 23 in the style transfer model, and then pass through the first residual layer 24, the second residual layer 25, and the third residual layer.
  • the difference layer 26 then passes through the first up-sampling layer 27, the second up-sampling layer 28, and finally the output layer 29.
  • the build-up layer 23 includes a mirror filling layer 31, a deep convolution layer 32, a point-wise convolution layer 33, an instance normalization layer 34, and a nonlinear activation layer 35.
  • the number of convolution kernels of the deep convolution layer 32 in the first convolution layer 21 is 32, the size of the convolution kernel is 9 ⁇ 9, and the step size is 1; the depth convolution layer 32 in the second convolution layer 22
  • the number of convolution kernels is 64, the size of the convolution kernel is 3x3, and the step size is 2.
  • the number of convolution kernels of the deep convolution layer 32 in the third convolution layer 23 is 128, the size of the convolution kernel is 3x3, and the step size Is 2.
  • the depth separable convolution layer is used to perform a depth convolution operation on the image to be migrated to obtain a first output feature; perform a point-by-point convolution operation on the first output feature to obtain a second output feature.
  • the depth separable convolutional layer includes a deep convolutional layer 32 and a pointwise convolutional layer 33.
  • Depth separable convolution is an operation method that decomposes standard convolution into deep convolution and a 1x1 convolution, that is, point-by-point convolution. Depth separable convolution can significantly reduce the parameters and computational complexity of the style transfer model.
  • the above-mentioned deep convolution operation on the image to be migrated can obtain the first output feature which can be run on the deep convolution layer 32.
  • the above-mentioned convolution operation on the first output feature point-by-point, and the second output feature can be run on the point-wise convolution layer. Run on 33.
  • the image to be migrated can be Reflect Padding on the mirror filling layer 31, and the padding is [[0,0], [padding, padding], [padding, padding] , [0,0]], the size of the padding is the size of the convolution kernel divided by 2 and rounded to obtain the filled image to be migrated, and then input the filled image to be migrated into the depth convolutional layer 32.
  • the instance is grouped into one layer 34, which is used to normalize the second output feature based on the transfer style parameter to obtain the third output feature.
  • the transfer style parameter refers to the parameter of the transfer style corresponding to the image to be transferred.
  • the second output feature can be normalized by the following instance normalization (Instance Normalization, IN) feature formula:
  • c represents the feature map obtained after convolution of the image to be transferred
  • c mean represents the mean value of c
  • c std represents the standard deviation of c
  • v mean represents the mean value of the feature map of the style image
  • v std represents the feature map of the style image Standard deviation.
  • the channel numbers of v mean , v std , c mean , and c std are the same, v mean is initialized to a vector of all 0s, and v std is initialized to a vector of all 1s.
  • the training of the style transfer model is the training of v mean and v std .
  • the non-linear activation layer 35 is used to perform non-linear operations on the third output feature to obtain the fourth output feature.
  • the nonlinear operation can be performed through the nonlinear activation function RELU.
  • the respective structural schematic diagrams of the first residual layer 24, the second residual layer 25, and the third residual layer 26 are shown in FIG. 4.
  • the residual layer includes two convolutional layers 41. After the input passes through the two convolutional layers 41, The result of is added to the input to get the output. This structure is conducive to the stability and convergence of the style transfer model.
  • the number of convolution kernels for deep convolution in the convolution layer 41 included in the first residual layer 24, the second residual layer 25, and the third residual layer 26 are all 128, the size of the convolution kernel is all 3x3, and the step size is Both are 1.
  • the first upsampling layer 27 and the second upsampling layer 28 include a convolutional layer and a nearest neighbor upsampling layer.
  • the nearest neighbor upsampling layer is used to perform an interpolation operation on the fourth output feature to obtain the fifth output feature, and the resolution of the fifth output feature is greater than the resolution of the fourth output feature. After the nearest neighbor upsampling layer expands the resolution of the fourth output feature by 2 times, the fifth output feature is obtained.
  • the embodiment of the present application uses the nearest neighbor upsampling , Can effectively avoid the checkerboard effect.
  • the number of convolution kernels for deep convolution in the first upsampling layer 27 is 64, the size of the convolution kernel is 3x3, and the step size is 2; the number of convolution kernels for deep convolution in the second upsampling layer 28 is 32, The size of the convolution kernel is 3x3 and the step size is 2.
  • the output layer 29 is used for outputting a migration image after performing a convolution operation on the fifth output feature.
  • the convolution kernel of the depth convolution in the convolution layer included in the output layer 29 is proficiently 3, the size of the convolution kernel is 9 ⁇ 9, the step size is 1, and the activation function is tanh.
  • the style transfer model adopts deep separable convolution, which reduces the computing time of the style transfer model; the style transfer model adopts nearest neighbor up-sampling, which effectively avoids the checkerboard effect.
  • the image to be migrated includes a foreground image and a background image.
  • the computer device may add the pixel values of the pixels in the same position of the foreground style transfer image and the background style transfer image to generate an output image.
  • the foreground style transfer image refers to an image obtained after performing style transfer processing on a foreground image
  • a background style transfer image refers to an image obtained after performing style transfer processing on a background image
  • the foreground image and the background image can be obtained in the following manner:
  • the original image is segmented through the image segmentation model to obtain the foreground gray matrix and the background gray matrix; the original image is respectively multiplied with the foreground gray matrix and the background gray matrix to obtain the foreground image and the background image.
  • the computer device can segment the original image through the image segmentation model to obtain the foreground grayscale matrix.
  • the area where element 1 in the foreground grayscale matrix is located represents the foreground image; the computer device compares the original image with the foreground grayscale matrix. Multiply to obtain the foreground image; then the computer equipment reverses the foreground grayscale matrix to obtain the background grayscale matrix.
  • the area where element 1 in the background grayscale matrix is located represents the background image, and the computer equipment multiplies the original image with the background grayscale matrix To get the background image.
  • the computer device can segment the original image through the image segmentation model to obtain the background grayscale matrix.
  • the area where element 1 in the background grayscale matrix is located represents the background image, and the computer device converts the original image Multiply the background gray matrix to obtain the background image; then the computer device reverses the background gray matrix to obtain the foreground gray matrix.
  • the area where element 1 in the foreground gray matrix is located represents the foreground image, and the computer device compares the original image with The foreground grayscale matrix is multiplied to obtain the foreground image.
  • the computer device can obtain the original image, read it in in the form of RGB (Red Green Blue, red, green, and blue) to obtain the image matrix I h, w, c , h and w are the height and width of the original image, c represents the number of channels, c is 3, the original image is represented as an RGB three-channel image.
  • RGB Red Green Blue, red, green, and blue
  • the values in O h, w include 0 and 1, and the position of 1 is the portrait ,
  • the portrait P h, w, c can be calculated by the following formula: Represents the multiplication of the corresponding position elements of the two matrices. Invert the portrait grayscale matrix, that is, the value of 1 becomes 0, and the value of 0 becomes 1, to obtain the background grayscale matrix O′ h,w , and the background image B h,w,c can be calculated by the following formula:
  • the embodiment of the present application only uses the area where element 1 in the background gray matrix or the foreground gray matrix is located to represent the background image or the foreground image.
  • the background gray matrix or the foreground The area where the element 0 in the gray matrix is located represents the background image or the foreground image. The embodiment of the application does not limit this.
  • the image processing method provided in the embodiment of the present application further includes the following steps:
  • the image undergoes style transfer processing.
  • the computer equipment obtains the original image; obtains the portrait and background image based on the original image through the portrait segmentation model; determines whether the portrait undergoes style transfer processing, if the portrait undergoes style transfer processing, obtains the first style selection information, and then selects the information based on the first style , Get the portrait style transfer image; if the portrait does not undergo style transfer processing, it is still the original portrait; determine whether the background image undergoes style transfer processing, if the background image undergoes style transfer processing, obtain the second style selection information, and then based on the second Style selection information to obtain the background style transfer image; if the background image is not subjected to style transfer processing, it is still the original background image, and finally based on the above-mentioned portrait style transfer image or original portrait, and background style transfer image or original background image, the output image is obtained .
  • the portrait segmentation model needs to have a higher accuracy for the segmentation effect to be good. Otherwise, the edge segmentation of the portrait will be unclear.
  • the portrait segmentation is combined with the style transfer. Style transfer has strong style characteristics, which to a certain extent conceals the problem of unclear segmentation of portrait contour edges, so that the portrait segmentation model can be used normally without high accuracy, eliminating the need for optimization and maintenance of the portrait segmentation model.
  • FIG. 7 shows a block diagram of an image processing apparatus provided by an embodiment of the present application.
  • the apparatus has functions for implementing the above-mentioned method examples. The functions may be implemented by hardware or by hardware executing corresponding software.
  • the apparatus 700 may include: an image segmentation module 710, an image processing module 720, and an image generation module 730.
  • the image segmentation module 710 is configured to perform segmentation processing on the original image to obtain a foreground image and a background image;
  • the image processing module 720 is configured to perform style transfer processing on the image to be transferred based on the style selection information to obtain the transferred image; wherein the image to be transferred includes at least one of the following: the foreground image, the background image, and the
  • the style selection information includes n types of migration styles, where n is a positive integer;
  • the image generation module 730 is configured to generate an output image based on the migration image.
  • the transferred image is obtained, and the output image is generated based on the transferred image.
  • the embodiment of the present application The foreground image or the background image or the foreground image and the background image can be individually subjected to style transfer processing. Compared with the superimposition of a layer of filter on the upper layer of the original image in the related technology, the effect diversity of the output image is improved.
  • the image processing module 720 is configured to:
  • the image processing module 720 includes: a style selection unit 721 and an image processing unit 722.
  • the style selection unit 721 is configured to select m transfer styles from the n transfer styles, and the m is a positive integer less than or equal to the n.
  • the image processing unit 722 is configured to perform style transfer processing on the image to be transferred based on the m transfer styles through the style transfer model to obtain the transferred image.
  • the image processing unit 722 is configured to:
  • the product of the weight of the i-th transfer style and the style parameter of the i-th transfer style is determined as the target of the i-th transfer style Style parameter, said i is a positive integer less than or equal to said m;
  • the image to be migrated includes the foreground image and the background image;
  • the device 700 further includes: a model selection module 740;
  • the model selection module 740 is configured to select a model that supports the first style selection information and the second style selection information for style transfer processing from the style transfer model set as the style transfer model, and the style transfer model set includes At least one model, the first style selection information represents a migration style corresponding to the foreground image, and the second style selection information represents a migration style corresponding to the background image.
  • the image to be migrated includes the foreground image and the background image;
  • the device 700 further includes: a model selection module 740.
  • Model selection module 740 for:
  • a model that supports the use of first style selection information for style transfer processing is selected from a set of style transfer models as a first style transfer model, the set of style transfer models includes at least one model, and the first style selection information represents the foreground A transfer style corresponding to the image, where the first style transfer model is used to perform style transfer processing on the foreground image;
  • a model that supports the use of second style selection information for style transfer processing is selected from the set of style transfer models as the second style transfer model, where the second style selection information represents the transfer style corresponding to the background image, and the second style transfer model
  • the style transfer model is used to perform style transfer processing on the background image.
  • the device 700 further includes: a data acquisition module 750, a function determination module 760, and a model training module 770.
  • the data acquisition module 750 is configured to acquire training data of the style transfer model, the training data includes at least one training sample, the training sample includes a training image and a style image, and the style image refers to a style transfer model.
  • the image processing module 720 is further configured to perform style transfer processing on the training image through the style transfer model to obtain a training transfer image;
  • the function determining module 760 is configured to determine the value of the loss function based on the content feature of the training image, the content feature of the training migration image, the style feature of the training migration image, and the style feature of the style image;
  • the model training module 770 is configured to train the style transfer model based on the value of the loss function to obtain the trained style transfer model.
  • the function determining module 760 is configured to:
  • the value of the loss function is determined based on the value of the content loss function and the value of the style loss function.
  • the style transfer model includes a depth separable convolutional layer, an instance grouping layer, a nonlinear activation layer, a nearest neighbor upsampling layer, and an output layer;
  • the depth separable convolution layer is used to perform a depth convolution operation on the image to be transferred to obtain a first output feature; perform a point-by-point convolution operation on the first output feature to obtain a second output feature;
  • the examples are grouped into one layer for normalizing the second output feature based on the transfer style parameter to obtain a third output feature, where the transfer style parameter refers to the transfer style parameter corresponding to the image to be transferred ;
  • the non-linear activation layer is used to perform a non-linear operation on the third output feature to obtain a fourth output feature
  • the nearest neighbor upsampling layer is used to perform an interpolation operation on the fourth output feature to obtain a fifth output feature, the resolution of the fifth output feature is greater than the resolution of the fourth output feature;
  • the output layer is configured to output the migration image after performing a convolution operation on the fifth output feature.
  • the image to be migrated includes the foreground image and the background image;
  • the image generation module 730 is used to:
  • the foreground style transfer image refers to an image obtained after performing style transfer processing on the foreground image
  • the background style transfer image refers to an image obtained after performing style transfer processing on the background image
  • the image segmentation module 710 is configured to:
  • the original image is respectively multiplied by the foreground grayscale matrix and the background grayscale matrix to obtain the foreground image and the background image.
  • the device 700 further includes: an instruction acquisition module 780 and an information acquisition module 790.
  • the instruction acquisition module 780 is used to acquire an image confirmation instruction
  • the information acquisition module 790 is configured to acquire first style selection information if the image confirmation instruction indicates that the image to be transferred includes the foreground image;
  • the information obtaining module 790 is further configured to obtain second style selection information if the image confirmation instruction indicates that the image to be transferred includes the background image;
  • the information obtaining module 790 is further configured to obtain the first style selection information and the second style selection information if the image confirmation instruction indicates that the image to be transferred includes the foreground image and the background image ;
  • the first style selection information is used to perform style transfer processing on the foreground image
  • the second style selection information is used to perform style transfer processing on the background image
  • FIG. 9 shows a structural block diagram of a computer device provided by an embodiment of the present application.
  • the computer can be a terminal or a server.
  • the computer device in the embodiment of the present application may include one or more of the following components: a processor 910 and a memory 920.
  • the processor 910 may include one or more processing cores.
  • the processor 910 uses various interfaces and lines to connect various parts of the entire computer device, and executes the computer by running or executing instructions, programs, code sets, or instruction sets stored in the memory 920, and calling data stored in the memory 920.
  • the processor 910 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA).
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 910 may integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly deals with operating systems and applications, etc.; the modem is used to deal with wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 910,
  • processor 910 executes the program instructions in the memory 920, the methods provided in the foregoing method embodiments are implemented.
  • the memory 920 may include random access memory (RAM) or read-only memory (ROM).
  • the memory 920 includes a non-transitory computer-readable storage medium.
  • the memory 920 may be used to store instructions, programs, codes, code sets or instruction sets.
  • the memory 920 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing the operating system, instructions for at least one function, instructions for implementing the foregoing method embodiments, etc.; storage data area It can store data created based on the use of computer equipment, etc.
  • the structure of the above-mentioned computer device is only illustrative.
  • the computer device may include more or fewer components, such as a display screen, etc., which is not limited in this embodiment.
  • FIG. 9 does not constitute a limitation on the computer device 900, and may include more or fewer components than shown, or combine certain components, or adopt different component arrangements.
  • a computer-readable storage medium is also provided, and a computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor of a computer device to implement the above-mentioned image processing method. Each step in the embodiment.
  • a computer program product is also provided, which is used to implement the above-mentioned image processing method when the computer program product is executed.
  • the “and/or” mentioned in this article describes the association relationship of the associated objects, and means that there can be three kinds of relationships, for example, A and/or B can mean that there is A alone, and there are both A and B, there are three cases of B alone.
  • the character "/" generally indicates that the associated objects before and after are in an "or” relationship.
  • the numbering of the steps described in this article only exemplarily shows a possible order of execution among the steps. In some other embodiments, the above steps may also be executed out of the order of numbers, such as two differently numbered ones. The steps are executed at the same time, or the two steps with different numbers are executed in the reverse order from the figure, which is not limited in the embodiment of the present application.

Abstract

本申请实施例提供了一种图像处理方法、装置、设备及存储介质，涉及图像处理技术领域。所述方法包括：对原始图像进行分割处理，得到前景图像和背景图像；基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像；其中，待迁移图像包括前景图像和/或背景图像，风格选择信息包括n种迁移风格，n为正整数；基于迁移图像，生成输出图像。本申请实施例提高了输出图像的效果多样性。

Description

图像处理方法、装置、设备及存储介质
本申请要求于2019年12月02日提交的申请号为201911212737.9、发明名称为“图像处理方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及图像处理技术领域，特别涉及一种图像处理方法、装置、设备及存储介质。
背景技术
随着技术的发展，用户可以对原始图像进行编辑处理，从而生成一个新的图像。
在相关技术中，用户可以对原始图像进行风格迁移处理，从而生成一个新的图像。风格迁移处理是指将原始图像作为内容图像，选择另一图像作为风格图像，利用风格迁移算法生成新的图像，该新的图像的内容与内容图像相似，风格与风格图像相似的处理。一般通过在原始图像的上层增加一层滤镜来对该原始图像进行风格迁移处理。
发明内容
本申请实施例提供一种图像处理方法、装置、设备及存储介质。所述技术方案如下：
一方面，本申请实施例提供一种图像处理方法，所述方法包括：
对原始图像进行分割处理，得到前景图像和背景图像；
基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像；其中，所述待迁移图像包括以下至少一项：所述前景图像、所述背景图像，所述风格选择信息包括n种迁移风格，所述n为正整数；
基于所述迁移图像，生成输出图像。
另一方面，本申请实施例提供一种图像处理装置，所述装置包括：
图像分割模块，用于对原始图像进行分割处理，得到前景图像和背景图像；
图像处理模块，用于基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像；其中，所述待迁移图像包括以下至少一项：所述前景图像、所述背景图像，所述风格选择信息包括n种迁移风格，所述n为正整数；
图像生成模块，用于基于所述迁移图像，生成输出图像。
另一方面，本申请实施例提供一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器存储有计算机程序，所述计算机程序由所述处理器加载并执行以实现如上述方面所述的图像处理方法。
又一方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序由处理器加载并执行以实现如上述方面所述的图像处理方法。
附图说明
图1是本申请一个实施例提供的图像处理方法的流程图；
图2是本申请一个实施例提供的风格迁移模型的结构示意图；
图3是本申请一个实施例提供的卷积层的结构示意图；
图4是本申请一个实施例提供的残差层的结构示意图；
图5是本申请一个实施例提供的图像分割方法的流程图；
图6是本申请另一个实施例提供的图像处理方法的流程图；
图7是本申请一个实施例提供的图像处理装置的框图；
图8是本申请另一个实施例提供的图像处理装置的框图；
图9是本申请一个实施例提供的计算机设备的结构框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
在本申请实施例中，各步骤的执行主体可以是计算机设备，计算机设备是指具备计算和处理能力的电子设备。计算机设备可以是终端，例如终端可以是手机、平板电脑、多媒体播放设备或其他便携式电子设备；计算机设备也可以是服务器，服务器可以是一台服务器或者是由多台服务器组成的服务器集群。
下面，对本申请的实施例进行介绍说明。
本申请实施例提供了一种图像处理方法，所述方法包括：
对原始图像进行分割处理，得到前景图像和背景图像；
基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像；其中，所述待迁移图像包括以下至少一项：所述前景图像、所述背景图像，所述风格选择信息包括n种迁移风格，所述n为正整数；
基于所述迁移图像，生成输出图像。
在示意性实施例中，所述基于所述风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像，包括：
通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
在示意性实施例中，所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像，包括：
从所述n种迁移风格中选取m种迁移风格，所述m为小于或等于所述n的正整数；
通过所述风格迁移模型基于所述m种迁移风格对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
在示意性实施例中，所述通过所述风格迁移模型基于所述m种迁移风格对所述待迁移图像进行风格迁移处理，得到所述迁移图像，包括：
对于所述m种迁移风格中的第i种迁移风格，将所述第i种迁移风格的权重与所述第i种迁移风格的风格参数的乘积，确定为所述第i种迁移风格的目标风格参数，所述i为小于或等于所述m的正整数；
将所述m种迁移风格中的各种迁移风格的目标风格参数之和，确定为所述待迁移图像对应的迁移风格参数；
通过所述风格迁移模型基于所述迁移风格参数对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
在示意性实施例中，所述待迁移图像包括所述前景图像和所述背景图像；
所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像之前，还包括：
从风格迁移模型集合中选取支持采用第一风格选择信息和第二风格选择信息进行风格迁移处理的模型作为所述风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征所述前景图像对应的迁移风格，所述第二风格选择信息表征所述背景图像对应的迁移风格。
在示意性实施例中，所述待迁移图像包括所述前景图像和所述背景图像；
所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像之前，还包括：
从风格迁移模型集合中选取支持采用第一风格选择信息进行风格迁移处理的模型作为第一风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征所述前景图像对应的迁移风格，所述第一风格迁移模型用于对所述前景图像进行风格迁移处理；
从所述风格迁移模型集合中选取支持采用第二风格选择信息进行风格迁移处理的模型作为第二风格迁移模型，所述第二风格选择信息表征所述背景图像对应的迁移风格，所述第二风格迁移模型用于对所述背景图像进行风格迁移处理。
在示意性实施例中，所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像之前，还包括：
获取所述风格迁移模型的训练数据，所述训练数据包括至少一个训练样本，所述训练样本包括训练图像和风格图像，所述风格图像是指在进行风格迁移处理时作为参考风格的图像；
通过所述风格迁移模型对所述训练图像进行风格迁移处理，得到训练迁移图像；
基于所述训练图像的内容特征、所述训练迁移图像的内容特征、所述训练迁移图像的风格特征、所述风格图像的风格特征，确定损失函数的值；
基于所述损失函数的值对所述风格迁移模型进行训练，得到训练完成的所述风格迁移模型。
在示意性实施例中，所述基于所述训练图像的内容特征、所述训练迁移图像的内容特征、所述训练迁移图像的风格特征、所述风格图像的风格特征，确定损失函数的值，包括：
基于所述训练图像的内容特征和所述训练迁移图像的内容特征，确定内容损失函数的值；
基于所述训练迁移图像的风格特征和所述风格图像的风格特征，确定风格损失函数的值；
基于所述内容损失函数的值和所述风格损失函数的值，确定所述损失函数的值。
在示意性实施例中，所述风格迁移模型中包括深度可分离卷积层、实例归一层、非线性激活层、最近邻上采样层和输出层；
所述深度可分离卷积层，用于对所述待迁移图像进行深度卷积运算，得到第一输出特征；对所述第一输出特征进行逐点卷积运算，得到第二输出特征；
所述实例归一层，用于基于迁移风格参数，对所述第二输出特征进行归一化，得到第三输出特征，所述迁移风格参数是指所述待迁移图像对应的迁移风格的参数；
所述非线性激活层，用于对所述第三输出特征进行非线性运算，得到第四输出特征；
所述最近邻上采样层，用于对所述第四输出特征进行插值运算，得到第五输出特征，所述第五输出特征的分辨率大于所述第四输出特征的分辨率；
所述输出层，用于对所述第五输出特征进行卷积运算后，输出所述迁移图像。
在示意性实施例中，所述待迁移图像包括所述前景图像和所述背景图像；
所述基于所述迁移图像，生成输出图像，包括：
将前景风格迁移图像和背景风格迁移图像在同一位置像素的像素值相加，生成所述输出图像；
其中，所述前景风格迁移图像是指对所述前景图像进行风格迁移处理后得到的图像，所述背景风格迁移图像是指对所述背景图像进行风格迁移处理后得到的图像。
在示意性实施例中，所述对原始图像进行分割处理，得到前景图像和背景图像，包括：
通过图像分割模型对所述原始图像进行分割处理，得到前景灰度矩阵和背景灰度矩阵；
将所述原始图像分别与所述前景灰度矩阵和所述背景灰度矩阵相乘，得到所述前景图像和所述背景图像。
在示意性实施例中，所述基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像之前，还包括：
获取图像确认指令；
若所述图像确认指令指示所述待迁移图像包括所述前景图像，则获取第一风格选择信息；
若所述图像确认指令指示所述待迁移图像包括所述背景图像，则获取第二风格选择信息；
若所述图像确认指令指示所述待迁移图像包括所述前景图像和所述背景图像，则获取所述第一风格选择信息和所述第二风格选择信息；
其中，所述第一风格选择信息用于对所述前景图像进行风格迁移处理，所述第二风格选择信息用于对所述背景图像进行风格迁移处理。
请参考图1，其示出了本申请一个实施例提供的图像处理方法的流程图。该方法可以包括如下几个步骤。
步骤101，对原始图像进行分割处理，得到前景图像和背景图像。
原始图像可以是任意一个图像，前景图像是指原始图像中靠近前沿的人或物对应的图像，背景图像是指原始图像中衬托主体事物的景象对应的图像。
步骤102，基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像。
在本申请实施例中，待迁移图像是指需要进行风格迁移处理的图像，待迁移图像包括以下至少一项：前景图像、背景图像，也即，待迁移图像可以包括前景图像，或者，待迁移图像可以包括背景图像，或者，待迁移图像可以包括前景图像和背景图像。当待迁移图像包括前景图像时，计算机设备基于风格选择信息对前景图像进行风格迁移处理，得到前景风格迁移图像；当待迁移图像包括背景图像时，计算机设备基于风格选择信息对背景图像进行风格迁移处理，得到背景风格迁移图像；当待迁移图像包括前景图像和背景图像时，计算机设备对前景图像和背景图像进行风格迁移处理，得到前景风格迁移图像和背景风格迁移图像。
风格选择信息包括n种迁移风格，n为正整数。示例性地，风格选择信息可以是风格选择向量，其中，风格选择向量包括n维元素，每一个元素对应一种迁移风格，当该元素的值为0时，表明该元素对应的迁移风格并未被作为待迁移图像对应的迁移风格，待迁移对应的迁移风格是指待迁移图像进行风格迁移处理时使用的迁移风格。例如，假设风格选择信息包括3种迁移风格：第一迁移风格、第二迁移风格和第三迁移风格，则风格选择向量可以表示成[0,0,1]，[0,0,1]表示用第三迁移风格对待迁移图像进行风格迁移处理；风格选择向量还可以表示成[0,1,0]，[0,1,0]表示用第二迁移风格对待迁移图像进行风格迁移处理；风格选择向量还可以表示成[1,0,0]，[1,0,0]表示用第一迁移风格对待迁移图像进行风格迁移处理。
步骤103，基于迁移图像，生成输出图像。
输出图像是指内容与原始图像相似，风格与迁移图像相似的图像。当待迁移图像包括前景图像时，计算机设备基于前景风格迁移图像和原始图像，生成输出图像；当待迁移图像包括背景图像时，计算机设备基于背景风格迁移图像和原始图像，生成输出图像；当待迁移图像包括前景图像和背景图像时，计算机设备基于前景风格迁移图像和背景风格迁移图像，生成输出图像。
综上所述，本申请实施例提供的技术方案中，通过基于风格选择信息对前景图像和/或背景图像进行风格迁移处理，得到迁移图像，并基于该迁移图像生成输出图像，本申请实施例可以单独对前景图像或背景图像或前景图像和背景图像进行风格迁移处理，相较于相关技术中的在原始图像的上层叠加一层滤镜，提高了输出图像的效果多样性。
在示意性实施例中，计算机设备通过风格迁移模型基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像。
风格迁移模型是指对待迁移图像进行风格迁移处理的模型。示例性地，风格迁移模型可以通过深度学习的方式进行风格迁移处理。风格迁移模型可以实现单风格迁移处理，也可以实现多风格迁移处理。风格迁移模型可以在GPU(Graphics Processing Unit，图形处理器)上运行，在GPU上运行风格迁移模型可以提高风格迁移模型的运算速度。
在示意性实施例中，上述步骤可以包括如下子步骤：
第一、从n种迁移风格中选取m种迁移风格，m为小于或等于n的正整数；
用户可以从n种迁移风格中选取m种迁移风格作为待迁移图像对应的迁移风格。例如，用户可以从3种迁移风格中选取任意1种或任意2种或3种迁移风格作为待迁移图像对应的迁移风格。
第二、通过风格迁移模型基于m种迁移风格对待迁移图像进行风格迁移处理，得到迁移图像。
示例性地，迁移图像可以通过如下方式得到：
1、对于m种迁移风格中的第i种迁移风格，将第i种迁移风格的权重与第i种迁移风格的风格参数的乘积，确定为第i种迁移风格的目标风格参数，i为小于或等于m的正整数；
第i种迁移风格的权重用于指示该第i种迁移风格在m种迁移风格中所占的比重。示例性地，第i种迁移风格的权重越大，表明该第i种迁移风格在迁移图像中的风格越明显。第i种迁移风格的权重可以由用户自行设置，也可以由计算机设备预先设置好。迁移风格的风格参数是指表征迁移风格的参数，风格参数可以包括迁移风格的均值和迁移风格的标准差，在其它可能的实现方式中，风格参数还可以包括其它参数，例如，风格参数可以包括迁移风格的方差。假设风格选择信息为[0,0,1]，即存在3种迁移风格：第一迁移风格、第二迁移风格和第三迁移风格，第一迁移风格的权重为0，第二迁移风格的权重为0，第三迁移风格的权重为1，也即，用户选取第三迁移风格作为待迁移图像对应的迁移风格。第三迁移风格的均值为0.5、标准差为0.7，第三迁移风格的目标均值为1*0.5＝0.5、目标标准差为1*0.7＝0.7。
假设风格选择信息为[0.5,0.5,0]，即第一迁移风格的权重为0.5，第二迁移风格的权重为0.5，第三迁移风格的权重为0，也即，用户选择第一迁移风格和第二迁移风格作为待迁移图像对应的迁移风格。假设第一迁移风格的均值为0.2，标准差为0.4；第二迁移风格的均值为0.3，标准差为0.6。此时，第一迁移风格的目标均值为0.5*0.2＝0.1、目标标准差为0.5*0.4＝0.2，第二迁移风格的目标均值为0.5*0.3＝0.15、目标标准差为0.5*0.6＝0.3。
2、将m种迁移风格中的各种迁移风格的目标风格参数之和，确定为待迁移图像对应的迁移风格参数；
当目标风格参数包括目标均值和目标标准差时，将m种迁移风格中的各种迁移风格的目标均值之和，确定为待迁移图像对应的迁移均值；将m种迁移风格中的各种迁移风格的目标标准差之和，确定为待迁移图像对应的迁移标准差。
仍然以上述示例为例，当风格选择信息为[0,0,1]时，待迁移图像对应的迁移均值为0.5、迁移标准差为0.7。
当风格选择信息为[0.5,0.5,0]时，待迁移图像对应的迁移均值为0.1+0.15＝0.25、迁移标准差为0.2+0.3＝0.5。
当待迁移图像包括前景图像和背景图像时，前景图像对应有第一风格选择信息，背景图像对应有第二风格选择信息。此时，前景图像对应的迁移风格参数和背景图像对应的迁移风格参数的计算方式与上述计算方式类似，此处不再赘述。
3、通过风格迁移模型基于迁移风格参数对待迁移图像进行风格迁移处理，得到迁移图像。
综上所述，本申请实施例提供的技术方案中，通过风格迁移模型基于风格选择信息对待迁移图像进行风格迁移处理，风格迁移模型可以实现单风格迁移处理，也可以实现多风格迁移处理，提高了迁移图像的多样性。
将m种迁移风格中的各种迁移风格的目标风格参数之和，确定为待迁移图像对应的迁移风格参数，可以实现多风格迁移。通过风格迁移模型基于风格选择信息和待迁移图像即可生成迁移图像，操作简便。
在示意性实施例中，待迁移图像包括前景图像和背景图像。在通过风格迁移模型对前景图像和背景图像进行风格迁移处理之前，需要先确定风格迁移模型，风格迁移模型可以通过如下方式确定：
在一个示例中，从风格迁移模型集合中选取支持采用第一风格选择信息和第二风格选择信息进行风格迁移处理的模型作为风格迁移模型。
在本申请实施例中，风格迁移模型集合中包括至少一个模型，第一风格选择信息表征前景图像对应的迁移风格，第二风格选择信息表征背景图像对应的迁移风格。第一风格选择信息和第二风格选择信息可以相同，也可以不同，即前景图像和背景图像对应的迁移风格可以相同，也可以不同。示例性地，第一风格选择信息和第二风格选择信息包括的迁移风格的数目相同，都为n。
通过选取支持第一风格选择信息和第二风格选择信息的风格迁移模型对待迁移图像进行风格迁移处理，计算机设备只需要调用一个风格迁移模型即可实现对前景图像和背景图像的风格迁移处理，减轻了计算机设备的存储压力。
在另一个示例中，从风格迁移模型集合中选取支持采用第一风格选择信息进行风格迁移处理的模型作为第一风格迁移模型，从风格迁移模型集合中选取支持采用第二风格选择信息进行风格迁移处理的模型作为第二风格迁移模型。
在本申请实施例中，风格迁移模型集合中包括至少一个模型，第一风格选择信息表征前景图像对应的迁移风格，第一风格迁移模型用于对前景图像进行风格迁移处理。第二风格选择信息表征背景图像对应的迁移风格，第二风格迁移模型用于对背景图像进行风格迁移处理。示例性地，第一风格迁移模型的运算时间在风格迁移模型集合中是最短的，或者，第一风格迁移模型的运算精度在风格迁移模型集合中是最高的。示例性地，第二风格迁移模型的运算时间在风格迁移模型集合中是最短的，或者，第二风格迁移模型的运算精度在风格迁移模型集合中是最高的。
示例性地，运算时间表征了运算效率，选取运算时间最短的模型作为风格迁移模型，可以保证风格迁移模型的运算效率。
示例性地，运算精度可以基于风格迁移模型输出的迁移图像与风格图像之间的风格匹配度确定，风格匹配度越高，表明运算精度越高；反之，风格匹配度越低，表明运算精度越低。选取运算精度最高的模型作为风格迁移模型，可以应用于对运算精度要求较高的业务场景中。第一风格迁移模型和第二风格迁移模型的选取可以基于前景图像和背景图像的不同需求确定。例如，前景图像对运算精度要求较高，则可以从风格迁移模型集合中选取支持第一风格信息，且运算精度最高的模型作为第一风格迁移模型；背景图像对运算时间要求较高，则可以从风格迁移模型集合中选取支持第二风格信息，且运算时间最短的模型作为第二风格迁移模型。
综上所述，本申请实施例提供的技术方案中，通过选取运算时间最短的模型作为风格迁移模型，保证了风格迁移模型的运算效率；通过选取运算精度最高的模型作为风格迁移模型，可以应用于对运算精度要求较高的业务场景中。通过上述两种不同依据确定风格迁移模型，风格迁移模型的选取更为灵活。
示例性地，在通过风格迁移模型对待迁移图像进行风格迁移处理之前，需要先对风格迁移模型进行训练，其训练过程如下：
第一、获取风格迁移模型的训练数据；
在本申请实施例中，训练数据包括至少一个训练样本，训练样本包括训练图像和风格图像，风格图像是指在进行风格迁移处理时作为参考风格的图像，训练图像可以是任意一个图像，例如，训练图像可以是前景图像，也可以是背景图像，本申请实施例对此不作限定。
第二、通过风格迁移模型对训练图像进行风格迁移处理，得到训练迁移图像；
训练迁移图像是指对训练图像进行风格迁移处理后得到的图像。示例性地，在对风格迁移模型进行训练时，风格迁移模型是基于单风格对训练图像进行风格迁移处理的，即，风格选择信息为one-hot(独热)型，例如[0,0,1]。
第三、基于训练图像的内容特征、训练迁移图像的内容特征、训练迁移图像的风格特征、 风格图像的风格特征，确定损失函数的值；
训练图像的内容特征用于表征训练图像包含的图像内容，训练迁移图像的内容特征用于表征训练迁移图像包含的图像内容，训练迁移图像的风格特征用于表征训练迁移图像的风格，风格图像的风格特征用于表征风格图像的风格。
可选地，计算机设备基于训练图像的内容特征和训练迁移图像的内容特征，确定内容损失函数的值；基于训练迁移图像的风格特征和风格图像的风格特征，确定风格损失函数的值；基于内容损失函数的值和风格损失函数的值，确定损失函数的值。
示例性地，计算机设备可以利用VGG(Visual Geometry Group，视觉几何组)-19提取训练图像和训练迁移图像的内容特征，例如，将VGG-19中的relu4_1(激励4_1)层的输出特征作为训练图像和训练迁移图像的内容特征。
内容损失函数
Figure PCTCN2020130090-appb-000001
可以通过如下公式计算：
Figure PCTCN2020130090-appb-000002
其中，
Figure PCTCN2020130090-appb-000003
表示训练迁移图像的内容特征，
Figure PCTCN2020130090-appb-000004
表示训练图像的内容特征，l表示层数，
Figure PCTCN2020130090-appb-000005
表示训练图像中的第l层第i行第j列的特征值，
Figure PCTCN2020130090-appb-000006
表示训练迁移图像中的第l层第i行第j列的特征值。
示例性地，计算机设备可以利用VGG-19提取训练迁移图像和风格图像的风格特征，例如，选取VGG-19中的relu1_1(激励1_1)层、relu2_1(激励2_1)层、relu3_1(激励3_1)层、relu4_1(激励4_1)层的输出特征作为训练迁移图像和风格图像的风格特征。
可选地，在计算风格损失函数之前，先计算特征感知矩阵，特征感知矩阵
Figure PCTCN2020130090-appb-000007
c′的计算公式如下：
Figure PCTCN2020130090-appb-000008
其中，C j表示第j层的通道数，H j表示第j层的长度，W j表示第j层的宽度，φ j(x) h，w，c表示VGG-19第j层的特征值，φ j(x) h，w，c′表示φ j(x) h，w，c的转置。
风格损失函数
Figure PCTCN2020130090-appb-000009
基于上述特征感知矩阵计算如下：
Figure PCTCN2020130090-appb-000010
其中，
Figure PCTCN2020130090-appb-000011
表示训练迁移图像的特征感知矩阵，
Figure PCTCN2020130090-appb-000012
表示风格图像的特征感知矩阵，F表示范数。
在可能的实现方式中，将内容损失函数的值和风格损失函数的值之和，确定为损失函数的值。损失函数L(θ)的计算公式如下：
Figure PCTCN2020130090-appb-000013
其中，α表示内容损失函数在损失函数中的权重，β表示风格损失函数在损失函数中的 权重，
Figure PCTCN2020130090-appb-000014
表示内容损失函数，
Figure PCTCN2020130090-appb-000015
表示风格损失函数，当α越大时，训练迁移图像的内容特征越明显；当β越大时，训练迁移图像的风格特征越浓厚。
第四、基于损失函数的值对风格迁移模型进行训练，得到训练完成的风格迁移模型。
当损失函数的值小于预设阈值时，停止对风格迁移模型的训练，得到训练完成的风格迁移模型。当然，在其他可能的实现方式中，当训练次数达到预设次数时，停止对风格迁移模型的训练，得到训练完成的风格迁移模型。示例性地，计算机设备基于损失函数的值对风格迁移模型中包括的风格参数进行训练。
综上所述，本申请实施例提供的技术方案中，通过基于训练图像的内容特征、训练迁移图像的内容特征、训练迁移图像的风格特征和风格图像的风格特征，对风格迁移模型进行训练，使得最终训练得到的风格迁移模型的精度更高。
如图2所示，其示出了本申请一个实施例提供的风格迁移模型的架构示意图。待迁移图像会经过风格迁移模型中的第一卷积层21、第二卷积层22、第三卷积层23，然后经过第一残差层24、第二残差层25、第三残差层26，再然后经过第一上采样层27、第二上采样层28，最后经过输出层29。
示例性地，第一卷积层21、第二卷积层22、第三卷积层23各自的结构如图3所示，第一卷积层21、第二卷积层22和第三卷积层23包括镜像填充层31、深度卷积层32、逐点卷积层33、实例归一层34和非线性激活层35。
示例性地，第一卷积层21中的深度卷积层32的卷积核数量为32、卷积核大小为9x9、步长为1；第二卷积层22中的深度卷积层32的卷积核数量为64、卷积核大小为3x3、步长为2；第三卷积层23中的深度卷积层32的卷积核数量为128、卷积核大小为3x3、步长为2。
深度可分离卷积层，用于对待迁移图像进行深度卷积运算，得到第一输出特征；对第一输出特征进行逐点卷积运算，得到第二输出特征。可选地，深度可分离卷积层包括深度卷积层32和逐点卷积层33。深度可分离卷积是一种将标准卷积分解成深度卷积以及1个1x1的卷积即逐点卷积的运算方式，深度可分离卷积可以显著减少风格迁移模型的参数及运算量。上述对待迁移图像进行深度卷积运算，得到第一输出特征可以在深度卷积层32上运行，上述对第一输出特征进行逐点卷积运行，得到第二输出特征可以在逐点卷积层33上运行。在对待迁移图像进行深度卷积运算之前，可以先在镜像填充层31上对待迁移图像进行镜像填充(Reflect Padding)，填充为[[0,0]，[padding，padding]，[padding，padding]，[0,0]]，padding的大小为卷积核大小除以2取整，得到填充完毕的待迁移图像，然后再将上述填充完毕的待迁移图像输入深度卷积层32。
实例归一层34，用于基于迁移风格参数，对第二输出特征进行归一化，得到第三输出特征，迁移风格参数是指待迁移图像对应的迁移风格的参数。在实例归一层34上基于迁移风格参数，对第二输出特征进行归一化可以通过如下实例归一化(Instance Normalization，IN)feature的公式实现：
Figure PCTCN2020130090-appb-000016
其中，c表示待迁移图像卷积后得到的特征图，c mean表示c的均值，c std表示c的标准差，v mean表示风格图像的特征图的均值，v std表示风格图像的特征图的标准差。v mean、v std、c mean、c std的通道数一致，v mean初始化为全0向量，v std初始化为全1向量。对风格迁移模型的训练即是对v mean和v std的训练。
非线性激活层35，用于对第三输出特征进行非线性运算，得到第四输出特征。示例性地，可以通过非线性激活函数RELU进行非线性运算。
第一残差层24、第二残差层25和第三残差层26各自的结构示意图如图4所示，残差层包括两层卷积层41，输入经过两层卷积层41后的结果与输入相加，得到输出，该结构有利于风格迁移模型的稳定和收敛。第一残差层24、第二残差层25和第三残差层26包括的卷积层41中的深度卷积的卷积核数量均为128、卷积核大小均为3x3、步长均为1。
第一上采样层27和第二上采样层28包括卷积层和最近邻上采样层。最近邻上采样层，用于对第四输出特征进行插值运算，得到第五输出特征，第五输出特征的分辨率大于第四输出特征的分辨率。最近邻上采样层将第四输出特征的分辨率扩大2倍后，得到第五输出特征，相较于相关技术中的通过反卷积的方式进行上采样，本申请实施例通过最近邻上采样，可以有效避免棋盘格效应。第一上采样层27中的深度卷积的卷积核数量为64、卷积核大小为3x3、步长为2；第二上采样层28中的深度卷积的卷积核数量为32、卷积核大小为3x3、步长为2。
输出层29，用于对第五输出特征进行卷积运算后，输出迁移图像。输出层29包括的卷积层中的深度卷积的卷积核熟练为3、卷积核大小为9x9、步长为1、激活函数为tanh。
综上所述，本申请实施例提供的技术方案中，风格迁移模型采用深度可分离卷积，降低了风格迁移模型的运算时间；风格迁移模型采用最近邻上采样，有效避免了棋盘格效应。
在示意性实施例中，待迁移图像包括前景图像和背景图像。此时，计算机设备可以将前景风格迁移图像和背景风格迁移图像在同一位置像素的像素值相加，生成输出图像。
在本申请实施例中，前景风格迁移图像是指对前景图像进行风格迁移处理后得到的图像，背景风格迁移图像是指对背景图像进行风格迁移处理后得到的图像。
示例性地，前景图像和背景图像可以通过如下方式获得：
通过图像分割模型对原始图像进行分割处理，得到前景灰度矩阵和背景灰度矩阵；将原始图像分别与前景灰度矩阵和背景灰度矩阵相乘，得到前景图像和背景图像。
可选地，计算机设备可以通过图像分割模型对原始图像进行分割处理，得到前景灰度矩阵，前景灰度矩阵里的元素1所在的区域表征前景图像；计算机设备将原始图像与前景灰度矩阵相乘，得到前景图像；然后计算机设备将前景灰度矩阵取反，得到背景灰度矩阵，背景灰度矩阵里的元素1所在的区域表征背景图像，计算机设备将原始图像与背景灰度矩阵相乘，得到背景图像。当然，在其他可能的实现方式中，计算机设备可以通过图像分割模型对原始图像进行分割处理，得到背景灰度矩阵，背景灰度矩阵里的元素1所在的区域表征背景图像，计算机设备将原始图像与背景灰度矩阵相乘，得到背景图像；然后计算机设备将背景灰度矩阵取反，得到前景灰度矩阵，前景灰度矩阵里的元素1所在的区域表征前景图像，计算机设备将原始图像与前景灰度矩阵相乘，得到前景图像。
示例性地，假设前景图像为人像，则如图5所示，计算机设备可以获取原始图像，以RGB(Red Green Blue，红绿蓝)的方式读入，得到图像矩阵I h，w，c，h和w分别为原始图像的高度和宽度，c代表通道数，c为3，将原始图像表示为RGB三通道图像。将上述图像矩阵输入人像分割模型中，得到人像灰度矩阵O h，w，h和w与上述原始图像的高度和宽度一致，O h， w中的值包括0和1，1的位置为人像，人像P h，w，c可以通过如下公式计算得到：
Figure PCTCN2020130090-appb-000017
表示两个矩阵相对应的位置元素相乘。将人像灰度矩阵取反，即1的值变为0,0的值变为1，得到背景灰度矩阵O′ h，w，背景图像B h，w，c可以通过如下公式计算得到：
Figure PCTCN2020130090-appb-000018
需要说明的是，本申请实施例仅以背景灰度矩阵或前景灰度矩阵中的元素1所在的区域表征背景图像或前景图像，在其他可能的实现方式中，可以以背景灰度矩阵或前景灰度矩阵中的元素0所在的区域表征背景图像或前景图像。本申请实施例对此不作限定。
示例性地，在基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像之前，本申请实施例提供的图像处理方法还包括如下步骤：
获取图像确认指令；若图像确认指令指示待迁移图像包括前景图像，则获取第一风格选择信息；若图像确认指令指示待迁移图像包括背景图像，则获取第二风格选择信息；若图像确认指令指示待迁移图像包括前景图像和背景图像，则获取第一风格选择信息和第二风格选择信息；其中，第一风格选择信息用于对前景图像进行风格迁移处理，第二风格选择信息用于对背景图像进行风格迁移处理。
如图6所示，以前景图像为人像为例进行介绍说明本申请实施例提供的图像处理方法的实现流程。计算机设备获取原始图像；通过人像分割模型基于原始图像，得到人像和背景图像；判断人像是否进行风格迁移处理，若人像进行风格迁移处理，则获取第一风格选择信息，然后基于第一风格选择信息，得到人像风格迁移图像；若人像不进行风格迁移处理，则仍然是原人像；判断背景图像是否进行风格迁移处理，若背景图像进行风格迁移处理，则获取第二风格选择信息，然后基于第二风格选择信息，得到背景风格迁移图像；若背景图像不进行风格迁移处理，则仍然是原背景图像，最后基于上述人像风格迁移图像或原人像，以及背景风格迁移图像或原背景图像，得到输出图像。
在相关技术中，人像分割模型需要有较高的精度，分割的效果才会好，否则会造成人像边缘分割不清楚，本申请实施例提供的技术方案中，将人像分割与风格迁移结合，由于风格迁移具有强烈的风格特征，在一定程度上掩盖了人像轮廓边缘分割不清楚的问题，使得人像分割模型不需要很高的精度也可以正常使用，省去了人像分割模型的优化维护工作。
下述为本申请装置实施例，可以用于执行本申请方法实施例。对于本申请装置实施例中未披露的细节，请参照本申请方法实施例。
请参考图7，其示出了本申请一个实施例提供的图像处理装置的框图，该装置具有实现上述方法示例的功能，所述功能可以由硬件实现，也可以由硬件执行相应的软件实现。该装置700可以包括：图像分割模块710、图像处理模块720和图像生成模块730。
所述图像分割模块710，用于对原始图像进行分割处理，得到前景图像和背景图像；
所述图像处理模块720，用于基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像；其中，所述待迁移图像包括以下至少一项：所述前景图像、所述背景图像，所述风格选择信息包括n种迁移风格，所述n为正整数；
所述图像生成模块730，用于基于所述迁移图像，生成输出图像。
综上所述，本申请实施例提供的技术方案中，通过基于风格选择信息对前景图像和/或背景图像进行风格迁移处理，得到迁移图像，并基于该迁移图像生成输出图像，本申请实施例可以单独对前景图像或背景图像或前景图像和背景图像进行风格迁移处理，相较于相关技术中的在原始图像的上层叠加一层滤镜，提高了输出图像的效果多样性。
在示意性实施例中，所述图像处理模块720，用于：
通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
在示意性实施例中，如图8所示，所述图像处理模块720，包括：风格选择单元721和图像处理单元722。
风格选择单元721，用于从所述n种迁移风格中选取m种迁移风格，所述m为小于或等于所述n的正整数。
图像处理单元722，用于通过所述风格迁移模型基于所述m种迁移风格对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
在示意性实施例中，所述图像处理单元722，用于：
对于所述m种迁移风格中的第i种迁移风格，将所述第i种迁移风格的权重与所述第i 种迁移风格的风格参数的乘积，确定为所述第i种迁移风格的目标风格参数，所述i为小于或等于所述m的正整数；
将所述m种迁移风格中的各种迁移风格的目标风格参数之和，确定为所述待迁移图像对应的迁移风格参数；
通过所述风格迁移模型基于所述迁移风格参数对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
在示意性实施例中，所述待迁移图像包括所述前景图像和所述背景图像；
所述装置700，还包括：模型选择模块740；
所述模型选择模块740，用于从风格迁移模型集合中选取支持采用第一风格选择信息和第二风格选择信息进行风格迁移处理的模型作为所述风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征所述前景图像对应的迁移风格，所述第二风格选择信息表征所述背景图像对应的迁移风格。
在示意性实施例中，所述待迁移图像包括所述前景图像和所述背景图像；
所述装置700，还包括：模型选择模块740。
模型选择模块740，用于：
从风格迁移模型集合中选取支持采用第一风格选择信息进行风格迁移处理的模型作为第一风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征所述前景图像对应的迁移风格，所述第一风格迁移模型用于对所述前景图像进行风格迁移处理；
从所述风格迁移模型集合中选取支持采用第二风格选择信息进行风格迁移处理的模型作为第二风格迁移模型，所述第二风格选择信息表征所述背景图像对应的迁移风格，所述第二风格迁移模型用于对所述背景图像进行风格迁移处理。
所述装置700，还包括：数据获取模块750、函数确定模块760和模型训练模块770。
所述数据获取模块750，用于获取所述风格迁移模型的训练数据，所述训练数据包括至少一个训练样本，所述训练样本包括训练图像和风格图像，所述风格图像是指在进行风格迁移处理时作为参考风格的图像；
所述图像处理模块720，还用于通过所述风格迁移模型对所述训练图像进行风格迁移处理，得到训练迁移图像；
所述函数确定模块760，用于基于所述训练图像的内容特征、所述训练迁移图像的内容特征、所述训练迁移图像的风格特征、所述风格图像的风格特征，确定损失函数的值；
所述模型训练模块770，用于基于所述损失函数的值对所述风格迁移模型进行训练，得到训练完成的所述风格迁移模型。
在示意性实施例中，所述函数确定模块760，用于：
基于所述训练图像的内容特征和所述训练迁移图像的内容特征，确定内容损失函数的值；
基于所述训练迁移图像的风格特征和所述风格图像的风格特征，确定风格损失函数的值；
基于所述内容损失函数的值和所述风格损失函数的值，确定所述损失函数的值。
在示意性实施例中，所述风格迁移模型中包括深度可分离卷积层、实例归一层、非线性激活层、最近邻上采样层和输出层；
所述深度可分离卷积层，用于对所述待迁移图像进行深度卷积运算，得到第一输出特征；对所述第一输出特征进行逐点卷积运算，得到第二输出特征；
所述实例归一层，用于基于迁移风格参数，对所述第二输出特征进行归一化，得到第三输出特征，所述迁移风格参数是指所述待迁移图像对应的迁移风格的参数；
所述非线性激活层，用于对所述第三输出特征进行非线性运算，得到第四输出特征；
所述最近邻上采样层，用于对所述第四输出特征进行插值运算，得到第五输出特征，所述第五输出特征的分辨率大于所述第四输出特征的分辨率；
所述输出层，用于对所述第五输出特征进行卷积运算后，输出所述迁移图像。
在示意性实施例中，所述待迁移图像包括所述前景图像和所述背景图像；
所述图像生成模块730，用于：
将前景风格迁移图像和背景风格迁移图像在同一位置像素的像素值相加，生成所述输出图像；
其中，所述前景风格迁移图像是指对所述前景图像进行风格迁移处理后得到的图像，所述背景风格迁移图像是指对所述背景图像进行风格迁移处理后得到的图像。
在示意性实施例中，所述图像分割模块710，用于：
通过图像分割模型对所述原始图像进行分割处理，得到前景灰度矩阵和背景灰度矩阵；
将所述原始图像分别与所述前景灰度矩阵和所述背景灰度矩阵相乘，得到所述前景图像和所述背景图像。
在示意性实施例中，所述装置700，还包括：指令获取模块780和信息获取模块790。
所述指令获取模块780，用于获取图像确认指令；
所述信息获取模块790，用于若所述图像确认指令指示所述待迁移图像包括所述前景图像，则获取第一风格选择信息；
所述信息获取模块790，还用于若所述图像确认指令指示所述待迁移图像包括所述背景图像，则获取第二风格选择信息；
所述信息获取模块790，还用于若所述图像确认指令指示所述待迁移图像包括所述前景图像和所述背景图像，则获取所述第一风格选择信息和所述第二风格选择信息；
其中，所述第一风格选择信息用于对所述前景图像进行风格迁移处理，所述第二风格选择信息用于对所述背景图像进行风格迁移处理。
需要说明的是，上述实施例提供的装置在实现其功能时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将设备的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的装置与方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
请参考图9，其示出了本申请一个实施例提供的计算机设备的结构框图。在可能的实现方式中，该计算机可以是终端，也可以是服务器。
本申请实施例中的计算机设备可以包括一个或多个如下部件：处理器910和存储器920。
处理器910可以包括一个或者多个处理核心。处理器910利用各种接口和线路连接整个计算机设备内的各个部分，通过运行或执行存储在存储器920内的指令、程序、代码集或指令集，以及调用存储在存储器920内的数据，执行计算机设备的各种功能和处理数据。可选地，处理器910可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器910可集成中央处理器(Central Processing Unit，CPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统和应用程序等；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器910中，单独通过一块芯片进行实现。
可选地，处理器910执行存储器920中的程序指令时实现上述各个方法实施例提供的方法。
存储器920可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory，ROM)。可选地，该存储器920包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器920可用于存储指令、程序、代码、代码集或指令集。存储器920可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于至少一个功能的指令、用于实现上述各个方法实施例的指令等； 存储数据区可存储根据计算机设备的使用所创建的数据等。
上述计算机设备的结构仅是示意性的，在实际实现时，计算机设备可以包括更多或更少的组件，比如：显示屏等，本实施例对此不作限定。
本领域技术人员可以理解，图9中示出的结构并不构成对计算机设备900的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
在示例性实施例中，还提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序由计算机设备的处理器加载并执行以实现上述图像处理方法实施例中的各个步骤。
在示例性实施例中，还提供了一种计算机程序产品，当该计算机程序产品被执行时，其用于实现上述图像处理方法。
应当理解的是，在本文中提及的“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。另外，本文中描述的步骤编号，仅示例性示出了步骤间的一种可能的执行先后顺序，在一些其它实施例中，上述步骤也可以不按照编号顺序来执行，如两个不同编号的步骤同时执行，或者两个不同编号的步骤按照与图示相反的顺序执行，本申请实施例对此不作限定。
以上所述仅为本申请的示例性实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    对原始图像进行分割处理，得到前景图像和背景图像；
    基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像；其中，所述待迁移图像包括以下至少一项：所述前景图像、所述背景图像，所述风格选择信息包括n种迁移风格，所述n为正整数；
    基于所述迁移图像，生成输出图像。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像，包括：
    通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
  3. 根据权利要求2所述的方法，其特征在于，所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像，包括：
    从所述n种迁移风格中选取m种迁移风格，所述m为小于或等于所述n的正整数；
    通过所述风格迁移模型基于所述m种迁移风格对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
  4. 根据权利要求3所述的方法，其特征在于，所述通过所述风格迁移模型基于所述m种迁移风格对所述待迁移图像进行风格迁移处理，得到所述迁移图像，包括：
    对于所述m种迁移风格中的第i种迁移风格，将所述第i种迁移风格的权重与所述第i种迁移风格的风格参数的乘积，确定为所述第i种迁移风格的目标风格参数，所述i为小于或等于所述m的正整数；
    将所述m种迁移风格中的各种迁移风格的目标风格参数之和，确定为所述待迁移图像对应的迁移风格参数；
    通过所述风格迁移模型基于所述迁移风格参数对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
  5. 根据权利要求2所述的方法，其特征在于，所述待迁移图像包括所述前景图像和所述背景图像；
    所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像之前，还包括：
    从风格迁移模型集合中选取支持采用第一风格选择信息和第二风格选择信息进行风格迁移处理的模型作为所述风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征所述前景图像对应的迁移风格，所述第二风格选择信息表征所述背景图像对应的迁移风格。
  6. 根据权利要求2所述的方法，其特征在于，所述待迁移图像包括所述前景图像和所述背景图像；
    所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像之前，还包括：
    从风格迁移模型集合中选取支持采用第一风格选择信息进行风格迁移处理的模型作为第一风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征 所述前景图像对应的迁移风格，所述第一风格迁移模型用于对所述前景图像进行风格迁移处理；
    从所述风格迁移模型集合中选取支持采用第二风格选择信息进行风格迁移处理的模型作为第二风格迁移模型，所述第二风格选择信息表征所述背景图像对应的迁移风格，所述第二风格迁移模型用于对所述背景图像进行风格迁移处理。
  7. 根据权利要求2所述的方法，其特征在于，所述通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像之前，还包括：
    获取所述风格迁移模型的训练数据，所述训练数据包括至少一个训练样本，所述训练样本包括训练图像和风格图像，所述风格图像是指在进行风格迁移处理时作为参考风格的图像；
    通过所述风格迁移模型对所述训练图像进行风格迁移处理，得到训练迁移图像；
    基于所述训练图像的内容特征、所述训练迁移图像的内容特征、所述训练迁移图像的风格特征、所述风格图像的风格特征，确定损失函数的值；
    基于所述损失函数的值对所述风格迁移模型进行训练，得到训练完成的所述风格迁移模型。
  8. 根据权利要求7所述的方法，其特征在于，所述基于所述训练图像的内容特征、所述训练迁移图像的内容特征、所述训练迁移图像的风格特征、所述风格图像的风格特征，确定损失函数的值，包括：
    基于所述训练图像的内容特征和所述训练迁移图像的内容特征，确定内容损失函数的值；
    基于所述训练迁移图像的风格特征和所述风格图像的风格特征，确定风格损失函数的值；
    基于所述内容损失函数的值和所述风格损失函数的值，确定所述损失函数的值。
  9. 根据权利要求2至8任一项所述的方法，其特征在于，所述风格迁移模型中包括深度可分离卷积层、实例归一层、非线性激活层、最近邻上采样层和输出层；
    所述深度可分离卷积层，用于对所述待迁移图像进行深度卷积运算，得到第一输出特征；对所述第一输出特征进行逐点卷积运算，得到第二输出特征；
    所述实例归一层，用于基于迁移风格参数，对所述第二输出特征进行归一化，得到第三输出特征，所述迁移风格参数是指所述待迁移图像对应的迁移风格的参数；
    所述非线性激活层，用于对所述第三输出特征进行非线性运算，得到第四输出特征；
    所述最近邻上采样层，用于对所述第四输出特征进行插值运算，得到第五输出特征，所述第五输出特征的分辨率大于所述第四输出特征的分辨率；
    所述输出层，用于对所述第五输出特征进行卷积运算后，输出所述迁移图像。
  10. 根据权利要求1所述的方法，其特征在于，所述待迁移图像包括所述前景图像和所述背景图像；
    所述基于所述迁移图像，生成输出图像，包括：
    将前景风格迁移图像和背景风格迁移图像在同一位置像素的像素值相加，生成所述输出图像；
    其中，所述前景风格迁移图像是指对所述前景图像进行风格迁移处理后得到的图像，所述背景风格迁移图像是指对所述背景图像进行风格迁移处理后得到的图像。
  11. 根据权利要求1所述的方法，其特征在于，所述对原始图像进行分割处理，得到前景图像和背景图像，包括：
    通过图像分割模型对所述原始图像进行分割处理，得到前景灰度矩阵和背景灰度矩阵；
    将所述原始图像分别与所述前景灰度矩阵和所述背景灰度矩阵相乘，得到所述前景图像 和所述背景图像。
  12. 根据权利要求1所述的方法，其特征在于，所述基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像之前，还包括：
    获取图像确认指令；
    若所述图像确认指令指示所述待迁移图像包括所述前景图像，则获取第一风格选择信息；
    若所述图像确认指令指示所述待迁移图像包括所述背景图像，则获取第二风格选择信息；
    若所述图像确认指令指示所述待迁移图像包括所述前景图像和所述背景图像，则获取所述第一风格选择信息和所述第二风格选择信息；
    其中，所述第一风格选择信息用于对所述前景图像进行风格迁移处理，所述第二风格选择信息用于对所述背景图像进行风格迁移处理。
  13. 一种图像处理装置，其特征在于，所述装置包括：
    图像分割模块，用于对原始图像进行分割处理，得到前景图像和背景图像；
    图像处理模块，用于基于风格选择信息对待迁移图像进行风格迁移处理，得到迁移图像；其中，所述待迁移图像包括以下至少一项：所述前景图像、所述背景图像，所述风格选择信息包括n种迁移风格，所述n为正整数；
    图像生成模块，用于基于所述迁移图像，生成输出图像。
  14. 根据权利要求13所述的装置，其特征在于，所述图像处理模块，用于：
    通过风格迁移模型基于所述风格选择信息对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
  15. 根据权利要求14所述的装置，其特征在于，所述图像处理模块，包括：
    风格选择单元，用于从所述n种迁移风格中选取m种迁移风格，所述m为小于或等于所述n的正整数；
    图像处理单元，用于通过所述风格迁移模型基于所述m种迁移风格对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
  16. 根据权利要求15所述的装置，其特征在于，所述图像处理单元，用于：
    对于所述m种迁移风格中的第i种迁移风格，将所述第i种迁移风格的权重与所述第i种迁移风格的风格参数的乘积，确定为所述第i种迁移风格的目标风格参数，所述i为小于或等于所述m的正整数；
    将所述m种迁移风格中的各种迁移风格的目标风格参数之和，确定为所述待迁移图像对应的迁移风格参数；
    通过所述风格迁移模型基于所述迁移风格参数对所述待迁移图像进行风格迁移处理，得到所述迁移图像。
  17. 根据权利要求14所述的装置，其特征在于，所述待迁移图像包括所述前景图像和所述背景图像；
    所述装置，还包括：模型选择模块，用于：
    从风格迁移模型集合中选取支持采用第一风格选择信息和第二风格选择信息进行风格迁移处理的模型作为所述风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征所述前景图像对应的迁移风格，所述第二风格选择信息表征所述背景图像对应的迁移风格。
  18. 根据权利要求14所述的装置，其特征在于，所述待迁移图像包括所述前景图像和所述背景图像；
    所述装置，还包括：模型选择模块，用于：
    从风格迁移模型集合中选取支持采用第一风格选择信息进行风格迁移处理的模型作为第一风格迁移模型，所述风格迁移模型集合中包括至少一个模型，所述第一风格选择信息表征所述前景图像对应的迁移风格，所述第一风格迁移模型用于对所述前景图像进行风格迁移处理；
    从所述风格迁移模型集合中选取支持采用第二风格选择信息进行风格迁移处理的模型作为第二风格迁移模型，所述第二风格选择信息表征所述背景图像对应的迁移风格，所述第二风格迁移模型用于对所述背景图像进行风格迁移处理。
  19. 一种计算机设备，其特征在于，所述计算机设备包括处理器和存储器，所述存储器存储有计算机程序，所述计算机程序由所述处理器加载并执行以实现如权利要求1至12任一项所述的图像处理方法。
  20. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机程序，所述计算机程序由处理器加载并执行以实现如权利要求1至12任一项所述的图像处理方法。
PCT/CN2020/130090 2019-12-02 2020-11-19 图像处理方法、装置、设备及存储介质 WO2021109876A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20896430.4A EP4064189A1 (en) 2019-12-02 2020-11-19 Image processing method, apparatus and device, and storage medium
US17/824,043 US20220284638A1 (en) 2019-12-02 2022-05-25 Method for image processing, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911212737.9 2019-12-02
CN201911212737.9A CN110956654B (zh) 2019-12-02 2019-12-02 图像处理方法、装置、设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/824,043 Continuation US20220284638A1 (en) 2019-12-02 2022-05-25 Method for image processing, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2021109876A1 true WO2021109876A1 (zh) 2021-06-10

Family

ID=69979258

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/130090 WO2021109876A1 (zh) 2019-12-02 2020-11-19 图像处理方法、装置、设备及存储介质

Country Status (4)

Country Link
US (1) US20220284638A1 (zh)
EP (1) EP4064189A1 (zh)
CN (1) CN110956654B (zh)
WO (1) WO2021109876A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114842342A (zh) * 2022-05-16 2022-08-02 网思科技股份有限公司 一种基于人工智能的杂乱场景检测方法、装置及相关设备
CN116452414A (zh) * 2023-06-14 2023-07-18 齐鲁工业大学(山东省科学院) 一种基于背景风格迁移的图像和谐化方法及系统

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3702958B1 (en) * 2019-02-26 2023-04-05 Identy Inc. Method for verifying the identity of a user by identifying an object within an image that has a biometric characteristic of the user and separating a portion of the image comprising the biometric characteristic from other portions of the image
CN110956654B (zh) * 2019-12-02 2023-09-19 Oppo广东移动通信有限公司 图像处理方法、装置、设备及存储介质
CN111494946B (zh) * 2020-04-23 2021-05-18 腾讯科技（深圳）有限公司 图像处理方法、装置、设备及计算机可读存储介质
CN111583168A (zh) * 2020-06-18 2020-08-25 上海眼控科技股份有限公司 图像合成方法、装置、计算机设备和存储介质
CN111652830A (zh) * 2020-06-28 2020-09-11 Oppo广东移动通信有限公司 图像处理方法及装置、计算机可读介质及终端设备
CN111768425B (zh) * 2020-07-23 2021-08-10 腾讯科技（深圳）有限公司 图像处理方法、装置及设备
CN112101546B (zh) * 2020-09-01 2024-01-05 浙江大学 生成多样化结果的风格迁移方法、装置、计算机设备和存储介质
CN111951157A (zh) * 2020-09-02 2020-11-17 深圳传音控股股份有限公司 图像处理方法、设备及存储介质
CN114494566A (zh) * 2020-11-09 2022-05-13 华为技术有限公司 图像渲染方法和装置
CN113160033B (zh) * 2020-12-28 2023-04-28 武汉纺织大学 服装风格迁移系统与方法
CN113012082A (zh) * 2021-02-09 2021-06-22 北京字跳网络技术有限公司 图像显示方法、装置、设备及介质
CN113052757A (zh) * 2021-03-08 2021-06-29 Oppo广东移动通信有限公司 图像处理方法、装置、终端及存储介质
CN117808934A (zh) * 2022-09-29 2024-04-02 华为技术有限公司 一种数据处理方法及相关设备

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107613360A (zh) * 2017-09-20 2018-01-19 北京奇虎科技有限公司 视频数据实时处理方法及装置、计算设备
CN108205813A (zh) * 2016-12-16 2018-06-26 微软技术许可有限责任公司 基于学习网络的图像风格化
US20190057527A1 (en) * 2017-08-17 2019-02-21 Adobe Systems Incorporated Digital Media Environment for Style-Aware Patching in a Digital Image
CN109636712A (zh) * 2018-12-07 2019-04-16 北京达佳互联信息技术有限公司 图像风格迁移及数据存储方法、装置和电子设备
CN109863537A (zh) * 2016-10-21 2019-06-07 谷歌有限责任公司 风格化输入图像
CN110210347A (zh) * 2019-05-21 2019-09-06 赵森 一种基于深度学习的彩色套层剪纸智能设计方法
CN110222722A (zh) * 2019-05-14 2019-09-10 华南理工大学 交互式图像风格化处理方法、系统、计算设备及存储介质
WO2020025000A1 (en) * 2018-08-01 2020-02-06 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and apparatus for image processing
CN110830706A (zh) * 2018-08-08 2020-02-21 Oppo广东移动通信有限公司 图像处理方法和装置、存储介质、电子设备
CN110909790A (zh) * 2019-11-20 2020-03-24 Oppo广东移动通信有限公司 图像的风格迁移方法、装置、终端及存储介质
CN110956654A (zh) * 2019-12-02 2020-04-03 Oppo广东移动通信有限公司 图像处理方法、装置、设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190279075A1 (en) * 2018-03-09 2019-09-12 Nvidia Corporation Multi-modal image translation using neural networks
CN108537776A (zh) * 2018-03-12 2018-09-14 维沃移动通信有限公司 一种图像风格迁移模型生成方法及移动终端
CN108596830B (zh) * 2018-04-28 2022-04-22 国信优易数据股份有限公司 一种图像风格迁移模型训练方法以及图像风格迁移方法

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109863537A (zh) * 2016-10-21 2019-06-07 谷歌有限责任公司 风格化输入图像
CN108205813A (zh) * 2016-12-16 2018-06-26 微软技术许可有限责任公司 基于学习网络的图像风格化
US20190057527A1 (en) * 2017-08-17 2019-02-21 Adobe Systems Incorporated Digital Media Environment for Style-Aware Patching in a Digital Image
CN107613360A (zh) * 2017-09-20 2018-01-19 北京奇虎科技有限公司 视频数据实时处理方法及装置、计算设备
WO2020025000A1 (en) * 2018-08-01 2020-02-06 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and apparatus for image processing
CN110830706A (zh) * 2018-08-08 2020-02-21 Oppo广东移动通信有限公司 图像处理方法和装置、存储介质、电子设备
CN109636712A (zh) * 2018-12-07 2019-04-16 北京达佳互联信息技术有限公司 图像风格迁移及数据存储方法、装置和电子设备
CN110222722A (zh) * 2019-05-14 2019-09-10 华南理工大学 交互式图像风格化处理方法、系统、计算设备及存储介质
CN110210347A (zh) * 2019-05-21 2019-09-06 赵森 一种基于深度学习的彩色套层剪纸智能设计方法
CN110909790A (zh) * 2019-11-20 2020-03-24 Oppo广东移动通信有限公司 图像的风格迁移方法、装置、终端及存储介质
CN110956654A (zh) * 2019-12-02 2020-04-03 Oppo广东移动通信有限公司 图像处理方法、装置、设备及存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114842342A (zh) * 2022-05-16 2022-08-02 网思科技股份有限公司 一种基于人工智能的杂乱场景检测方法、装置及相关设备
CN114842342B (zh) * 2022-05-16 2023-01-24 网思科技集团有限公司 一种基于人工智能的杂乱场景检测方法、装置及相关设备
CN116452414A (zh) * 2023-06-14 2023-07-18 齐鲁工业大学(山东省科学院) 一种基于背景风格迁移的图像和谐化方法及系统
CN116452414B (zh) * 2023-06-14 2023-09-08 齐鲁工业大学(山东省科学院) 一种基于背景风格迁移的图像和谐化方法及系统

Also Published As

Publication number Publication date
CN110956654B (zh) 2023-09-19
CN110956654A (zh) 2020-04-03
EP4064189A1 (en) 2022-09-28
US20220284638A1 (en) 2022-09-08

Similar Documents

Publication Publication Date Title
WO2021109876A1 (zh) 图像处理方法、装置、设备及存储介质
US11200638B2 (en) Image style transform methods and apparatuses, devices and storage media
CN109670558B (zh) 使用深度学习的数字图像完成
US20210150678A1 (en) Very high-resolution image in-painting with neural networks
CN110232661B (zh) 基于Retinex和卷积神经网络的低光照彩色图像增强方法
US11544905B2 (en) Method and apparatus for providing virtual clothing wearing service based on deep-learning
KR100924689B1 (ko) 모바일 기기의 이미지 변환 장치 및 방법
CN110909790A (zh) 图像的风格迁移方法、装置、终端及存储介质
CN110598781A (zh) 图像处理方法、装置、电子设备及存储介质
CN110533594B (zh) 模型训练方法、图像重建方法、存储介质及相关设备
JP2022547769A (ja) 機械学習を用いた画像カラー化
WO2018153264A1 (zh) 一种图像处理的方法及装置
WO2021115242A1 (zh) 一种超分辨率图像处理方法以及相关装置
WO2021052028A1 (zh) 图像颜色迁移方法、装置、计算机设备和存储介质
US20230306685A1 (en) Image processing method, model training method, related apparatuses, and program product
CN111127309A (zh) 肖像风格迁移模型训练方法、肖像风格迁移方法以及装置
CN113052923B (zh) 色调映射方法、装置、电子设备和存储介质
CN112070854B (zh) 一种图像生成方法、装置、设备及存储介质
JP7463186B2 (ja) 情報処理装置、情報処理方法及びプログラム
CN114862729A (zh) 图像处理方法、装置、计算机设备和存储介质
WO2023207741A1 (zh) 一种元宇宙场景素材的建模方法及相关装置
CN116843566A (zh) 色调映射方法、装置、显示设备和存储介质
US11200708B1 (en) Real-time color vector preview generation
CN114004751A (zh) 一种图像处理方法及其相关设备
CN114708144B (zh) 图像数据处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20896430

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020896430

Country of ref document: EP

Effective date: 20220624

NENP Non-entry into the national phase

Ref country code: DE