WO2023005357A1 - 图像风格迁移模型的训练方法、图像风格迁移方法及装置 - Google Patents

图像风格迁移模型的训练方法、图像风格迁移方法及装置 Download PDF

Info

Publication number
WO2023005357A1
WO2023005357A1 PCT/CN2022/093126 CN2022093126W WO2023005357A1 WO 2023005357 A1 WO2023005357 A1 WO 2023005357A1 CN 2022093126 W CN2022093126 W CN 2022093126W WO 2023005357 A1 WO2023005357 A1 WO 2023005357A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
style
sample image
sample
transfer model
Prior art date
Application number
PCT/CN2022/093126
Other languages
English (en)
French (fr)
Inventor
白须
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2023005357A1 publication Critical patent/WO2023005357A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image

Definitions

  • Embodiments of the present disclosure relate to the field of computers, and in particular, to a training method for an image style transfer model, an image style transfer method and a device.
  • Image style transfer refers to an image processing technique that renders an image into a painting with a specific artistic style.
  • the existing image style transfer technology to transfer the target image to the image of the style sample image in the transfer process, it is generally realized by using the mapping changes between image domains. That is, the target image is taken as one image domain, and the style sample image is taken as another image domain, and the mapping relationship between the two image domains is learned by using the GAN network model, so that the target image can be reconstructed and the style sample image is obtained.
  • a certain style of image is learned by using the GAN network model.
  • the target image of a specific style obtained in this way is simply mapped to the color style of the image domain, but there is no more representation of the image block surface and line sense in the style sample image, so that the obtained specific style The block and line sense of the style image is insufficient, and the style effect is not good.
  • the embodiments of the present disclosure provide a training method for an image style transfer model, an image style transfer method and an apparatus.
  • the present disclosure provides a method for training an image style transfer model, including:
  • an image style transfer method including:
  • the target image is input into a trained image style transfer model to perform image style transfer processing, and the trained image style transfer model is the training of the image style transfer model according to any one of the first aspect obtained by the method;
  • a style image corresponding to the target image is obtained.
  • the present disclosure provides a training device for an image style transfer model, including:
  • the first acquisition module is configured to perform style preprocessing on the real sample image to obtain a style sample image corresponding to the real sample image;
  • the first processing module is configured to perform color clustering processing and line blurring processing on the style sample image respectively, to obtain a color clustering map and a line blurring map of the style sample image;
  • the training module is used to use the real sample image and the style sample image to train the image style transfer model to be trained, and use the color clustering map of the style sample image and the line blur map of the style sample image Supervising the training of the image style transfer model to be trained to obtain the trained image style transfer model.
  • the present disclosure provides a terminal, including:
  • An acquisition module configured to acquire a target image
  • a processing module configured to input the target image into a trained image style transfer model to perform image style transfer processing, and the trained image style transfer model is the image according to any one of the first aspect Obtained by the training method of the style transfer model;
  • the acquiring module is further configured to acquire a style image corresponding to the target image.
  • an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;
  • the memory stores computer-executable instructions
  • the at least one processor executes the computer-executed instructions stored in the memory, so that the at least one processor executes the above-mentioned first aspect and various possible training methods related to the image style transfer model in the first aspect, and /or, execute the image style transfer method described in the above second aspect and various possible related aspects of the second aspect.
  • the embodiments of the present disclosure provide a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the above-mentioned first aspect and the first aspect are realized.
  • various possible training methods related to the image style transfer model, and/or, implement the above-mentioned second aspect and various possible image style transfer methods related to the second aspect are realized.
  • the embodiments of the present disclosure provide a computer program product, including computer instructions.
  • the computer instructions are executed by a processor, the above-mentioned first aspect and various possible image style transfers related to the first aspect can be realized.
  • an embodiment of the present disclosure provides a computer program.
  • the computer program When the computer program is executed by a processor, it implements the above-mentioned first aspect and various possible training methods related to the image style transfer model in the first aspect. And/or, implement the image style transfer method described in the above second aspect and various possible related aspects of the second aspect.
  • the image style transfer model training method, image style transfer method and device use the real sample image and its corresponding style sample image to train the image style transfer model to be trained, and use the style sample image
  • the color cluster map and the line blur map supervise the training of the image style transfer model to be trained, and obtain the trained image style transfer model.
  • the image style transfer model not only the real sample image and the style sample image are used to determine the mapping relationship of the image on the image domain, but also the color clustering map and the line fuzzy map are used to analyze the image block surface and line sense of the image. Supervision and representation are carried out, so that the style image output by the trained image style transfer model has a better sense of block and line, and the style transfer effect has been well improved.
  • FIG. 1 is a schematic diagram of a network architecture based on the present disclosure
  • FIG. 2 is a schematic flowchart of a training method for an image style transfer model provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of a data flow when training an image style transfer model provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of data flow when training a style image generation model provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic flowchart of an image style transfer method provided by an embodiment of the present disclosure.
  • FIG. 6 is a structural block diagram of a training device for an image style transfer model provided by an embodiment of the present disclosure
  • FIG. 7 is a structural block diagram of a terminal provided by an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present disclosure.
  • GAN networks have been more and more widely used in computer vision research, and image style transfer is just a practical application of applying GAN networks to the field of computer vision.
  • the transfer process of the target image to the image style of the style sample image is generally realized by using the mapping changes between image domains.
  • the style features in the style image can be extracted by using the trained style transfer model, and mixed with the target image to reconstruct the target image, and the target image with style features can be obtained .
  • the style target image obtained in this way only maps the color style of the image domain, but does not have more representations for the image block surface and line sense in the style sample image, so that the obtained style The block and line sense of the target image is insufficient, and the style effect is not good.
  • the inventor first thought that on the basis of using the target image of the prior art and the style image of a specific style to train the style transfer model, it is also possible to increase the supervision of the model on the color block surface features and line features , that is to use the color clustering map and the line fuzzy map to supervise and represent the image block surface and line sense, so that the style image generated by the style transfer model presents a stronger block surface and line sense, so that The style image output by the trained image style transfer model has a better sense of block and line, and the effect of style transfer has been greatly improved.
  • FIG. 1 is a schematic diagram of a network architecture on which the present disclosure is based.
  • the network architecture shown in FIG. 1 may specifically include at least one terminal 1 and a server 2 .
  • the terminal 1 may specifically include hardware devices such as user mobile phones, smart home devices, tablet computers, and wearable devices.
  • the server 2 may specifically be a server or a server cluster arranged in the cloud, and a training device for an image style transfer model for training an image style transfer model is integrated or installed in the server or server cluster, so that the training device for the image style transfer model is Hardware or software for executing the image style transfer method of the present disclosure.
  • a trained image style transfer model may be deployed in the terminal 1, through which the image style transfer model can be used to perform style transfer processing on the target image selected by the user, so as to obtain a style image with a specific style.
  • the server 2 can send the configuration files and configuration parameters related to the trained image style transfer model to the terminal 1, so that the terminal 1 can call the trained image style pre-stored locally or in the service cloud
  • the transfer model is used to perform style transfer processing on the image and display the style image.
  • the image style transfer model trained by the server 2 is deployed on the terminal 1, so that the terminal 1 can use the image style transfer model to implement the target image uploaded or taken by the user through the terminal 1 on the basis of providing basic application functions for the user. Perform style transfer processing and display.
  • the image style transfer model training device for training the image style transfer model can also be deployed in the terminal 1 or other terminals for training the image style transfer model. This disclosure does not limit the device for training the image style transfer model.
  • FIG. 2 is a schematic flowchart of a method for training an image style transfer model provided by an embodiment of the present disclosure.
  • the training method of the image style transfer model provided by the embodiment of the present disclosure includes:
  • Step 101 Perform style preprocessing on real sample images to obtain style sample images corresponding to the real sample images;
  • Step 102 performing color clustering and line blurring processing on the style sample image respectively, to obtain a color clustering map and a line blurring map of the style sample image;
  • Step 103 Use the real sample image and the style sample image to train the image style transfer model to be trained, and use the color clustering map of the style sample image and the line blur map of the style sample image to supervise the training of the image style transfer model to be trained, and obtain the trained image style transfer model.
  • the training method of the image style transfer model provided in this embodiment is executed by the aforementioned image style transfer model training device.
  • paired training images are used for the image style transfer model for training.
  • the image style transfer model training apparatus will obtain a real sample image as the real sample image in step 101 .
  • the image style transfer model training device can also perform style preprocessing on the real sample image to obtain the style sample image of the preset style, wherein the preset style It may be the aforementioned specific style (for example, cartoon style).
  • the image style transfer model training device may also perform color clustering processing and line blurring processing on the style sample images respectively, so as to obtain the color clustering graph and the line blurring graph of the style sample images.
  • the line blurring processing on the style sample image may specifically be performing edge smoothing processing on the style sample image based on Gaussian filtering, so as to remove the sense of line in the style sample image, smooth the pixel data, and obtain a line blur map.
  • performing line blurring processing on the image can blur the lines in the style sample image and present a mosaic effect.
  • Performing color clustering processing on the style sample image may specifically include: performing image segmentation and merging processing on the style sample image, so as to divide the style sample image into multiple regions according to the image content of the style sample image, wherein each The regions are image regions with the same image content in the style sample image; then, based on the color in each region, color mean value processing is performed on each region of the style sample image.
  • the color clustering process can be realized based on the Felzenszwalb algorithm.
  • the Felzenszwalb algorithm is a greedy clustering algorithm based on a graph.
  • the pixels are clustered to achieve image The effect of segmentation and merging to obtain several regions based on the graph content of the image.
  • the image style transfer model training device can perform color mean value processing for each region, that is, take the region as a unit, perform mean value processing on the color values of all pixels belonging to the region to obtain the color mean value, and use the color mean value to replace each pixel in the region.
  • the original color value of each pixel so that the image appears blocky.
  • the sense of color clustering in the style sample image can be enhanced.
  • the real sample image and its corresponding style sample image can be used to train the image style transfer model to be trained, and the color clustering map and line blur map of the style sample image can be used for training
  • the training of the image style transfer model is supervised, and the trained image style transfer model is obtained.
  • the image style transfer model can specifically include a generative confrontation network
  • the training of the image style transfer model is generally a process of multiple rounds of training, that is, the real sample image and its corresponding style sample image, and the color clustering of the style sample image
  • the image and the blurred line image will be repeatedly input into the image style transfer model to train the image style transfer model multiple times.
  • the present disclosure does not limit the number of times of training.
  • the image style transfer model may include a first generation adversarial network, and the first generation adversarial network includes a first generator and a first discriminator.
  • Use the real sample image and its corresponding style sample image to train the image style transfer model to be trained, and use the color clustering map of the style sample image and the line blur map to train the image style transfer model to be trained
  • the training of the real sample image is supervised, comprising: inputting the real sample image to the first generator to obtain an intermediate image of the real sample image; aggregating the color of the intermediate image of the real sample image and the style sample image
  • the class diagram, the line fuzzy image of the style sample image and the style sample image corresponding to the real sample image are input to a first discriminator to obtain a first discrimination result; according to the first discrimination result, the first
  • the generator performs parameter adjustment, and returns to the step of inputting the real sample image to the first generator until the image style transfer model converges.
  • FIG. 3 is a schematic diagram of a data flow when training an image style transfer model provided by an embodiment of the present disclosure.
  • the image style transfer model includes a first generator and a first discriminator.
  • the first generator is configured to process the real sample image input to the first generator based on weight parameters, and output an intermediate image of the real sample image.
  • the first generator receives the real sample image and outputs an intermediate image, and then the intermediate image and the style sample image corresponding to the real sample image are simultaneously input into the first discriminator for the first discriminator to compare These two types of images are subjected to discrimination processing. That is, the intermediate image of the real sample image and the style sample image corresponding to the real sample image are discriminated by the first discriminator to obtain a processing result.
  • the first discriminator can easily distinguish the style sample image corresponding to the real sample image as a real image, while the intermediate image is a fake image; at this time, the first discriminator feeds back the discrimination result to the first generator
  • the generator is used for parameter adjustment by the first generator, and generation of the next intermediate image based on the next real sample image.
  • the purpose of training the model is to enable the intermediate image output by the first generator to confuse the first discriminator, so that the discriminant result of the first discriminator on the intermediate image is different from the style sample image corresponding to the real sample image.
  • the results are consistent, that is, when the intermediate image is input to the first discriminator, the first discriminator can discriminate the intermediate image as a real image, and at this time, the first generator and the first discriminator complete the training.
  • the color feature difference between the intermediate image and the color cluster map of the style sample image may be determined, and the line feature difference between the intermediate image and the line fuzzy map of the style sample image may be determined; and then The difference between the two features is fed back to the first generator, so that the first generator performs parameter adjustment, and the intermediate image generated after the parameter adjustment and its corresponding color cluster map have a color feature difference smaller than that before the parameter adjustment.
  • the color feature difference and at the same time make the line feature difference between the intermediate image generated after the parameter adjustment and the corresponding line blur image smaller than the line feature difference before the parameter adjustment.
  • the supervision can be implemented based on a loss function, that is, a loss function based on VGG19-bn to output the discrimination result.
  • the first discriminator uses the VGG19-bn in it to extract the high-level features of the intermediate image of the real sample image and the color cluster map of the style sample image respectively, and obtain the corresponding color feature loss Value, calculate the color feature difference; synchronous or asynchronous, the first discriminator also uses VGG19-bn to extract the high-level features of the intermediate image of the real sample image and the line blur map of the style sample image, respectively, and obtain the corresponding line features
  • the loss value is calculated to obtain the line feature difference.
  • the image style transfer model training apparatus adjusts the parameters of the first generator according to the color feature difference, the line feature difference, and the processing result, so that the first generator is optimized. Since the optimization process is repeated, in this way, the intermediate image generated by the image style transfer model will gradually have a specific style, for example, cartoon style. At the same time, due to the two feature parameters (color feature difference and line feature Loss value) supervision, the block surface and line sense of the generated intermediate image will also be gradually enhanced.
  • the trained image style transfer model can use the first generator after repeated parameter adjustment to process the target image, generate a corresponding style image, and realize the style of the image.
  • Migration processing, and the style sample image obtained by the style transfer processing has a better sense of block and line, and the effect of style transfer is better.
  • the image quality of the style sample images used to train the image style transfer model determines the training quality of the model to a certain extent.
  • the supervision on the direction of block surface and line sense can also be emphasized during the process of generating style sample images from real sample images, so that The style sample image is a picture with a certain sense of block and line, which further facilitates the model to approach the training direction of block and line.
  • the real sample image may be processed by using a trained style image generation model to obtain a style sample image corresponding to the real sample image.
  • style image generation model is a generation confrontation network including a second generator and a second discriminator;
  • style preprocessing is performed on the obtained real sample image to obtain a style sample image corresponding to the real sample image, including:
  • obtaining a training sample image obtaining a reference style sample image; performing line blurring processing on the reference style sample image to obtain a blurred sample image of the reference style sample image; inputting the training sample image to the second generator , to obtain an intermediate sample image of the training sample image; perform color clustering processing on the intermediate sample image to obtain a color clustering sample image of the intermediate sample image; use the intermediate sample image, the reference style sample image , the fuzzy sample map of the reference style sample image and the color clustering sample map of the intermediate sample image are used to train the second discriminator to obtain a second discrimination result; according to the second discrimination result, the first The second generator performs parameter adjustment, and returns to the step of inputting the training sample image to the second generator until the style image generation model converges to the real sample image.
  • a style image generation model will be constructed for generating Style sample images corresponding to real sample images.
  • FIG. 4 is a schematic diagram of a data flow when training a style image generation model provided by an embodiment of the present disclosure.
  • the style image generation model includes a second generator and a second discriminator.
  • a generative model of style images can be trained. Specifically, the training sample image and the reference style sample image may be obtained first.
  • the reference style sample image includes an image with a preset style, such as a cartoon style image.
  • the reference style sample images can be random and large in number.
  • the training sample images may include realistic images such as landscape images and still life images.
  • the training device for the image style transfer model may perform line blurring processing on the reference style sample image to obtain a blurred sample image of the reference style sample image.
  • performing line blurring processing on the reference style sample image may specifically include performing edge smoothing processing on the reference style sample image based on Gaussian filtering, so that the sense of lines in the reference style sample image is removed, the pixel data is smoothed, and a line blur image is obtained. , its effect can be similar to the effect of processing the style sample image in the foregoing embodiments.
  • the training sample image is input to the second generator to obtain an intermediate sample image.
  • each area is an image area with the same image content in the image to be processed; then, based on the color in each area, the color mean value processing is performed on each area of the intermediate sample image, and the effect can be similar to the aforementioned style Processing effect on the sample image.
  • the training of the style image generation model in this embodiment is realized based on non-paired graphs.
  • use the second discriminator to discriminate the fuzzy sample image using the intermediate sample image, the reference style sample image, and the reference style sample image, and obtain a method based on the line style Discrimination results of features.
  • the second discriminator can recognize that the blurred sample image of the reference style sample image is a fake image, and the reference style sample image is a real image, while the intermediate sample image generated by the second generator will be discriminated is a pseudo-graph, such discrimination results will be fed back to the second generator, so that the weight parameters of the second generator can be optimized.
  • the second generator will be continuously optimized until the intermediate sample image generated by the second generator will be judged as a real image by the second discriminator.
  • the second generator can be used to generate a A style sample image with a certain sense of line.
  • the color cluster sample map of the intermediate sample image obtained above can also be used to supervise the model.
  • VGG19-bn in the second discriminator to extract the color features in the intermediate sample image and the color features in the color cluster sample map of the intermediate sample image, and calculate the difference between them.
  • the parameters of the second generator are adjusted according to the difference, so that the adjusted second generator can generate an intermediate sample image with the smallest color feature difference. That is, using the second discriminator to perform discriminative processing on the fuzzy sample image using the intermediate sample image, the reference style sample image, and the reference style sample image to obtain a discriminant result based on line style features; and determine the The feature difference between the intermediate sample image and the color cluster sample map of the intermediate sample image in the color style feature; according to the difference between the discrimination result based on the line style feature and the color style feature, the second generator Make parameter adjustments.
  • the trained style image generation model can use the second generator after repeated parameter adjustment to process the real sample image to generate the corresponding style sample image, and the style sample image It has a certain sense of line and block surface, so that the image style transfer model can be trained by using such style sample images, and an image style transfer model with better training effect can be obtained.
  • the image style transfer method uses the real sample image and its corresponding style sample image to train the image style transfer model to be trained, and uses the color clustering diagram of the style sample image and the line
  • the fuzzy graph supervises the training of the image style transfer model to be trained, and obtains the trained image style transfer model.
  • the image style transfer model not only the real sample image and the style sample image are used to determine the mapping relationship of the image on the image domain, but also the color clustering map and the line fuzzy map are used to improve the image block surface and line sense. Supervised and represented, so that the style sample image output by the trained image style transfer model has a better sense of block and line, and the effect of style transfer has been greatly improved.
  • FIG. 5 is a schematic flowchart of an image style transfer method provided by an embodiment of the present disclosure. As shown in FIG. 5 , the method includes:
  • Step 201 acquiring a target image
  • Step 202 input the target image into the trained image style transfer model, so as to perform the style transfer processing of the target image;
  • Step 203 Obtain a style image corresponding to the target image.
  • the execution body of the image style transfer method provided in this embodiment is the aforementioned terminal, and in some embodiments of the present disclosure, the image style transfer model trained through the aforementioned embodiments will be deployed on the terminal. During deployment, the configuration file and related data of the image style transfer model can be sent from the server to the terminal, so that installation and loading can be completed on the terminal and used.
  • the user can send the target image that needs style transfer processing to the model of the terminal through the port of the application APP, so that the model can perform style transfer processing on the target image, and obtain the style image of the processed target image.
  • the image style transfer model involved in the embodiments of the present disclosure may be obtained through training based on any of the foregoing embodiments, and the training process of the model will not be repeated in this embodiment.
  • the terminal may adopt a block processing method when processing the target image.
  • inputting the target image into the trained image style transfer model to perform image style transfer processing includes: performing block processing on the target image to obtain multiple target blocks; wherein, each The combination of target blocks constitutes the target image, and there is an overlapping area between adjacent target blocks; the target blocks are respectively input into the trained image style transfer model for processing, so as to obtain the style transfer blocks corresponding to the target blocks; and splicing the style transfer blocks to obtain a style image corresponding to the target image.
  • the target image to be processed is segmented to obtain several target blocks; wherein, the combination of each target block constitutes the target image, and the adjacent target blocks There is an overlapping area between them.
  • the target image A is segmented, the target block A1, the target block A2 and the target block A3 are obtained, wherein there is a certain overlapping area between the target block A1 and the target block A2, and the target block A2 There is a certain overlapping area between A3 and the target block A3, and the combination of each target block will cover the entire target image A.
  • the terminal may use the trained image style transfer model to process each target block respectively, to obtain a style image block corresponding to each target block. For example, the terminal will perform style transfer processing on the target block A1 , target block A2 and target block A3 respectively to obtain corresponding style image blocks B1 , style image blocks B2 and style image blocks B3 .
  • the terminal will combine style image block B1, style image block B2, and style image block B3 to obtain style image B.
  • the splicing of overlapping areas will adopt linear interpolation processing.
  • the pixel value of the overlapping area on the style image block B1 and the pixel value of the overlapping area on the style image block B2 can be interpolated , get the pixel value of the overlapping area. In this way, the output style image can be made smoother and the effect is better.
  • the style image of the target image output by the trained image style transfer model has higher robustness and better style effect; in addition, the terminal can directly use the image style image obtained by the server for training.
  • the scale of the image style transfer model deployed on the terminal can be effectively controlled, which also reduces the load on the terminal when performing processing.
  • FIG. 6 is a structural block diagram of an image style transfer model training device provided by an embodiment of the present disclosure.
  • the training device of described image style transfer model comprises:
  • An acquisition module 11 configured to perform style preprocessing on the real sample image to obtain a style sample image corresponding to the real sample image;
  • a processing module 12 configured to perform color clustering processing and line blurring processing on the style sample image, respectively, to obtain a color clustering map and a line blurring map of the style sample image;
  • the training module 13 is used to use the real sample image and the style sample image to train the image style transfer model to be trained, and use the color clustering diagram of the style sample image and the blurred lines of the style sample image Fig. 1 supervises the training of the image style transfer model to be trained, and obtains the trained image style transfer model.
  • the image style transfer model includes a first generation adversarial network, and the first generation adversarial network includes a first generator and a first discriminator;
  • the training module 13 is configured to input the real sample image to the first generator to obtain the intermediate image of the real sample image; combine the intermediate image of the real sample image, the color clustering image of the style sample image, the line blur image of the style sample image and the The style sample image corresponding to the real sample image is input to the first discriminator to obtain the first discriminant result; the parameters of the first generator are adjusted according to the first discriminant result, and the real The step of inputting sample images to the first generator until the image style transfer model converges.
  • the training module 13 is configured to use the first discriminator to discriminate the intermediate image of the real sample image and the style sample image corresponding to the real sample image to obtain a processing result; the color feature difference between the intermediate image and the color clustering map of the style sample image; and determine the line feature difference between the intermediate image and the line fuzzy map of the style sample image; according to the color feature difference, The line feature difference and the processing result are used to adjust the parameters of the first generator.
  • the acquiring module 11 is configured to use the trained style image generation model to process the real sample image to obtain the style sample image corresponding to the real sample image.
  • the obtaining module 11 is used to obtain a training sample image; obtain a reference style sample image; perform line blurring processing on the reference style sample image to obtain a blurred sample image of the reference style sample image; input the training sample image to the second generator to obtain an intermediate sample image of the training sample image; perform color clustering processing on the intermediate sample image to obtain a color clustering sample map of the intermediate sample image; use the intermediate sample image , the reference style sample image, the fuzzy sample image of the reference style sample image, and the color cluster sample image of the intermediate sample image are used to train the second discriminator to obtain a second discriminant result; according to the second discriminator Adjust the parameters of the second generator based on the result of the second discrimination, and return to the step of inputting the training sample image to the second generator until the style image generation model converges.
  • the acquiring module 11 is configured to: use the second discriminator to discriminate the blurred sample image using the intermediate sample image, the reference style sample image, and the reference style sample image, to obtain Discrimination results based on line style features; and determining the feature difference between the intermediate sample image and the color cluster sample map of the intermediate sample image in color style features; according to the discrimination results based on line style features and the color style features The difference is to adjust the parameters of the second generator.
  • the processing module 12 is configured to perform image segmentation and merging processing on the image to be processed when performing color clustering processing, so as to divide the image to be processed into multiple Areas, wherein each area is an image area with the same image content in the image to be processed; based on the color in each area, color mean value processing is performed on each area of the image to be processed, wherein the image to be processed
  • the images include said style sample images and/or intermediate sample images.
  • the processing module 12 is configured to perform edge smoothing processing based on Gaussian filtering on the image to be processed when performing line blurring processing, wherein the image to be processed includes the style sample image and/or Refer to style sample images.
  • the image style transfer model training device uses the real sample image and its corresponding style sample image to train the image style transfer model to be trained, and uses the color clustering map of the style sample image and The line blur map supervises the training of the image style transfer model to be trained, and obtains the trained image style transfer model.
  • the image style transfer model not only the real sample image and the style sample image are used to determine the mapping relationship of the image on the image domain, but also the color clustering map and the line fuzzy map are used to improve the image block surface and line sense. Supervised and represented, so that the style sample image output by the trained image style transfer model has a better sense of block and line, and the effect of style transfer has been greatly improved.
  • FIG. 7 is a structural block diagram of a terminal provided in an embodiment of the present disclosure. For ease of description, only the parts related to the embodiments of the present disclosure are shown. Referring to Figure 7, the terminal includes:
  • a processing module 22 configured to input the target image into a trained image style transfer model to perform style transfer processing of the target image; wherein, the trained image style transfer model is according to any one of the first aspect Obtained by the training method of the described image style transfer model;
  • the obtaining module 21 is further configured to obtain a style image corresponding to the target image.
  • the processing module 22 is specifically configured to perform block processing on the target image to obtain a plurality of target blocks; wherein, the combination of each target block constitutes the target image, and the adjacent target There is an overlapping area between the blocks; the target blocks are respectively input into the trained image style transfer model for processing, so as to obtain the style transfer blocks corresponding to the target blocks;
  • the acquisition module 21 is configured to combine the style transfer blocks to obtain the style image corresponding to the target image.
  • processing module 22 is further configured to: use linear interpolation to process the overlapping area.
  • the electronic device provided in this embodiment can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and details will not be repeated here in this embodiment.
  • the electronic device 900 may be a terminal device or a media library.
  • the terminal equipment may include but not limited to mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA for short), tablet computers (Portable Android Device, PAD for short), portable multimedia players (Portable Media Player, referred to as PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), mobile terminals such as wearable electronic devices, and fixed terminals such as digital TVs, desktop computers, and smart home devices.
  • PDA Personal Digital Assistant
  • PMP portable multimedia players
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • mobile terminals such as wearable electronic devices
  • fixed terminals such as digital TVs, desktop computers, and smart home devices.
  • the electronic device shown in FIG. 8 is only an embodiment, and should not limit the functions and application scope of the embodiments of the present disclosure.
  • an electronic device 900 may include a processor 901 (such as a central processing unit, a graphics processing unit, etc.) for performing the above method, which may be stored in a read-only memory (Read Only Memory, ROM for short) 902 according to the Various appropriate actions and processes are executed by the program loaded from the storage device 908 or the program loaded into the random access memory (Random Access Memory, RAM for short) 903. In the RAM 903, various programs and data necessary for the operation of the electronic device 900 are also stored.
  • the processor 901, ROM 902, and RAM 903 are connected to each other through a bus 904.
  • An input/output (Input/Output, I/O for short) interface 905 is also connected to the bus 904 .
  • an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; ), a speaker, a vibrator, etc.
  • a storage device 908 including, for example, a magnetic tape, a hard disk, etc.
  • the communication means 909 may allow the electronic device 900 to perform wireless or wired communication with other devices to exchange data. While FIG. 8 shows electronic device 900 having various means, it should be understood that it is not a requirement to implement or have all of the illustrated means, and more or fewer means may instead be implemented or provided.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program is used to execute the methods shown in the flow charts according to the embodiments of the present disclosure.
  • program code may be downloaded and installed from a network via communication means 909, or from storage means 908, or from ROM 902.
  • the processor 901 When the computer program is executed by the processor 901, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.
  • the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, Random Access Memory (RAM), Read-Only Memory (Read-Only Memory).
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device .
  • the program code contained on the computer readable medium can be transmitted by any appropriate medium, including but not limited to: electric wire, optical cable, radio frequency (Radio Frequency, RF for short), etc., or any suitable combination of the above.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is made to execute the methods shown in the above-mentioned embodiments.
  • Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or media library.
  • the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external A computer (connected via the Internet, eg, using an Internet service provider).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the unit does not constitute a limitation of the unit itself under certain circumstances, for example, the first obtaining unit may also be described as "a unit for obtaining at least two Internet Protocol addresses".
  • exemplary types of hardware logic components include: Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA for short), Application Specific Integrated Circuit (ASIC for short), application specific standard product (Application Specific Standard Product, ASSP for short), System On Chip (SOC for short), Complex programmable logic device (CPLD for short), etc.
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, Random Access Memory (RAM), Read-Only Memory (RAM), ROM), Erasable Programmable Read-Only Memory (EPROM for short), flash memory, optical fiber, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM for short), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • RAM Random Access Memory
  • RAM Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • flash memory optical fiber
  • Compact Disc Read-Only Memory Compact Disc Read-Only Memory
  • CD-ROM Compact Disc Read-Only Memory
  • a method for training an image style transfer model includes:
  • the image style transfer model includes a first generation confrontation network, and the first generation confrontation network includes a first generator and a first discriminator;
  • the training of the image style transfer model to be trained is supervised, including:
  • the intermediate image of the real sample image, the color clustering map of the style sample image, the line blur map of the style sample image, and the style sample image corresponding to the real sample image are input into The first discriminator obtains the discriminant results, including:
  • the parameter adjustment of the first generator according to the discrimination result includes:
  • performing style preprocessing on the real sample image to obtain the style sample image corresponding to the real sample image includes:
  • the real sample image is processed by using the trained style image generation model to obtain a style sample image corresponding to the real sample image.
  • the style image generation model includes a second generation adversarial network, and the second generation adversarial network includes a second generator and a second discriminator;
  • the method also includes:
  • the intermediate sample image, the reference style sample image, the fuzzy sample map of the reference style sample image, and the color cluster sample map of the intermediate sample image to perform the second discriminator Training to obtain the second discriminant result, including:
  • the parameter adjustment of the second generator according to the discrimination result includes:
  • Parameter adjustment is performed on the second generator according to the difference between the discrimination result based on the line style feature and the color style feature.
  • the color clustering process includes:
  • the image to be processed includes the style sample image and/or the intermediate sample image.
  • the line blurring process includes:
  • the image to be processed is subjected to edge smoothing based on Gaussian filtering
  • the image to be processed includes the style sample image and/or a reference style sample image.
  • an image style transfer method includes:
  • the target image is input into a trained image style transfer model to perform style transfer processing of the target image, and the trained image style transfer model is the image style transfer model according to any one of the aforementioned first aspects obtained by the training method;
  • a style image corresponding to the target image is obtained.
  • the inputting the target image into the trained image style transfer model to perform image style transfer processing includes:
  • the style image corresponding to the target image is obtained by splicing the style transfer blocks.
  • the overlapping regions are processed using linear interpolation.
  • an image style transfer model training device includes:
  • An acquisition module configured to perform style preprocessing on the real sample image to obtain a style sample image corresponding to the real sample image
  • a processing module configured to perform color clustering processing and line blurring processing on the style sample image respectively, to obtain a color clustering map and a line blurring map of the style sample image;
  • the training module is used to use the real sample image and the style sample image to train the image style transfer model to be trained, and use the color clustering map of the style sample image and the line blur map of the style sample image Supervising the training of the image style transfer model to be trained to obtain the trained image style transfer model.
  • the image style transfer model includes a first generation confrontation network, and the first generation confrontation network includes a first generator and a first discriminator;
  • the training module is used to input the real sample image to the first generator to obtain an intermediate image of the real sample image; cluster the color of the intermediate image of the real sample image and the style sample image
  • the image, the blurred line image of the style sample image, and the style sample image corresponding to the real sample image are input to the first discriminator to obtain a first discrimination result; according to the first discrimination result, the first generated
  • the generator performs parameter adjustment, and returns to the step of inputting the real sample image into the first generator until the image style transfer model converges.
  • the training module is specifically configured to use the first discriminator to discriminate the intermediate image of the real sample image and the style sample image corresponding to the real sample image to obtain a processing result; the color feature difference between the intermediate image and the color clustering map of the style sample image; and determine the line feature difference between the intermediate image and the line fuzzy map of the style sample image; according to the color feature difference, The line feature difference and the processing result are used to adjust the parameters of the first generator.
  • the acquisition module is configured to use the trained style image generation model to process the real sample image to obtain the style sample image corresponding to the real sample image.
  • the obtaining module is used to obtain a training sample image; obtain a reference style sample image; perform line blurring processing on the reference style sample image to obtain a blurred sample image of the reference style sample image; input the training sample image to The second generator obtains an intermediate sample image of the training sample image; performs color clustering processing on the intermediate sample image to obtain a color cluster sample map of the intermediate sample image; uses the intermediate sample image, The reference style sample image, the fuzzy sample image of the reference style sample image, and the color cluster sample image of the intermediate sample image are used to train the second discriminator to obtain a second discriminant result; according to the second As a result of the discrimination, the parameters of the second generator are adjusted, and the step of inputting the training sample image into the second generator is returned until the style image generation model converges.
  • the acquisition module is configured to: use the second discriminator to perform discriminative processing on the fuzzy sample image using the intermediate sample image, the reference style sample image, and the reference style sample image, and obtain the Discrimination results of line style features; and determining the feature difference between the intermediate sample image and the color cluster sample map of the intermediate sample image in color style features; according to the discrimination results based on line style features and the color style features difference, and adjust the parameters of the second generator.
  • the processing module when performing color clustering processing, is configured to perform image segmentation and merging processing on the image to be processed, so as to divide the image to be processed into multiple regions according to the image content of the image to be processed , wherein, each area is an image area with the same image content in the image to be processed; based on the color in each area, color mean value processing is performed on each area of the image to be processed, wherein the image to be processed
  • the style sample images and/or intermediate sample images are included.
  • the processing module is configured to perform edge smoothing processing based on Gaussian filtering on the image to be processed when performing line blurring processing, wherein the image to be processed includes the style sample image and/or reference Style sample images.
  • a terminal includes:
  • An acquisition module configured to acquire a target image
  • a processing module configured to input the target image into a trained image style transfer model to perform style transfer processing of the target image; wherein, the trained image style transfer model is according to any one of the aforementioned first aspects Obtained by the training method of the described image style transfer model;
  • the obtaining module is also used to obtain a style image corresponding to the target image.
  • the processing module is specifically configured to perform block processing on the target image to obtain multiple target blocks; wherein, the combination of each target block constitutes the target image, and adjacent target images There is an overlapping area between the blocks; the target blocks are respectively input into the trained image style transfer model for processing, so as to obtain the style transfer blocks corresponding to the target blocks;
  • the acquisition module is used to combine the style transfer blocks to obtain the style image corresponding to the target image.
  • the processing module is further configured to: use linear interpolation to process the overlapping area.
  • an electronic device includes: at least one processor and a memory;
  • the memory stores computer-executable instructions
  • the at least one processor executes the computer-implemented instructions stored in the memory, such that the at least one processor performs the method as described in any one of the preceding items.
  • a computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the following is implemented: The method described in any one of the foregoing.
  • a computer program product includes computer instructions, and when the computer instructions are executed by a processor, implement the method as described in any one of the preceding items.
  • a computer program when executed by a processor, implements the method described in any one of the preceding items.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种图像风格迁移模型的训练方法、图像风格迁移方法及装置。方法利用真实样本图像和其对应的风格样本图像，对待训练的图像风格迁移模型进行训练，并利用风格样本图像的颜色聚类图以及线条模糊图对待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。由于在对图像风格迁移模型进行训练时不仅利用真实样本图像和风格样本图像确定图像在图像域上映射关系，还利用颜色聚类图以及线条模糊图对图像在图像块面感和线条感上进行了监督和表征，从而使得训练完毕的图像风格迁移模型所输出的风格图像有着较佳的块面感和线条感，风格迁移效果得到了良好提升。

Description

图像风格迁移模型的训练方法、图像风格迁移方法及装置
相关申请的交叉引用
本申请要求于2021年7月28日提交的、申请号为2021108596963、名称为“图像风格迁移模型的训练方法、图像风格迁移方法及装置”的中国专利申请的优先权，其全部内容通过引用并入本文。
技术领域
本公开实施例涉及计算机领域，尤其涉及一种图像风格迁移模型的训练方法、图像风格迁移方法及装置。
背景技术
近年来，GAN网络开始被越来越广泛地应用在计算机视觉研究工作中，图像风格迁移正是将GAN网络运用到计算机视觉领域的一种实际应用。图像风格迁移是指将一张图片渲染为具有特定艺术风格的画作的图像处理技术。
利用现有的图像风格迁移技术将目标图像迁移至风格样本图像风格的图像的迁移过程中，一般是利用图像域之间的映射变化实现的。即，将目标图像作为一个图像域，将风格样本图像作为另一个图像域，利用GAN网络模型学习该两个图像域之间的映射关系，从而能够对目标图像进行重建，得到具有风格样本图像的特定风格的图像。
但是，利用这样的方式获得的特定风格的目标图像只是单纯对于图像域的色彩风格进行了映射，而对于风格样本图像中的图像块面感和线条感却没有更多的表征，使得获得的特定风格图像的块面感和线条感不足，风格效果不佳。
发明内容
针对上述问题，本公开实施例提供了一种图像风格迁移模型的训练方法、图像风格迁移方法及装置。
第一方面，本公开提供了一种图像风格迁移模型的训练方法，包括：
对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
第二方面，本公开提供了一种图像风格迁移方法，包括：
获取目标图像；
将所述目标图像输入至训练后的图像风格迁移模型中，以进行图像的风格迁移处理，所述训练后的图像风格迁移模型是根据第一方面任一项所述的图像风格迁移模型的训练方法所得到的；以及
获得所述目标图像对应的风格图像。
第三方面，本公开提供了一种图像风格迁移模型的训练装置，包括：
第一获取模块，用于对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
第一处理模块，用于对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
训练模块，用于利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
第四方面，本公开提供了一种终端，包括：
获取模块，用于获取目标图像；
处理模块，用于将所述目标图像输入至训练后的图像风格迁移模型中，以进行图像的风格迁移处理，所述训练后的图像风格迁移模型是根据第一方面任一项所述的图像风格迁移模型的训练方法所得到的；
所述获取模块，还用于获得所述目标图像对应的风格图像。
第五方面，本公开实施例提供一种电子设备，包括：至少一个处理器和存储器；
所述存储器存储计算机执行指令；
所述至少一个处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如上述第一方面以及第一方面各种可能的涉及所述的图像风格迁移模型的训练方法，和/或，执行如上述第二方面以及第二方面各种可能的涉及所述的图像风格迁移方法。
第六方面，本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上述第一方面以及第一方面各种可能的涉及所述的图像风格迁移模型的训练方法，和/或，实现如上述第二方面以及第二方面各种可能的涉及所述的图像风格迁移方法。
第七方面，本公开实施例提供一种计算机程序产品，包括计算机指令，所述计算机指令被处理器执行时，实现如上述第一方面以及第一方面各种可能的涉及所述的图像风格迁移模型的训练方法，和/或，实现如上述第二方面以及第二方面各种可能的涉及所述的图像风格迁移方法。
第八方面，本公开实施例提供一种计算机程序，所述计算机程序被处理器执行时，实现如上述第一方面以及第一方面各种可能的涉及所述的图像风格迁移模型的训练方法，和/或，实现如上述第二方面以及第二方面各种可能的涉及所述的图像风格迁移方法。
本公开实施例提供的图像风格迁移模型的训练方法、图像风格迁移方法及装置，由于利用真实样本图像和其对应的风格样本图像，对待训练的图像风格迁移模型进行训练，并利用风格样本图像的颜色聚类图以及线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。由于在对图像风格迁移模型进行训练时，不仅利用真实样本图像和风格样本图像确定图像在图像域上映射关系，还利用颜色聚类图以及线条模糊 图对图像在图像块面感和线条感上进行了监督和表征，从而使得训练完毕的图像风格迁移模型所输出的风格图像有着较佳的块面感和线条感，风格迁移效果得到了良好提升。
附图说明
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开所基于的一种网络架构的示意图；
图2为本公开实施例提供的一种图像风格迁移模型的训练方法的流程示意图；
图3为本公开实施例提供的一种对图像风格迁移模型进行训练时的数据流示意图；
图4为本公开实施例提供的一种对风格图像生成模型进行训练时的数据流示意图；
图5为本公开实施例提供的一种图像风格迁移方法的流程示意图；
图6为本公开实施例提供的图像风格迁移模型的训练装置的结构框图；
图7为本公开实施例提供的终端的结构框图；
图8为本公开实施例提供的电子设备的硬件结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
近年来，GAN网络开始被越来越广泛地应用在计算机视觉研究工作中，图像风格迁移正是将GAN网络运用到计算机视觉领域的一种实际应用。
利用现有的图像风格迁移技术，将目标图像迁移至风格样本图像风格的图像的迁移过程中，一般是利用图像域之间的映射变化实现的。
具体来说，在实现图像风格迁移时，可利用训练完毕的风格迁移模型将风格图像中的风格特征提取出来，并与目标图像进行内容混合以对目标图像进行重建，得到具有风格特征的目标图像。
在现有技术中，为了实现对风格迁移模型的训练，一般会采集大量的原始图像以及具有特定风格的风格图像，然后利用这些原始图像以及风格图像对生成对抗模型进行训练，以使得训练后的生成对抗模型可对目标图像进行处理，得到与风格图像风格相同的目标图像对应的风格图像，从而完成对于目标图像的风格迁移处理。
但是，利用这样的方式获得的风格的目标图像只是单纯对于图像域的色彩风格进行了映射，而对于风格样本图像中的图像块面感和线条感却没有更多的表征，使得获得的风格的目标图像的块面感和线条感不足，风格效果不佳。
针对这样的问题，发明人首先想到，在利用现有技术的目标图像和特定风格的风格图像对风格迁移模型进行训练的基础上，还可增加对模型在颜色块面特征和线条特征上的监督，即利用颜色聚类图以及线条模糊图对图像在图像块面感和线条感上进行监督和表征，以使得风格迁移模型所生成的风格图像呈现更强的块面感和线条感，从而使得训练完毕的图像风格迁移模型所输出的风格图像有着更佳的块面感和线条感，风格迁移效果得到了良好提升。
参考图1，图1为本公开所基于的一种网络架构的示意图，该图1所示网络架构具体可包括至少一个终端1以及服务器2。
其中，终端1具体可以包括用户手机、智能家居设备、平板电脑、可穿戴设备等硬件设备。
服务器2可具体为设置在云端的服务器或者服务器集群，其服务器或服务器集群中集成或安装有用于训练图像风格迁移模型的图像风格迁移模型的训练装置，以使该图像风格迁移模型的训练装置为用于执行本公开图像风格迁移方法的硬件或软件。
其中，终端1中可以布设有训练完毕的图像风格迁移模型，通过该图像风格迁移模型，终端1可将用户选中的目标图像进行风格迁移处理，以得到具有特定风格的风格图像。
在使用时，通过网络，服务器2可将与训练完毕的图像风格迁移模型相关的配置文件以及配置参数下发至终端1，以使终端1可调用预存在本地或服务云端的训练完毕的图像风格迁移模型，进行对图像的风格迁移处理和风格图像的显示。
终端1上部署有通过服务器2训练得到的图像风格迁移模型，以使得终端1在为用户提供基本应用功能的基础上，还可利用图像风格迁移模型实现对于用户通过终端1上传或者拍摄的目标图像进行风格迁移处理和展示。
此外，训练图像风格迁移模型的图像风格迁移模型的训练装置也可以部署于终端1或者其他终端中，用于训练图像风格迁移模型，本公开对训练图像风格迁移模型的设备不进行限制。
下面将针对本公开提供的图像风格迁移方法进行进一步说明：
第一方面，图2为本公开实施例提供的一种图像风格迁移模型的训练方法的流程示意图。参考图2，本公开实施例提供的图像风格迁移模型的训练方法，包括：
步骤101、对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
步骤102、对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
步骤103、利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
需要说明的是，本实施例提供的图像风格迁移模型的训练方法的执行主体为前述的图像风格迁移模型的训练装置。
为了能够使得训练后的图像风格迁移模型具有质量较佳的输出图像，在本实施方式中，对于图像风格迁移模型将采用配对训练图像，以进行训练。
具体的，首先，图像风格迁移模型训练装置会获取真实样本图像，以作为步骤101中的真实样本图像。
为了实现对于图像风格迁移模型的配对训练，图像风格迁移模型训练装置在获得真实样本图像之后，还可以对真实样本图像进行风格预处理，以得到预设风格的风格样本图像，其中的预设风格可为前述的特定风格(例如，卡通风格)。
与现有技术不同是的，如前所述的，为了使得训练后的图像风格迁移模型对于特定风格(例如，卡通风格)具有更佳的图像风格迁移能力，在本公开提供的实施方式中，会进一步利用风格样本图像的颜色聚类图以及风格样本图像的线条模糊图对模型在图像块面感和线条感上进行监督训练。
具体的，在训练前，图像风格迁移模型训练装置还可以对风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图。
对于风格样本图像进行线条模糊化处理具体可为对风格样本图像进行基于高斯滤波的边缘平滑处理，从而将风格样本图像中的线条感去除，使得像素数据平滑，得到线条模糊图。
一般来说，对该图进行线条模糊化处理，可使得风格样本图像中的线条模糊，呈现马赛克效果。
对于风格样本图像进行颜色聚类处理具体可包括：对风格样本图像进行图像分割处理和合并处理，以根据风格样本图像的图内容将所述待风格样本图像划分为多个区域，其中，每一区域为风格样本图像中的具有相同图内容的图像区域；然后，基于每一区域中的颜色，对风格样本图像的各区域分别进行颜色均值处理。
具体来说，对于颜色聚类处理可基于Felzenszwalb算法实现，Felzenszwalb算法是一种基于图的贪心聚类算法，通过对于图像中每一像素的相似度进行分析，以对像素进行聚类起到图像分割和合并的效果，得到基于图像的图内容的若干区域。然后，图像风格迁移模型训练装置可以针对每一区域进行颜色均值处理，即以区域为单位，将属于该区域的全部像素的颜色值进行均值处理得到颜色均值，并用该颜色均值取代该区域的每个像素的原颜色值，从而使得图像呈现块面感。
通过对该图进行颜色聚类处理，可将该风格样本图像中的颜色聚集感更强。
当完成上述对风格样本图像的处理之后，可以利用真实样本图像和其对应的风格样本图像，对待训练的图像风格迁移模型进行训练，并利用风格样本图像的颜色聚类图以及线条模糊图对待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
具体的，图像风格迁移模型具体可包括生成对抗网络，对图像风格迁移模型的训练一般是多次循环训练的过程，即，真实样本图像和其对应的风格样本图像、风格样本图像的颜色聚类图以及线条模糊图会多次重复输入至图像风格迁移模型中，以对图像风格迁移模型进行多次训练。根据实际需求，本公开对训练的次数是不进行限制的。
图像风格迁移模型可以包括第一生成对抗网络，第一生成对抗网络包括第一生成器和第一判别器。利用真实样本图像和其对应的风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，包括：将所述真实样本图像输入至所述第一生 成器，得到所述真实样本图像的中间图；将所述真实样本图像的中间图像、所述风格样本图像的颜色聚类图、所述风格样本图像的线条模糊图以及所述真实样本图像对应的风格样本图像，输入至第一判别器，得到第一判别结果；根据所述第一述判别结果对所述第一生成器进行参数调整，并返回所述将所述真实样本图像输入至所述第一生成器的步骤，直至所述图像风格迁移模型收敛。
具体的，图3为本公开实施例提供的一种对图像风格迁移模型进行训练时的数据流示意图。参考图3，图像风格迁移模型中包括第一生成器和第一判别器。
其中，第一生成器用于基于权重参数对输入至第一生成器的真实样本图像进行处理，并输出真实样本图像的中间图像。
在训练的初期，第一生成器接收真实样本图像并输出一中间图像，然后，该中间图像以及真实样本图像对应的风格样本图像被同时输入至第一判别器中，以供第一判别器对该两种图像进行判别处理。即，利用所述第一判别器对所述真实样本图像的中间图像，以及所述真实样本图像对应的风格样本图像进行判别处理，得到处理结果。
由于在训练的初期，第一判别器能够轻易的判别出真实样本图像对应的风格样本图像为真图，而中间图像为伪图；此时，第一判别器将该判别结果反馈给第一生成器，以供第一生成器进行参数调整，并基于下一真实样本图像进行下一中间图像的生成。
可知的是，对于模型的训练目的在于，使得第一生成器输出的中间图像能够迷惑第一判别器，使第一判别器对中间图的判别结果与对真实样本图像对应的风格样本图像的判别结果一致，即当中间图像被输入至第一判别器时，第一判别器可以将该中间图像判别为真图，此时第一生成器和第一判别器完成训练。
在上述训练的同时，为了实现对于图像在块面感和线条感上的监督，还需要利用前述得到的风格样本图像的颜色聚类图和风格样本图像的颜色聚类图对于第一生成器在中间图像的高层特征进行监督。
具体的，可确定所述中间图像和所述风格样本图像的颜色聚类图的颜色特征差值，以及确定所述中间图像和所述风格样本图像的线条模糊图的线条特征差值；然后将该两种特征差值反馈给第一生成器，以使得第一生成器进行调参，并使得调参之后所生成的中间图与其相应的颜色聚类图在颜色特征差值小于调参之前的颜色特征差值，同时使得调参之后所生成的中间图与其相应的线条模糊图在线条特征差值小于调参之前的线条特征差。
进一步的，该监督可基于损失函数实现，即基于VGG19-bn的损失函数以对判别结果进行输出。其中，当第一生成器生成中间图像之后，第一判别器利用其中的VGG19-bn分别提取真实样本图像的中间图像以及风格样本图像的颜色聚类图的高层特征，分别得到相应的颜色特征损失值，计算得到颜色特征差值；同步的或异步的，第一判别器还利用VGG19-bn分别提取真实样本图像的中间图像以及风格样本图像的线条模糊图的高层特征，分别得到相应的线条特征损失值，计算得到线条特征差值。
换句话说，图像风格迁移模型训练装置根据所述颜色特征差值、所述线条特征差值、以及所述处理结果，对所述第一生成器进行参数调整，使得第一生成器得到优化。由于优化过程是重复进行的，通过这样的方式，会使得图像风格迁移模型所生成的中间图像 逐步具有特定风格，例如，卡通风格，同时，由于对两个特征参数(颜色特征差值和线条特征损失值)的监督，生成的中间图像的块面感和线条感也将逐步得到增强。
通过采用上述方式对图像风格迁移模型进行训练，能够使得训练完毕的图像风格迁移模型能在利用反复调参后的第一生成器对目标图像进行处理，生成相应的风格图像，实现对图像的风格迁移处理，且风格迁移处理得到的风格样本图像有着更佳的块面感和线条感，风格迁移效果更佳。
在上述实施方式中，用于对图像风格迁移模型进行训练的风格样本图像的图像质量在一定程度上决定了模型的训练质量。为了进一步提高模型在块面感和线条感方向的训练效果，可选实施方式中，还可对于在真实样本图像生成风格样本图像的过程中加重对于块面感和线条感方向的监督，以使得风格样本图像是具有一定块面感和线条感的图，进一步有利于模型向着块面感和线条感的训练方向靠近。
具体的，可利用训练好的风格图像生成模型对所述真实样本图像进行处理，得到真实样本图像对应的风格样本图像。
进一步的，所述风格图像生成模型为包括第二生成器和第二判别器的生成对抗网络；
相应的，对获得的真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像，包括：
获得训练样本图像；获取参考风格样本图像；对所述参考风格样本图像进行线条模糊化处理，得到所述参考风格样本图像的模糊样本图；将所述训练样本图像输入至所述第二生成器，得到所述训练样本图像的中间样本图像；对所述中间样本图像进行颜色聚类处理，得到所述中间样本图像的颜色聚类样本图；利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图以及所述中间样本图像的颜色聚类样本图对所述第二判别器进行训练，得到第二判别结果；根据所述第二判别结果对所述第二生成器进行参数调整，并返回将所述训练样本图像输入至所述第二生成器的步骤直至所述风格图像生成模型收敛真实样本图像。
如前所述的，为了使得训练后的风格图像生成模型能够在块面感和线条感上有着更好的图像处理效果，在本公开提供的实施方式中将构建风格图像生成模型以用于生成真实样本图像对应的风格样本图像。
图4为本公开实施例提供的一种对风格图像生成模型进行训练时的数据流示意图。参考图4，风格图像生成模型中包括第二生成器和第二判别器。
首先，可以对风格图像生成模型进行训练。具体的，可以先获得训练样本图像，以及参考风格样本图像。
其中，参考风格样本图像包括具有预设风格的图像，如卡通风格的图像。其中参考风格样本图像可以随机且大量的。
训练样本图像则可以包括风景图像、静物图像等的写实图像。
值得说明的是，上述获得的训练样本图像和参考风格样本图像之间的内容不存在关联，即其二者并不为配对关系。
然后，图像风格迁移模型的训练装置可以对参考风格样本图像进行线条模糊化处理，得到所述参考风格样本图像的模糊样本图。其中，对于参考风格样本图像进行线条模糊化处理具体可包括对参考风格样本图像进行基于高斯滤波的边缘平滑处理，从而使得参 考风格样本图像中的线条感去除，使得像素数据平滑，得到线条模糊图，其效果可类似于前述实施方式中对风格样本图像的处理效果。
再后，如图4所示的，将训练样本图像输入至所述第二生成器，得到中间样本图像。
然后，对所述中间样本图像进行颜色聚类处理，得到所述中间样本图像的颜色聚类样本图。其中，对于中间样本图像的颜色聚类处理，与前述方式类似，即对中间样本图像进行图像分割处理和合并处理，以根据中间样本图像的图内容将所述待处理图划分为多个区域，其中，每一区域为待处理图像中的具有相同图内容的图像区域；然后，基于每一区域中的颜色，对中间样本图像的各区域分别进行颜色均值处理，其效果可类似于前述对风格样本图像的处理效果。
与对图像风格迁移模型进行基于配对图的训练不同的是，对于本实施方式中的风格图像生成模型的训练是基于非配对图实现的。在训练时，继续参考图4，要利用所述第二判别器对利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图进行判别处理，得到基于线条风格特征的判别结果。
具体的，在训练的初始阶段，第二判别器能够识别出参考风格样本图像的模糊样本图为伪图，参考风格样本图像为真图，而第二生成器所生成的中间样本图像将被判别为伪图，这样的判别结果将被反馈至第二生成器，以使第二生成器的权重参数得到优化。
随着训练的推进，第二生成器将被不断的优化，直至第二生成器所生成的中间样本图像将被第二判别器判别为真图，此处，得到第二生成器可用于生成具有一定线条感的风格样本图像。
此外，为了使得第二生成器生成的风格样本图像具有块面感，本实施方式中，还可以利用前述获得的中间样本图像的颜色聚类样本图，对模型进行监督。
即，利用第二判别器中的VGG19-bn分别对中间样本图像中的颜色特征以及对中间样本图像的颜色聚类样本图中的颜色特征进行提取，并计算二者差值。根据该差值对第二生成器进行调参，以使调参后的第二生成器能够生成使得颜色特征差值最小的中间样本图像。即，利用所述第二判别器对利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图进行判别处理，得到基于线条风格特征的判别结果；以及确定所述中间样本图像和所述中间样本图像的颜色聚类样本图在颜色风格特征的特征差值；根据基于线条风格特征的判别结果和所述颜色风格特征的差值，对所述第二生成器进行参数调整。
通过采用上述方式对风格图像生成模型进行训练，能够使得训练完毕的风格图像生成模型在利用反复调参后的第二生成器对真实样本图像进行处理，生成相应的风格样本图像，且风格样本图像是具有一定线条感和块面感的，从而使得利用这样的风格样本图像对图像风格迁移模型进行训练，能够得到训练效果较好的图像风格迁移模型。
本公开实施例提供的图像风格迁移方法，由于利用真实样本图像和其对应的风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。由于在对图像风格迁移模型进行训练时不仅利用真实样本图像和风格样本图像确定图像在图像域上映射关系，还利用颜色聚类图以及线条模糊图对图像 在图像块面感和线条感上进行了监督和表征，从而使得训练完毕的图像风格迁移模型所输出的风格样本图像有着较佳的块面感和线条感，风格迁移效果得到了良好提升。
在上述实施方式的基础上，图5为本公开实施例提供的一种图像风格迁移方法的流程示意图，如图5所示的，该方法包括：
步骤201、获取目标图像；
步骤202、将所述目标图像输入至训练后的图像风格迁移模型中，以进行目标图像的风格迁移处理；
步骤203、获得所述目标图像对应的风格图像。
需要说明的是，本实施例的提供的图像风格迁移方法的执行主体为前述的终端，在本公开的一些实施例中，终端上将部署有通过前述实施例训练得到的图像风格迁移模型。在部署时，该图像风格迁移模型的配置文件和相关数据可以由服务器下发至终端，以在终端完成安装和加载，并进行使用。
用户可通过将需要进行风格迁移处理的目标图像，通过应用APP的端口发送至终端的模型，以供模型对目标图像进行风格迁移处理，获得处理后的目标图像的风格图像。
其中，本公开实施方式中所涉及的图像风格迁移模型可以是基于前述任一实施方式训练所获得的，本实施方式对其模型的训练过程不再进行赘述。
可选实施方式中，为了进一步提高终端在处理目标图像时的处理效率，减少终端内存损耗，在终端处理目标图像时可采用分块处理的方式。
具体的，将所述目标图像输入至训练后的图像风格迁移模型中，以进行图像的风格迁移处理，包括：对所述目标图像进行分块处理，以得到多个目标图块；其中，各目标图块的组合构成所述目标图像，且相邻的目标图块之间具有重叠区域；将所述各目标图块分别输入至所述训练后的图像风格迁移模型中进行处理，以得到所述各目标图块对应的风格迁移图块；以及拼接所述各风格迁移图块获得所述目标图像对应的风格图像。
在终端获得目标图像之后，首先，对所述待处理的目标图像进行分割处理，以得到若干目标图块；其中，各目标图块的组合构成所述目标图像，且相邻的目标图块之间具有重叠区域。例如，将目标图像A进行分割处理后，得到目标图块A1、目标图块A2和目标图块A3，其中，目标图块A1和目标图块A2之间具有一定的重叠区域，目标图块A2和目标图块A3之间具有一定的重叠区域，而将各目标图块组合起来将覆盖整个目标图像A。
然后，终端可以利用所述训练后的图像风格迁移模型对所述各目标图块分别进行处理，得到各目标图块对应的风格图像块。例如，终端将对目标图块A1、目标图块A2和目标图块A3分别进行风格迁移处理，以得到相应的风格图像块B1、风格图像块B2和风格图像块B3。
最后，拼接所述各风格图像块获得所述目标图对应的风格图像。例如，终端将拼接风格图像块B1、风格图像块B2和风格图像块B3得到风格图像B。
在上述拼接过程中，对于相邻的风格图像块来说，重叠区域的拼接将采用线性插值处理，例如，由于目标图块A1和A2之间具有一定重叠区域，相应的，风格图像块B1和B2之间也具有相同的重叠区域，为了使得输出的风格图像完整和流畅，可将该重叠区 域在风格图像块B1上的像素值和该重叠区域在风格图像块B2上的像素值进行插值处理，得到重叠区域的像素值。通过这样的方式能够使得输出的风格图像更为流畅，效果更好。
本公开实施例提供的图像风格迁移方法，利用训练后的图像风格迁移模型输出的目标图像的风格图像的鲁棒性较高，风格效果较佳；此外，终端可直接使用由服务器进行训练获得的训练后的图像风格迁移模型，由于终端无需承担图像风格迁移模型的训练处理，部署在终端上的图像风格迁移模型的规模可以得到有效控制，也使得终端在执行处理时的负载降低。
对应于上文实施例的图像风格迁移方法，图6为本公开实施例提供的图像风格迁移模型的训练装置的结构框图。为了便于说明，仅示出了与本公开实施例相关的部分。参照图6，所述图像风格迁移模型的训练装置包括：
获取模块11，用于对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
处理模块12，用于对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
训练模块13，用于利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
可选的，所述图像风格迁移模型包括第一生成对抗网络，所述第一生成对抗网络包括第一生成器和第一判别器；所述训练模块13，用于将所述真实样本图像输入至所述第一生成器，得到所述真实样本图像的中间图；将所述真实样本图像的中间图像、所述风格样本图像的颜色聚类图、所述风格样本图像的线条模糊图以及所述真实样本图像对应的风格样本图像，输入至第一判别器，得到第一判别结果；根据所述第一述判别结果对所述第一生成器进行参数调整，并返回所述将所述真实样本图像输入至所述第一生成器的步骤，直至所述图像风格迁移模型收敛。
可选的，所述训练模块13，用于利用所述第一判别器对所述真实样本图像的中间图像，以及所述真实样本图像对应的风格样本图像进行判别处理，得到处理结果；确定所述中间图像和所述风格样本图像的颜色聚类图的颜色特征差值；以及确定所述中间图像和所述风格样本图像的线条模糊图的线条特征差值；根据所述颜色特征差值、所述线条特征差值、以及所述处理结果，对所述第一生成器进行参数调整。
可选的，所述获取模块11，用于利用训练后的风格图像生成模型对所述真实样本图像进行处理，得到所述真实样本图像对应的风格样本图像。
所述获取模块11用于获得训练样本图像；获取参考风格样本图像；对所述参考风格样本图像进行线条模糊化处理，得到所述参考风格样本图像的模糊样本图；将所述训练样本图像输入至所述第二生成器，得到所述训练样本图像的中间样本图像；对所述中间样本图像进行颜色聚类处理，得到所述中间样本图像的颜色聚类样本图；利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图以及所述中间样本图像的颜色聚类样本图对所述第二判别器进行训练，得到第二判别结果；根据所述 第二判别结果对所述第二生成器进行参数调整，并返回将所述训练样本图像输入至所述第二生成器的步骤直至所述风格图像生成模型收敛。
可选的，所述获取模块11，用于：利用所述第二判别器对利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图进行判别处理，得到基于线条风格特征的判别结果；以及确定所述中间样本图像和所述中间样本图像的颜色聚类样本图在颜色风格特征的特征差值；根据基于线条风格特征的判别结果和所述颜色风格特征的差值，对所述第二生成器进行参数调整。
可选的，所述处理模块12在执行颜色聚类处理时，用于对待处理图像进行图像分割处理和合并处理，以根据所述待处理图像的图内容将所述待处理图像划分为多个区域，其中，每一区域为待处理图像中的具有相同图内容的图像区域；基于每一区域中的颜色，对所述待处理图像的各区域分别进行颜色均值处理，其中，所述待处理图像包括所述风格样本图像和/或中间样本图像。
可选的，所述处理模块12在执行线条模糊化处理时，用于对所述待处理图像进行基于高斯滤波的边缘平滑处理，其中，所述待处理图像包括所述风格样本图像和/或参考风格样本图像。
本公开实施例提供的图像风格迁移模型的训练装置，由于利用真实样本图像和其对应的风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。由于在对图像风格迁移模型进行训练时不仅利用真实样本图像和风格样本图像确定图像在图像域上映射关系，还利用颜色聚类图以及线条模糊图对图像在图像块面感和线条感上进行了监督和表征，从而使得训练完毕的图像风格迁移模型所输出的风格样本图像有着较佳的块面感和线条感，风格迁移效果得到了良好提升。
对应于上文实施例的图像风格迁移方法，图7为本公开实施例提供的终端的结构框图。为了便于说明，仅示出了与本公开实施例相关的部分。参照图7，所述终端包括：
获取模块21，用于获取目标图像；
处理模块22，用于将所述目标图像输入至训练后的图像风格迁移模型中，以进行目标图像的风格迁移处理；其中，所述训练后的图像风格迁移模型是根据第一方面任一项所述的图像风格迁移模型的训练方法所得到的；
所述获取模块21，还用于获得所述目标图像对应的风格图像。
可选的，所述处理模块22，具体用于对所述目标图像进行分块处理，以得到多个目标图块；其中，各目标图块的组合构成所述目标图像，且相邻的目标图块之间具有重叠区域；将所述各目标图块分别输入至所述训练后的图像风格迁移模型中进行处理，以得到所述各目标图块对应的风格迁移图块；
所述获取模块21，用于拼接所述各风格迁移图块获得所述目标图对应的风格图像。
可选的，处理模块22，还用于：采用线性插值处理所述重叠区域。
本实施例提供的电子设备，可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，本实施例此处不再赘述。
参考图8，其示出了适于用来实现本公开实施例的电子设备900的结构示意图，该电子设备900可以为终端设备或媒体库。其中，终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，简称PDA)、平板电脑(Portable Android Device，简称PAD)、便携式多媒体播放器(Portable Media Player，简称PMP)、车载终端(例如车载导航终端)、可穿戴电子设备等等的移动终端以及诸如数字TV、台式计算机、智能家居设备等等的固定终端。图8示出的电子设备仅仅是一个实施例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，电子设备900可以包括用于执行上述方法的处理器901(例如中央处理器、图形处理器等)，其可以根据存储在只读存储器(Read Only Memory，简称ROM)902中的程序或者从存储装置908加载到随机访问存储器(Random Access Memory，简称RAM)903中的程序而执行各种适当的动作和处理。在RAM 903中，还存储有电子设备900操作所需的各种程序和数据。处理器901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(Input/Output，简称I/O)接口905也连接至总线904。
通常，以下装置可以连接至I/O接口905：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置906；包括例如液晶屏幕(Liquid Crystal Display，简称LCD)、扬声器、振动器等的输出装置907；包括例如磁带、硬盘等的存储装置908；以及通信装置909。通信装置909可以允许电子设备900与其他设备进行无线或有线通信以交换数据。虽然图8示出了具有各种装置的电子设备900，但是应理解的是，并不要求实施或具备所有示出的装置，可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，所述计算机程序包含用于执行根据本公开实施例所述的各流程图所示的方法的程序代码。在这样的实施例中，所述计算机程序可以通过通信装置909从网络上被下载和安装，或者从存储装置908被安装，或者从ROM 902被安装。在所述计算机程序被处理器901执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(Random Access Memory，简称RAM)、只读存储器(Read-Only Memory，简称ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，简称EPROM)、闪存、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，简称CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括 但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency，简称RF)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备执行上述实施例所示的方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或媒体库上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，简称LAN)或广域网(Wide Area Network，简称WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(Field-Programmable Gate Array，简称FPGA)、专用集成电路(Application Specific Integrated Circuit，简称ASIC)、专用标准产品(Application Specific Standard Product，简称ASSP)、片上系统(System On Chip，简称SOC)、复杂可编程逻辑设备(Complex programmable logic device，简称CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限 于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体实施例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(Random Access Memory，简称RAM)、只读存储器(Read-Only Memory，简称ROM)、可擦除可编程只读存储器(Erasable Programmable Read Only Memory，简称EPROM)、快闪存储器、光纤、便捷式紧凑盘只读存储器(Compact Disc Read-Only Memory，简称CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
以下是本公开的一些实施例。
第一方面，根据本公开的一个或多个实施例，一种图像风格迁移模型的训练方法，包括：
对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
可选的，所述图像风格迁移模型包括第一生成对抗网络，所述第一生成对抗网络包括第一生成器和第一判别器；
所述利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，包括：
将所述真实样本图像输入至所述第一生成器，得到所述真实样本图像的中间图像；
将所述真实样本图像的中间图像、所述风格样本图像的颜色聚类图、所述风格样本图像的线条模糊图以及所述真实样本图像对应的风格样本图像，输入至第一判别器，得到第一判别结果；
根据所述第一述判别结果对所述第一生成器进行参数调整，并返回所述将所述真实样本图像输入至所述第一生成器的步骤，直至所述图像风格迁移模型收敛。
可选的，所述将所述真实样本图像的中间图像、所述风格样本图像的颜色聚类图、所述风格样本图像的线条模糊图以及所述真实样本图像对应的风格样本图像，输入至第一判别器，得到判别结果，包括：
利用所述第一判别器对所述真实样本图像的中间图像，以及所述真实样本图像对应的风格样本图像进行判别处理，得到处理结果；
确定所述中间图像和所述风格样本图像的颜色聚类图的颜色特征差值；以及
确定所述中间图像和所述风格样本图像的线条模糊图的线条特征差值；
相应的，所述根据所述判别结果对所述第一生成器进行参数调整，包括：
根据所述颜色特征差值、所述线条特征差值、以及所述处理结果，对所述第一生成器进行参数调整。
可选的，所述对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像，包括：
利用训练后的风格图像生成模型对所述真实样本图像进行处理，得到所述真实样本图像对应的风格样本图像。
可选的，所述风格图像生成模型包括第二生成对抗网络，所述第二生成对抗网络包括第二生成器和第二判别器；
所述方法还包括：
获得训练样本图像；
获取参考风格样本图像；
对所述参考风格样本图像进行线条模糊化处理，得到所述参考风格样本图像的模糊样本图；
将所述训练样本图像输入至所述第二生成器，得到所述训练样本图像的中间样本图像；
对所述中间样本图像进行颜色聚类处理，得到所述中间样本图像的颜色聚类样本图；
利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图以及所述中间样本图像的颜色聚类样本图对所述第二判别器进行训练，得到第二判别结果；
根据所述第二判别结果对所述第二生成器进行参数调整，并返回将所述训练样本图像输入至所述第二生成器的步骤直至所述风格图像生成模型收敛。
可选的，所述利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图以及所述中间样本图像的颜色聚类样本图对所述第二判别器进行训练，得到第二判别结果，包括：
利用所述第二判别器对所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图进行判别处理，得到基于线条风格特征的判别结果；以及
确定所述中间样本图像和所述中间样本图像的颜色聚类样本图在颜色风格特征的特征差值；
相应的，所述根据所述判别结果对所述第二生成器进行参数调整，包括：
根据基于线条风格特征的判别结果和所述颜色风格特征的差值，对所述第二生成器进行参数调整。
可选的，所述颜色聚类处理，包括：
对待处理图像进行图像分割处理和合并处理，以根据所述待处理图像的图内容将所述待处理图像划分为多个区域，其中，每一区域为待处理图像中的具有相同图内容的图像区域；
基于每一区域中的颜色，对所述待处理图像的各区域分别进行颜色均值处理，
其中，所述待处理图像包括所述风格样本图像和/或中间样本图像。
可选的，所述线条模糊化处理，包括：
对待处理图像进行基于高斯滤波的边缘平滑处理，
其中，所述待处理图像包括所述风格样本图像和/或参考风格样本图像。
第二方面，根据本公开的一个或多个实施例，一种图像风格迁移方法，包括：
获取目标图像；
将所述目标图像输入至训练后的图像风格迁移模型中，以进行目标图像的风格迁移处理，所述训练后的图像风格迁移模型是根据前述第一方面任一项所述的图像风格迁移模型的训练方法所得到的；以及
获得所述目标图像对应的风格图像。
可选的，所述将所述目标图像输入至训练后的图像风格迁移模型中，以进行图像的风格迁移处理，包括：
对所述目标图像进行分块处理，以得到多个目标图块；其中，各目标图块的组合构成所述目标图像，且相邻的目标图块之间具有重叠区域；
将所述各目标图块分别输入至所述训练后的图像风格迁移模型中进行处理，以得到所述各目标图块对应的风格迁移图块；以及
拼接所述各风格迁移图块获得所述目标图对应的风格图像。
可选的，还包括：
采用线性插值处理所述重叠区域。
第三方面，根据本公开的一个或多个实施例，一种图像风格迁移模型的训练装置，包括：
获取模块，用于对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
处理模块，用于对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
训练模块，用于利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
可选的，所述图像风格迁移模型包括第一生成对抗网络，所述第一生成对抗网络包括第一生成器和第一判别器；
所述训练模块用于将所述真实样本图像输入至所述第一生成器，得到所述真实样本图像的中间图像；将所述真实样本图像的中间图像、所述风格样本图像的颜色聚类图、所述风格样本图像的线条模糊图以及所述真实样本图像对应的风格样本图像，输入至第一判别器，得到第一判别结果；根据所述第一述判别结果对所述第一生成器进行参数调整，并返回所述将所述真实样本图像输入至所述第一生成器的步骤，直至所述图像风格迁移模型收敛。
可选的，所述训练模块，具体用于利用所述第一判别器对所述真实样本图像的中间图像，以及所述真实样本图像对应的风格样本图像进行判别处理，得到处理结果；确定所述中间图像和所述风格样本图像的颜色聚类图的颜色特征差值；以及确定所述中间图像和所述风格样本图像的线条模糊图的线条特征差值；根据所述颜色特征差值、所述线条特征差值、以及所述处理结果，对所述第一生成器进行参数调整。
可选的，所述获取模块，用于利用训练后的风格图像生成模型对所述真实样本图像进行处理，得到所述真实样本图像对应的风格样本图像。
所述获取模块用于获得训练样本图像；获取参考风格样本图像；对所述参考风格样本图像进行线条模糊化处理，得到所述参考风格样本图像的模糊样本图；将所述训练样本图像输入至所述第二生成器，得到所述训练样本图像的中间样本图像；对所述中间样本图像进行颜色聚类处理，得到所述中间样本图像的颜色聚类样本图；利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图以及所述中间样本图像的颜色聚类样本图对所述第二判别器进行训练，得到第二判别结果；根据所述第二判别结果对所述第二生成器进行参数调整，并返回将所述训练样本图像输入至所述第二生成器的步骤直至所述风格图像生成模型收敛。
可选的，所述获取模块，用于：利用所述第二判别器对利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图进行判别处理，得到基于线条风格特征的判别结果；以及确定所述中间样本图像和所述中间样本图像的颜色聚类样本图在颜色风格特征的特征差值；根据基于线条风格特征的判别结果和所述颜色风格特征的差值，对所述第二生成器进行参数调整。
可选的，所述处理模块在执行颜色聚类处理时，用于对待处理图像进行图像分割处理和合并处理，以根据所述待处理图像的图内容将所述待处理图像划分为多个区域，其中，每一区域为待处理图像中的具有相同图内容的图像区域；基于每一区域中的颜色，对所述待处理图像的各区域分别进行颜色均值处理，其中，所述待处理图像包括所述风格样本图像和/或中间样本图像。
可选的，所述处理模块在执行线条模糊化处理时，用于对所述待处理图像进行基于高斯滤波的边缘平滑处理，其中，所述待处理图像包括所述风格样本图像和/或参考风格样本图像。
第四方面，根据本公开的一个或多个实施例，一种终端，包括：
获取模块，用于获取目标图像；
处理模块，用于将所述目标图像输入至训练后的图像风格迁移模型中，以进行目标图像的风格迁移处理；其中，所述训练后的图像风格迁移模型是根据前述第一方面任一项所述的图像风格迁移模型的训练方法所得到的；
所述获取模块还用于获得所述目标图像对应的风格图像。
可选的，所述处理模块，具体用于对所述目标图像进行分块处理，以得到多个目标图块；其中，各目标图块的组合构成所述目标图像，且相邻的目标图块之间具有重叠区域；将所述各目标图块分别输入至所述训练后的图像风格迁移模型中进行处理，以得到所述各目标图块对应的风格迁移图块；
所述获取模块用于拼接所述各风格迁移图块获得所述目标图对应的风格图像。
可选的，处理模块还用于：采用线性插值处理所述重叠区域。
第五方面，根据本公开的一个或多个实施例，一种电子设备，包括：至少一个处理器和存储器；
所述存储器存储计算机执行指令；
所述至少一个处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如前述任一项所述的方法。
第六方面，根据本公开的一个或多个实施例，一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如前述任一项所述的方法。
第七方面，根据本公开的一个或多个实施例，一种计算机程序产品，包括计算机指令，所述计算机指令被处理器执行时，实现如前述任一项所述的方法。
第八方面，根据本公开的一个或多个实施例，一种计算机程序，所述计算机程序被处理器执行时，实现如前述任一项所述的方法。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的实施例形式。

Claims (17)

  1. 一种图像风格迁移模型的训练方法，其特征在于，包括：
    对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
    对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
    利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
  2. 根据权利要求1所述的图像风格迁移模型的训练方法，其特征在于，所述图像风格迁移模型包括第一生成对抗网络，所述第一生成对抗网络包括第一生成器和第一判别器；
    所述利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，包括：
    将所述真实样本图像输入至所述第一生成器，得到所述真实样本图像的中间图像；
    将所述真实样本图像的中间图像、所述风格样本图像的颜色聚类图、所述风格样本图像的线条模糊图以及所述真实样本图像对应的风格样本图像，输入至第一判别器，得到第一判别结果；
    根据所述第一述判别结果对所述第一生成器进行参数调整，并返回所述将所述真实样本图像输入至所述第一生成器的步骤，直至所述图像风格迁移模型收敛。
  3. 根据权利要求2所述的图像风格迁移模型的训练方法，其特征在于，所述将所述真实样本图像的中间图像、所述风格样本图像的颜色聚类图、所述风格样本图像的线条模糊图以及所述真实样本图像对应的风格样本图像，输入至第一判别器，得到第一判别结果，包括：
    利用所述第一判别器对所述真实样本图像的中间图像，以及所述真实样本图像对应的风格样本图像进行判别处理，得到处理结果；
    确定所述中间图像和所述风格样本图像的颜色聚类图的颜色特征差值；以及
    确定所述中间图像和所述风格样本图像的线条模糊图的线条特征差值；
    相应的，所述根据所述第一判别结果对所述第一生成器进行参数调整，包括：
    根据所述颜色特征差值、所述线条特征差值、以及所述处理结果，对所述第一生成器进行参数调整。
  4. 根据权利要求1-3中任一项所述的图像风格迁移模型的训练方法，其特征在于，所述对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像，包括：
    利用训练后的风格图像生成模型对所述真实样本图像进行处理，得到所述真实样本图像对应的风格样本图像。
  5. 根据权利要求4所述的图像风格迁移模型的训练方法，其特征在于，所述风格图像生成模型包括第二生成对抗网络，所述第二生成对抗网络包括第二生成器和第二判别器；
    所述方法还包括：
    获得训练样本图像；
    获取参考风格样本图像；
    对所述参考风格样本图像进行线条模糊化处理，得到所述参考风格样本图像的模糊样本图；
    将所述训练样本图像输入至所述第二生成器，得到所述训练样本图像的中间样本图像；
    对所述中间样本图像进行颜色聚类处理，得到所述中间样本图像的颜色聚类样本图；
    利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图以及所述中间样本图像的颜色聚类样本图对所述第二判别器进行训练，得到第二判别结果；
    根据所述第二判别结果对所述第二生成器进行参数调整，并返回将所述训练样本图像输入至所述第二生成器的步骤直至所述风格图像生成模型收敛。
  6. 根据权利要求5所述的图像风格迁移模型的训练方法，其特征在于，所述利用所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图以及所述中间样本图像的颜色聚类样本图对所述第二判别器进行训练，得到第二判别结果，包括：
    利用所述第二判别器对所述中间样本图像、所述参考风格样本图像、所述参考风格样本图像的模糊样本图进行判别处理，得到基于线条风格特征的判别结果；以及
    确定所述中间样本图像和所述样本图的颜色聚类样本图在颜色风格特征的特征差值；
    相应的，所述根据所述第二判别结果对所述第二生成器进行参数调整，包括：
    根据基于线条风格特征的判别结果和所述颜色风格特征的差值，对所述第二生成器进行参数调整。
  7. 根据权利要求1-6中任一项所述的图像风格迁移模型的训练方法，其特征在于，所述颜色聚类处理，包括：
    对待处理图像进行图像分割处理和合并处理，以根据所述待处理图像的图内容将所述待处理图像划分为多个区域，其中，每一区域为待处理图像中的具有相同图内容的图像区域；
    基于每一区域中的颜色，对所述待处理图像的各区域分别进行颜色均值处理，
    其中，所述待处理图像包括所述风格样本图像和/或所述中间样本图像。
  8. 根据权利要求1-6中任一项所述的图像风格迁移模型的训练方法，其特征在于，所述线条模糊化处理，包括：
    对待处理图像进行基于高斯滤波的边缘平滑处理，
    其中，所述待处理图像包括所述风格样本图像和/或所述参考风格样本图像。
  9. 一种图像风格迁移方法，其特征在于，包括：
    获取目标图像；
    将所述目标图像输入至训练后的图像风格迁移模型中，以进行目标图像的风格迁移处理，所述训练后的图像风格迁移模型是根据权利要求1-8中任一项所述的图像风格迁移 模型的训练方法所得到的；以及
    获得所述目标图像对应的风格图像。
  10. 根据权利要求9所述的图像风格迁移方法，其特征在于，所述将所述目标图像输入至训练后的图像风格迁移模型中，以进行目标图像的风格迁移处理，包括：
    对所述目标图像进行分块处理，以得到多个目标图块；其中，各目标图块的组合构成所述目标图像，且相邻的目标图块之间具有重叠区域；
    将所述各目标图块分别输入至所述训练后的图像风格迁移模型中进行处理，以得到所述各目标图块对应的风格迁移图块；以及
    拼接所述各风格迁移图块获得所述目标图对应的风格图像。
  11. 根据权利要求10所述的图像风格迁移方法，其特征在于，还包括：
    采用线性插值处理所述重叠区域。
  12. 一种图像风格迁移模型的训练装置，其特征在于，包括：
    获取模块，用于对真实样本图像进行风格预处理，得到所述真实样本图像对应的风格样本图像；
    处理模块，用于对所述风格样本图像分别进行颜色聚类处理以及线条模糊化处理，得到所述风格样本图像的颜色聚类图以及线条模糊图；
    训练模块，用于利用所述真实样本图像和所述风格样本图像，对待训练的图像风格迁移模型进行训练，并利用所述风格样本图像的颜色聚类图以及所述风格样本图像的线条模糊图对所述待训练的图像风格迁移模型的训练进行监督，获得训练后的图像风格迁移模型。
  13. 一种终端，其特征在于，包括：
    获取模块，用于获取目标图像；
    处理模块，用于将所述目标图像输入至训练后的图像风格迁移模型中，以进行目标图像的风格迁移处理，所述训练后的图像风格迁移模型是根据权利要求1-8中任一项所述的图像风格迁移模型的训练方法所得到的；
    所述获取模块，还用于获得所述目标图像对应的风格图像。
  14. 一种电子设备，其中，包括：
    至少一个处理器；以及
    存储器；
    所述存储器存储计算机执行指令；
    所述至少一个处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如权利要求1-8中任一项所述的图像风格迁移模型的训练方法，和/或，如权利要求9-11中任一项所述的图像风格迁移方法。
  15. 一种计算机可读存储介质，其中，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如权利要求1-8中任一项所述的图像风格迁移模型的训练方法，和/或，如权利要求9-11中任一项所述的图像风格迁移方法。
  16. 一种计算机程序产品，包括计算机指令，其特征在于，所述计算机指令被处理器执行时实现如权利要求1-8中任一项所述的图像风格迁移模型的训练方法，和/或，如权利要求9-11中任一项所述的图像风格迁移方法。
  17. 一种计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1-8中任一项所述的图像风格迁移模型的训练方法，和/或，如权利要求9-11中任一项所述的图像风格迁移方法。
PCT/CN2022/093126 2021-07-28 2022-05-16 图像风格迁移模型的训练方法、图像风格迁移方法及装置 WO2023005357A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110859696.3A CN115689865A (zh) 2021-07-28 2021-07-28 图像风格迁移模型的训练方法、图像风格迁移方法及装置
CN202110859696.3 2021-07-28

Publications (1)

Publication Number Publication Date
WO2023005357A1 true WO2023005357A1 (zh) 2023-02-02

Family

ID=85057947

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/093126 WO2023005357A1 (zh) 2021-07-28 2022-05-16 图像风格迁移模型的训练方法、图像风格迁移方法及装置

Country Status (2)

Country Link
CN (1) CN115689865A (zh)
WO (1) WO2023005357A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111862274A (zh) * 2020-07-21 2020-10-30 有半岛(北京)信息科技有限公司 生成对抗网络训练方法、图像风格迁移方法及装置
CN112581361A (zh) * 2020-12-30 2021-03-30 北京达佳互联信息技术有限公司 图像风格迁移模型的训练方法、图像风格迁移方法及装置
US20210110588A1 (en) * 2019-10-11 2021-04-15 Robert G. Adamson, III Mobile application for object recognition, style transfer and image synthesis, and related systems, methods, and apparatuses
CN112734627A (zh) * 2020-12-24 2021-04-30 北京达佳互联信息技术有限公司 图像风格迁移模型的训练方法、图像风格迁移方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210110588A1 (en) * 2019-10-11 2021-04-15 Robert G. Adamson, III Mobile application for object recognition, style transfer and image synthesis, and related systems, methods, and apparatuses
CN111862274A (zh) * 2020-07-21 2020-10-30 有半岛(北京)信息科技有限公司 生成对抗网络训练方法、图像风格迁移方法及装置
CN112734627A (zh) * 2020-12-24 2021-04-30 北京达佳互联信息技术有限公司 图像风格迁移模型的训练方法、图像风格迁移方法及装置
CN112581361A (zh) * 2020-12-30 2021-03-30 北京达佳互联信息技术有限公司 图像风格迁移模型的训练方法、图像风格迁移方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEN JIE: "Research Image Style Transfer Algorithm Based on Generative Adversarial Networks", CHINA MASTER'S' THESES FULL-TEXT DATABASE (ELECTRONIC JOURNAL)-INFORMATION & TECHNOLOGY), TIANJIN POLYTECHNIC UNIVERSITY, CN, no. 4, 15 April 2021 (2021-04-15), CN , XP093029246, ISSN: 1674-0246 *

Also Published As

Publication number Publication date
CN115689865A (zh) 2023-02-03

Similar Documents

Publication Publication Date Title
CN110766777B (zh) 虚拟形象的生成方法、装置、电子设备及存储介质
WO2020155907A1 (zh) 用于生成漫画风格转换模型的方法和装置
WO2022166872A1 (zh) 一种特效展示方法、装置、设备及介质
CN111476871B (zh) 用于生成视频的方法和装置
US10580143B2 (en) High-fidelity 3D reconstruction using facial features lookup and skeletal poses in voxel models
CN110070896B (zh) 图像处理方法、装置、硬件装置
CN111414879A (zh) 人脸遮挡程度识别方法、装置、电子设备及可读存储介质
WO2022132032A1 (zh) 人像图像处理方法及装置
WO2023143129A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN114898177B (zh) 缺陷图像生成方法、模型训练方法、设备、介质及产品
WO2024120223A1 (zh) 图像处理方法、装置、设备、存储介质及计算机程序产品
CN113902636A (zh) 图像去模糊方法及装置、计算机可读介质和电子设备
CN115731341A (zh) 三维人头重建方法、装置、设备及介质
CN113658065A (zh) 图像降噪方法及装置、计算机可读介质和电子设备
CN113538304B (zh) 图像增强模型的训练方法及装置、图像增强方法及装置
CN110689478B (zh) 图像风格化处理方法、装置、电子设备及可读介质
CN113610720A (zh) 视频去噪方法及装置、计算机可读介质和电子设备
CN110069641B (zh) 图像处理方法、装置和电子设备
CN110619602B (zh) 一种图像生成方法、装置、电子设备及存储介质
WO2023005357A1 (zh) 图像风格迁移模型的训练方法、图像风格迁移方法及装置
WO2020155984A1 (zh) 人脸表情图像处理方法、装置和电子设备
WO2023138441A1 (zh) 视频生成方法、装置、设备及存储介质
WO2022262473A1 (zh) 图像处理方法、装置、设备及存储介质
CN110209861A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN110555799A (zh) 用于处理视频的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22847970

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE