CN115272146B - Stylized image generation method, system, device and medium - Google Patents
Stylized image generation method, system, device and medium Download PDFInfo
- Publication number
- CN115272146B CN115272146B CN202210893426.9A CN202210893426A CN115272146B CN 115272146 B CN115272146 B CN 115272146B CN 202210893426 A CN202210893426 A CN 202210893426A CN 115272146 B CN115272146 B CN 115272146B
- Authority
- CN
- China
- Prior art keywords
- image
- stylized
- processing
- head
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012545 processing Methods 0.000 claims abstract description 114
- 238000005520 cutting process Methods 0.000 claims abstract description 50
- 238000003708 edge detection Methods 0.000 claims abstract description 22
- 238000000926 separation method Methods 0.000 claims abstract description 17
- 238000003860 storage Methods 0.000 claims description 19
- 230000004927 fusion Effects 0.000 claims description 10
- 230000007797 corrosion Effects 0.000 claims description 5
- 238000005260 corrosion Methods 0.000 claims description 5
- 230000003190 augmentative effect Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 12
- 230000006870 function Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 230000001965 increasing effect Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- CCEKAJIANROZEO-UHFFFAOYSA-N sulfluramid Chemical group CCNS(=O)(=O)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)F CCEKAJIANROZEO-UHFFFAOYSA-N 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a stylized image generation method, a stylized image generation system, stylized image generation equipment and a stylized image generation medium, wherein the method comprises the following steps: acquiring an image to be processed, and performing background region separation processing on the image to be processed to determine a character image and a head image; carrying out edge detection processing on the figure image according to the head image, determining an expansion parameter, carrying out expansion cutting processing on the figure image according to the expansion parameter, and determining a cut image set; performing stylization processing on the cutting image set to determine a stylized image set; and merging the stylized image sets according to an interpolation and superposition method to determine a target stylized image. The invention can output more stable and smooth stylized effect by an interpolation superposition method on the basis of keeping the key features of the portrait and optimizing the style effect of the head, and can be widely applied to the technical field of image processing.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a stylized image generation method, a stylized image generation system, stylized image generation equipment and a stylized image generation medium.
Background
In recent years, with the development of non-homogeneous tokens, the production of personal-specific digital content has been gradually increased, and stylized character image creation is an important component of digital content production. In the current method for generating a stylized image, a hidden vector of an overall image is generated, stylized processing is performed on the hidden vector, and then the hidden vector is decoded and restored to a result image. The method is simple and convenient in operation mode, the styles of the generated images are uniform, but the method is easily influenced by non-human face areas in the images, and the problems of edge overlapping, facial dislocation, insufficient aesthetic feeling and the like easily occur.
In summary, how to improve the generation effect of the stylized image is a technical problem that needs to be solved by those skilled in the art at present.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, a system, a device, and a medium for generating a stylized image, so as to achieve an effect of enhancing the generation of the stylized image.
In one aspect, the present invention provides a stylized image generating method, comprising:
acquiring an image to be processed;
carrying out background region separation processing on the image to be processed to determine a character image and a head image;
performing edge detection processing on the figure image according to the head image to determine an expansion parameter;
performing extended clipping processing on the figure image according to the extended parameters to determine a clipped image set;
performing stylization processing on the cutting image set to determine a stylized image set;
and merging the stylized image sets according to an interpolation and superposition method to determine a target stylized image.
Optionally, the performing background region separation processing on the image to be processed to determine a person image and a head image includes:
carrying out background separation processing on the image to be processed to determine a character image;
and performing head region identification and cutting processing on the person image to determine a head image.
Optionally, the performing, according to the head image, edge detection on the human image to determine an expansion parameter includes:
acquiring the figure image as a current comparison image;
stylizing the head image and the current comparison image respectively to determine a head stylized image and a current comparison stylized image;
performing edge detection processing on the head stylized image to determine a head image edge value;
performing edge detection processing on the current contrast stylized image to determine a contrast image edge value;
determining an edge difference value according to the edge value of the head image and the edge value of the comparison image, when the edge difference value is greater than a difference threshold value, performing reduction processing on the current comparison image, returning to perform stylization processing on the head image and the current comparison image respectively, and determining the stylized head image and the stylized current comparison image until the edge difference value is less than or equal to the difference threshold value;
and calculating the pixel difference value of the head stylized image and the current comparison stylized image, and determining an expansion parameter.
Optionally, the performing extended cropping processing on the human image according to the extended parameters to determine a cropped image set includes:
performing head region clipping processing on the figure image, determining a current clipping image and storing the current clipping image to a clipping image set;
performing expansion processing on the current cutting image according to the expansion parameters, determining an expanded cutting image and storing the expanded cutting image in the cutting image set;
and when the extended clipping image is smaller than the character image, acquiring the extended clipping image as a current clipping image, returning to the step of performing extension processing on the current clipping image according to the extension parameters, determining the extended clipping image and storing the extended clipping image in the clipping image set until the extended clipping image is larger than or equal to the character image.
Optionally, the stylizing the cropped image set to determine a stylized image set includes:
acquiring the image to be processed;
performing stylization processing on the image to be processed to determine a stylized background image;
performing stylization processing on each image in the cutting image set respectively to determine a first image set;
and adding the stylized background image into the first image set, and determining the stylized image set.
Optionally, the merging the stylized image sets according to an interpolation and superposition method to determine a target stylized image includes:
acquiring a first image set and a stylized background image in the stylized image set, wherein the first image set comprises a plurality of image sets which are sequentially ordered from small to large;
sequentially carrying out pixel color interpolation superposition processing on adjacent images in the first image set to determine a fused image;
and performing image embedding processing on the fusion image according to the stylized background image to determine a target stylized image.
Optionally, the performing, according to the stylized background image, image embedding processing on the fused image to determine a target stylized image includes:
performing image embedding processing on the fused image according to the stylized background image to determine a first image;
and carrying out corrosion expansion processing on the first image to determine a target stylized image.
On the other hand, the embodiment of the invention also provides a stylized image generation system, which comprises:
the first module is used for acquiring an image to be processed;
the second module is used for carrying out background region separation processing on the image to be processed and determining a character image and a head image;
the third module is used for carrying out edge detection processing on the figure image according to the head image and determining an expansion parameter;
the fourth module is used for carrying out expansion cutting processing on the figure image according to the expansion parameters and determining a cutting image set;
a fifth module, configured to perform stylization processing on the clipped image set, and determine a stylized image set;
and the sixth module is used for carrying out merging processing on the stylized image set according to an interpolation and superposition method and determining a target stylized image.
In another aspect, an embodiment of the present invention further discloses an electronic device, where the electronic device includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for implementing connection communication between the processor and the memory, and the program, when executed by the processor, implements the method as described above.
In another aspect, an embodiment of the present invention further discloses a storage medium, which is a computer-readable storage medium for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the foregoing method.
In another aspect, an embodiment of the present invention further discloses a computer program product or a computer program, where the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The computer instructions may be read by a processor of a computer device from a computer-readable storage medium, and the computer instructions executed by the processor cause the computer device to perform the foregoing method.
Compared with the prior art, the invention adopting the technical scheme has the following technical effects: according to the invention, the image to be processed is obtained, the background area separation processing is carried out on the image to be processed, the figure image and the head image are determined, and the stylized influence of background colors can be reduced through the processing mode of background area separation; in addition, the invention carries out edge detection processing on the figure image according to the head image, determines an expansion parameter, carries out expansion cutting processing on the figure image according to the expansion parameter, determines a cutting image set, carries out stylization processing on the cutting image set, determines a stylized image set, can construct a multi-level image through the expansion parameter and then carries out stylization processing, improves the aesthetic degree of the image, and improves the smoothness among the multi-level images; moreover, the stylized image set is merged according to an interpolation and superposition method, the target stylized image is determined, and the multilayer images can be fused through the interpolation and superposition method, so that the problems of edges and dislocation among the multilayer images are reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a stylized image generation method according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
The stylized character image is used as an important component for digital content production, and all large Internet macros have research and provide related image generation capabilities including Aries, tencent, baidu, volcano engines and the like. The current character stylization generation framework of the human figure mainly comprises the following two schemes:
1. the overall stylization scheme is as follows: and generating an implicit vector of the whole image, performing stylization processing on the implicit vector, and then decoding and restoring the implicit vector into a result image.
2. The human face matting and splicing scheme is as follows: and determining a face region and cutting out the face region through face key point identification. And then stylizing the original picture, stylizing the face area, and finally pasting the processed face area back.
The operation mode of the whole stylization scheme is simple and convenient, the style of the generated image is uniform, but the generated image is easily influenced by a non-face area in the image, and the finally obtained effect is not beautiful enough. The human face matting and splicing scheme has relatively good effect on partial processing of human faces, but certain errors exist in human face region identification and human face segmentation, and when partial pictures are used for pasting human face regions, due to the errors between the processed human faces and original human faces, the finally generated portrait edges are easy to have overlapping traces, such as neck dislocation, double chin and the like, so that the generation effect is influenced.
With the increasing popularity of personal exclusive digital content, the demand and scenes for generating digital images by using stylization capability are increasing, and the stability and the attractiveness of stylization effect are insufficient, so that the use experience and the popularization speed of the capability are influenced. The invention aims to provide a stylized image generation method which is applied to a stylized character image generation process. The optimal stylized area can be identified, cut and selected on the basis of reserving key portrait features of the user picture, and stable stylized image generation capacity is achieved through multi-layer stylized design and an interpolation superposition processing method.
Referring to fig. 1, an embodiment of the present invention provides a stylized image generating method, including:
s101, acquiring an image to be processed;
s102, performing background region separation processing on the image to be processed to determine a person image and a head image;
s103, performing edge detection processing on the figure image according to the head image, and determining an expansion parameter;
s104, performing extended cropping processing on the character image according to the extended parameters to determine a cropped image set;
s105, performing stylization processing on the cutting image set to determine a stylized image set;
and S106, merging the stylized image sets according to an interpolation and superposition method, and determining a target stylized image.
The embodiment of the invention performs background region separation processing on the to-be-processed image input by a user or acquired by an artificial intelligence method, so that the portrait and the background in the to-be-processed image are segmented, and the influence of the color of the background region is reduced. Then, the embodiment of the invention further identifies the head area in the character image, and calculates the stylized edge difference by carrying out edge detection on the character image and the head image to obtain the appropriate head area expansion parameter. Then, the embodiment of the invention expands and cuts the head area based on the expansion parameters, and forms a multi-level image area from the head to the whole to obtain a stylized image set containing the multi-level image. Finally, the multi-layer images in the stylized image set are stylized respectively, and the stylized multi-layer images are spliced and overlapped to obtain the target stylized image.
Further preferably, in step S102, the performing a background region separation process on the image to be processed to determine a human image and a head image includes:
carrying out background separation processing on the image to be processed to determine a character image;
and performing head region identification and cutting processing on the character image to determine a head image.
In the process of stylized model training, in order to make the generated picture style as close to the target style as possible, the color coding loss (YUV loss) is increased, and the brightness and the color of the whole generated stylized image are made as close to the target style as possible. When the brightness and color of the background in the picture are too different from those of the portrait, the color of the face part in the generated image is deviated from the real color. In order to solve the above problems, in the embodiments of the present invention, a portrait is separated from an image to be processed through a portrait matting model to obtain a portrait image, which can suppress interference of a background on a portrait part, so that a stylized image is better generated. Because the portrait stylized model is trained mostly based on head paired images, after the person image is obtained, the embodiment of the invention performs head region recognition and cropping processing on the person image to obtain a head image.
In the embodiment of the invention, the portrait matting is realized based on the open source P3M-NET model, and more precise and stable matting capability is realized by expanding the P3M-10k data set and embedding various background images. The P3M-NET model adopts a multi-task framework to decompose a portrait matting task into two subtasks of semantic segmentation and detail matting, and fusion and information interaction of each subtask and corresponding different-level codes are realized through a deep bidirectional feature fusion module, a shallow bidirectional feature fusion module and a three-party feature fusion module. And (3) performing head region identification and clipping on the whole image of the separated portrait by using a dlib tool, wherein the dlib is a modern C + + tool box which comprises machine learning algorithms and tools for creating complex software in C + + to solve practical problems. The method is widely applied to the fields of machine learning, deep learning and image processing, including robots, embedded equipment, mobile phones and large-scale high-performance computing environments.
Further preferably, the performing edge detection processing on the human figure image according to the head image and determining an expansion parameter includes:
acquiring the figure image as a current comparison image;
performing stylization processing on the head image and the current comparison image respectively to determine a head stylized image and a current comparison stylized image;
performing edge detection processing on the head stylized image to determine a head image edge value;
performing edge detection processing on the current contrast stylized image to determine a contrast image edge value;
determining an edge difference value according to the edge value of the head image and the edge value of the comparison image, when the edge difference value is greater than a difference threshold value, performing reduction processing on the current comparison image, returning to perform stylization processing on the head image and the current comparison image respectively, and determining the stylized head image and the stylized current comparison image until the edge difference value is less than or equal to the difference threshold value;
and calculating the pixel difference value of the head stylized image and the current comparison stylized image, and determining an expansion parameter.
The current stylization capability mainly generates an anti-network, such as cartonongan, comixGAN, animeGAN, and the like, and there is a great difference in processing effect for a header region and an entire region. Since the portrait stylized model is trained based on paired head images in most cases, when the stylized image is generated, the head image is inferred better in most cases, but the overall image is inferred worse. If the head region and the whole region are respectively stylized and then spliced, the effect of splicing the stylized image of the head and the whole stylized image can be influenced due to the difference of the stylized image in texture and color, such as head and neck dislocation, limb fracture, color fracture and the like.
Therefore, the embodiment of the invention reduces the texture difference generated by stylizing the pictures with different sizes by calculating the proper head area expansion parameter. Specifically, in the embodiment of the present invention, the person image obtained by segmentation in the above step is used as a current comparison image, and the stylized head image and the current comparison image are respectively performed to obtain a stylized head image and a stylized current comparison image. After the head image and the current comparison image are stylized, respectively calculating the edge values of the two stylized images based on Sobel operators to obtain the edge value of the head image and the edge value of the comparison image. Subtracting the edge value of the head image from the edge value of the comparison image to obtain an edge difference value, and judging the size of the edge difference value; and when the edge difference value is larger than the difference threshold value, reducing the current comparison image, returning to the step of performing stylization processing on the head image and the current comparison image respectively, and determining the head stylized image and the current comparison stylized image until the edge difference value is smaller than or equal to the difference threshold value. And calculating the pixel difference value of the head stylized image and the current contrast stylized image to obtain an expansion parameter. The expansion parameter is the pixel difference value obtained by subtracting the head stylized image from the current contrast stylized image and left, right, upper and lower pixel values of the residual image. The expansion parameter is used for expanding the pixel values along the up, down, left and right directions of the image to obtain a multilayer image. It should be noted that the stylized generation model described above may be used for stylizing in the embodiments of the present invention, but the head image and the current comparison image need to be stylized separately using the same stylized generation model. The difference threshold is a preset threshold, which is generally set to 60, so that if the difference threshold is too low, the expanded area is small, and the whole stylization process can be completed by performing area expansion and stylization for many times; and if the difference threshold is too high, the expanded area is large, and the situation of edge dislocation is easy to occur.
Further preferably, the performing augmented cropping processing on the human image according to the augmented parameters to determine a cropped image set includes:
performing head region clipping processing on the figure image, determining a current clipping image and storing the current clipping image into a clipping image set;
performing expansion processing on the current cutting image according to the expansion parameters, determining an expanded cutting image and storing the expanded cutting image into the cutting image set;
and when the extended clipping image is smaller than the character image, acquiring the extended clipping image as a current clipping image, returning to the step of performing extension processing on the current clipping image according to the extension parameters, determining the extended clipping image and storing the extended clipping image in the clipping image set until the extended clipping image is larger than or equal to the character image.
The method and the device for clipping the human image have the advantages that the head area clipping processing is carried out on the human image, the current clipping image is obtained and stored in the clipping image set, and the current clipping image is the first-layer image. And then, according to the expansion parameters obtained in the step, performing expansion processing on the current cut image to obtain an expanded cut image, and storing the expanded cut image into a cut image set, wherein the expanded cut image is a second image. And judging the size of the extended cutting image, when the extended cutting image is smaller than the figure image, taking the extended cutting image as the current cutting image, returning to the step of performing extended processing on the current cutting image according to the extended parameters, determining the extended cutting image and storing the extended cutting image in a cutting image set until the extended cutting image is larger than or equal to the figure image, namely, extending the first layer image (namely the head area) to the second layer image based on the extended parameters, and circularly repeating the extending step on the basis of the newly obtained layered image area until the extended image area covers the whole figure image. Finally, the cut image set stores cut multilayer images.
Further preferably, the stylizing the cropped image set to determine the stylized image set includes:
acquiring the image to be processed;
performing stylization processing on the image to be processed to determine a stylized background image;
performing stylization processing on each image in the cutting image set respectively to determine a first image set;
and adding the stylized background image into the first image set to determine the stylized image set.
The embodiment of the invention converts the image to be processed to generate the stylized background image, thereby facilitating the subsequent superposition of the background area. Meanwhile, the embodiment of the invention respectively performs stylization processing on each image in the cut image set to obtain a first image set, wherein the first image set is a multi-layer stylized image set from the head to the whole area. In the embodiment of the invention, a lightweight generation confrontation model AnimeGAN with less network parameters is adopted to carry out stylization processing operation on the multi-layer images in the cut image set. The generator of the animagegan model is a symmetric codec network, and all convolutional layers in the discriminator are standard convolutions.
Further, as a preferred embodiment, the merging the stylized image sets according to an interpolation and superposition method to determine a target stylized image includes:
acquiring a first image set and a stylized background image in the stylized image set, wherein the first image set comprises a plurality of image sets which are sequentially ordered from small to large;
sequentially carrying out pixel color interpolation superposition processing on adjacent images in the first image set to determine a fused image;
and performing image embedding processing on the fused image according to the stylized background image to determine a target stylized image.
The edge parts in the stylized image set are basically consistent, the textures of the multi-layer images can be well connected together, but the color parts are different, so that the situation of color separation occurs after the multi-layer images are combined. Based on this, when the multi-layer stylized images are merged, the images of adjacent layers are fused by an interpolation and superposition method, and an example is shown in which the image of the nth layer and the image of the (N + 1) th layer are fused: and carrying out interpolation transformation on the pixel color values of the image of the Nth layer, wherein the interpolation transformation is carried out firstly in the transverse direction and then in the longitudinal direction.
Wherein, the horizontal transformation formula is as follows:
the longitudinal transformation formula is:
in the formula (I), the compound is shown in the specification,
RGB color value, representing a point in an image of the Nth layer>
The RGB color values of a point corresponding to the (N + 1) th layer image are represented, and x and y respectively represent the abscissa and the coordinate of the point in the N layer image; />
Representing an intermediate value obtained after the point of the Nth layer image is subjected to transverse color value conversion; />
Representing the final color value of the N layer image after the point transformation; w is a N Width, h, of the image of the Nth layer N Indicating a high for the nth layer image.
According to the embodiment of the invention, the transformed N layer image is embedded into the (N + 1) layer image according to the original left coordinate and the upper coordinate position to form a fused image, and the colors of the two layers of images can be well fused together. And sequentially carrying out pixel color interpolation superposition processing on adjacent images in the first image set according to the method to obtain a fused image. And finally, carrying out fusion embedding on the fusion image and the background stylized picture to obtain a final target stylized image.
Further preferably, the image embedding processing on the fused image according to the stylized background image to determine a target stylized image includes:
performing image embedding processing on the fused image according to the stylized background image to determine a first image;
and carrying out corrosion expansion processing on the first image to determine a target stylized image.
Because the portrait segmented by the portrait segmentation model has noise and unsmooth edges, the embodiment of the invention leads the transition between the portrait and the background to be more natural and smooth edges by carrying out corrosion expansion operation on the first image, and finally obtains the stylized conversion image of the original image to be processed.
An embodiment of the present invention further provides a stylized image generation system, including:
the first module is used for acquiring an image to be processed;
the second module is used for carrying out background region separation processing on the image to be processed and determining a character image and a head image;
the third module is used for carrying out edge detection processing on the figure image according to the head image and determining an expansion parameter;
the fourth module is used for carrying out expansion cutting processing on the figure image according to the expansion parameters and determining a cutting image set;
a fifth module, configured to perform stylization processing on the clipped image set, and determine a stylized image set;
and the sixth module is used for carrying out merging processing on the stylized image set according to an interpolation and superposition method and determining a target stylized image.
Corresponding to the method of fig. 1, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for implementing connection communication between the processor and the memory, wherein the program, when executed by the processor, implements the method as described above.
In correspondence with the method of fig. 1, the embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage, and the storage medium stores one or more programs, which are executable by one or more processors to implement the method as described above.
The embodiment of the invention also discloses a computer program product or a computer program, which comprises computer instructions, and the computer instructions are stored in a computer readable storage medium. The computer instructions may be read by a processor of a computer device from a computer-readable storage medium, and executed by the processor to cause the computer device to perform the method illustrated in fig. 1.
In summary, the embodiments of the present invention have the following advantages:
1. in addition, by corrosion expansion operation, noise and unsmooth edge conditions existing in portrait segmentation are reduced, and transition between the portrait and the background is natural and smooth.
2. According to the embodiment of the invention, the edge difference is calculated through the Sobel operator, so that the optimal head portrait expansion parameter is determined, the head is constructed to form a multi-level image integrally, the stylized attractiveness of the head region is ensured, and the smooth connection between the images in different levels is also ensured.
3. The embodiment of the invention realizes the fusion of the multilayer images by an interpolation superposition method, avoids the problems of side seams and dislocation among the multilayer images and finishes the stable output of the whole stylization effect.
In alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flow charts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are performed independently.
Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the described functions and/or features may be integrated in a single physical device and/or software module, or one or more functions and/or features may be implemented in a separate physical device or software module. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be understood within the ordinary skill of an engineer, given the nature, function, and internal relationship of the modules. Accordingly, those skilled in the art can, using ordinary skill, practice the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative of and not intended to limit the scope of the invention, which is defined by the appended claims and their full scope of equivalents.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Further, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (9)
1. A stylized image generation method, the method comprising:
acquiring an image to be processed;
carrying out background region separation processing on the image to be processed to determine a character image and a head image;
carrying out edge detection processing on the figure image according to the head image, and determining an expansion parameter;
performing extended clipping processing on the figure image according to the extended parameters to determine a clipped image set;
performing stylization processing on the cutting image set to determine a stylized image set;
merging the stylized image sets according to an interpolation and superposition method to determine a target stylized image;
wherein, the edge detection processing is performed on the human image according to the head image, and the determining of the expansion parameters includes:
acquiring the figure image as a current comparison image;
performing stylization processing on the head image and the current comparison image respectively to determine a head stylized image and a current comparison stylized image;
performing edge detection processing on the head stylized image to determine a head image edge value;
performing edge detection processing on the current contrast stylized image to determine a contrast image edge value;
determining an edge difference value according to the edge value of the head image and the edge value of the comparison image, when the edge difference value is greater than a difference threshold value, performing reduction processing on the current comparison image, returning to perform stylization processing on the head image and the current comparison image respectively, and determining the stylized head image and the stylized current comparison image until the edge difference value is less than or equal to the difference threshold value;
and calculating the pixel difference value of the head stylized image and the current comparison stylized image, and determining an expansion parameter.
2. The method of claim 1, wherein the performing background region separation processing on the image to be processed to determine a human image and a head image comprises:
carrying out background separation processing on the image to be processed to determine a character image;
and performing head region identification and cutting processing on the person image to determine a head image.
3. The method of claim 1, wherein the determining a cropped image set by performing an augmented cropping process on the human image according to the augmented parameters comprises:
performing head region clipping processing on the figure image, determining a current clipping image and storing the current clipping image into a clipping image set;
performing expansion processing on the current cutting image according to the expansion parameters, determining an expanded cutting image and storing the expanded cutting image into the cutting image set;
and when the expanded cutting image is smaller than the character image, acquiring the expanded cutting image as a current cutting image, returning to the step of performing expansion processing on the current cutting image according to the expansion parameters, determining the expanded cutting image and storing the expanded cutting image in the cutting image set until the expanded cutting image is larger than or equal to the character image.
4. The method of claim 1, wherein said stylizing said cropped image set to determine a stylized image set comprises:
acquiring the image to be processed;
performing stylization processing on the image to be processed to determine a stylized background image;
performing stylization processing on each image in the cutting image set respectively to determine a first image set; and adding the stylized background image into the first image set, and determining the stylized image set.
5. The method according to any one of claims 1-4, wherein said merging the set of stylized images according to interpolation and superposition to determine a target stylized image comprises:
acquiring a first image set and a stylized background image in the stylized image set, wherein the first image set comprises a plurality of image sets which are sequentially ordered from small to large;
sequentially carrying out pixel color interpolation superposition processing on adjacent images in the first image set to determine a fused image;
and performing image embedding processing on the fusion image according to the stylized background image to determine a target stylized image.
6. The method of claim 5, wherein the performing image embedding processing on the fused image according to the stylized background image to determine a target stylized image comprises:
performing image embedding processing on the fusion image according to the stylized background image to determine a first image;
and carrying out corrosion expansion processing on the first image to determine a target stylized image.
7. A stylized image generation system, the system comprising:
the first module is used for acquiring an image to be processed;
the second module is used for carrying out background region separation processing on the image to be processed and determining a character image and a head image;
the third module is used for carrying out edge detection processing on the figure image according to the head image and determining an expansion parameter;
the fourth module is used for carrying out expansion cutting processing on the figure image according to the expansion parameters and determining a cutting image set;
a fifth module, configured to perform stylization processing on the clipped image set, and determine a stylized image set;
a sixth module, configured to perform merging processing on the stylized image set according to an interpolation and superposition method, and determine a target stylized image;
the third module is configured to perform edge detection processing on the human image according to the head image, and determine an expansion parameter, where the edge detection processing includes:
acquiring the figure image as a current comparison image;
stylizing the head image and the current comparison image respectively to determine a head stylized image and a current comparison stylized image;
performing edge detection processing on the head stylized image to determine a head image edge value;
performing edge detection processing on the current contrast stylized image to determine a contrast image edge value;
determining an edge difference value according to the edge value of the head image and the edge value of the comparison image, reducing the current comparison image when the edge difference value is greater than a difference threshold value, returning to the step of performing stylization processing on the head image and the current comparison image respectively, and determining the head stylized image and the current comparison stylized image until the edge difference value is less than or equal to the difference threshold value;
and calculating the pixel difference value of the head stylized image and the current comparison stylized image, and determining an expansion parameter.
8. An electronic device, characterized in that the electronic device comprises a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for enabling a connection communication between the processor and the memory, which program, when executed by the processor, realizes the steps of the method according to any one of claims 1 to 6.
9. A storage medium, being a computer readable storage medium, for computer readable storage, characterized in that the storage medium stores one or more programs executable by one or more processors to implement the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210893426.9A CN115272146B (en) | 2022-07-27 | 2022-07-27 | Stylized image generation method, system, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210893426.9A CN115272146B (en) | 2022-07-27 | 2022-07-27 | Stylized image generation method, system, device and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115272146A CN115272146A (en) | 2022-11-01 |
| CN115272146B true CN115272146B (en) | 2023-04-07 |
Family
ID=83770122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210893426.9A Active CN115272146B (en) | 2022-07-27 | 2022-07-27 | Stylized image generation method, system, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115272146B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117237397A (en) * | 2023-07-13 | 2023-12-15 | 天翼爱音乐文化科技有限公司 | Portrait segmentation method, system, equipment and storage medium based on feature fusion |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111028137A (en) * | 2018-10-10 | 2020-04-17 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN111986076A (en) * | 2020-08-21 | 2020-11-24 | 深圳市慧鲤科技有限公司 | Image processing method and device, interactive display device and electronic equipment |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| RU2677573C2 (en) * | 2017-04-05 | 2019-01-17 | Общество С Ограниченной Ответственностью "Яндекс" | System and method of adding stylized properties to image |
| CN109829353B (en) * | 2018-11-21 | 2023-04-18 | 东南大学 | Face image stylizing method based on space constraint |
| CN112215854A (en) * | 2020-10-19 | 2021-01-12 | 珠海金山网络游戏科技有限公司 | Image processing method and device |
| CN113763233A (en) * | 2021-08-04 | 2021-12-07 | 深圳盈天下视觉科技有限公司 | Image processing method, server and photographing device |
| CN114037692A (en) * | 2021-11-16 | 2022-02-11 | 深圳传音控股股份有限公司 | Image processing method, mobile terminal and storage medium |
| CN114049473A (en) * | 2021-11-18 | 2022-02-15 | 维沃移动通信有限公司 | Image processing method and device |
-
2022
- 2022-07-27 CN CN202210893426.9A patent/CN115272146B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111028137A (en) * | 2018-10-10 | 2020-04-17 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN111986076A (en) * | 2020-08-21 | 2020-11-24 | 深圳市慧鲤科技有限公司 | Image processing method and device, interactive display device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115272146A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11170558B2 (en) | Automatic rigging of three dimensional characters for animation | |
| US11393152B2 (en) | Photorealistic real-time portrait animation | |
| US8872850B2 (en) | Juxtaposing still and dynamic imagery for cliplet creation | |
| US7450758B2 (en) | Stylization of video | |
| US9779774B1 (en) | Generating semantically meaningful video loops in a cinemagraph | |
| JP4500614B2 (en) | Image-based rendering and editing method and apparatus | |
| Kort | Computer aided inbetweening | |
| US11727617B2 (en) | Single image-based real-time body animation | |
| CN116724330A (en) | High resolution portrait stylized framework using hierarchical variational encoder | |
| US20220222897A1 (en) | Portrait editing and synthesis | |
| US7974470B2 (en) | Method and apparatus for processing an image | |
| CN113261013A (en) | System and method for realistic head rotation and facial animation synthesis on mobile devices | |
| Agarwala | Snaketoonz: A semi-automatic approach to creating cel animation from video | |
| CN105956995B (en) | A kind of face appearance edit methods based on real-time video eigen decomposition | |
| CN115004236A (en) | Photo-level realistic talking face from audio | |
| US20220156987A1 (en) | Adaptive convolutions in neural networks | |
| US11157773B2 (en) | Image editing by a generative adversarial network using keypoints or segmentation masks constraints | |
| CN115272146B (en) | Stylized image generation method, system, device and medium | |
| CN115457169A (en) | Voice-driven human face animation generation method and system | |
| Noris et al. | Temporal noise control for sketchy animation | |
| US6201542B1 (en) | Three-dimensional polygon display apparatus using progressive polygon data | |
| Zhao et al. | Cartoon image processing: a survey | |
| Ueno et al. | Continuous and Gradual Style Changes of Graphic Designs with Generative Model | |
| JP5473096B2 (en) | Modeling concave surfaces in image-based visual hulls | |
| JPWO2008023819A1 (en) | Computer system and operation control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |