CN112069769A - Intelligent character effect migration method for special effect characters - Google Patents

Abstract

The invention provides an intelligent character effect migration method for special effect characters, which comprises the following steps: training a mask extraction sub-network to extract a decoration element mask by utilizing a training data set, and training a basic special effect migration sub-network to migrate a basic character special effect; special effect word D with decorative elements_yAnd its paired character pattern picture C_yObtaining a decorative element mask M from the trained mask extraction sub-network_y(ii) a Will D_yAnd its paired character pattern picture C_yTarget font picture C_xInputting the result into a trained basic special effect migration sub-network to obtain a result S of basic special effect migration and decoration element elimination_x(ii) a Using M_y，C_yAnd C_xCarrying out element recombination to fuse decorative elements into S_xObtaining the moved special effect character D with the decorative elements corresponding to the target character pattern_x. The method can transfer the decorative elements of the character special effect while transferring the special effect of the character, and cannot cause the loss and distortion of the decorative elements.

Description

Intelligent character effect migration method for special effect characters

Technical Field

The invention belongs to the field of character special effect style migration, and particularly relates to an intelligent character effect migration method for special effect characters.

Background

The special effect word style migration aims to migrate the style of a given special effect word to a given font, and realize the batch generation of the special effect word. In recent years, special effect word style migration has received increasing academic attention.

The special effect word style migration method can be divided into three categories. Conventional approaches based on block fusion implement style migration by finding similar texture blocks on a given special effect word that match the current glyph block. To better preserve the legibility of glyphs, this type of approach typically computes a priori on the glyph structure and considers the similarity on the glyphs when finding similar blocks. The method based on the global statistic generally extracts the features by means of a deep neural network pre-trained on a classification task, then calculates the global statistic of the features, and enables a migration result to be consistent with a special effect of a character to be migrated on the global statistic in the modes of iteration, training a forward network and the like. The method based on the deep neural network firstly collects a pair training set, then builds a deep neural network framework, trains the network on the training set, and finally applies the trained network to the special effect of the character to be migrated to realize character effect migration.

However, the existing method for transferring the style of special effect words cannot process the decorative elements on the special effect of words, which can cause the loss and distortion of the decorative elements.

Disclosure of Invention

Aiming at the technical problems, the invention provides an intelligent character effect transferring method for special effect characters, which can transfer the special effects of the characters and simultaneously transfer the decorative elements of the characters without losing and distorting the decorative elements.

The technical scheme adopted by the invention is as follows:

an intelligent word effect migration method for special effect words comprises the following steps:

training a mask extraction sub-network to extract a decoration element mask by utilizing a training data set, and training a basic special effect migration sub-network to migrate a basic character special effect;

special effect word D with decorative elements_yAnd its paired character pattern picture C_yTo trainExtracting the sub-network from the good mask to obtain a decoration element mask M_y；

Will D_yAnd its paired character pattern picture C_yTarget font picture C_xInputting the result into a trained basic special effect migration sub-network to obtain a result S of basic special effect migration and decoration element elimination_x；

Using M_y，C_yAnd C_xCarrying out element recombination to fuse decorative elements into S_xObtaining the moved special effect character D with the decorative elements corresponding to the target character pattern_x。

Further, the training data set comprises a synthesized special effect word picture D with decoration and a matched character pattern picture C thereof, and a collected special effect word picture D with decoration and manually marked character pattern_wAnd the paired character pattern picture C_w(ii) a Wherein the picture D is obtained by randomly adding the decoration element picture to a collected or synthesized special effect word picture obtained by randomly combining the collected texture pictures to a character picture containing no special effect.

Furthermore, the mask extraction sub-network adopts a U-net network structure, and a discriminator network netSegD for judging a picture domain where the input picture is located is built, wherein the netSegD comprises four convolution modules, and each convolution module comprises a convolution layer and a linear rectification function.

Further, the training method for mask extraction sub-network comprises the following steps:

combining the D and the C in the channel dimension and inputting the mask to extract the sub-network to obtain the decoration element mask

Simultaneously extracting the feature P of the penultimate layer;

will D_wAnd C_wMerging the input masks in the channel dimension to extract sub-networks to obtain the decorative element mask

Simultaneously extracting features P of the penultimate layer_w；

According to

P、P_wAnd setting a loss function to obtain a trained mask extraction sub-network.

Further, the basic special effect migration sub-network adopts a multi-scale framework structure and is trained sequentially under three resolutions of 64 × 64, 128 × 128 and 256 × 256.

Further, the training method of the basic special effect migration sub-network comprises the following steps:

will D_y、C_y、C_xInputting the data into a basic special effect migration sub-network, merging the data in a channel domain to obtain a migration result S_x；

According to S_xAnd setting a loss function to obtain a trained basic special effect migration sub-network.

Further, the method for recombining the elements comprises the following steps:

using DenseCrF for M_yOptimizing;

will M_yScaling to a certain resolution, clustering by using DBSCAN, and obtaining different decorative elements and masks thereof by searching for connected shapes;

for in C_yEach decorative element belonging to the area E is searched to obtain the corresponding decorative element C_xThereby the decorative element is moved from D to E_yIs copied to S_xAdjusting the size of the decorative element to obtain D_x。

Further, for in C_yEach decorative element belonging to the area E is searched according to the following formula to obtain the corresponding decorative element in the area C_xAt the suitable position E':

wherein M is_guide(·)，

And M_Exi(. respectively) indicates the regions in M_guide，

And M_ExiThe sum of the values of (a) and (b). M_guideThe obtaining method comprises the following steps: at C_yUpper computation level prior M_HorAnd vertical prior M_VerNormalized to [0,1 ]]Interval, fuzzy with Gaussian fuzzy kernel; at C_yUpper computation distribution prior M_Dis(ii) a Will obscure M_HorAnd M_VerAnd M_DisCombining in channel dimensions to yield M_guide；

Is at C_xThe same as M_guideThe method for obtaining.

A smart word effect migration system for special effect words, comprising:

the mask extraction sub-network module is used for inputting special effect characters with decorative elements and paired character patterns of the special effect characters and obtaining a decorative element mask from the special effect characters;

a basic special effect migration sub-network module, which is used for inputting special effect words with decorative elements, paired font pictures thereof and target font pictures, migrating the basic special effects of the special effect words to the target font, eliminating the decorative elements of the special effect words and outputting results;

and the element recombination module is used for fusing the decoration elements on the result output by the basic special effect migration sub-network module by utilizing the mask obtained by the mask extraction sub-network module to obtain the migrated special effect characters with the decoration elements corresponding to the target character patterns.

The method comprises the steps of firstly extracting a sub-network through a shade to obtain the shade of the decoration element for separating the character special effect and the decoration element, then carrying out basic character special effect migration through a basic special effect migration sub-network, and finally recombining the decoration element and the special effect character after the basic character special effect migration according to the distribution of the decoration element to obtain a special effect migration result. The method can transfer the decorative elements of the character special effect while transferring the special effect of the character, and cannot cause the loss and distortion of the decorative elements.

Drawings

Fig. 1 is a block diagram of a mask extraction sub-network used in the present invention.

Fig. 2 is a block diagram of a basic effect migration sub-network used in the present invention.

FIG. 3 is a block diagram of a smart word effect migration framework used in the present invention.

Detailed Description

In order to make the aforementioned and other features and advantages of the invention more comprehensible, embodiments accompanied with figures are described in detail below.

The embodiment discloses an intelligent word effect migration method for special effect words, which is specifically described as follows:

step 1: a set of character-effect picture pair data sets is collected, and a group of picture pairs consists of a special-effect character picture and a corresponding character-shape picture without a special effect. The special effect words can be generated in batches through a Photoshop special effect word synthesis action. The data set contains 60 special effect character styles, each style contains 19 fonts of English capital and lower case letters, and the total number of the special effect character picture pairs is 988 fonts. The resolution of the special effect word picture and the font picture is 320 multiplied by 320. 85% of the picture pairs are divided as a training set, and 15% of the picture pairs are divided as a test set. 300 cartoon or geometric texture pictures are collected, and synthesized special effect words are obtained through random combination to be used as supplement of training. Thereafter, 4000 sheets of vector graphics were collected as decorative elements. In the network training process, the decorative element pictures are randomly added to the collected or synthesized special effect word pictures to synthesize special effect words with decorations, and the special effect words are used as a synthesized special effect word domain. 1000 special effect character pictures are collected from channels such as a network and the like, and characters are manually marked to serve as a real special effect character domain.

Step 2: a mask extraction subnetwork is constructed.

The network structure is shown in fig. 1. The mask extraction sub-network adopts a U-net network structure. In the special effect word domain, firstly, the synthesized special effect word D with decoration and the matched word pattern picture C are merged and input in the channel dimensionNetwork, extracting sub-network from input data by mask to obtain mask of decoration elements

And simultaneously extracting the feature P of the penultimate layer. In the real special effect word field, firstly, the special effect word D with decoration is put_wCharacter pattern picture C matched with it_wMerging input networks in channel dimension, extracting sub-networks from input data through masks to obtain masks of decorative elements

Simultaneously extracting features P of the penultimate layer_w. Meanwhile, a discriminator network netSegD is established. netSegD consists of four convolution modules, each containing a convolution layer and a linear rectification function. netSegD outputs a determination of the picture domain in which the input picture is located.

And step 3: the training mask extracts the sub-network.

Total loss function L_maskThe device consists of two parts:

L_mask＝λ_segL_seg+λ_advL_adv，

parameter lambda_segIs set to 1, lambda_advSet to 0.01. L is_segExtract the loss function for the mask:

parameter lambda_L1Is set to 1, lambda_PerSet to 1, M is the labeled decoration element mask of the special effect word picture, VGG (·) is a five-layer extracted feature in ReLU1, ReLU2, ReLU3, ReLU4, ReLU5 using a pre-trained classification network VGG.

L_advExtract the penalty-fighting function for the mask:

L_adv＝-log(netSegD(P_w))，

netSegD(P_w) Inputting P for arbiter netSegD_wThe resulting output. netSegD is trained in judging defeatedAnd (3) in the picture domain from which the features come, wherein the loss function is as follows:

and 4, step 4: and building a basic special effect migration network.

The network structure is as shown in fig. 2, a multi-scale frame structure is adopted, and the network is difficult to directly train on high resolution, so that a training strategy with gradually improved resolution is adopted, the training difficulty is gradually increased from easy to difficult, and specifically, the network is trained sequentially under three resolutions of 64 × 64, 128 × 128 and 256 × 256. Inputting the special effect word D with synthesized decoration elements_yWhich is paired with a font picture C_yAnd target font picture C_x. The three input pictures are merged in a channel domain, and a migration result S is obtained after the three input pictures pass through a basic special effect migration network_x。

And 5: and training a basic special effect migration network. Total loss function L_transferConsists of two parts, using WGAN-GP:

parameter lambda_L1Is set to 1, lambda_advSet to 0.01. L is_advTo combat the loss function:

in the above formula, the first and second carbon atoms are,

meaning that the mean is calculated for X for distribution Y. D represents a discriminator network, and D (a, b, c, D) represents the discrimination result of a after the input of a, b, c, D by the discriminator network. The arbiter network adopts the network structure of PatchGAN.

Is a real target special effect word in the data set,

in distribution

And S_xTo be sampled uniformly.

Step 6: and building an intelligent word effect migration system framework. The trained mask extraction sub-network and the base special effects migration sub-network are combined as in fig. 3. Within the framework of the system, a given special effect word D to be migrated with decorative elements is first of all transferred_yAnd its paired character pattern picture C_yInputting the information into the trained mask extraction sub-network to obtain a decoration element mask M_y. Then the special effect word D with the decorative elements is put into_yWhich is paired with a font picture C_yAnd target font picture C_xInputting the data into a basic special effect migration sub-network to obtain a result S of basic special effect migration and decoration element elimination_x. Then carrying out element recombination to obtain the special effect character D with the decorative elements after the final style migration_x。

The element recombination part is firstly passed through C_yCalculating the level prior M_Hor. Definition of x_y,minIs C_yThe pixel position, x, of the foreground font at the leftmost end of each upper line_y,maxThe rightmost end thereof. Then generate

Wherein, K_w＝0.06(x_y,max-x_y,min)。

Thereafter, M is generated by recursion_Hor：

x_y,centerIs represented by C_yOf foreground fonts on each lineThe pixel position of the center.

Then M is added_HorNormalized to [0,1 ]]Interval, and fuzzy by Gaussian fuzzy kernel to obtain final M_Hor。

Similarly, by C_yCalculating vertical prior M_Ver. Definition of y_x,minIs C_yThe pixel position of the foreground font at the top of each column, y_x,maxFor its lowest end, an estimate of the vertical variation of the glyph may then be generated

Wherein, K'_w＝0.06(y_x,max-y_x,min)。

Thereafter, M is generated by recursion_Ver：

y_x,centerIs represented by C_yThe center of the foreground font of each column is located at the pixel position.

Then M is added_VerNormalized to [0,1 ]]Interval, and fuzzy by Gaussian fuzzy kernel to obtain final M_Ver。

At C_yUpper computation distribution prior M_Dis：

M_Dis＝(1-Dis(x,y))⁵，

Dis (x, y) denotes the glyph distribution prior calculated by Yang et al (cf. Shuai Yang, Jianying Liu, Zhouhui Lian, and Zongming Guo. "Awesound type: Statistics-Based Text Effects Transfer", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, Jul. 2017.).

Combining M in channel dimension_Hor、M_Ver、M_DisTo obtainTo total prior M_guide. Similarly, at C_xThe total prior is obtained by the upper calculation

Setting a superposed recording matrix M_ExiWhen the element position has a decorative element, M_Exi(x, y) is 0, otherwise M_Exi(x,y)＝1。

The element recombination was performed by first using DenseCrF on M_yAnd (6) optimizing. Then, at M_yTo get M_yScaled to 5 × 5 resolution and clustered with DBSCAN. Different decorative elements and their shades are obtained by finding connected shapes. For each decorative element, note it at C_yUpper part belongs to the area E and is searched to obtain the corresponding area C_xAt the suitable position E':

M_guide(·)，

and M_Exi(. respectively) indicates the regions in M_guide，

And M_ExiThe sum of the values of (a) and (b).

Decorative elements from D_yIs copied to S_xIf the superposition area of the decoration element and the font is reduced after copying, the decoration element is slightly reduced and is closer to the font, otherwise, the decoration element is slightly enlarged and is further away from the font, and the special effect character D with the decoration element after final style migration is obtained_x。

And 7: special effect word D to be migrated with decorative element_yAnd its corresponding font picture C_yTarget font picture C_xInputting the system frame to obtain the special effect character D with decorative elements after the final style migration_x。

The above embodiments are only intended to illustrate the technical solution of the present invention and not to limit the same, and a person skilled in the art can modify the technical solution of the present invention or substitute the same without departing from the spirit and scope of the present invention, and the scope of the present invention should be determined by the claims.

Claims (10)

1. An intelligent word effect migration method for special effect words comprises the following steps:

training a mask extraction sub-network to extract a decoration element mask by utilizing a training data set, and training a basic special effect migration sub-network to migrate a basic character special effect;

special effect word D with decorative elements_yAnd its paired character pattern picture C_yObtaining a decorative element mask M from the trained mask extraction sub-network_y；

Will D_yAnd its paired character pattern picture C_yTarget font picture C_xInputting the result into a trained basic special effect migration sub-network to obtain a result S of basic special effect migration and decoration element elimination_x；

Using M_y，C_yAnd C_xCarrying out element recombination to fuse decorative elements into S_xObtaining the moved special effect character D with the decorative elements corresponding to the target character pattern_x。

2. The method of claim 1, wherein the mask extraction sub-network is a U-net network structure, and a discriminator network netSegD is constructed for determining a picture domain in which the input picture is located, the netSegD comprising four convolution modules, each convolution module comprising a convolution layer and a linear rectification function.

3. The method of claim 2, wherein the training data set comprises a composite decorated special effect word picture D and its counterpart glyph picture C, and a collected and artificially annotated glyph decorated special effect word picture D_wAnd itPaired character pattern picture C_w(ii) a Wherein the picture D is obtained by randomly adding the decoration element picture to a collected or synthesized special effect word picture obtained by randomly combining the collected texture pictures to a character picture containing no special effect.

4. A method as claimed in claim 3, wherein the training method of mask extraction sub-networks comprises the steps of:

combining the D and the C in the channel dimension and inputting the mask to extract the sub-network to obtain the decoration element mask

Simultaneously extracting the feature P of the penultimate layer;

will D_wAnd C_wMerging the input masks in the channel dimension to extract sub-networks to obtain the decorative element mask

Simultaneously extracting features P of the penultimate layer_w；

According to

P、P_wAnd setting a loss function to obtain a trained mask extraction sub-network.

5. The method of claim 4, wherein the method is based on

P、P_wThe method of setting the loss function is as follows:

the loss function comprises a mask extraction loss function L_segSum mask extraction penalty function L_advThe total loss function L is composed of the two loss functions_mask：

L_mask＝λ_segL_seg+λ_advL_adv，

Wherein λ is_segIs set to 1, lambda_advSet to 0.01;

wherein λ is_L1Is set to 1, lambda_PerSetting as 1, M is a labeled decoration element mask of a special effect character picture, and VGG (·) is a feature extracted in five layers of ReLU1, ReLU2, ReLU3, ReLU4 and ReLU5 by using a pre-trained classification network VGG;

L_adv＝-log(netSegD(P_w))，

among them, netSegD (P)_w) Inputting P for arbiter netSegD_wThe resulting output, netSegD, is trained in the picture domain from which the input features are judged to come from, with a loss function of:

6. the method of claim 1, wherein the base special effect migration sub-network employs a multi-scale framework structure, and the sequential training of the network is performed by using a training strategy with resolution gradually increased, including sequential training at three resolutions of 64 x 64, 128 x 128, and 256 x 256.

7. Method according to claim 1 or 2, characterized in that the training of the base effect migration subnetwork comprises the following steps:

will D_y、C_y、C_xInputting the data into a basic special effect migration sub-network, merging the data in a channel domain to obtain a migration result S_x；

According to S_xAnd setting a loss function to obtain a trained basic special effect migration sub-network.

8. The method of claim 1, wherein the recombination of elements comprises the steps of:

using DenseCrF for M_yOptimizing;

will M_yScaling to a certain resolution, clustering by using DBSCAN, and obtaining different decorative elements and masks thereof by searching for connected shapes;

for in C_yEach decorative element belonging to the area E is searched to obtain the corresponding decorative element C_xThereby the decorative element is moved from D to E_yIs copied to S_xAdjusting the size of the decorative element to obtain D_x。

9. The method of claim 8, wherein for at C_yEach decorative element belonging to the area E is searched according to the following formula to obtain the corresponding decorative element in the area C_xAt the suitable position E':

wherein M is_guide(·)，

And M_Exi(. respectively) indicates the regions in M_guide，

And M_ExiThe sum of the values of (a) and (b).

M_guideThe obtaining method comprises the following steps: at C_yUpper computation level prior M_HorAnd vertical prior M_VerNormalized to [0,1 ]]Interval, fuzzy with Gaussian fuzzy kernel; at C_yUpper computation distribution prior M_Dis(ii) a Will obscure M_HorAnd M_yerAnd M_DisCombining in channel dimensions to yield M_guide；

Is at C_xThe same as M_guideThe method for obtaining.

10. A smart word effect migration system for special effect words, comprising:

the mask extraction sub-network module is used for inputting special effect characters with decorative elements and paired character patterns of the special effect characters and obtaining a decorative element mask from the special effect characters;

a basic special effect migration sub-network module, which is used for inputting special effect words with decorative elements, paired font pictures thereof and target font pictures, migrating the basic special effects of the special effect words to the target font, eliminating the decorative elements of the special effect words and outputting results;

and the element recombination module is used for fusing the decoration elements on the result output by the basic special effect migration sub-network module by utilizing the mask obtained by the mask extraction sub-network module to obtain the migrated special effect characters with the decoration elements corresponding to the target character patterns.

Images