CN117236284A - Font generation method and device based on adaptation of style information and content information - Google Patents

Abstract

The invention discloses a font generation method and device based on the adaptation of style information and content information. The method includes: obtaining a current source font picture and a current reference style font picture set, and combining the current source font picture and the current reference style font picture set. Input it into the preset font generation model for calculation to generate a target style font image corresponding to the current source font image. This solution automatically generates target style font pictures through a pre-built preset font generation model, which improves the efficiency of generating target style font pictures; moreover, the preset font generation model makes full use of the style characteristics of multiple current reference style font pictures. The accuracy of generating target style font images is improved; at the same time, since the preset number of historical reference style font images are randomly selected, the preset font generation model can learn more features, thereby further improving the generation target Style font image accuracy.

Description

Font generation method and device based on adaptation of style information and content information

Technical Field

The present invention relates to the field of computer technology, and in particular to a font generation method and device based on adaptation of style information and content information.

Background Art

Chinese fonts are huge in number and complex in structure, and the styles of each font vary greatly. Since manual font design is time-consuming and labor-intensive, and requires professional designers to design, the font generation task has become an important research direction.

In the related art, when generating fonts of different styles, the strokes, components, structures and other information are manually annotated, and the annotated information is used as prior information to generate fonts of the required style. However, there is a problem of low generation efficiency when generating fonts using the above method.

Summary of the invention

The present invention aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of the present invention proposes a font generation method based on adaptation of style information and content information, the method comprising:

Obtaining a current source font image and a current reference style font image set; wherein the current reference style font image set includes a preset number of randomly selected current reference style font images, each current reference style font image corresponds to different content information, and each current reference style font image corresponds to the same style information;

The current source font image and the current reference style font image set are input into a preset font generation model for calculation to generate a target style font image corresponding to the current source font image; wherein the content information of the target style font image is the same as the content information of the current source font image, and the style information of the target style font image is the same as the style information of each current reference style font image.

In a possible implementation, the preset font generation model includes a content encoder, a style encoder, a style content feature adaptation module and a decoder, and the current source font image and the current reference style font image set are input into the preset font generation model for calculation to generate a target style font image corresponding to the current source font image, including:

Inputting the current source font image and the current reference style font image set into a preset font generation model, performing feature extraction on the current source font image through a content encoder, and generating a first content feature;

Extracting features from the current reference style font picture set through a style encoder to generate a first style feature set; wherein the first style feature set includes first style features corresponding to each current reference style font picture;

The first content feature and the first style feature set are fused by a style content feature adaptation module to generate a fused feature;

The fused features are decoded through the decoder to generate the target style font image.

In a possible implementation, the first content feature and the first style feature set are fused by the style content feature adaptation module to generate a fused feature, including:

The first content feature and the first style feature set are connected by the style content feature adaptation module to generate a connection feature; the connection feature includes a second content feature corresponding to the first content feature and a second style feature set corresponding to the first style feature set, and the second style feature set includes a second style feature corresponding to each first style feature;

Calculating a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set;

Based on the first weight and the second weight, a weighted fusion process is performed on the second content feature and the second style feature set to generate a fusion feature.

In a possible implementation, calculating a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set includes:

Sum the connected features to generate combined features;

Perform global average pooling on the combined features to generate a feature vector;

Compress the feature vector to generate compressed features;

Performing conversion processing on the compressed features to generate a first probability distribution corresponding to the second content feature and a second probability distribution corresponding to the second style feature set;

Based on the first probability distribution and the second probability distribution, a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set are calculated respectively.

In a possible implementation manner, the preset number of the multiple current reference style font images is less than or equal to six current reference style font images.

In a possible implementation, the process of constructing the preset font generation model includes:

Acquire a training sample set and a target image; wherein the training sample set includes a historical source font image and a plurality of types of historical reference style font image sets, each type of historical reference style font image set corresponds to different style information, the historical reference style font image set includes a preset number of randomly selected historical reference style font images, each historical reference style font image corresponds to different content information, and each historical reference style font image corresponds to the same style information;

Input the training sample set and the target image into the initial font generation model for training to obtain the output result;

An overall loss value is calculated based on the output result, and the model parameters are updated according to the overall loss value, and a preset font generation model is generated based on the updated model parameters.

In a possible implementation, calculating the overall loss value based on the output result includes:

Based on the output results and the target image, calculate the mean absolute error loss value;

Calculate the content adversarial loss value based on the output results and historical source font images;

For various types of historical reference style font image collections, calculate the style adversarial loss value based on the output results and the historical reference style font image collections;

Based on the mean absolute error loss value, content adversarial loss value and style adversarial loss value, an overall loss value is generated.

A second aspect of the present invention provides a font generation device based on adaptation of style information and content information, the device comprising:

An acquisition module, used to acquire a current source font image and a current reference style font image set; wherein the current reference style font image set includes a preset number of current reference style font images randomly selected, each current reference style font image corresponds to different content information, and each current reference style font image corresponds to the same style information;

A generation module is used to input the current source font image and the current reference style font image set into a preset font generation model for calculation, and generate a target style font image corresponding to the current source font image; wherein the content information of the target style font image is the same as the content information of the current source font image, and the style information of the target style font image is the same as the style information of each current reference style font image.

In a possible implementation, the preset font generation model includes a content encoder, a style encoder, a style content feature adaptation module and a decoder, and the above generation module is specifically used for:

Inputting the current source font image and the current reference style font image set into a preset font generation model, performing feature extraction on the current source font image through a content encoder, and generating a first content feature;

Extracting features from the current reference style font picture set through a style encoder to generate a first style feature set; wherein the first style feature set includes first style features corresponding to each current reference style font picture;

The first content feature and the first style feature set are fused by a style content feature adaptation module to generate a fused feature;

The fused features are decoded through the decoder to generate the target style font image.

In a possible implementation manner, the generation module is further used to:

The first content feature and the first style feature set are connected by the style content feature adaptation module to generate a connection feature; the connection feature includes a second content feature corresponding to the first content feature and a second style feature set corresponding to the first style feature set, and the second style feature set includes a second style feature corresponding to each first style feature;

Calculating a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set;

Based on the first weight and the second weight, a weighted fusion process is performed on the second content feature and the second style feature set to generate a fusion feature.

In a possible implementation manner, the generation module is further used to:

Sum the connected features to generate combined features;

Perform global average pooling on the combined features to generate a feature vector;

Compress the feature vector to generate compressed features;

Performing conversion processing on the compressed features to generate a first probability distribution corresponding to the second content feature and a second probability distribution corresponding to the second style feature set;

Based on the first probability distribution and the second probability distribution, a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set are calculated respectively.

In a possible implementation manner, the preset number of the multiple current reference style font images is less than or equal to six current reference style font images.

In a possible implementation manner, the above-mentioned font generation device based on adaptation of style information and content information is further used for:

Acquire a training sample set and a target image; wherein the training sample set includes a historical source font image and a plurality of types of historical reference style font image sets, each type of historical reference style font image set corresponds to different style information, the historical reference style font image set includes a preset number of randomly selected historical reference style font images, each historical reference style font image corresponds to different content information, and each historical reference style font image corresponds to the same style information;

Input the training sample set and the target image into the initial font generation model for training to obtain the output result;

An overall loss value is calculated based on the output result, and the model parameters are updated according to the overall loss value, and a preset font generation model is generated based on the updated model parameters.

In a possible implementation manner, the above-mentioned font generation device based on adaptation of style information and content information is further used for:

Based on the output results and the target image, calculate the mean absolute error loss value;

Calculate the content adversarial loss value based on the output results and historical source font images;

For various types of historical reference style font image collections, calculate the style adversarial loss value based on the output results and the historical reference style font image collections;

Based on the mean absolute error loss value, content adversarial loss value and style adversarial loss value, an overall loss value is generated.

A third aspect of the present invention proposes an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the font generation method based on adaptation of style information and content information as described in the first aspect.

A fourth aspect of the present invention proposes a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the font generation method based on adaptation of style information and content information as described in the first aspect.

The embodiments of the present invention have the following beneficial effects:

The embodiment of the present invention provides a font generation method based on the adaptation of style information and content information, the method comprising: obtaining a current source font image and a current reference style font image set; wherein the current reference style font image set includes a preset number of current reference style font images randomly selected, each current reference style font image corresponds to different content information, and each current reference style font image corresponds to the same style information; the current source font image and the current reference style font image set are input into a preset font generation model for calculation to generate a target style font image corresponding to the current source font image. This scheme automatically generates a target style font image through a pre-constructed preset font generation model, thereby improving the efficiency of generating a target style font image; and the preset font generation model makes full use of the style features of multiple current reference style font images, and improves the accuracy of generating a target style font image by effectively matching these style features with the content features of the current source font image; at the same time, since the preset number of historical reference style font images are randomly selected, the preset font generation model can learn more features, thereby further improving the accuracy of generating a target style font image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a different font style provided in an embodiment of the present application;

FIG2 is a block diagram of a computer device provided in an embodiment of the present application;

FIG3 is a flowchart of a method for generating a font based on adaptation of style information and content information provided by an embodiment of the present invention;

FIG4 is a flow chart of generating a target style font image provided by an embodiment of the present invention;

FIG5 is an overall framework diagram of generating a target style font image provided by an embodiment of the present invention;

FIG6 is a flow chart of generating fusion features provided by an embodiment of the present invention;

FIG7 is a flow chart of calculating a first weight and a second weight provided by an embodiment of the present invention;

FIG8 is a flow chart of constructing a preset font generation model provided by an embodiment of the present invention;

FIG9 is a flow chart of calculating an overall loss value provided by an embodiment of the present invention;

FIG10 is a schematic diagram of the results of an ablation experiment provided in an embodiment of the present application;

FIG11 is a schematic diagram of test results of a preset font generation model and other models for an unseen character set of a seen font provided by an embodiment of the present application;

FIG12 is a schematic diagram of test results of a preset font generation model and other models for a seen character set of an unseen font and an unseen character set of an unseen font provided by an embodiment of the present application;

FIG. 13 is a structural block diagram of a font generation device based on adaptation of style information and content information provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

There are a large number of Chinese fonts with complex structures, and the styles of each font vary greatly. Since manual font design is time-consuming and labor-intensive, and requires professional designers to design, font generation has become an important research direction. In related technologies, when generating fonts of different styles, the strokes, components, structures and other information are manually annotated, and the annotated information is used as prior information to generate the font of the required style.

Specifically, the existing manual font design technology is not only time-consuming and labor-intensive, but also needs to be completed by professionals. In addition, the current mainstream small sample generation model is based on the style-content separation paradigm, which is mainly based on font information such as strokes, components, and structures. It does not take into account that the style characteristics behind each character in the same style are different. As shown in Figure 1, Figure 1 is a schematic diagram of a different font style provided in an embodiment of the present application. It can be seen that different font styles are mainly reflected in shape, thickness, angle, curvature, etc. These factors have different effects on visual presentation and emotional communication. Therefore, the efficiency and accuracy of style font image generation using existing methods are low.

In view of this, the present application proposes a font generation method and device based on the adaptation of style information and content information, which automatically generates a target style font image through a pre-constructed preset font generation model, thereby improving the efficiency of generating the target style font image; and, the preset font generation model makes full use of the style features of multiple current reference style font images, and improves the accuracy of generating the target style font image by effectively matching these style features with the content features of the current source font image; at the same time, since a preset number of multiple historical reference style font images are randomly selected, the preset font generation model can learn more features, thereby further improving the accuracy of generating the target style font image.

In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values may be based on additional conditions or values beyond the described values in practice.

The font generation method based on adaptation of style information and content information provided in the present application can be applied to a computer device (electronic device), and the computer device can be a server or a terminal, wherein the server can be a single server or a server cluster composed of multiple servers. The embodiments of the present application do not make specific limitations on this, and the terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices.

Taking the computer device as a server as an example, FIG2 shows a block diagram of a server. As shown in FIG2, the server may include a processor and a memory connected via a system bus. The processor of the server is used to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a font generation method based on the adaptation of style information and content information is implemented.

Those skilled in the art will understand that the structure shown in FIG. 2 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. Optionally, the server may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

It should be noted that the execution subject of the embodiments of the present application may be a computer device or a font generation device based on adaptation of style information and content information. The following method embodiments will be described with a computer device as the execution subject.

FIG3 is a flowchart of a method for generating a font based on adaptation of style information and content information provided by an embodiment of the present invention. As shown in FIG3 , the method includes the following steps:

Step 302: Obtain the current source font image and the current reference style font image set.

The current reference style font picture set includes a preset number of current reference style font pictures randomly selected, each current reference style font picture corresponds to different content information, and each current reference style font picture corresponds to the same style information.

Therefore, when generating a target font style image, it is necessary to first obtain the current source font image and the current reference style font image set. The current source font image contains the corresponding content information, that is, what specific characters are, and the current reference style font image set contains the corresponding style information, such as different font styles such as Kaiti and Songti, so that the target style font image can be generated based on the current source font image and the current reference style font image set.

Step 304: Input the current source font image and the current reference style font image set into a preset font generation model for calculation to generate a target style font image corresponding to the current source font image.

The content information of the target style font image is the same as the content information of the current source font image, and the style information of the target style font image is the same as the style information of each current reference style font image.

In some optional embodiments, the above-mentioned preset font generation model is a model for generating a target style font picture based on the current source font picture and the current reference style font picture set. The preset font generation model may include a content encoder, a style encoder, a style content feature adaptation module and a decoder. The current source font picture and the current reference style font picture set are input into the preset font generation model for calculation. When a target style font picture corresponding to the current source font picture is generated, as shown in FIG. 4, FIG. 4 is a flowchart of generating a target style font picture provided by an embodiment of the present invention, including:

Step 402: Input the current source font image and the current reference style font image set into a preset font generation model, perform feature extraction on the current source font image through a content encoder, and generate a first content feature.

Step 404: extract features from the current reference style font image set through a style encoder to generate a first style feature set.

Step 406: The first content feature and the first style feature set are fused by the style content feature adaptation module to generate a fused feature.

Step 408: decode the fused features through a decoder to generate a target style font image.

Please refer to 5, which is an overall framework diagram of generating a target style font image provided by an embodiment of the present invention. The current source font image is also the content image _xc , and the current reference style font image set is also the style image , the content encoder is E _c , the style encoder is _Es , and the decoder is D .

After the current source font image and the current reference style font image set are input into the preset font generation model, the current source font image can be firstly feature extracted by the content encoder E _c to generate a first content feature f _c , the size of which is C×H×W, where C represents the number of channels, and H and W represent the height and width of the first content feature f _{c ,} respectively. In addition, the current reference style font image set can be feature extracted by the style encoder _Es to generate a first style feature set _fi . The first style feature set _fi includes the first style features corresponding to each current reference style font image, i.e., f ₁ to f _k , and the size of the first style feature set _fi is k×C×H×W, where k represents the maximum number of current reference style font images.

Thus, the first content feature f _c and the first style feature set _fi can be fused by the style content feature adaptation module to generate a fused feature. In some optional embodiments, as shown in FIG6 , FIG6 is a flow chart of generating a fused feature provided by an embodiment of the present invention, including:

Step 602: Connect the first content feature and the first style feature set through the style content feature adaptation module to generate a connection feature.

Step 604: Calculate a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set.

Step 606: Based on the first weight and the second weight, perform weighted fusion processing on the second content feature and the second style feature set to generate a fusion feature.

Among them, after the style content feature adaptation module performs a connection operation on the first content feature and the first style feature set to generate a connection feature, the connection feature includes the second content feature corresponding to the first content feature f _c , and the second style feature set corresponding to the first style feature set _fi The second style feature set The second style features corresponding to each first style feature are included, that is, the second style features corresponding to f ₁ to f _k respectively. to The size of the connected features is (k+1)×C×H×W.

Next, the first weight corresponding to the second content feature and the second weight corresponding to the second style feature set may be calculated, and finally the second content feature and the second style feature set may be weighted fused using the first weight and the second weight to generate a fused feature.

In this embodiment, since the current reference style font image set contains multiple current reference style font images, the preset font generation model makes full use of the style features of the multiple current reference style font images, and achieves excellent font generation effect by effectively matching these style features with the content features of the current source font image; and, by effectively fusing the second content feature and the second style feature set, the expressiveness and diversity of the generated target style font image are further enhanced. This fusion process not only improves the model's ability to generate fonts, but also can produce font designs with unique style and artistic sense, providing a new and effective solution for the field of small sample font generation.

In some optional embodiments, as shown in FIG. 7 , FIG. 7 is a flowchart of calculating a first weight and a second weight provided by an embodiment of the present invention, including:

Step 702: sum the connection features to generate a combined feature.

Step 704: Perform global average pooling processing on the combined features to generate a feature vector.

Step 706: compress the feature vector to generate a compressed feature.

Step 708: convert the compressed features to generate a first probability distribution corresponding to the second content feature and a second probability distribution corresponding to the second style feature set.

Step 710: Calculate a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set based on the first probability distribution and the second probability distribution.

Among them, the connection features can be summed up first to generate a combined feature U. Optionally, the features in the connection features can be added together to obtain the combined feature U. The size of the combined feature U is C×H×W, which can be calculated by formula (1). Then, the combined feature U can be subjected to global average pooling to extract global information, and the size of the combined feature U can be reduced to generate a feature vector S. The size of the feature vector S is C×1×1, which can be calculated by formula (2).

(1)

(2)

In order to further reduce the size of the feature vector S and improve efficiency, a simple fully connected layer (FC) can be used to compress the feature vector to generate a compressed feature Z whose size is D×1×1.

In addition, the compressed features can be transformed through softmax mapping to generate a first probability distribution corresponding to the second content feature and a second probability distribution corresponding to the second style feature set. Furthermore, based on the first probability distribution and the second probability distribution, the first weight w _c corresponding to the second content feature can be calculated, which can be calculated by formula (3), and the second weight w _i corresponding to the second style feature set can be calculated, which is to The corresponding w ₁ to w _k can be calculated by formula (4).

(3)

(4)

in, represents a first probability distribution corresponding to the second content feature, represents a second probability distribution corresponding to the second style feature set.

Next, the second content feature and the second style feature set can be weighted fused based on the first weight and the second weight to generate a fused feature F, the size of which is C×H×W. Optionally, the second content feature and the second style feature set can be multiplied by the first weight and the second weight respectively to calculate the contribution of each branch to the final fused feature F. In essence, this process involves weighted fusion of feature information from each branch under the guidance of their respective weights, with the goal of creating a unified and comprehensive feature representation. Specifically, it can be calculated using formula (5).

(5)

Finally, after decoding the fused feature F through the decoder D, the target style font image y can be generated.

In some optional embodiments, the preset font generation model is a model for generating a target style font image based on a current source font image and a current reference style font image set. The construction process of the preset font generation model is shown in FIG8 , which is a flow chart of constructing a preset font generation model provided by an embodiment of the present invention, including:

Step 802: Obtain a training sample set and a target image.

Step 804: input the training sample set and the target image into the initial font generation model for training to obtain an output result.

Step 806: Calculate the overall loss value based on the output result, update the model parameters according to the overall loss value, and generate a preset font generation model based on the updated model parameters.

The training sample set includes historical source font images and various types of historical reference style font image sets, each type of historical reference style font image set corresponds to different style information, and the historical reference style font image set includes a preset number of randomly selected historical reference style font images, each historical reference style font image corresponds to different content information, and each historical reference style font image corresponds to the same style information. It is a pre-set standard image used to calculate the corresponding loss value during the model training process. It should be noted that the historical source font image can also be represented by x _c , and the historical reference style font image collection can also be represented by X _s .

When the training sample set and the target image are input into the initial font generation model for training and the output result is obtained, the output result can also be represented as y, and the specific process of obtaining the output result can refer to the process of generating the target style font image y when using the model in the above embodiment, which will not be repeated here.

In some optional embodiments, when calculating the overall loss value based on the output result, as shown in FIG. 9 , FIG. 9 is a flow chart of calculating the overall loss value provided by an embodiment of the present invention, including:

Step 902: Calculate the mean absolute error loss value based on the output result and the target image.

Step 904: Calculate the content resistance loss value based on the output result and the historical source font image.

Step 906: For various types of historical reference style font image sets, calculate the style adversarial loss value based on the output result and the historical reference style font image set.

Step 908: Generate an overall loss value based on the mean absolute error loss value, the content adversarial loss value, and the style adversarial loss value.

Among them, the above-mentioned preset font generation model is a generative adversarial network architecture as a whole. The generator network generates new data by learning the distribution of the training sample set, while the discriminator network tries to distinguish the data generated by the generator from the real training data. The two networks compete with each other. Finally, the generator network can generate new data similar to the training data.

During the training process, the relationship between the generator and the discriminator is constrained by the mean absolute error loss value, i.e., the _L1 loss and the adversarial loss. The adversarial loss consists of the content adversarial loss and the style adversarial loss. Thus, the above-mentioned preset font generation model may also include a style discriminator D _s and a content discriminator D _c . The style discriminator D _s is used to calculate the style adversarial loss value. , the content discriminator D _c is used to calculate the content adversarial loss value .

Next, based on the output result and the target image, the mean absolute error loss value L ₁ can be calculated, which is the pixel-level difference between the output result and the target image. Specifically, it can be calculated using formula (6).

(6)

in, Indicates when From When distributed, the expectation of the formula ×.

Then, based on the output results and historical source font images, calculate the content adversarial loss value , content adversarial loss value Generate loss from content and content identification loss Composition, loss identification by content , the discriminator can better identify whether the generated image, that is, the output result, has the same content as the historical source font image, through the content generation loss , the generator can make the output result closer to the content of the historical source font image. Specifically, the content adversarial loss value can be calculated by formula (7): , the content generation loss is calculated by formula (8) , the content identification loss is calculated by formula (9): .

(7)

(8)

(9)

in, Indicates when From When distributed, the expectation of the formula ×. Indicates that The resulting value that is input into the content discriminator. Indicates when From When distributed, the expectation of the formula ×. Indicates that The resulting value that is input into the content discriminator.

In addition, for various types of historical reference style font image collections, the style adversarial loss value can be calculated based on the output results and the historical reference style font image collections. Style adversarial loss Generating loss from style and style discrimination loss Composition, through style identification loss , the discriminator can better identify whether the generated image, that is, whether the output result has the same style as the historical reference style font image, through the style generation loss , the generator can make the output result closer to the style of the historical reference font image. Specifically, the style adversarial loss value can be calculated by formula (10): , the style generation loss is calculated by formula (11) , the style identification loss is calculated by formula (12) .

(10)

(11)

(12)

in, Indicates when From When distributed, the expectation of the formula ×. Indicates that The resulting value that is input into the style discriminator. Indicates when From When distributed, the expectation of the formula ×. Indicates that The resulting value that is input into the style discriminator.

Finally, the overall loss value L _total can be generated based on the mean absolute error loss value L ₁ , the content adversarial loss value L _advc and the style adversarial loss value L _advs . Specifically, it can be calculated using formula (13).

(13)

in, It means trying to minimize the formula from the generator’s perspective. It means trying to maximize the formula from the perspective of the discriminator, and λ _adv represents a hyperparameter that can be customized in advance. Represents a number ranging from 1 to 100.

In this embodiment, the overall loss value is obtained by combining the three loss values of mean absolute error loss value, content adversarial loss value and style adversarial loss value, so that the parameters of the model can be adjusted more reasonably and accurately, so that the final accuracy of your preset font generation model is higher.

In some optional embodiments, during the use of the preset font generation model, the preset number of multiple current reference style font images is less than or equal to six current reference style font images. Similarly, during the construction of the preset font generation model, the various types of historical reference style font image sets include a preset number of randomly selected historical reference style font images that are also less than or equal to six.

In this embodiment, the preset font generation model can be trained with a small amount of sample data, and the user only needs to provide a small amount of sample characters to generate a rich variety of new fonts, which greatly improves the efficiency and creativity of font design; in addition, since a preset number of historical reference style font images are randomly selected, the preset font generation model can learn more features, thereby improving the accuracy of generating target style font images.

Moreover, the ablation experiment shows that the effect is optimal when there are six historical reference style font pictures in the same type of historical reference style font picture set, as shown in Figure 10, which is a schematic diagram of the results of an ablation experiment provided in an embodiment of the present application. Among them, the X-axis represents the number k of historical reference style font pictures in the same type of historical reference style font picture set, and the Y-axis represents the image similarity evaluation index (Fréchet Inception Distance, referred to as FID), which is a measure used to calculate the distance between the feature vectors of the real image and the generated image. If the FID value is smaller, the similarity is higher, and the best case is FID=0, and the two images are the same.

In addition, in order to evaluate the preset font generation model, we can use the seen fonts seen characters (SFSC) as training and the seen fonts unseen characters (SFUC) as testing. We divide the other task into two subtasks: unseen fonts unseen characters (UFSC) and unseen fonts unseen characters (UFUC), and use the model trained in the seen fonts seen characters task to directly test these two subtasks.

Thus, as shown in Figure 11, Figure 11 is a schematic diagram of the test results of a preset font generation model provided in an embodiment of the present application and other models for an unseen character set of a seen font, and Figure 12 is a schematic diagram of the test results of a preset font generation model provided in an embodiment of the present application and other models for an unseen character set of an unseen font and an unseen character set of an unseen font. Among them, the framed parts in Figures 11 and 12 are fonts with poor generation effects, and the second to last row is the generation result corresponding to the preset font generation model provided in the present application. It can be seen that the generation result of the preset font generation model in the present application is good, and the last row is the target image. The other rows are the test results of other models for an unseen character set of an unseen font and an unseen character set of an unseen font. In addition, the left half of Figure 12 separated by a vertical line is the test result corresponding to the unseen character set of an unseen font, and the right half is the test result corresponding to the seen character set of an unseen font.

In an embodiment of the present application, the method includes: obtaining a current source font image and a current reference style font image set; wherein the current reference style font image set includes a preset number of randomly selected current reference style font images, each current reference style font image corresponds to different content information, and each current reference style font image corresponds to the same style information; the current source font image and the current reference style font image set are input into a preset font generation model for calculation to generate a target style font image corresponding to the current source font image. This scheme automatically generates a target style font image through a pre-constructed preset font generation model, thereby improving the efficiency of generating a target style font image; and, the preset font generation model makes full use of the style features of multiple current reference style font images, and improves the accuracy of generating a target style font image by effectively matching these style features with the content features of the current source font image; at the same time, since a preset number of multiple historical reference style font images are randomly selected, the preset font generation model can learn more features, thereby further improving the accuracy of generating a target style font image.

It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

FIG. 13 is a structural block diagram of a font generation device based on adaptation of style information and content information provided by an embodiment of the present invention.

As shown in FIG. 13 , the font generation device 1300 based on adaptation of style information and content information includes:

The acquisition module 1302 is used to obtain the current source font image and the current reference style font image set; wherein the current reference style font image set includes a preset number of randomly selected current reference style font images, each of which corresponds to different content information, and each of which corresponds to the same style information.

The generation module 1304 is used to input the current source font image and the current reference style font image set into a preset font generation model for calculation, and generate a target style font image corresponding to the current source font image; wherein the content information of the target style font image is the same as the content information of the current source font image, and the style information of the target style font image is the same as the style information of each of the current reference style font images.

Regarding the device in the above embodiment, the specific way in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here. Each module in the above font generation device based on the adaptation of style information and content information can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations of the above modules.

In one embodiment of the present application, a computer device is provided, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

Obtaining a current source font image and a current reference style font image set; wherein the current reference style font image set includes a preset number of randomly selected current reference style font images, each current reference style font image corresponds to different content information, and each current reference style font image corresponds to the same style information;

The current source font image and the current reference style font image set are input into a preset font generation model for calculation to generate a target style font image corresponding to the current source font image; wherein the content information of the target style font image is the same as the content information of the current source font image, and the style information of the target style font image is the same as the style information of each current reference style font image.

In one embodiment of the present application, the preset font generation model includes a content encoder, a style encoder, a style content feature adaptation module and a decoder, and the processor further implements the following steps when executing the computer program:

Inputting the current source font image and the current reference style font image set into a preset font generation model, performing feature extraction on the current source font image through a content encoder, and generating a first content feature;

Extracting features from the current reference style font picture set through a style encoder to generate a first style feature set; wherein the first style feature set includes first style features corresponding to each current reference style font picture;

The first content feature and the first style feature set are fused by a style content feature adaptation module to generate a fused feature;

The fused features are decoded through the decoder to generate the target style font image.

In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

The first content feature and the first style feature set are connected by the style content feature adaptation module to generate a connection feature; the connection feature includes a second content feature corresponding to the first content feature and a second style feature set corresponding to the first style feature set, and the second style feature set includes a second style feature corresponding to each first style feature;

Calculating a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set;

Based on the first weight and the second weight, a weighted fusion process is performed on the second content feature and the second style feature set to generate a fusion feature.

In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

Sum the connected features to generate combined features;

Perform global average pooling on the combined features to generate a feature vector;

Compress the feature vector to generate compressed features;

Performing conversion processing on the compressed features to generate a first probability distribution corresponding to the second content feature and a second probability distribution corresponding to the second style feature set;

Based on the first probability distribution and the second probability distribution, a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set are calculated respectively.

In one embodiment of the present application, the preset number of the multiple current reference style font pictures is less than or equal to six current reference style font pictures.

In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

Acquire a training sample set and a target image; wherein the training sample set includes a historical source font image and a plurality of types of historical reference style font image sets, each type of historical reference style font image set corresponds to different style information, the historical reference style font image set includes a preset number of randomly selected historical reference style font images, each historical reference style font image corresponds to different content information, and each historical reference style font image corresponds to the same style information;

Input the training sample set and the target image into the initial font generation model for training to obtain the output result;

An overall loss value is calculated based on the output result, and the model parameters are updated according to the overall loss value, and a preset font generation model is generated based on the updated model parameters.

In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

Based on the output results and the target image, calculate the mean absolute error loss value;

Calculate the content adversarial loss value based on the output results and historical source font images;

For various types of historical reference style font image collections, calculate the style adversarial loss value based on the output results and the historical reference style font image collections;

Based on the mean absolute error loss value, content adversarial loss value and style adversarial loss value, an overall loss value is generated.

The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, which will not be repeated here.

In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

Obtaining a current source font image and a current reference style font image set; wherein the current reference style font image set includes a preset number of randomly selected current reference style font images, each current reference style font image corresponds to different content information, and each current reference style font image corresponds to the same style information;

The current source font image and the current reference style font image set are input into a preset font generation model for calculation to generate a target style font image corresponding to the current source font image; wherein the content information of the target style font image is the same as the content information of the current source font image, and the style information of the target style font image is the same as the style information of each current reference style font image.

In one embodiment of the present application, the preset font generation model includes a content encoder, a style encoder, a style content feature adaptation module and a decoder, and when the computer program is executed by the processor, the following steps are also implemented:

Inputting the current source font image and the current reference style font image set into a preset font generation model, performing feature extraction on the current source font image through a content encoder, and generating a first content feature;

Extracting features from the current reference style font picture set through a style encoder to generate a first style feature set; wherein the first style feature set includes first style features corresponding to each current reference style font picture;

The first content feature and the first style feature set are fused by a style content feature adaptation module to generate a fused feature;

The fused features are decoded through the decoder to generate the target style font image.

In one embodiment of the present application, when the computer program is executed by a processor, the following steps are implemented:

The first content feature and the first style feature set are connected by the style content feature adaptation module to generate a connection feature; the connection feature includes a second content feature corresponding to the first content feature and a second style feature set corresponding to the first style feature set, and the second style feature set includes a second style feature corresponding to each first style feature;

Calculating a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set;

Based on the first weight and the second weight, a weighted fusion process is performed on the second content feature and the second style feature set to generate a fusion feature.

In one embodiment of the present application, when the computer program is executed by a processor, the following steps are implemented:

Sum the connected features to generate combined features;

Perform global average pooling on the combined features to generate a feature vector;

Compress the feature vector to generate compressed features;

Performing conversion processing on the compressed features to generate a first probability distribution corresponding to the second content feature and a second probability distribution corresponding to the second style feature set;

Based on the first probability distribution and the second probability distribution, a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set are calculated respectively.

In one embodiment of the present application, the preset number of the multiple current reference style font pictures is less than or equal to six current reference style font pictures.

In one embodiment of the present application, when the computer program is executed by a processor, the following steps are implemented:

Acquire a training sample set and a target image; wherein the training sample set includes a historical source font image and a plurality of types of historical reference style font image sets, each type of historical reference style font image set corresponds to different style information, the historical reference style font image set includes a preset number of randomly selected historical reference style font images, each historical reference style font image corresponds to different content information, and each historical reference style font image corresponds to the same style information;

Input the training sample set and the target image into the initial font generation model for training to obtain the output result;

An overall loss value is calculated based on the output result, and the model parameters are updated according to the overall loss value, and a preset font generation model is generated based on the updated model parameters.

In one embodiment of the present application, when the computer program is executed by a processor, the following steps are implemented:

Based on the output results and the target image, calculate the mean absolute error loss value;

Calculate the content adversarial loss value based on the output results and historical source font images;

For various types of historical reference style font image collections, calculate the style adversarial loss value based on the output results and the historical reference style font image collections;

Based on the mean absolute error loss value, content adversarial loss value and style adversarial loss value, an overall loss value is generated.

The computer-readable storage medium provided in this embodiment has similar implementation principles and technical effects to those of the above method embodiments, and will not be described in detail here.

Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and/or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (8)

1. A font generation method based on adaptation of style information and content information, characterized in that the method includes: Obtain the current source font picture and the current reference style font picture set; wherein the current reference style font picture set includes a randomly selected preset number of current reference style font pictures, and each of the current reference style font pictures corresponds to different content information, and each of the current reference style font pictures corresponds to the same style information; The current source font picture and the current reference style font picture set are input into the preset font generation model, and the content encoder in the preset font generation model performs feature extraction on the current source font picture to generate First content characteristics; The style encoder in the preset font generation model performs feature extraction on the current reference style font picture set to generate a first style feature set; wherein the first style feature set includes the same as each of the current reference The first style feature corresponding to the style font picture; The first content feature and the first style feature set are connected through the style content feature adaptation module in the preset font generation model to generate a connection feature; the connection feature includes the first content feature and the first style feature set. a second content feature corresponding to the content feature, and a second style feature set corresponding to the first style feature set, where the second style feature set includes a second style feature corresponding to each of the first style features; Calculate a first weight corresponding to the second content feature and a second weight corresponding to the second set of style features; Based on the first weight and the second weight, perform a weighted fusion process on the second content feature and the second style feature set to generate a fusion feature; The decoder in the preset font generation model decodes the fusion features to generate a target style font picture corresponding to the current source font picture; the content information of the target style font picture is consistent with the current source font picture. The content information of the font pictures is the same, and the style information of the target style font picture is the same as the style information of each of the current reference style font pictures. 2. The method according to claim 1, characterized in that the calculation of the first weight corresponding to the second content feature and the second weight corresponding to the second style feature set includes: Perform summation processing on the connection features to generate combined features; Perform global average pooling processing on the combined features to generate feature vectors; Perform compression processing on the feature vector to generate compressed features; Convert the compression features to generate a first probability distribution corresponding to the second content feature and a second probability distribution corresponding to the second style feature set; Based on the first probability distribution and the second probability distribution, a first weight corresponding to the second content feature and a second weight corresponding to the second style feature set are respectively calculated. 3. The method according to claim 1 or 2, characterized in that the preset number of multiple current reference style font pictures is less than or equal to six current reference style font pictures. 4. The method according to claim 1 or 2, characterized in that the construction process of the preset font generation model includes: Obtain a training sample set and a target image; wherein, the training sample set includes historical source font images and multiple types of historical reference style font image sets, and various types of historical reference style font image sets correspond to different style information, The collection of historical reference style font pictures includes a preset number of randomly selected historical reference style font pictures, each of the historical reference style font pictures corresponds to different content information, and each of the historical reference style font pictures corresponds to the same style information; Input the training sample set and the target image into the initial font generation model for training to obtain an output result; Calculate an overall loss value based on the output result, update model parameters based on the overall loss value, and generate the preset font generation model based on the updated model parameters. 5. The method of claim 4, wherein calculating the overall loss value based on the output result includes: Based on the output result and the target image, calculate an average absolute error loss value; Based on the output result and the historical source font image, calculate the content confrontation loss value; For various types of historical reference style font picture sets, calculate a style confrontation loss value based on the output result and the historical reference style font picture set; The overall loss value is generated based on the mean absolute error loss value, the content adversarial loss value, and the style adversarial loss value. 6. A font generation device based on adaptation of style information and content information, characterized in that the device includes: The acquisition module is used to obtain the current source font picture and the current reference style font picture set; wherein the current reference style font picture set includes a randomly selected preset number of current reference style font pictures, each of the current reference style font pictures is The style font pictures correspond to different content information, and each of the current reference style font pictures corresponds to the same style information; A generation module, configured to input the current source font picture and the current reference style font picture set into a preset font generation model, and use the content encoder in the preset font generation model to encode the current source font picture. Perform feature extraction to generate first content features; perform feature extraction on the current reference style font picture set through the style encoder in the preset font generation model to generate a first style feature set; wherein, the first style The feature set includes a first style feature corresponding to each of the current reference style font pictures; the first content feature and the first style feature are configured through a style content feature adaptation module in the preset font generation model. The set performs a connection operation to generate connection features; the connection features include a second content feature corresponding to the first content feature, and a second style feature set corresponding to the first style feature set, and the second The style feature set includes second style features corresponding to each of the first style features; calculating a first weight corresponding to the second content feature, and a second weight corresponding to the second style feature set; based on The first weight and the second weight perform a weighted fusion process on the second content feature and the second style feature set to generate a fusion feature; the decoder in the preset font generation model performs a weighted fusion process on all the features. The fusion features are decoded to generate a target style font picture corresponding to the current source font picture; the content information of the target style font picture is the same as the content information of the current source font picture, and the target style font picture is The style information of is the same as the style information of each of the current reference style font pictures. 7. An electronic device, characterized in that the electronic device includes a processor and a memory, and the memory stores at least one instruction, at least a program, a code set or an instruction set, the at least one instruction, the at least A program, the code set or the instruction set is loaded and executed by the processor to implement the font generation method based on the adaptation of style information and content information as described in any one of claims 1-5. 8. A computer-readable storage medium, characterized in that the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code The set or instruction set is loaded and executed by the processor to implement the font generation method based on the adaptation of style information and content information as described in any one of claims 1 to 5.