CN113449787A - Chinese character stroke structure-based font library completion method and system - Google Patents
Chinese character stroke structure-based font library completion method and system Download PDFInfo
- Publication number
- CN113449787A CN113449787A CN202110701088.XA CN202110701088A CN113449787A CN 113449787 A CN113449787 A CN 113449787A CN 202110701088 A CN202110701088 A CN 202110701088A CN 113449787 A CN113449787 A CN 113449787A
- Authority
- CN
- China
- Prior art keywords
- chinese character
- stroke
- font
- type number
- chinese
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/126—Character encoding
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The invention discloses a font library completion method and system based on Chinese character stroke structure, concretely, determining Chinese character stroke type and number according to K stroke type classification results, creating K dimension stroke type number vector for each Chinese character, providing Chinese character font structure coding network according to ideographic description sequence defined by Unicode standard, including two coders and two decoders, obtaining high quality stroke type number coding and Chinese character font structure coding, representing Chinese character content characteristics by the two codes together, capable of more essentially representing Chinese character content, directly using related coder freezing parameters in the Chinese character font structure coding network in the font generation network, generating target font image by the decoder in combination with target font style characteristics, the generated Chinese character image has obvious advantages in visual effect, Chinese character content structure, font style and FID evaluation index, the high-precision Chinese character generation method has the advantages that the high-precision Chinese character content structure is kept, and one-to-many high-quality font generation is realized.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method and a system for complementing a font library based on a stroke structure of a Chinese character.
Background
As one of the oldest characters in the world, Chinese characters have complex font structures and various font styles. In the artificial intelligence era, people can contact and use different fonts every day, however, the manual development and design of a character font is time-consuming and labor-consuming, the design of the character font is more a difficult work requiring professional technology and heavy load, and the character font needs to be subjected to a plurality of complex processes such as character coding, stroke structure design and the like, and the design costs at least more than one year and is even longer, but the font style contained in the current Chinese font library cannot meet the actual character using requirements in three fields of cultural transmission, business and multimedia, and the demand of more and more users on personalized Chinese character expression is increasingly strengthened, so that the research is faster, and the high-quality font generation technology is the development trend of the font industry.
With the continuous improvement of generation countermeasure networks, a good effect is achieved in image style migration application, a font generation task can be regarded as an image style migration problem, and a new thought is provided for automatically realizing font generation. Meanwhile, the presentation mode of the Chinese character is a two-dimensional image, the image processing can be carried out on the two-dimensional image by utilizing the deep learning technology, the traditional font generation method mainly focuses on local expression characteristics such as strokes of the Chinese character and the like, the whole writing style of the Chinese character is not learned, the generation result of the existing model is poor when the font style span is large, and the generation efficiency and the generalization capability are still to be improved.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a method and a system for completing a font library based on a Chinese character stroke structure, the generated Chinese character image has obvious advantages in visual effect, Chinese character content structure, font style and FID evaluation index, the Chinese character content structure is retained with high precision, and the problem of poor generated result when the font style span is large in the existing model is solved.
In order to achieve the purpose, the invention adopts the technical scheme that: a font library completion method based on a Chinese character stroke structure comprises the following specific processes: classifying the stroke types of the common Chinese characters based on a common Chinese character data set, determining K stroke types to obtain the common Chinese characters and the stroke types corresponding to the common Chinese characters in a matching way, and carrying out digital coding on the stroke types corresponding to the common Chinese characters to establish K-dimensional stroke type number vectors to obtain the stroke type number vectors corresponding to the common Chinese characters;
on the basis of the number vector of the stroke types of the Chinese characters in the common Chinese character data set, combining a simkai word stock, and finally expanding to obtain a stroke type number vector data set of the Chinese characters;
designing a Chinese character font structure coding network model based on a self-coder principle, wherein a source Chinese character is provided with a Chinese character ideogram description sequence file according to a Cjklib project, an operator and a component of the ideogram description sequence corresponding to each Chinese character are arranged in the Chinese character ideogram description sequence file, each component corresponds to a Unicode code, and a Chinese character font structure description language and a Chinese character stroke type number vector are obtained by combining a data set of the Chinese character stroke type number vector;
taking a Chinese character image and two component images of the Chinese character as input, obtaining stroke type number codes and Chinese character font structure codes by considering stroke type coding loss functions and Chinese character font structure coding loss functions, and combining the stroke type number codes and the Chinese character font structure codes of the Chinese character to represent the content of the Chinese character to obtain content characteristic codes of the Chinese character;
introducing style feature codes into the content feature codes of the Chinese characters, adopting a font generation model to reconstruct the target font Chinese characters, and realizing one-to-many font style migration, namely completing a font library.
Classifying the stroke types of the common Chinese characters based on a common Chinese character data set, determining K stroke types to obtain the common Chinese characters and the stroke types corresponding to the common Chinese characters in a matching way, and carrying out digital coding on the stroke types corresponding to the common Chinese characters to establish K-dimensional stroke type number vectors to obtain the stroke type number vectors corresponding to the common Chinese characters; the method comprises the following specific steps:
on the basis of acquiring regular script image data, screening stroke sequences and types according to stroke sequences and types of 482 Chinese characters provided by a project Cjklib, acquiring 2100 stroke pictures based on 215 Chinese characters coexisting in a common Chinese character data set, and finally predefining 32 stroke types according to the strokes of the coexisting Chinese characters according to the existing 32 stroke types and the comprehensive consideration of the form and the occurrence number of 2100 strokes;
screening and classifying the images of 32 stroke types by adopting a ResNet-50 network, deleting the stroke types with low use frequency to obtain 31 stroke types, and obtaining the common Chinese characters and the stroke types corresponding to the common Chinese characters; the stroke types and the number of the stroke types are digitally coded, and for a given Chinese character x, the stroke type number vector is defined as a 31-dimensional vector c e { n ∈ [ ]i}31Wherein n isiIs the number of times that the i (i-1, 2, …,31) th type of stroke occurs in the Chinese character, and if a certain type of stroke does not exist, niAnd (5) obtaining a stroke type number vector corresponding to the common Chinese character.
On the basis of the Chinese character stroke type number vector in the common Chinese character data set, a simkai word stock is combined, and finally the stroke type number vector data set of the Chinese character is obtained by expansion, which is concretely as follows: the common Chinese character stroke type number vector is used as a data set, a training set and a test set are divided according to a preset proportion, a Chinese character image is input based on a Resnet-50 network framework, a stroke type number vector corresponding to the Chinese character image is output, a label of a data loading part is modified into the stroke type number vector of a known Chinese character, a sigmoid function is changed into a ReLU activation function, a cross entropy loss of a loss function is replaced by an L1 loss, the test set is the rest Chinese character images after training and verification of the preset number of Chinese character images and the stroke type number vectors thereof, and finally the stroke type number vector of the Chinese character is obtained and is used as the data set of the Chinese character stroke type number vector.
The self-encoder principle design Chinese character font structure encoding network comprises a Chinese character stroke type number encoder E1, a conventional encoder E2, a decoder D1 and a decoder D2; the Chinese character font structure coding network structure is as follows: obtaining a Chinese character stroke type number code S based on a Chinese character stroke type number encoder E1; the real Chinese character A is passed through conventional self-encoder E2, and passed through decoder D1 under the guidance of Chinese character stroke type number code S to reconstruct Chinese character fake _ A, and Chinese character component parts A1 and A2 are passed through encoder E2 to obtain two implicit content codes of Chinese character component parts A1 and A2, and then passed through full-connection network to make ideographic character description sequence IDS and two component partsThe implicit content coding features are combined into the structural coding of the Chinese character A, and finally the Chinese character fake _ A is reconstructed through a decoder D2 under the guidance of the Chinese character stroke type number coding SST。
The font generation model comprises a Chinese character stroke type number encoder E1, a Chinese character font structure encoder E2, a style characteristic encoder E3 of a target font, a decoder and a discriminator; the Chinese character stroke type number encoder E1 is used for extracting stroke type quantity characteristics of the source font Chinese character to generate Chinese character stroke type number codes, and the Chinese character font structure encoder E2 is used for extracting font structure characteristics to generate Chinese character font structure hidden codes.
Introducing style feature codes into content feature codes of Chinese characters, adopting a font generation model to reconstruct the target font Chinese characters and realizing one-to-many font style migration, which comprises the following steps: the method comprises the steps of taking a source font Chinese character image and a target font image as input, generating Chinese character stroke type number codes and Chinese character font structure hidden codes by adopting a font generation model, learning styles of a plurality of Chinese characters of a target font, learning styles of 4 Chinese characters of the same font during a specific experimental process so as to generate style characteristic vectors, carrying out feature fusion on the Chinese character stroke type number codes, the Chinese character font structure codes and the style characteristic codes, carrying out a plurality of times of upsampling operations, and reconstructing a converted target font Chinese character image.
The system for completing the font library based on the stroke structure of the Chinese character comprises a stroke type number vector construction module, a data set expansion module, a Chinese character structure and stroke type acquisition module, a Chinese character content characteristic coding module and a font style migration module;
the stroke type number vector construction module classifies the stroke types of the common Chinese characters based on a common Chinese character data set, determines K stroke types, obtains the common Chinese characters and stroke pictures and stroke types corresponding to the common Chinese characters in a matching mode, digitally encodes the stroke types corresponding to the common Chinese characters to create K-dimensional stroke type number vectors, and obtains the stroke type number vectors corresponding to the common Chinese characters;
the data set expansion module is used for finally expanding to obtain a stroke type number vector data set of the Chinese character by combining a simkai word stock on the basis of the Chinese character stroke type number vector in the common Chinese character data set;
the Chinese character structure and stroke type acquisition module is used for acquiring a Chinese character font structure description language and a stroke type number vector by combining a data set of the Chinese character stroke type number vector according to a Chinese character ideogram description sequence file provided by a Cjklib project, wherein each component corresponds to an operator and a component of an ideogram description sequence of each Chinese character, and each component corresponds to a Unicode;
the Chinese character content characteristic coding module is used for designing a Chinese character font structure coding network model based on a self-coder principle, taking the Chinese character image and two component images thereof as input, considering a stroke type coding loss function and a Chinese character font structure coding loss function to obtain stroke type number codes and Chinese character font structure codes, and combining the stroke type number codes and the Chinese character font structure codes of Chinese characters to represent the content of the Chinese characters to obtain the content characteristic codes of the Chinese characters;
the font style migration module is used for introducing style feature codes into the content feature codes of the Chinese characters, adopting a font generation model to reconstruct the target font Chinese characters, realizing one-to-many font style migration and further completing a font library.
A computer device comprises a processor and a memory, wherein the memory is used for storing a computer executable program, the processor reads part or all of the computer executable program from the memory and executes the computer executable program, and when the processor executes part or all of the computer executable program, the font library completion method based on the Chinese character stroke structure can be realized.
A computer readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method for complementing a font library based on a stroke structure of a chinese character according to the present invention can be implemented.
Compared with the prior art, the invention has at least the following beneficial effects:
the Chinese character image generated by the method has obvious advantages in visual effect, Chinese character content structure, font style and FID evaluation index, the Chinese character content structure is retained with high precision, the effectiveness and high-quality generated result of the method are verified, and the problem that the generated result is not good when the font style span is large in the existing model is solved, so that one-to-many high-quality font generation is realized. The invention takes the number vector of the stroke types of the Chinese characters and the structure coding of the character patterns of the Chinese characters as the characteristics of the content of the Chinese characters together, can more essentially represent the content of the Chinese characters, and takes the content as the guide information to realize the generation of one-to-many characters, can improve the efficiency and the quality of the task of generating the characters, and has very important guidance and research significance for the task of generating various Chinese character fonts.
Drawings
FIG. 1 is a schematic diagram of a font generation network framework based on a stroke structure of a Chinese character.
FIG. 2 shows the 31 stroke types of Chinese characters finally defined by the present invention.
FIG. 3 is a schematic diagram of a network framework for encoding Chinese character font structure according to the present invention.
FIG. 4 is a network output result of Chinese character pattern structure coding according to the present invention.
FIG. 5 is a schematic diagram of a font generation model designed by the present invention.
FIG. 6 shows a comparison of the results of generating a font image using the model of the present invention and three comparison methods.
FIG. 7 is a comparison of the details of the model of the present invention and the AGIS-Net generation results.
Fig. 8 is an image contrast generated for an ablation experiment with parameter freezing and retraining.
Fig. 9 is a comparison of ablation experiments with different loss functions removed.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings.
The font generation task may be considered as an image generation or image style migration problem, and both input and output are performed in an image form in the font generation model.
The invention mainly characterizes the Chinese character content characteristics by the stroke number codes and the character shape structure codes of the Chinese characters together as prior information and combines style characteristic vectors to realize one-to-many Chinese character font generation, namely, one Chinese character font is converted into a plurality of fonts at the same time, and the high-quality effect of generating images is met.
Specifically, the ResNet-50 network is used for classifying stroke types, 31 stroke types are finally determined, the stroke type and the number of the stroke types of each Chinese character are obtained, and a 31-dimensional stroke type number vector of each Chinese character is created. Then according to 12 ideographic character description operational characters defined by the Unicode standard, a Chinese character font structure coding network is designed by utilizing the principle of a self-coder, the network comprises two coders and two decoders, a source Chinese character image and a component image of the Chinese character are respectively input, the two are mutually constrained, stroke type coding loss and Chinese character font structure coding loss are designed, and therefore a high-quality Chinese character font structure coding is obtained. The Chinese character stroke type number code and the Chinese character structure code form Chinese character content characteristics together, so that one Chinese character content can be expressed in a low-dimensional mode. And finally, designing a font generation network on the basis of a Chinese character font structure coding network, referring to fig. 1, wherein the font generation network comprises a generator and a discriminator, the generator adopts a self-encoder structure, and is provided with three encoders and a decoder, the Chinese character content characteristics and the target font style characteristics are subjected to characteristic fusion in the decoder, and a target font image is generated by reconstruction. The experimental results also prove that the method of the invention is superior to other methods in the prior art. The target font image generated by the font generation network designed by the invention has good quality effects on the stroke type, the font structure and the font style of the Chinese character, and can meet the style migration of various fonts.
The invention provides a new Chinese character content representation method, and simultaneously introduces Chinese character stroke type number vectors and Chinese character font structure description codes to jointly form Chinese character content characteristics, so that one Chinese character content can be represented in a low-dimensionality manner.
In the encoding of the number of stroke types of the Chinese characters, 9574 stroke images of the regular script Chinese characters are firstly extracted, the stroke types are classified through Resnet-50 network, 31 stroke types are redefined, and reference is made to fig. 2.
The invention utilizes the prior information of strokes to guide the font generation, needs the stroke types as detailed as possible on the premise of ensuring the generation efficiency, and generates Chinese characters more accurately. According to the method, 9574 regular script Chinese characters provided by a project Makemeahanzi are used as reference data, a source project provides a svg (scalable vector graphics) file, the vector file contains the relative positions of strokes of the Chinese characters, pictures of the strokes of each Chinese character can be obtained by using regular expression matching extraction, 112617 pictures are obtained in total, the pictures of the strokes are named decimal Unicode coding + stroke order, and finally the svg format of the Chinese characters and the strokes is converted into the png image format, wherein the size of the images is 256 multiplied by 256.
The stroke classification is carried out to obtain stroke types of Chinese characters, the type and the number of strokes of each Chinese character are determined, a Chinese character stroke type number vector is created, and on the basis of obtaining the image data of a regular script, the stroke sequences and the types of 482 Chinese characters provided by a project Cjklib are provided, wherein the project provides 52 stroke types, the classified stroke types are more, a plurality of strokes have higher similarity, data redundancy can be caused by direct utilization, and the complexity is increased. The regular script data set is required to be screened, 215 Chinese characters coexist in the regular script data set, 2100 stroke pictures are obtained in total, 32 stroke types are defined according to the Chinese character stroke order rule specified by the state, the form and the number of the appearing 2100 strokes are comprehensively considered, and finally 32 stroke types are predefined according to the 215 Chinese character strokes, namely, horizontal stroke, vertical stroke, dot stroke, left falling stroke, transverse bending, right falling stroke, transverse hook, transverse bending, vertical bending, lifting, left falling stroke, vertical lifting, vertical bending hook, horizontal falling stroke hook, inclined hook, vertical bending hook, left falling stroke, transverse oblique hook, transverse bending, vertical bending left falling stroke, transverse bending hook, transverse bending hook, transverse bending left falling stroke, transverse bending hook, transverse bending hook, transverse bending left falling stroke and transverse bending. And the strokes of the same type in 2100 pictures are sorted together, the pictures are sorted from large to small according to the number of the pictures of the strokes of each type, and the category labels are named as 0,1,2 and 31 in sequence. And dividing the data set according to the ratio of 8:2, and respectively using the data set as a training set and a verification set of the stroke classification network, wherein the training set comprises 1680 stroke images, and the verification set comprises 420 stroke images.
The stroke image is classified by adopting a ResNet-50 network, the center of the stroke image is cut to 224 multiplied by 224 when data is loaded, the ResNet-50 network passes through 5 stages from input to output and comprises 49 convolution layers and a full connection layer, each layer needs Batch Norm to carry out normalization operation, and ReLU activation function is used for improving the fitting capability of the model; and finally, the image feature matrix is output through full connection, and corresponding class probability is obtained through Softmax, so that classification is carried out. The input image size of the network is 224 x 224, and in order to adapt to the data set of the stroke image, the final fully-connected layer of ResNet-50 is improved, the input of the original fully-connected layer is input into a linear layer with 256 output units again, then the ReLU layer and the Dropout layer are connected, finally the 256 x 32 linear layer is connected, and the output is 32-channel Softmax layer.
The invention uses a cross entropy loss function of Softmax, and calculates the cross entropy loss according to the Softmax function as follows:
wherein y iscIs the true distribution value; p is a radical ofcProbability values calculated for the Softmax function.
Resnet can effectively solve the problem of model degradation, reduce the difficulty of model deep-level network training, has better identification precision and strong performance, and obtains good results in large-scale competitions such as image classification, target detection, segmentation and the like.
Setting model parameters, setting 35 epochs, setting the learning rate to be 0.0001 and the batch size to be 8, updating gradient parameters by adopting an ADAM optimization operator, dividing a training set and a verification set according to the 8:2 ratio, and finally setting the maximum Top-1Accuracy on the training set to be 96.36 percent and the maximum Top-1Accuracy on the verification set to be 97.81 percent.
And the last residual stroke pictures are tested and integrated in batches, so that the efficiency can be improved, and the difficulty of manually screening wrong results can be reduced. The strokes are sorted to determine 9574 Chinese characters and their corresponding stroke types, which are to be used as data preprocessing for font generation, in preparation for font generation. After the stroke type is determined, the network is not used, only the final classification result is concerned, and therefore, the invention does not carry out a comparison experiment of a classification model. Finally, after testing, the stroke is found to be transversely folded and folded
In order to reduce the calculation amount of the later experimental parameters and the occupied memory, the preset stroke type of 'transverse folding and folding' is deleted, and meanwhile, the corresponding stroke of the Chinese character 'convex' is also deleted, so that 31 types of strokes are remained. Compared with the stroke type classification standard mentioned above, the stroke type classification standard takes the trend of the initial stroke and the falling stroke during stroke writing into consideration, defines the strokes of 'right-falling' and 'horizontal-falling', 'left-falling' and 'vertical-falling', 'horizontal-inclined hook' and 'horizontal-bending hook' into different types, but takes the great similarity of the strokes of 'vertical-bending left-falling' and 'vertical-bending' of the strokes of 9574 Chinese characters, totally appears 53 times in 112617 strokes, combines the two types of strokes into one type of 'vertical-bending left-falling', and can reduce the calculation amount of the following experimental parameters under the condition of ensuring that the generation effect is not influenced. Finally, the stroke types are divided into 31 types, the 31 types of strokes cover 9573 Chinese characters in the previous regular script data set, the stroke type labels are sequentially named as 1,2, … and 31 according to the sequence that the occurrence times of all the strokes of each type are sequenced from large to small, and finally, the stroke types are classified into 31 typesThe results are shown in FIG. 2. It can be seen from the final classification result and the occurrence frequency thereof that the use frequency of the strokes of different types in the Chinese character is greatly different, the occurrence frequency of the four strokes of 'horizontal, vertical, dot and left-falling' is more than 10000, and the occurrence frequency of the three strokes of 'horizontal bending hook, horizontal bending hook and horizontal bending left-falling' is less than 50. The strokes of Chinese characters are mainly focused on 6 kinds of strokes of 'horizontal, vertical, point, left-falling, transverse-folding and right-falling', and 6 kinds of basic strokes of 'horizontal, vertical, left-falling, point, right-falling and right-falling' only have the difference of 'point' and 'transverse-folding', so that other kinds of strokes can be considered to be obtained by the combination and transformation of the basic strokes. Thus, 9573 Chinese characters and stroke pictures and stroke types matched with the Chinese characters can be obtained.
For a given Chinese character x, the stroke type number vector is defined as a 31-dimensional vector c e { n }i}31Wherein n isiIs the number of times that the i (i-1, 2, …,31) th type of stroke occurs in the Chinese character, and if a certain type of stroke does not exist, ni0. Obtaining a stroke type number vector corresponding to 9573 Chinese characters; on the basis of the obtained 9573 Chinese character stroke type number vectors, the Chinese character stroke type number vectors are regarded as a data set, a training set and a verification set are also divided according to the 8:2 ratio, the input is a Chinese character image, and the output is the corresponding stroke type number vectors. The Resnet-50 network framework is also utilized, which is considered a regression problem. Modifying a label of a data loading part into a stroke type number vector of a known Chinese character, modifying a sigmoid function into a ReLU activation function, replacing cross entropy loss of a loss function with L1 loss, verifying that 9573 Chinese character images and stroke type number vectors thereof are known through training, testing sets are the rest 11329 Chinese character images, and finally obtaining 20902 stroke type number vector codes of Chinese characters, wherein the stroke type number vectors are stored in a json file for reading convenience, and the format is defined as { "decimal Unicode of Chinese characters": [ Stroke type number vector]}。
The invention provides a new font generation network model which comprises a generator and a discriminator, wherein the generator comprises a Chinese character stroke type number encoder, a Chinese character font structure encoder, a style encoder and a decoder. Specifically, the stroke type and the number of the Chinese character are determined according to the classification result of the 31 stroke types, and a 31-dimensional stroke type number vector is created for each Chinese character. Then, according to the description sequence of ideograph defined by Unicode standard, a Chinese character font structure coding network is designed, including two coders and two decoders, in order to obtain high-quality stroke type number coding and Chinese character font structure coding. The two codes are used for representing the Chinese character content characteristics together, so that the Chinese character content can be represented more essentially. And finally, directly using the freezing parameters of the relevant encoder in the Chinese character font structure coding network in a font generation network, and generating a target font image through a decoder by combining the style characteristics of the target font.
In the Chinese character font structure coding, 12 font description languages defined according to the Unicode standard and 21126 font structure descriptions of Chinese characters provided by the Cjklib project are used as a data set, then a Chinese character font structure coding network comprising three branches is designed, the first is a stroke type number coder, the second is a self-coder from a Chinese character image to a Chinese character image, and finally, two components forming the Chinese characters are self-coders from the Chinese character image, and common constraint enables the coders to generate high-quality structural codes.
The ideographic character description sequence defined by referring to the Unicode specification is a Chinese character structure description grammar, which adopts the operation mode of prefix representation, the structure type is used as prefix operator in front, the corresponding number of Chinese character components are used as operands in the back to describe and characterize Chinese characters, for example, Chinese character 'visit' is described as
A Chinese character "grid" or "square" is also known as a Chinese character, and is also known as a Unicode. The invention also provides the Unicode encoding of each operator, with reference to the 12 fixed structure types defined by the Unicode specification. As shown in table 1, the last overlay structure is used on the unibody characters where there is a cross-relationship. The ideographic description sequence only defines the 12 fixed structures, does not specify the types of the components, does not need to consider the specific forms of the components, has higher flexibility,and these operators are named 0,1, …,11 in order of Unicode encoding.
TABLE 1
The invention regards the left-middle-right structure and the upper-middle-lower structure in the structure type defined by the Unicode code as the combination of the left-right structure and the upper-lower structure of two steps, thus except the covering structure of the single-body character, the Chinese character can be gradually decomposed into two parts, there are 17717 Chinese characters which are composed of two parts, and the covering structures of the upper-middle-lower structure, the left-middle-right structure and the single character are not considered. Combining the stroke type number vector created by the invention, the Chinese character font structure codes and the stroke type numbers can be obtained, and the Chinese character font structure codes and the stroke type numbers are also stored in a json file, and the defined format is { "decimal Unicode of Chinese characters": { "decimal Unicode of Chinese characters": [ Chinese character stroke type number vector ], "decimal Unicode for component 1": [ stroke type number vector of part 1 ], "decimal Unicode of part 2": the stroke type number vector of [ element 2 ], "IDS": operator number, so that the structure coding network can directly read the json file when reading data and then load the image.
The Chinese character font structure coding network of the invention uses a self-coder, as shown in fig. 3, the input is Chinese character images A, A1 and A2, wherein A1 and A2 are the constituent elements of A, which jointly form Chinese character A. The invention relates to a Chinese character coding method, which comprises two encoders E1 and E2 and two decoders D1 and D2, wherein the encoder is regarded as having 3 branches, the first branch is a Chinese character stroke type number encoder E1, the Chinese character stroke type number encoder E1 is used for obtaining Chinese character stroke type number codes, the stroke type encoder E1 which is also designed in the invention is not limited on 2 ten thousand Chinese characters in a simkai word stock, stroke type number codes S can be obtained for any Chinese character except the data set, the second branch is a conventional self-encoder, a real Chinese character A passes through the encoder E2, and a Chinese character fake _ A is reconstructed through the decoder D1 under the guidance of the Chinese character stroke type number codes S. Third stepThe branch III is that the Chinese character component parts A1 and A2 obtain two implicit content codes of the Chinese character component parts A1 and A2 through the coder E2, then the ideograph description sequence IDS and the implicit content code characteristics of the two component parts are fused into the structure code of the Chinese character A through the full-connection network, and finally the Chinese character fake _ A is reconstructed through the decoder D2 under the guidance of the Chinese character stroke type number code SSTThe encoder learns the character pattern structure characteristics of the Chinese characters better, the generated structure encoding can reconstruct the Chinese characters better, and parameters can be frozen in a font generation model directly by using the two encoders trained in the step.
As an example, the encoder E1 and E2 have the same network structure and are composed of 6 convolution modules, each convolution module has a convolution layer, an example normalization layer and a leakage-based relu activation function layer, and each convolution module uses a residual error connection mode in order to make the encoder learn more implicit information. The original input size of the image is 128 × 128 × 1, and a single-channel black-and-white image is subjected to a convolution kernel with a size of 7 × 07 to reduce the dimension of the image to a feature map of 64 × 164 × 232, and then subjected to downsampling of the feature map by 5 convolution kernels of 3 × 33, so that the output of the encoder is a feature map of 2 × 42 × 5256. The two decoders D1 and D2 have the same network structure and share parameters, and there are 6 upsampling modules, each of which is composed of a deconvolution layer, an instance normalization layer and a ReLU activation layer, and the upsampling modules are also connected by using residual errors. The input of the decoder is a2 × 62 × 256 feature map, which is up-sampled to a 64 × 64 × 32 feature map by 53 × 3 deconvolution kernels, and then up-sampled to the original image size of 128 × 128 × 1 by 7 × 7 convolution kernels. Here, two implicit codes of A1 and A2 and an ideographic description sequence IDS are spliced together and input into a full-connection network, the input is a spliced 256 x 3-dimensional vector, and the output is a 256-dimensional structural code ASTThe output result of the glyph structure coding network refers to fig. 4.
The invention designs stroke type number coding loss to restrict the stroke type number coding obtained by the coder E1, and in order to ensure more stable training, an L1 loss function is used. The image of the chinese character a is directly subjected to the fake _ a image reconstructed by the self-encoder, and the chinese character component a1,a2 self-encoder reconstructed fake _ ASTThe image is respectively compared with the real Chinese character A image to calculate the L1 loss to restrict the reconstructed image quality of the decoder, and the image can also be regarded as the Chinese character font structure coding loss to indirectly restrict the quality generated by the coder. Defining c as the number code of the real Chinese character stroke types,
for the number obtained by encoder E1, xAIn order to input a real image of a chinese character,
is the reconstructed chinese character of the second branch from the encoder,
the third branch is the Chinese character reconstructed according to the Chinese character font structure coding, and the total objective function of the Chinese character font structure coding network is defined as follows:
wherein λA,λAST,λSEAre respective weights;
loss of L1 for image fake _ A and real Hanzi image A;
is an image fake _ ASTAnd L1 loss of the real Hanzi image A;
for the number coding loss of the stroke types of the Chinese characters,
the goal of network training is to minimize
Based on the Chinese character stroke type number coding and the Chinese character font structure coding, a target font style coder is added to connect the content characteristic coding and the style characteristic coding, the reconstructed target font Chinese character can be obtained through a decoder, and the true and false characters are judged through a discriminator, so that one-to-many font style migration is realized.
Referring to fig. 5, the font generation model designed by the present invention includes a generator and a discriminator, the generator is provided with three encoders E1, E2, E3 and a decoder, which can be divided into two branches: the first branch uses encoders E1 and E2 to respectively and correspondingly extract stroke type quantity characteristics and character pattern structural characteristics of source font Chinese characters to generate Chinese character stroke type number codes and Chinese character pattern structure hidden codes, network structures of the encoders E1 and E2 are all composed of convolution modules with the same layer number, the purpose is to represent Chinese characters in a layering mode, the Chinese character stroke type number codes and the Chinese character pattern structural codes generated by the encoders jointly form content characteristics of the Chinese characters, and low-dimensional representation of one Chinese character is achieved. As a preferred embodiment, the Chinese character stroke type number encoder E1 and the Chinese character font structure encoder E2 can directly utilize the network structure of the Chinese character font structure encoder of the invention, parameters are frozen, the network parameters are not optimized in the training process, the network parameters are simply used as a Chinese character content hierarchical representation extractor, and the effectiveness of the Chinese character font structure encoder trained in advance is further verified through an ablation experiment.
The second branch is that the encoder E3 extracts the style characteristics of the target font and learns the style of a plurality of Chinese characters of the target font. Specifically, the style of 4 Chinese characters in the same font is learned, style feature vectors are generated, stroke type codes obtained by coding of a coder E1 are coded, feature fusion is carried out on Chinese character font structure codes and style feature codes obtained by coding of a coder E2 in a decoder, multiple times of up-sampling operation are carried out, and the converted target font Chinese character images are reconstructed.
And finally, verifying authenticity through a discriminator, wherein the input of the discriminator is the target font Chinese character image and the target image reconstructed by the decoder.
The three encoders E1, E2 and E3 adopt the same network structure and are composed of 6 convolution modules, each convolution module is provided with a convolution layer, an example normalization layer and a LeaklyReLU activation function layer, and in order to enable the encoders to learn more implicit information, each convolution module uses a residual error connection mode. The original input size of the image is 128 × 128 × 1, and a single-channel black-and-white image is first subjected to a convolution kernel with a size of 7 × 7 to reduce the dimension of the image to a feature map of 64 × 64 × 32, and then subjected to downsampling of the feature map by 5 convolution kernels of 3 × 3, so that the final output of the encoder is a feature map of 2 × 2 × 256. The decoder is also provided with 6 upsampling modules, each of which consists of a deconvolution layer, an instance normalization layer and a ReLU activation layer, and the upsampling modules also use a residual concatenation method. The input of the decoder is a2 × 2 × 256 feature map, which is up-sampled to a 64 × 64 × 32 feature map by 53 × 3 deconvolution kernels, and then up-sampled to the original image size of 128 × 128 × 1 by 7 × 7 convolution kernels. The structure parameters of the coder and the decoder are the same as the network parameters of the Chinese character font structure coder. The discriminator adopts a common PatchGAN discriminator and comprises 4 convolution modules, the size of a convolution kernel of each convolution module is, an example normalization layer and a LeaklyReLU activation function layer are added, and finally, a full connection layer is arranged.
With respect to the loss function of the font generation network, the present invention constrains the generation of images using GAN's conventional antagonism loss, L1 loss, and context loss. Definition of
Is the generated target font image, y is the target font image, and the final objective function of the network is:
wherein is λ1,λ2,λ3Controlling each loss weight coefficient;
is the loss of the resistance to the disease,
use of
The loss is to make the training process more stable,
is a loss of context.
Finally, the character generation task can use the image quality evaluation standard, the invention mainly uses the FID as the evaluation index, and applies the FID to three different levels of the global image, the Chinese character content and the character styleGFocusing on the overall accuracy of the image, FIDCFocusing on the accuracy of the content, FIDSAccuracy of the style is a concern. Using the feature function to sum the real image y with the generated image
Modeled as a multivariate gaussian distribution, the difference of these two gaussian distributions can be measured by FID,
wherein
Generating an image; y is the true image;
is the mean of the feature vectors of the generated image; mu.syIs the mean value of the feature vectors of the real image; tr is the sum of the elements on the diagonal of the matrix;
isyWhich are covariance matrices that generate eigenvectors for the image and the real image, respectively. The FID considers more the connection between the generated image and the real image, and can measure the distance between the two images at a characteristic level, closer to human perception than the pixel-level metric. The smaller the FID, the closer the two distributions are, indicating a higher quality of the generated picture.
The invention takes simkai as the source font, which contains 20902 Chinese characters, which is also the data set used by the invention. 309 Chinese character ttf files are obtained from the internet, the ttf files of the fonts are converted into specific Chinese character images, 400 Chinese characters are randomly selected from a simkai data set, 309 font images of the 400 Chinese characters are generated, and 123600 data sets of the Chinese characters with the target font style are obtained. All image sizes are 128 x 128. 309 fonts with styles are divided into a training set, a verification set and a test set according to the proportion of 7:2:1, and 4 Chinese characters are randomly selected from each font to serve as style references of the font in the training process.
Comparing the result obtained by the method with the quality and evaluation indexes of generated images of the prior three methods of EMD, AGIS-Net and FUNIT, and respectively selecting calligraphy bodies such as 'squaring handwriting-positive Weber handwriting body', 'squaring handwriting Lijia simplified body' and the like from the test result image; handwriting such as 'square handwriting Tiange hard-tipped pen writing script', 'square silence brief' and the like; comparing 12 Chinese characters in the artistic characters such as 'mini simple sea rhyme', 'mini simple diamond heart' and the like, finding that the font image generated by the model designed by the invention is obviously superior to the generation results of EMD and FUNIT in the visual effects of the stroke type, the font structure and the font style of the Chinese characters, and the FUNIT generates more attention to the font style without paying more attention to the content structure of the Chinese characters. Especially, font images generated by EMD are fuzzy, the structure is disordered, and the distortion degree is large. The result of AGIS-Net is comparable to the model of the present invention, and has similar visual effects on the content and style of Chinese characters, but has the phenomenon of overlapping edge pixels under the condition of thicker strokes of the target font, such as the Chinese characters 'linden' and 'two' in artistic words. Secondly, the model of the invention is better in detail, referring to fig. 6 and 7, the dotted line in fig. 7 is the result generated by the model of the invention, the solid line is the result generated by AGIS-Net, the stroke edge pixel coincidence of the result of AGIS-Net can be obviously found, even the intermediate structure and the stroke of the Chinese character 'bodhi' and 'sudoku' can not be discerned in a fuzzy way, but the stroke boundary of the model of the invention generated results of handwriting and calligraphy is clearly distinguished, the font structure is clear, and the model is well reconstructed on the content and style of the Chinese character.
To further quantify the effectiveness of the font generation model of the present invention, FID indices are calculated for the four model-generated images and the true target font image, respectively, which can give a higher level of performance assessment over the entire data set. The invention applies the method to three different levels of global image, Chinese character content and font style, FIDGFocusing on the overall accuracy of the image, FIDCFocusing on the accuracy of the content, FIDSAccuracy of the style is a concern. The smaller the value of the FID, the closer to the target font image is indicated. From the calculation result, the model designed by the invention is in the global image FIDGThe calculated value is obviously smaller than that of the other three methods, the calculated index value of the EMD is the largest and is matched with the visual effect of the image displayed on the EMD, and the evaluation index further verifies that the model of the invention is superior to the AGIS-Net. In particular FIDCThe value of (A) is 18.7, which is significantly lower than the other three networks. Further verifies that the generated Chinese character font quality of the model is higher. The reason for this can be judged is that stroke type number coding and Chinese character font structure coding are added in the generation network to guide the generated Chinese characters to be well reconstructed on the content structure, so that the judgment of the designed Chinese character font structure coding network is effective, and FID index comparison results of the model designed by the invention and other three methods refer to Table 2.
TABLE 2
For further verifying the Chinese character font structure coding network coding designed by the inventionThe effectiveness of the code device, E2 encoder in the generator has designed two different training methods, one is as a new network, train encoder E2 get Chinese character font structure code directly again, only input the information of the description sequence of ideographic characters of the Chinese character in the network input; the other is that the invention directly uses the trained coder in the Chinese character font structure coding network, the structure coding network parameter is frozen, the network parameter is not optimized in the training process, and the network parameter is simply used as the Chinese character font structure coding extractor, thereby guiding the font generation. And other parameters are unchanged, and training tests are respectively carried out. Referring to fig. 8, from the visual effect, the styles of the two methods are not greatly different, but by carefully observing the characteristics of the content of the Chinese characters, the image font structure generated by directly using the encoder of the invention is well maintained, the character strokes and the structure boundaries are clearer, and the generated image quality is better. The retraining method has the advantages that the problems of stroke blurring, edge pixel overlapping and the like of the font with thicker strokes are solved, and particularly, the generated image of the font with thicker strokes is obvious. The qualitative judgment of the Chinese character font structure encoder of the invention is functional. The final comparison results are shown in table 3, it can be seen that the designed Chinese character font structure coding network is directly used, the parameters are frozen in the font generation network, and the difference of the generated Chinese character images in the content features is obviously smaller than the FID of the retraining coderGThe value of (c). With respect to FIDCWhen the freezing parameters are seen, the number of the Chinese character font structure encoder which is directly used for training is 6.4 points lower than that of the newly trained Chinese character font structure encoder, and the encoding method is applied to FIDSThe method has no great difference, only has great difference on Chinese character content, quantitatively judges that the encoder designed in the Chinese character font structure coding is effective from evaluation indexes, can well learn the font structure characteristics of Chinese characters, and leads the generated image to reconstruct the high-precision Chinese character content characteristics.
TABLE 3
Ablation experiments were performed from the perspective of the loss function, with the aim of investigating the effect of different loss functions in the network. The differences in the effects of generating the same target font are compared, with different penalty functions removed. From the comparison results of FIG. 9, it can be seen that removing any one of the loss functions affects the visual effect of the generated image, and removing the counteracting loss
Stroke distortion occurs, and obvious stroke blurring occurs to font with thicker strokes, such as oyster, linden and worries, which have obvious stroke ghost images; get rid of
The integral characteristics of the generated Chinese characters are still maintained, but the problem of stroke coincidence also exists,
the loss function effectively restricts the stroke details of the generated Chinese characters, and can punish the generation of local pixels of the image; the last column in FIG. 9 is to remove context loss
The generated image result is most obvious in quality reduction, the phenomena of fuzzy, stroke missing, structural disorder and the like are obviously seen for different fonts, particularly for target fonts with thinner strokes and fewer pixel points, the Chinese character image is difficult to construct under the constraint of no context loss, and even a blank image appears, for example, a Chinese character 'sparrow hawk (chi)' is directly a blank image under the condition of removing the context loss, so that the context loss is explained
The method plays the most important role in Chinese character image generation, context loss is in pixel level and feature level to restrict image reconstruction, and FID evaluation results of ablation experiments with different loss functions removed are referred to table 4.
TABLE 4
As a possibility, the invention provides a font library completion system based on a Chinese character stroke structure, which comprises a stroke type number vector construction module, a data set expansion module, a Chinese character structure and stroke type acquisition module, a Chinese character content characteristic coding module and a font style migration module;
the stroke type number vector construction module classifies the stroke types of the common Chinese characters based on a common Chinese character data set, determines K stroke types, obtains the common Chinese characters and stroke pictures and stroke types corresponding to the common Chinese characters in a matching mode, digitally encodes the stroke types corresponding to the common Chinese characters to create K-dimensional stroke type number vectors, and obtains the stroke type number vectors corresponding to the common Chinese characters;
the data set expansion module is used for finally expanding to obtain a stroke type number vector data set of the Chinese character by combining a simkai word stock on the basis of the Chinese character stroke type number vector in the common Chinese character data set;
the Chinese character structure and stroke type acquisition module is used for acquiring Chinese character font structure codes and stroke type numbers by combining a data set of Chinese character stroke type number vectors according to a Chinese character ideogram description sequence file provided by a Cjklib project, wherein the Chinese character ideogram description sequence file corresponds to an operator and a component of an ideogram description sequence of each Chinese character, and each component corresponds to a Unicode code;
the Chinese character content characteristic coding module is used for designing a Chinese character font structure coding network model based on a self-coder principle, taking Chinese character font structure coding and stroke type number as input, considering a stroke type coding loss function and a Chinese character font structure coding loss function to obtain stroke type number coding and Chinese character font structure coding, and combining the stroke type number coding and the Chinese character font structure coding of the Chinese character to represent the content of the Chinese character to obtain the content characteristic coding of the Chinese character;
the font style migration is used for introducing style feature codes into content feature codes of Chinese characters, adopting a font generation model to reconstruct target font Chinese characters, realizing one-to-many font style migration, and further completing a font library.
The invention can also provide a computer device, which comprises a processor and a memory, wherein the memory is used for storing the computer executable program, the processor reads part or all of the computer executable program from the memory and executes the computer executable program, and when the processor executes part or all of the computer executable program, the method for completing the font library based on the stroke structure of the Chinese character can be realized.
In another aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method for complementing a font library based on a stroke structure of a chinese character according to the present invention can be implemented.
The computer equipment can be an onboard computer, a notebook computer, a tablet computer, a desktop computer, a mobile phone or a workstation.
The processor may be a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), or an off-the-shelf programmable gate array (FPGA).
The memory of the invention can be an internal storage unit of a vehicle-mounted computer, a notebook computer, a tablet computer, a desktop computer, a mobile phone or a workstation, such as a memory and a hard disk; external memory units such as removable hard disks, flash memory cards may also be used.
Computer-readable storage media may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The computer-readable storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a Solid State Drive (SSD), or an optical disc. The Random Access Memory may include a resistive Random Access Memory (ReRAM) and a Dynamic Random Access Memory (DRAM).
The method realizes the completion of the font library, establishes a foundation for the subsequent construction of the personal writing font library based on the research content of the method, promotes the rehabilitation process of the patient by utilizing calligraphy treatment in the future, evaluates the rehabilitation degree of the operation by comparing the writing handwriting of the patient before and after the operation, and has great practical significance for the mental health identification and the operation rehabilitation evaluation of the patient by utilizing handwriting analysis. The method further explores the generalization capability and robustness of the fonts, has great application value in multiple fields of calligraphy sets, printed font completion and the like, cultural relic repair and the like, can assist in the design of the commercialized fonts, and has commercial value.
Claims (9)
1. The method for complementing the font library based on the stroke structure of the Chinese character is characterized by comprising the following specific processes:
classifying the stroke types of the common Chinese characters based on a common Chinese character data set, determining K stroke types to obtain the common Chinese characters and the stroke types corresponding to the common Chinese characters in a matching way, and carrying out digital coding on the stroke types corresponding to the common Chinese characters to establish K-dimensional stroke type number vectors to obtain the stroke type number vectors corresponding to the common Chinese characters;
on the basis of the number vector of the stroke types of the Chinese characters in the common Chinese character data set, combining a simkai word stock, and finally expanding to obtain a stroke type number vector data set of the Chinese characters;
designing a Chinese character font structure coding network model based on a self-coder principle, wherein a source Chinese character is provided with a Chinese character ideogram description sequence file according to a Cjklib project, an operator and a component of the ideogram description sequence corresponding to each Chinese character are arranged in the Chinese character ideogram description sequence file, each component corresponds to a Unicode code, and a Chinese character font structure description language and a Chinese character stroke type number vector are obtained by combining a data set of the Chinese character stroke type number vector;
taking a Chinese character image and two component images of the Chinese character as input, obtaining stroke type number codes and Chinese character font structure codes by considering stroke type coding loss functions and Chinese character font structure coding loss functions, and combining the stroke type number codes and the Chinese character font structure codes of the Chinese character to represent the content of the Chinese character to obtain content characteristic codes of the Chinese character;
introducing style feature codes into the content feature codes of the Chinese characters, adopting a font generation model to reconstruct the target font Chinese characters, and realizing one-to-many font style migration, namely completing a font library.
2. The method of claim 1, wherein the stroke types of the common Chinese characters are classified based on a common Chinese character data set, K stroke types are determined to obtain the common Chinese characters and the stroke types corresponding to the common Chinese characters, and the stroke types corresponding to the common Chinese characters are digitally encoded to create K-dimensional stroke type number vectors to obtain the stroke type number vectors corresponding to the common Chinese characters; the method comprises the following specific steps:
on the basis of acquiring regular script image data, screening stroke sequences and types according to stroke sequences and types of 482 Chinese characters provided by a project Cjklib, acquiring 2100 stroke pictures based on 215 Chinese characters coexisting in a common Chinese character data set, and finally predefining 32 stroke types according to the strokes of the coexisting Chinese characters according to the existing 32 stroke types and the comprehensive consideration of the form and the occurrence number of 2100 strokes;
screening and classifying the images of 32 stroke types by adopting a ResNet-50 network, deleting the stroke types with low use frequency to obtain 31 stroke types, and obtaining the common Chinese characters and the stroke types corresponding to the common Chinese characters; the stroke types and the number of the stroke types are digitally coded, and for a given Chinese character x, the stroke type number vector is defined as a 31-dimensional vector c e { n ∈ [ ]i}31Wherein n isiIs the number of times that the i (i-1, 2, …,31) th type of stroke occurs in the Chinese character, and if a certain type of stroke does not exist, niAnd (5) obtaining a stroke type number vector corresponding to the common Chinese character.
3. The method for completing a font library based on a Chinese character stroke structure according to claim 1, wherein a stroke type number vector data set of a Chinese character is finally obtained by expansion on the basis of a Chinese character stroke type number vector in the common Chinese character data set in combination with a simkai character library, and the method comprises the following specific steps: the common Chinese character stroke type number vector is used as a data set, a training set and a test set are divided according to a preset proportion, a Chinese character image is input based on a Resnet-50 network framework, a stroke type number vector corresponding to the Chinese character image is output, a label of a data loading part is modified into the stroke type number vector of a known Chinese character, a sigmoid function is changed into a ReLU activation function, a cross entropy loss of a loss function is replaced by an L1 loss, the test set is the rest Chinese character images after training and verification of the preset number of Chinese character images and the stroke type number vectors thereof, and finally the stroke type number vector of the Chinese character is obtained and is used as the data set of the Chinese character stroke type number vector.
4. The method of claim 1, wherein the self-encoder principle-designed chinese character font structure encoding network comprises a chinese character stroke number encoder E1, a conventional encoder E2, a decoder D1 and a decoder D2; the Chinese character font structure coding network structure is as follows: obtaining a Chinese character stroke type number code S based on a Chinese character stroke type number encoder E1; the real Chinese character A is passed through a conventional self-encoder E2, and is passed through a decoder D1 under the guidance of Chinese character stroke type number code S to reconstruct a Chinese character fake _ A, the Chinese character component parts A1 and A2 are passed through a coder E2 to obtain two implicit content codes of Chinese character component parts A1 and A2, then an ideographic character description sequence IDS and the implicit content code characteristics of two component parts are combined into a structure code of the Chinese character A by means of a full-connection network, and finally, the Chinese character fake _ A is passed through the decoder D2 under the guidance of the Chinese character stroke type number code S to reconstruct the Chinese character fake _ AST。
5. The method for complementing a font library based on a Chinese character stroke structure according to claim 1, wherein the font generation model comprises a Chinese character stroke type number encoder E1, a Chinese character font structure encoder E2, a style characteristic encoder E3 of a target font, a decoder and a discriminator; the Chinese character stroke type number encoder E1 is used for extracting stroke type quantity characteristics of the source font Chinese character to generate Chinese character stroke type number codes, and the Chinese character font structure encoder E2 is used for extracting font structure characteristics to generate Chinese character font structure hidden codes.
6. The method for completing a font library based on a stroke structure of a Chinese character according to claim 5, wherein style characteristic codes are introduced into content characteristic codes of the Chinese character, a font generation model is adopted to reconstruct a target font Chinese character, and one-to-many font style migration is realized, and the method specifically comprises the following steps: the method comprises the steps of taking a source font Chinese character image and a target font image as input, generating Chinese character stroke type number codes and Chinese character font structure hidden codes by adopting a font generation model, learning styles of a plurality of Chinese characters of a target font, learning styles of 4 Chinese characters of the same font during a specific experimental process so as to generate style characteristic vectors, carrying out feature fusion on the Chinese character stroke type number codes, the Chinese character font structure codes and the style characteristic codes, carrying out a plurality of times of upsampling operations, and reconstructing a converted target font Chinese character image.
7. The system for complementing the font library based on the stroke structure of the Chinese character is characterized by comprising a stroke type number vector construction module, a data set expansion module, a Chinese character structure and stroke type acquisition module, a Chinese character content characteristic coding module and a font style migration module;
the stroke type number vector construction module classifies the stroke types of the common Chinese characters based on a common Chinese character data set, determines K stroke types, obtains the common Chinese characters and stroke pictures and stroke types corresponding to the common Chinese characters in a matching mode, digitally encodes the stroke types corresponding to the common Chinese characters to create K-dimensional stroke type number vectors, and obtains the stroke type number vectors corresponding to the common Chinese characters;
the data set expansion module is used for finally expanding to obtain a stroke type number vector data set of the Chinese character by combining a simkai word stock on the basis of the Chinese character stroke type number vector in the common Chinese character data set;
the Chinese character structure and stroke type acquisition module is used for acquiring a Chinese character font structure description language and a stroke type number vector by combining a data set of the Chinese character stroke type number vector according to a Chinese character ideogram description sequence file provided by a Cjklib project, wherein each component corresponds to an operator and a component of an ideogram description sequence of each Chinese character, and each component corresponds to a Unicode;
the Chinese character content characteristic coding module is used for designing a Chinese character font structure coding network model based on a self-coder principle, taking the Chinese character image and two component images thereof as input, considering a stroke type coding loss function and a Chinese character font structure coding loss function to obtain stroke type number codes and Chinese character font structure codes, and combining the stroke type number codes and the Chinese character font structure codes of Chinese characters to represent the content of the Chinese characters to obtain the content characteristic codes of the Chinese characters;
the font style migration module is used for introducing style feature codes into the content feature codes of the Chinese characters, adopting a font generation model to reconstruct the target font Chinese characters, realizing one-to-many font style migration and further completing a font library.
8. A computer device is characterized by comprising a processor and a memory, wherein the memory is used for storing a computer executable program, the processor reads part or all of the computer executable program from the memory and executes the computer executable program, and when the processor executes part or all of the computer executable program, the font library completion method based on the Chinese character stroke structure can be realized according to any one of claims 1 to 7.
9. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for complementing a font library based on a stroke structure of a chinese character according to any one of claims 1 to 6 is implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110701088.XA CN113449787B (en) | 2021-06-23 | 2021-06-23 | Chinese character stroke structure-based font library completion method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110701088.XA CN113449787B (en) | 2021-06-23 | 2021-06-23 | Chinese character stroke structure-based font library completion method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113449787A true CN113449787A (en) | 2021-09-28 |
| CN113449787B CN113449787B (en) | 2022-12-09 |
Family
ID=77812380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110701088.XA Active CN113449787B (en) | 2021-06-23 | 2021-06-23 | Chinese character stroke structure-based font library completion method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113449787B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114530064A (en) * | 2022-02-22 | 2022-05-24 | 北京思明启创科技有限公司 | Calligraphy practicing method, calligraphy practicing device, calligraphy practicing equipment and calligraphy practicing storage medium based on video |
| CN114970447A (en) * | 2022-05-26 | 2022-08-30 | 华侨大学 | Chinese character font conversion method, device, equipment and storage medium |
| CN115497107A (en) * | 2022-09-30 | 2022-12-20 | 江西师范大学 | Zero-sample Chinese character recognition method based on stroke and radical decomposition |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101393645A (en) * | 2008-09-12 | 2009-03-25 | 浙江大学 | Hand-writing Chinese character computer generation and beautification method |
| CN104156721A (en) * | 2014-07-31 | 2014-11-19 | 南京师范大学 | Off-line Chinese character stroke extraction method based on template matching |
| CN106774973A (en) * | 2016-11-17 | 2017-05-31 | 浙江传媒学院 | A kind of font generation method of the structure and style based on Chinese character |
| CN108804397A (en) * | 2018-06-12 | 2018-11-13 | 华南理工大学 | A method of the Chinese character style conversion based on a small amount of target font generates |
| CN109558873A (en) * | 2018-12-03 | 2019-04-02 | 哈尔滨工业大学 | A kind of mode identification method based on this stack autoencoder network that changes |
| CN110033054A (en) * | 2019-03-14 | 2019-07-19 | 上海交通大学 | Personalized handwritten form moving method and system based on collaboration stroke optimization |
| CN110533737A (en) * | 2019-08-19 | 2019-12-03 | 大连民族大学 | The method generated based on structure guidance Chinese character style |
| CN111553246A (en) * | 2020-04-15 | 2020-08-18 | 山东大学 | Chinese character style migration method and system based on multi-task antagonistic learning network |
| CN111859852A (en) * | 2019-04-26 | 2020-10-30 | 普天信息技术有限公司 | Training device and method for Chinese character style migration model |
| US20200401938A1 (en) * | 2019-05-29 | 2020-12-24 | The Board Of Trustees Of The Leland Stanford Junior University | Machine learning based generation of ontology for structural and functional mapping |
| CN112163401A (en) * | 2020-10-22 | 2021-01-01 | 大连民族大学 | Chinese character font generation method based on compressed and excited GAN network |
| CN112800961A (en) * | 2021-01-28 | 2021-05-14 | 北京有竹居网络技术有限公司 | Stroke writing sequence detection method, device, medium and electronic equipment |
| US11017167B1 (en) * | 2018-06-29 | 2021-05-25 | Intuit Inc. | Misspelling correction based on deep learning architecture |
| CN113011337A (en) * | 2021-03-19 | 2021-06-22 | 山东大学 | Chinese character library generation method and system based on deep meta learning |
| CN113011431A (en) * | 2021-03-31 | 2021-06-22 | 华南理工大学 | Chinese character stroke segmentation and extraction method and system based on MaskRCNN |
-
2021
- 2021-06-23 CN CN202110701088.XA patent/CN113449787B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101393645A (en) * | 2008-09-12 | 2009-03-25 | 浙江大学 | Hand-writing Chinese character computer generation and beautification method |
| CN104156721A (en) * | 2014-07-31 | 2014-11-19 | 南京师范大学 | Off-line Chinese character stroke extraction method based on template matching |
| CN106774973A (en) * | 2016-11-17 | 2017-05-31 | 浙江传媒学院 | A kind of font generation method of the structure and style based on Chinese character |
| CN108804397A (en) * | 2018-06-12 | 2018-11-13 | 华南理工大学 | A method of the Chinese character style conversion based on a small amount of target font generates |
| US11017167B1 (en) * | 2018-06-29 | 2021-05-25 | Intuit Inc. | Misspelling correction based on deep learning architecture |
| CN109558873A (en) * | 2018-12-03 | 2019-04-02 | 哈尔滨工业大学 | A kind of mode identification method based on this stack autoencoder network that changes |
| CN110033054A (en) * | 2019-03-14 | 2019-07-19 | 上海交通大学 | Personalized handwritten form moving method and system based on collaboration stroke optimization |
| CN111859852A (en) * | 2019-04-26 | 2020-10-30 | 普天信息技术有限公司 | Training device and method for Chinese character style migration model |
| US20200401938A1 (en) * | 2019-05-29 | 2020-12-24 | The Board Of Trustees Of The Leland Stanford Junior University | Machine learning based generation of ontology for structural and functional mapping |
| CN110533737A (en) * | 2019-08-19 | 2019-12-03 | 大连民族大学 | The method generated based on structure guidance Chinese character style |
| CN111553246A (en) * | 2020-04-15 | 2020-08-18 | 山东大学 | Chinese character style migration method and system based on multi-task antagonistic learning network |
| CN112163401A (en) * | 2020-10-22 | 2021-01-01 | 大连民族大学 | Chinese character font generation method based on compressed and excited GAN network |
| CN112800961A (en) * | 2021-01-28 | 2021-05-14 | 北京有竹居网络技术有限公司 | Stroke writing sequence detection method, device, medium and electronic equipment |
| CN113011337A (en) * | 2021-03-19 | 2021-06-22 | 山东大学 | Chinese character library generation method and system based on deep meta learning |
| CN113011431A (en) * | 2021-03-31 | 2021-06-22 | 华南理工大学 | Chinese character stroke segmentation and extraction method and system based on MaskRCNN |
Non-Patent Citations (2)
| Title |
|---|
| BAOYAO ZHOU 等: "Easy generation of personal Chinese handwritten fonts", 《2011 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO》 * |
| 俞书世 等: "基于结构约束生成对抗网络的书法汉字生成", 《模式识别与人工智能》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114530064A (en) * | 2022-02-22 | 2022-05-24 | 北京思明启创科技有限公司 | Calligraphy practicing method, calligraphy practicing device, calligraphy practicing equipment and calligraphy practicing storage medium based on video |
| CN114970447A (en) * | 2022-05-26 | 2022-08-30 | 华侨大学 | Chinese character font conversion method, device, equipment and storage medium |
| CN115497107A (en) * | 2022-09-30 | 2022-12-20 | 江西师范大学 | Zero-sample Chinese character recognition method based on stroke and radical decomposition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113449787B (en) | 2022-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Altwaijry et al. | Arabic handwriting recognition system using convolutional neural network | |
| CN113449787B (en) | Chinese character stroke structure-based font library completion method and system | |
| CN111046784B (en) | Document layout analysis and identification method and device, electronic equipment and storage medium | |
| RU2661750C1 (en) | Symbols recognition with the use of artificial intelligence | |
| Davis et al. | Text and style conditioned GAN for generation of offline handwriting lines | |
| CN108304357B (en) | Chinese character library automatic generation method based on font manifold | |
| CN110276402B (en) | Salt body identification method based on deep learning semantic boundary enhancement | |
| Upchurch et al. | From a to z: supervised transfer of style and content using deep neural network generators | |
| EP3685312A1 (en) | Method and system for image content recognition | |
| CN106446954A (en) | Character recognition method based on depth learning | |
| CN112163401B (en) | Compression and excitation-based Chinese character font generation method of GAN network | |
| CN110114776A (en) | Use the system and method for the character recognition of full convolutional neural networks | |
| CN113807356B (en) | End-to-end low-visibility image semantic segmentation method | |
| Mattick et al. | SmartPatch: improving handwritten word imitation with patch discriminators | |
| Karunarathne et al. | Recognizing ancient sinhala inscription characters using neural network technologies | |
| CN113538359A (en) | System and method for finger vein image segmentation | |
| CN115457568A (en) | Historical document image noise reduction method and system based on generation countermeasure network | |
| Li et al. | SwordNet: Chinese character font style recognition network | |
| CN113962192B (en) | Method and device for generating Chinese character font generation model and Chinese character font generation method and device | |
| Jenckel et al. | Training LSTM-RNN with imperfect transcription: limitations and outcomes | |
| Cao et al. | FL-GAN: feature learning generative adversarial network for high-quality face sketch synthesis | |
| CN114241497A (en) | Table sequence identification method and system based on context attention mechanism | |
| Tang et al. | Two-stage image colorization via color codebook | |
| Sun et al. | End-to-end rubbing restoration using generative adversarial networks | |
| CN116311275B (en) | Text recognition method and system based on seq2seq language model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |