CN107644006B - Automatic generation method of handwritten Chinese character library based on deep neural network - Google Patents
Automatic generation method of handwritten Chinese character library based on deep neural network Download PDFInfo
- Publication number
- CN107644006B CN107644006B CN201710908121.XA CN201710908121A CN107644006B CN 107644006 B CN107644006 B CN 107644006B CN 201710908121 A CN201710908121 A CN 201710908121A CN 107644006 B CN107644006 B CN 107644006B
- Authority
- CN
- China
- Prior art keywords
- font
- character
- style
- chinese
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a method for automatically generating a Chinese character library in handwritten form based on a deep neural network, which comprises the steps of reconstructing character features of a small number of Chinese characters in handwritten form, establishing a character style migration network, estimating the character style features of the characters which are not written by a user through the character style migration network, combining the character content of a reference character with the writing style of the user, migrating the character content to a target handwritten form style, and generating a target character pattern picture, thereby obtaining a complete Chinese character library in the handwritten form in the character library. The method is an end-to-end generation method, does not need to extract strokes or components of Chinese characters and manual intervention, generates high-quality Chinese character patterns, greatly improves the efficiency of making the handwritten font library, enables the generation of the personalized font library to be simple and convenient, can meet the requirements of common people on the personalized handwritten font library, and accelerates the development process of the personalized font library.
Description
Technical Field
The invention relates to a computer graphic processing technology and an artificial intelligence technology, in particular to a method for automatically generating a Chinese character library in handwritten form based on a deep neural network.
Background
With the rapid development of the mobile internet, people have stronger and stronger intentions of pursuing beauty and individuation. Although the standard characters such as regular script and song font are convenient to use, the characters lack individuality. The popularity of social media such as microblogs, WeChats, and QQ has led young people to wish to use their own handwriting to reveal their personalities. Meanwhile, more and more calligraphy lovers hope to write electronic documents by using personalized handwriting on electronic mobile equipment such as computers, mobile phones and the like to communicate and achieve the effect of seeing words and seeing people. In addition, the personalized fonts can also be used for business design such as company logos and the like to highlight enterprise culture.
The western character set composed of latin letters, numbers, punctuation marks, etc. is small and can easily implement computer storage and encoding. The Chinese characters have complex structure and large quantity, and the commonly used GB2312 character set comprises 6763 simplified Chinese characters. At present, the Chinese font design and making technology in China is not advanced enough, the existing Chinese font making method mostly depends on manual experience and design, the automation degree is very low, generally, a calligraphy writer writes or a font designer makes hundreds to thousands of reference characters, all strokes and parts of all Chinese characters in a target font are included, and the edge outlines of the characters are stored by curves and straight lines. The font maker then processes and modifies the strokes and components of the reference word to generate a complete word stock. Finally, each Chinese character font is finely adjusted. The automatic program of the word stock manufacturing technology is low, the manufacturing period is long, and the efficiency is low due to a large amount of manual word shape adjustment and design.
In recent years, many researchers synthesize Chinese characters by multiplexing strokes or parts of Chinese characters, but the methods need to extract the strokes or the parts in advance, need manual intervention to ensure the correctness of an extraction result, and are not feasible for quickly making a personalized word stock.
With the development of deep learning, the deep neural network is applied to the generation of Chinese characters. The 'Rewrite' method recorded in the literature (YuchhenTian.2016. Rewrite: Neural Style Transfer For Chinese Fonts. (2016). RetrieveedNov 23,2016from https:// gitub.com/kaonashi-tyc/Rewrite) designs a trickle-drop network structure, can generate more standard Fonts, but a user needs to write thousands of Chinese characters, and has no good effect on the condition of writing Chinese characters and having large difference with the Style of the reference Fonts. The document (Zhouhui Lian, Bo Zhao, and Jian Jianguo Xiao.2016.automatic generation of large scale hand writing information about in Proc. SIGTRAPH ASIA 2016TB. ACM,12.) generates a complete Chinese library by modeling the writing style, but this method requires the automatic extraction of strokes or parts in advance. The zi2zi method described in the literature (YuchhenTian.2017. zi2zi: Master Chinese calligraph with Conditional additive networks. (2017). Retrieved Jun 3,2017from https:// githu.com/kaonashi-tyc/zi 2zi) generates the structure of the confrontation network and U-net based on the condition, synthesizes the Chinese character picture with a specific style by adopting the confrontation training mode, but still has the situations of fuzzy and false edges, generates the quality of the character pattern with low quality and cannot meet the requirements of practical application.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a method for automatically generating a handwritten Chinese character library based on a deep neural network, which comprises the steps of representing, reconstructing and transferring the font style, reconstructing the font features of a small number of handwritten Chinese characters written by a user, and estimating the font style features of the characters which are not written by the user; and establishing a font style migration network, combining the font content of the reference font with the writing style of the user, realizing the migration from the reference font style to the target handwriting style, and generating a target font picture, thereby obtaining a complete Chinese handwriting font library file.
The technical scheme adopted by the invention is as follows:
a method for automatically generating a Chinese character library in handwritten form based on a deep neural network comprises the steps of carrying out font characteristic reconstruction through a small number of Chinese characters in handwritten form, and estimating font style characteristics of the characters which are not written by a user; then combining the font content of the reference font with the writing style of the user through a font style migration network, migrating the reference font style to the target handwriting style, and generating a target font picture, thereby obtaining a complete Chinese handwriting font library file; mainly comprises the following six steps:
firstly, writing Chinese characters of a specified input set by a user, and taking a picture or scanning the picture;
secondly, dividing the picture into single Chinese character images, and normalizing the size of the single Chinese character images to be consistent with that of the reference character pattern picture; in the specific implementation of the invention, the size of the image is normalized to 224 multiplied by 224;
thirdly, extracting font characteristics of the Chinese characters written by the user through a pre-trained font identification network;
fourthly, estimating the font characteristics of the Chinese characters which are not written by the user through a font characteristic reconstruction network, learning the transformation relation from the reference Chinese character characteristics to the font characteristics of the corresponding Chinese characters with the writing style of the user, and reconstructing the font characteristics of the Chinese character set which is not written by the user; the reference font can be other fonts such as a regular font, a song font and a bold font.
Fifthly, respectively extracting the font content characteristics and the font style characteristics of the Chinese characters through a convolutional neural network, and realizing the migration from a reference Chinese character (such as a regular square font) to the writing style of the user through a font style migration network under the condition that the font content is not changed, so as to generate a Chinese character picture which is not written by the user in a complete character library;
and sixthly, combining the Chinese character pictures written by the user with the generated Chinese character pictures to obtain 6763 complete Chinese character pictures in the GB2312 Chinese character library, and vectorizing the Chinese characters to generate a personalized character library file with the writing style of the user.
Specifically, in the first step, 775 Chinese characters are selected according to the Chinese character use frequency and the Chinese character strokes and component structures in the GB2312 Chinese character library, an input set is formed, 50% of common Chinese characters can be covered, and all strokes and component types appearing in the GB2312 Chinese character library are included.
In the second step, the text image is subjected to direction correction and cutting to obtain a single Chinese character picture. The Chinese character picture is placed in the center of a square with the longer side of width and height as the side length, and then the picture is scaled to 224 multiplied by 224, and the aspect ratio of the original Chinese character is maintained.
In the third step, for each Chinese character font picture, the font depth characteristic is represented by the high-level characteristic of the pre-trained font identification network phi. Specifically, the font recognition network adopts a network structure of VGG16(Karen Simmonyan and Dndrew Zisserman.2014.Very deep related networks for large-scale image recognition. arXiv preprinting arXiv:1409.1556 (2014)), and trains on 100 font data. Specifically, 6763 Chinese character pictures of the GB2312 character set are generated by utilizing the ttf file of each font, and the picture size is 224 multiplied by 224. And adjusting parameters on the model trained by the ImageNet data set by taking half of the picture data as a training set and the other half of the picture data as a test set. The invention adopts the output of 14 multiplied by 512 dimensions of the conv5_3 layer (after the ReLU activation layer) of the font identification network to characterize the font style characteristics of Chinese characters.
In the fourth step, the font characteristics of the Chinese character font which is not written by the user are estimated through the font characteristic reconstruction network. In a font recognition network, Chinese characters of the same font are gathered together in a depth feature space, and a font feature reconstruction network learns a transformation relation R from a reference font feature to a user handwritten Chinese character font feature through a small number of Chinese characters written by the user. For Chinese characters that the user does not write, the font features can be estimated by the font feature reconstruction network (as shown in FIG. 3). The structure of the font features reconstruction network is similar to the encoder-decoder (as shown in fig. 2). The input of the character feature coder is a depth character feature phi obtained by a character recognition network through a reference character pattern xrelu5_3(x) The character font feature decoder comprises four downsampling layers, a vector obtained by coding is connected with a character font type vector and is sent to the character font feature decoder, and the character font type vector is a 64-dimensional random vector, so that a network can better distinguish each character font during training. The font characteristic decoder and the font characteristic encoder have symmetrical structures and comprise a series of up-sampling layers, and finally the estimated depth font characteristic h is obtaineds＝R(φrelu5_3(x) ). In order to reduce the information loss during encoding, the corresponding layers of the font property encoder and font property decoder are jump-connected.
In a fifth step, the font style migration network migrates the Chinese characters from the reference font style to the user handwriting style. The Chinese character can be regarded as the combination of the font content and the font style, and the invention respectively uses two convolution neural networks to code the font content and the font style. Specifically, the font characteristic reconstruction network in the fourth step can obtain the deep font style characteristic h of the Chinese character fonts. Meanwhile, the reference character pattern is processed by a content encoder (a series of down-sampling layers) to obtain a content vector h of the Chinese character patternc. Then, the Chinese character font content h will be representedcFont style hsAnd font type hfAre combined to form h ═ hs,hc,hf]The data is sent to five residual blocks and then passed through a decoder (a series of upsampling layers, which generates glyphs according to the specified style and content) to obtain a block with data to be usedThe user writes the style of Chinese character picture. Considering that the reference font and the user handwritten glyph have similar structural features, the lower layers of the content encoder are jump-connected to the corresponding layers of the decoder (as shown in fig. 2).
In the invention, the font style migration network utilizes the idea of generating an confrontation network, the input is a reference font (such as a regular font) font, and the output is a target handwritten font picture with the same font content
And respectively splicing the generated font image and the real font image written by the user with the corresponding reference font image to form an image, sending the image into a discriminator D to judge whether the image is generated or written by the user, and judging the type of the font.
The sizes of the input and output pictures of the font style migration network are 224 multiplied by 224, and the input pictures are respectively sent to a content encoder and a font characteristic reconstruction network to obtain font content and font characteristic codes. The content encoder contains 4 downsampled layers, each layer consisting of a convolution layer with convolution kernel size 5 x 5 and step size 2, batch normalization layer, and LeakyRelu. And then combining vectors representing the Chinese character font content, the font style and the font category, and sending the combined vectors into five residual blocks, wherein each residual block comprises two cascaded BN-Relu-Convolition structures. And finally, obtaining the generated picture through four upsampling layers, wherein each upsampling layer comprises a deconvolution layer with a convolution kernel size of 5 multiplied by 5 and a step length of 2, a batch normalization layer and Relu. The structure of the discriminator is a network structure in the literature (Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei AEfrost.2016. image-to-image transformation with conditional access network. arxiv preprritinxiv: 1611.07004 (2016)).
When a font style migration network is trained, the invention measures and generates the font by combining the countermeasure loss, the pixel point space loss and the style consistency loss
And the real target font written by the usery, updating the network parameters. The countermeasure loss function refers to the thought of generating the countermeasure network, and the discriminator simultaneously judges the genres of the generated fonts and the genres. Assuming that the reference font picture and the target font picture obey the joint probability distribution x, y-p as shown in formulas (1) and (2)data(x, y), reference glyph picture x obeys probability distribution pdata(x) Vector h representing the content of Chinese character fontcFont style vector hsAnd vector h of font classfThe combined vector h ═ hs,hc,hf]Obey to a probability distribution ph(h)，Ds(x, y) represents the result of the judgment of the character picture by the discriminator, Dc(x, y) represents the result of the judgment of the picture type by the discriminator.
And
representing the penalty of the arbiter for generating a glyph authenticity and font type decision, respectively. It is desirable for the discriminator that the network-generated glyphs be judged to be more likely to be false, and the font style migration network is expected to generate glyph images that are judged to be true, so the font style migration network minimization is desirable
Updating network parameters while arbiter maximization
And updating the network parameters.
To ensure that the generated font and the target font are in the pixel pointSimilarity of spaces, as shown in equation (3), by calculating the generated glyph image
L of the handwritten target character pattern picture ypixel(L1 distance) to measure their degree of similarity.
In addition, glyphs are generated by computation
The difference from the target glyph y in the depth feature space is used to measure the similarity, as shown in formula (4), which includes
Relu with y at font recognition network phi2_2、relu3_3And relu4_3The squared error loss of the activation value of the layer.
And finally, combining the three loss functions by a certain weight, wherein the loss function of the font style migration network is L as shown in a formula (5), and the parameters in the font style migration network are adjusted by minimizing L to obtain the optimal parameter value.
Wherein α, β and gamma are weight coefficients of the three loss functions respectively.
Due to the complexity of the network structure, instability of the generation of the anti-network training is generated, and the network convergence difficulty of the complex font is higher for each writing style. In order to accelerate the convergence speed, the invention is pre-trained by 20 square fonts, each of which contains 2000 common characters. When learning a particular handwriting style, only the parameters need to be fine-tuned on the pre-trained model.
In the sixth step, 775 Chinese characters written by the user are combined with 5988 generated Chinese characters to obtain 6763 Chinese characters of the complete GB 2312. And then carrying out Vectorization operation on 6763 Chinese characters (Pan W., Lian Z., Tang Y., XiaoJ., Skeleton-Guided Vectorization of Chinese calligraphic images, MMSP 2014, paper ID 19,2014) to obtain a word stock file in a TrueType format with the handwriting style of the user.
Compared with the prior art, the invention has the advantages that:
the invention provides a method for automatically generating a handwritten Chinese library based on a deep neural network, and provides a novel algorithm for font characteristic reconstruction and font style migration. The whole process does not need to extract strokes or components of the Chinese characters and manual intervention, and the method is an end-to-end generation method, greatly improves the efficiency of making the handwritten font library, and can meet the requirements of common people on the personalized handwritten font library. The method can generate high-quality Chinese character font without stroke or component extraction and manual intervention operation, greatly shortens the manufacturing period of the font library, improves the manufacturing efficiency, enables the generation of the personalized font library to be simple and convenient, and accelerates the development process of the personalized font library.
Drawings
FIG. 1 is a block diagram of the process for automatically generating a library of handwritten Chinese characters according to the present invention.
Fig. 2 is a network structure diagram of the automatic generation method of the character library in handwriting.
FIG. 3 is a schematic diagram of a font feature reconstruction process provided by the present invention;
wherein, (a) is the result of dimension reduction of the style characteristics of the deep font through t-SNE; (b) the method comprises the following steps of learning a schematic diagram of a conversion relation R of a reference font style to a handwriting style on a small number of Chinese characters written by a user; (c) the font style characteristics of the Chinese characters which are not written by the user and are obtained by transforming the relation R.
FIG. 4 is a comparison of experimental results of the method of the present invention and three other neural network-based generation methods (Rewrite, FontSL, zi2 zi);
wherein, the Rewrite can only generate fuzzy font pictures and can not ensure the correctness of the font; the fonts of three different handwriting styles generated by the FontSL have similarity, and the essential style characteristics of the fonts are not learned; zi2zi has false edges and unreasonable strokes; the method of the invention can generate high-quality font images with the handwriting style of the user.
FIG. 5 is a diagram of the rendering effect of the text in the Chinese character library with three different handwriting styles according to the embodiment of the present invention;
wherein the regular font in (a) is a Chinese character which is not written by the user; the corresponding character positions in (b) - (d) show the Chinese character patterns generated by the method of the invention.
Detailed Description
The invention will be further described by way of examples, without in any way limiting the scope of the invention, with reference to the accompanying drawings.
The invention provides a method for automatically generating a Chinese character library in handwritten form based on a deep neural network, which carries out font characteristic reconstruction through a small number of Chinese characters in handwritten form and estimates the font style characteristic of the characters which are not written by a user; and then combining the font content of the reference font with the writing style of the user through a font style migration network, migrating the reference font style to the target handwriting style, and generating a target font picture, thereby obtaining a complete Chinese handwriting font library file.
The flow chart and the network structure chart of the method of the invention are shown in the attached figures 1 and 2, and when the method is implemented specifically, the method comprises the following steps:
1) the user writes 775 Chinese characters of the appointed input set, takes pictures or scans the pictures, and uploads the pictures to the system;
2) and cutting the picture into single Chinese character images, and performing direction correction and cutting on the text images to obtain the single Chinese character pictures. The Chinese character picture is placed in the center of a square with the longer side of width and height as the side length, then the picture is scaled to 224 multiplied by 224, the width-height ratio of the original Chinese character is kept, and the picture is saved by the unicode code naming of the Chinese character.
3) Training a font characteristic reconstruction network, as shown in fig. 2, wherein the network input is a deep font characteristic obtained by referring to a font character through a font recognition network, and the target output is the deep font characteristic of a Chinese character font with a user writing style. In the training stage, 775 Chinese characters written by the user are used as training data, and network parameters are adjusted, so that the loss of square error between the network output and a target value is minimized. In the subsequent training process of the style migration network, the parameters of the font characteristic encoder are fixed, and the parameters of the font characteristic decoder are finely adjusted.
4) Fine-tuning the pre-trained font style migration network, as shown in fig. 2;
41) for each reference font x, the font style vector h is obtained by respectively sending the font style vector x to a font characteristic reconstruction network and a content encodersContent vector h of sum fontc；
42) Vector the font style hsContent vector h of fontcAnd font class vector hfCombining the two pictures, sending the combined pictures to a residual error network and a decoder to obtain a Chinese character picture which has the same character content as the reference character pattern x and has the writing style of the user
43) A loss function of the font style migration network is calculated.
a) To combat the loss
D is shown in formulas (1) and (2)s(x, y) represents the result of the judgment of the character picture by the discriminator, Dc(x, y) represents the result of the judgment of the picture type by the discriminator.
And
representing the penalty of the arbiter for generating a glyph authenticity and font type decision, respectively. It is desirable for the discriminator that the network-generated glyphs be judged to be more likely to be false, and the font style migration network is expected to generate glyph images that are judged to be true, so the font style migration network minimization is desirable
Updating network parameters while arbiter maximization
And updating the network parameters.
b) Loss of pixel space
Computationally generated glyph images
The distance from the L1 of the handwritten target glyph picture y yields the pixel point space loss.
c) Loss of style consistency
Generating glyphs by computation
Relu of font identification network phi with target font y2_2、relu3_3And relu4_3Loss of squared error of activation values of layers yields a loss of style consistency Lstyle。
Finally, these three errors are combined by a certain weight, usually set to α -1, β -100, and γ -15.
5) After the network training is finished, the font migration network can migrate the input arbitrary regular font to the Chinese character with the handwriting style of the user. 5988 regular font characters which are not written by the user are input into the network, and the regular font characters are output as font pictures which have the same font character content and the writing style of the user.
6) 755 Chinese character pictures written by the user and 5988 generated Chinese character pictures are combined to obtain 6763 complete Chinese character pictures in a GB2312 Chinese character library, and the Chinese characters are vectorized to generate a TrueType-format character library file with the writing style of the user.
FIG. 5 is a diagram of the rendering effect of the text of the Chinese character library of the three different handwriting styles according to the present invention. Wherein, (a) is the text rendering effect of the word stock generated by 775 Chinese characters written by the user, and (b) - (d) are the text rendering effect of the word stock obtained by the method of the invention. The experimental results show that the method can generate vivid character patterns with the handwriting style of the user, does not need any manual intervention and prior Chinese character information, is simple and efficient, and can meet the requirements of common people on the personalized handwriting character library.
The technical solutions in the embodiments of the present invention are clearly and completely described above with reference to the drawings in the embodiments of the present invention. It is to be understood that the described examples are only a few embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Claims (6)
1. A method for automatically generating a Chinese character library in handwritten form based on a deep neural network comprises the steps of reconstructing character features of a small number of Chinese characters in handwritten form, estimating character style features of characters which are not written by a user, combining character content of a reference character with the writing style of the user through a character style migration network, migrating the character content of the reference character to a target handwritten form style from the reference character style, generating a target character style picture, and obtaining a complete Chinese handwritten form library file; the method comprises the following steps:
firstly, writing Chinese characters of an appointed input set by a user, and obtaining a text picture by photographing or scanning;
secondly, segmenting the text picture to obtain a single Chinese character image, and normalizing the size of the single Chinese character image to be consistent with the size of the reference character pattern picture;
thirdly, extracting font characteristics of the Chinese characters written by the user through a pre-trained font identification network aiming at each Chinese character font picture written by the user;
fourthly, estimating the font characteristics of the Chinese characters which are not written by the user through a font characteristic reconstruction network, learning the transformation relation from the reference Chinese character characteristics to the font characteristics of the corresponding Chinese characters with the writing style of the user, and reconstructing the font characteristics of the Chinese character set which is not written by the user;
the font characteristic reconstruction network structure comprises a font characteristic encoder and a font characteristic decoder; specifically, a conversion relation R from the reference font characteristics to the handwriting Chinese character font characteristics of the user is learned through the Chinese characters written by the user, and the font characteristics of the Chinese characters which are not written by the user are estimated through the learned conversion relation;
fifthly, respectively extracting the character form content characteristics and the character form style characteristics of the Chinese characters through a convolutional neural network, realizing the transfer from the reference Chinese characters to the writing style of the user through a character form style transfer network under the condition that the character form content is not changed, and generating a Chinese character picture which is not written by the user in a complete character library;
the font style migration network migrates the Chinese characters from the reference font style to the handwriting style of the user; specifically, the content and style of Chinese character patterns are coded by two convolutional neural networks respectively; obtaining the deep font style characteristic h of the Chinese character font through the font style migration networks(ii) a Reference character shapeContent encoder obtains content vector h of Chinese character fontc(ii) a Then, the content vector h of Chinese character fontcCharacter style feature hsAnd font type hfThe vector combination of (a) is h ═ hs,hc,hf]Sending the Chinese character image into a residual block, and obtaining a Chinese character image with a user writing style through a decoder;
splicing the generated font picture and the real font picture written by the user with the corresponding reference font picture to form pictures, sending the pictures into a discriminator D, judging whether the pictures are generated or written by the user, and judging the type of the fonts at the same time;
the discriminator simultaneously judges the genres of true and false and font of the generated font, specifically, assumes that the reference font picture and the target font picture obey the joint probability distribution x, y-pdata(x, y), reference glyph picture x obeys probability distribution pdata(x) Vector h ═ hs,hc,hf]Obey to a probability distribution ph(h) (ii) a The loss of the judger for generating the font true and false and the font type judgment is expressed by the following expressions (1) and (2), respectively:
wherein D iss(x, y) represents the result of judging the truth of the character picture by the discriminator; dc(x, y) represents the result of the judgment of the image type by the discriminator;
and
respectively representing the loss of the judger for judging whether the generated font is true or false and the font type;
character pattern picture generated by calculation of formula (3)
L of the handwritten target character pattern picture ypixelDistance, which measures the degree of similarity between:
the font can also be generated by the calculation of the formula (4)
The difference from the target glyph y in the depth feature space is used for measuring the similarity between the two types:
wherein formula (4) comprises
Relu with y at font recognition network phi2_2、relu3_3And relu4_3Loss of square error of the activation value of the layer;
finally, combining the three loss functions through weights, and expressing the loss function L of the font style migration network as an expression (5):
wherein α, β and gamma are weight coefficients of the three loss functions respectively;
parameters in the font style migration network are adjusted by minimizing a loss function L of the font style migration network according to the formula (5), so that an optimal parameter value is obtained;
and sixthly, combining the Chinese character picture written by the user and the generated Chinese character picture to obtain a complete Chinese character picture of the Chinese character library, and carrying out vectorization, thereby generating a personalized character library file with the writing style of the user.
2. The method of claim 1, wherein in the first step, 775 chinese characters are selected based on the frequency of use of chinese characters and the composition of strokes and parts of the chinese character library in GB2312, covering 50% of commonly used chinese characters, and including all strokes and part types appearing in the chinese character library in GB2312, as the input set.
3. The method for automatically generating a handwritten Chinese character library based on a deep neural network as claimed in claim 1, wherein in the second step, specifically, the direction correction and the cutting are performed on the text picture to obtain a single Chinese character picture; the single Chinese character picture is placed in the center of a square with the longer side of width and height as the side length, and then the picture is scaled to the size of 224 multiplied by 224, and the aspect ratio of the original Chinese character is kept.
4. The method for automatically generating a Chinese character library in handwritten form based on deep neural network as claimed in claim 1, wherein in the third step, specifically, the font recognition network is trained on 100 font data by using VGG16 network structure to obtain a pre-trained font recognition network; and adopting the output of the Conv5_3 layer of the font identification network to represent the font style characteristics of the Chinese characters, thereby extracting the font characteristics of the Chinese characters written by the user.
5. The method of claim 1, wherein in the fourth step, the input of the font feature encoder of the font feature reconstruction network structure is the deep font feature phi obtained by the font recognition network through the reference glyph xrelu5_3(x) The font feature encoder comprises four down-sampling layers, the vector obtained by encoding is connected with the representing font category vector and is sent to the font feature decoder, and the font category vector is a 64-dimensional random vector, so that the network can better distinguish each font during training; character decoder and character coding deviceHas a symmetrical structure, comprises a plurality of upsampling layers and obtains an estimated depth font characteristic hs＝R(φrelu5_3(x))。
6. The method of claim 1, wherein in the fourth step, the font feature reconstruction network structure comprises a font feature encoder and a font feature decoder, and the font feature encoder and the font feature decoder are connected in a jump manner at corresponding layers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710908121.XA CN107644006B (en) | 2017-09-29 | 2017-09-29 | Automatic generation method of handwritten Chinese character library based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710908121.XA CN107644006B (en) | 2017-09-29 | 2017-09-29 | Automatic generation method of handwritten Chinese character library based on deep neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107644006A CN107644006A (en) | 2018-01-30 |
| CN107644006B true CN107644006B (en) | 2020-04-03 |
Family
ID=61122621
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710908121.XA Active CN107644006B (en) | 2017-09-29 | 2017-09-29 | Automatic generation method of handwritten Chinese character library based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107644006B (en) |
Families Citing this family (61)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108304357B (en) * | 2018-01-31 | 2020-09-08 | 北京大学 | Chinese character library automatic generation method based on font manifold |
| CN108459999B (en) * | 2018-02-05 | 2022-02-22 | 杭州时趣信息技术有限公司 | Font design method, system, equipment and computer readable storage medium |
| CN108388925A (en) * | 2018-03-06 | 2018-08-10 | 天津工业大学 | The anti-pattern collapse robust image generation method for generating network is fought based on New Conditions |
| CN108537743B (en) * | 2018-03-13 | 2022-05-20 | 杭州电子科技大学 | Face image enhancement method based on generation countermeasure network |
| CN108416390B (en) * | 2018-03-16 | 2019-11-01 | 西北工业大学 | Hand-written script recognition methods based on two-dimensional convolution dimensionality reduction |
| CN110363830B (en) * | 2018-04-10 | 2023-05-02 | 阿里巴巴集团控股有限公司 | Element image generation method, device and system |
| CN108550107A (en) * | 2018-04-27 | 2018-09-18 | Oppo广东移动通信有限公司 | A kind of image processing method, picture processing unit and mobile terminal |
| CN108615036B (en) * | 2018-05-09 | 2021-10-01 | 中国科学技术大学 | Natural scene text recognition method based on convolution attention network |
| CN108961428B (en) * | 2018-05-23 | 2023-05-26 | 杭州易现先进科技有限公司 | Style migration method, medium, device and computing equipment for three-dimensional actions |
| CN108804397B (en) * | 2018-06-12 | 2021-07-20 | 华南理工大学 | Chinese character font conversion generation method based on small amount of target fonts |
| CN109035318B (en) * | 2018-06-14 | 2021-11-30 | 西安电子科技大学 | Image style conversion method |
| CN108830245A (en) * | 2018-06-24 | 2018-11-16 | 海南大学 | Body method is write in personalized handwritten form replacement |
| CN109165376B (en) * | 2018-06-28 | 2023-07-18 | 西交利物浦大学 | Style character generation method based on small amount of samples |
| CN108960425B (en) * | 2018-07-05 | 2022-04-19 | 广东工业大学 | Rendering model training method, system, equipment, medium and rendering method |
| CN110738715B (en) * | 2018-07-19 | 2021-07-09 | 北京大学 | Automatic migration method of dynamic text special effect based on sample |
| CN109241750B (en) * | 2018-07-20 | 2023-04-07 | 海南大学 | Method for changing display style of individual English letters |
| CN109308475A (en) * | 2018-07-26 | 2019-02-05 | 北京百悟科技有限公司 | A kind of character recognition method and device |
| CN109064522A (en) * | 2018-08-03 | 2018-12-21 | 厦门大学 | The Chinese character style generation method of confrontation network is generated based on condition |
| CN109522975A (en) * | 2018-09-18 | 2019-03-26 | 平安科技（深圳）有限公司 | Handwriting samples generation method, device, computer equipment and storage medium |
| CN109522898A (en) * | 2018-09-18 | 2019-03-26 | 平安科技（深圳）有限公司 | Handwriting samples picture mask method, device, computer equipment and storage medium |
| CN109408776A (en) * | 2018-10-09 | 2019-03-01 | 西华大学 | A kind of calligraphy font automatic generating calculation based on production confrontation network |
| CN109447137B (en) * | 2018-10-15 | 2022-06-14 | 聚时科技(上海)有限公司 | Image local style migration method based on decomposition factors |
| CN109615671A (en) * | 2018-10-25 | 2019-04-12 | 北京中关村科金技术有限公司 | A kind of character library sample automatic generation method, computer installation and readable storage medium storing program for executing |
| CN109635883B (en) * | 2018-11-19 | 2023-04-18 | 北京大学 | Chinese character library generation method based on structural information guidance of deep stack network |
| CN109472249A (en) | 2018-11-22 | 2019-03-15 | 京东方科技集团股份有限公司 | A kind of method and device of determining script superiority and inferiority grade |
| CN109543777B (en) * | 2018-11-28 | 2020-10-27 | 中国科学院自动化研究所 | Method and system for evaluating writing quality of handwritten Chinese characters |
| CN110097615B (en) * | 2018-12-06 | 2021-01-19 | 北京大学 | Stylized and de-stylized artistic word editing method and system |
| CN112840312B (en) * | 2018-12-19 | 2023-05-26 | 深圳市欢太科技有限公司 | Font setting method and device |
| CN112805674B (en) * | 2018-12-19 | 2024-04-02 | 深圳市欢太科技有限公司 | Font setting method and device |
| CN112840305B (en) * | 2018-12-19 | 2023-11-28 | 深圳市欢太科技有限公司 | Font switching method and related product |
| CN110033054B (en) * | 2019-03-14 | 2021-05-25 | 上海交通大学 | Personalized handwriting migration method and system based on collaborative stroke optimization |
| CN110135530B (en) * | 2019-05-16 | 2021-08-13 | 京东方科技集团股份有限公司 | Method and system for converting Chinese character font in image, computer device and medium |
| CN110176050B (en) * | 2019-05-30 | 2023-05-09 | 清华大学深圳研究生院 | Aesthetic optimization method for text generated image |
| CN110335212B (en) * | 2019-06-28 | 2021-01-15 | 西安理工大学 | Defect ancient book Chinese character repairing method based on condition confrontation network |
| CN110414633B (en) * | 2019-07-04 | 2022-10-14 | 东南大学 | System and method for recognizing handwritten fonts |
| CN110570481A (en) * | 2019-07-31 | 2019-12-13 | 中国地质大学（武汉） | calligraphy word stock automatic repairing method and system based on style migration |
| CN112329803B (en) * | 2019-08-05 | 2022-08-26 | 北京大学 | Natural scene character recognition method based on standard font generation |
| CN111753493A (en) * | 2019-09-29 | 2020-10-09 | 西交利物浦大学 | Style character generation method containing multiple normalization processes based on small amount of samples |
| CN112784531B (en) * | 2019-11-05 | 2024-02-27 | 北京大学 | Chinese character form and character library generation method based on deep learning and component splicing |
| CN110969681B (en) * | 2019-11-29 | 2023-08-29 | 山东浪潮科学研究院有限公司 | Handwriting word generation method based on GAN network |
| CN110956678B (en) * | 2019-12-16 | 2022-02-22 | 北大方正集团有限公司 | Font processing method and device |
| CN111027550B (en) * | 2019-12-17 | 2023-09-08 | 新方正控股发展有限责任公司 | Method and device for adjusting visual center of gravity of word stock |
| CN111144066B (en) * | 2019-12-27 | 2022-02-18 | 北大方正集团有限公司 | Adjusting method, device and equipment for font of font library and storage medium |
| CN111553837B (en) * | 2020-04-28 | 2022-04-29 | 武汉理工大学 | Artistic text image generation method based on neural style migration |
| CN111539873B (en) * | 2020-05-06 | 2023-10-20 | 中国科学院自动化研究所 | Method and system for generating personal style handwriting word stock |
| CN111667006A (en) * | 2020-06-06 | 2020-09-15 | 大连民族大学 | Method for generating family font based on AttGan model |
| CN111667008A (en) * | 2020-06-08 | 2020-09-15 | 大连民族大学 | Personalized Chinese character font picture generation method based on feature fusion |
| CN112132916B (en) * | 2020-08-18 | 2023-11-14 | 浙江大学 | Seal cutting work customized design generating device for generating countermeasure network |
| CN112070658B (en) * | 2020-08-25 | 2024-04-16 | 西安理工大学 | Deep learning-based Chinese character font style migration method |
| CN112183027B (en) * | 2020-08-31 | 2022-09-06 | 同济大学 | Artificial intelligence based artwork generation system and method |
| CN112115678B (en) | 2020-09-21 | 2024-04-12 | 京东方科技集团股份有限公司 | Information display method and device, storage medium and electronic equipment |
| CN112417959A (en) * | 2020-10-19 | 2021-02-26 | 上海臣星软件技术有限公司 | Picture generation method and device, electronic equipment and computer storage medium |
| CN112381910A (en) * | 2020-12-28 | 2021-02-19 | 北京亿信华辰软件有限责任公司武汉分公司 | Handwriting stylization method for characters of printed body |
| CN113095038B (en) * | 2021-05-08 | 2024-04-16 | 杭州王道控股有限公司 | Font generation method and device for generating countermeasure network based on multi-task discriminator |
| CN113256750B (en) * | 2021-05-26 | 2023-06-23 | 武汉中科医疗科技工业技术研究院有限公司 | Medical image style reconstruction method, medical image style reconstruction device, computer equipment and storage medium |
| CN113553932B (en) * | 2021-07-14 | 2022-05-13 | 同济大学 | Calligraphy character erosion repairing method based on style migration |
| CN113515919B (en) * | 2021-09-14 | 2022-01-07 | 北京江融信科技有限公司 | Method and system for generating Chinese TrueType font |
| CN113807430B (en) * | 2021-09-15 | 2023-08-08 | 网易（杭州）网络有限公司 | Model training method, device, computer equipment and storage medium |
| CN113903043B (en) * | 2021-12-11 | 2022-05-06 | 绵阳职业技术学院 | Method for identifying printed Chinese character font based on twin metric model |
| CN114863527B (en) * | 2022-05-06 | 2024-03-19 | 杭州电子科技大学 | Makeup style migration method based on FP-SCGAN model |
| CN116416628A (en) * | 2023-06-06 | 2023-07-11 | 广州宏途数字科技有限公司 | Handwriting font recognition based method and recognition system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106384094A (en) * | 2016-09-18 | 2017-02-08 | 北京大学 | Chinese word stock automatic generation method based on writing style modeling |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140363082A1 (en) * | 2013-06-09 | 2014-12-11 | Apple Inc. | Integrating stroke-distribution information into spatial feature extraction for automatic handwriting recognition |
| CN103488711B (en) * | 2013-09-09 | 2017-06-27 | 北京大学 | A kind of method and system of quick Fabrication vector font library |
| CN106611172B (en) * | 2015-10-23 | 2019-11-08 | 北京大学 | A kind of Chinese character synthetic method based on style study |
-
2017
- 2017-09-29 CN CN201710908121.XA patent/CN107644006B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106384094A (en) * | 2016-09-18 | 2017-02-08 | 北京大学 | Chinese word stock automatic generation method based on writing style modeling |
Non-Patent Citations (1)
| Title |
|---|
| Chinese Typography Transfer;Jie Chang，Yujun Gu，Ya Zhang;《arXiv preprint arXiv：1707.04904》;20170716;第1-7页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107644006A (en) | 2018-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107644006B (en) | Automatic generation method of handwritten Chinese character library based on deep neural network | |
| CN109190722B (en) | Font style migration transformation method based on Manchu character picture | |
| Jiang et al. | Scfont: Structure-guided chinese font generation via deep stacked networks | |
| CN109635883B (en) | Chinese character library generation method based on structural information guidance of deep stack network | |
| Jiang et al. | Dcfont: an end-to-end deep chinese font generation system | |
| CN108304357B (en) | Chinese character library automatic generation method based on font manifold | |
| AU2019202063B2 (en) | Synthesizing new font glyphs from partial observations | |
| CN110443864B (en) | Automatic artistic font generation method based on single-stage small-amount sample learning | |
| CN114139495A (en) | Chinese font style migration method based on adaptive generation countermeasure network | |
| CN111428727B (en) | Natural scene text recognition method based on sequence transformation correction and attention mechanism | |
| CN111814566A (en) | Image editing method, image editing device, electronic equipment and storage medium | |
| CN114596566B (en) | Text recognition method and related device | |
| CN110473151B (en) | Partition convolution and correlation loss based dual-stage image completion method and system | |
| CN113553797A (en) | End-to-end consistent-style character and image generation method | |
| US20230215162A1 (en) | Method and apparatus for text-to-image generation using self-supervised discriminator to extract image feature | |
| Zhao et al. | Cartoon image processing: a survey | |
| CN114820303A (en) | Method, system and storage medium for reconstructing super-resolution face image from low-definition image | |
| CN112837210B (en) | Multi-shape variable-style face cartoon automatic generation method based on feature map segmentation | |
| CN111161266B (en) | Multi-style font generation method of variational self-coding machine based on vector quantization | |
| CN110097615B (en) | Stylized and de-stylized artistic word editing method and system | |
| CN117057310A (en) | Font generation method and device based on diffusion model | |
| CN113744158A (en) | Image generation method and device, electronic equipment and storage medium | |
| CN113344060A (en) | Text classification model training method, litigation shape classification method and device | |
| Huang et al. | Gentext: Unsupervised artistic text generation via decoupled font and texture manipulation | |
| KR102608266B1 (en) | Method and apparatus for generating image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |