US20160364633A1 - Font recognition and font similarity learning using a deep neural network - Google Patents
Font recognition and font similarity learning using a deep neural network Download PDFInfo
- Publication number
- US20160364633A1 US20160364633A1 US14/734,466 US201514734466A US2016364633A1 US 20160364633 A1 US20160364633 A1 US 20160364633A1 US 201514734466 A US201514734466 A US 201514734466A US 2016364633 A1 US2016364633 A1 US 2016364633A1
- Authority
- US
- United States
- Prior art keywords
- font
- image
- text
- cnn
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013528 artificial neural network Methods 0.000 title description 3
- 238000012360 testing method Methods 0.000 claims abstract description 84
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 68
- 238000012549 training Methods 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims description 28
- 239000013598 vector Substances 0.000 claims description 16
- 230000002194 synthesizing effect Effects 0.000 claims description 12
- 230000006835 compression Effects 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 9
- 230000009466 transformation Effects 0.000 claims description 8
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims 1
- 230000003094 perturbing effect Effects 0.000 claims 1
- 230000006870 function Effects 0.000 abstract description 12
- 230000001419 dependent effect Effects 0.000 abstract 1
- 230000015572 biosynthetic process Effects 0.000 description 23
- 238000003786 synthesis reaction Methods 0.000 description 23
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 8
- 210000002569 neuron Anatomy 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000003292 diminished effect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G06K9/66—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/24—Character recognition characterised by the processing or recognition method
- G06V30/242—Division of the character sequences into groups prior to recognition; Selection of dictionaries
- G06V30/244—Division of the character sequences into groups prior to recognition; Selection of dictionaries using graphical properties, e.g. alphabet type or font
- G06V30/245—Font recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
-
- G06K9/627—
-
- G06K9/6828—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/1914—Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/22—Cropping
Definitions
- Some systems have been developed for recognizing fonts and identifying similar fonts. However, such systems often use a limited amount and type of information that restricts their ability to recognize or identify similar fonts. In particular, these systems were developed with a small scale of font categories. As a result, these systems are unable to recognize fonts or identify similar fonts for fonts that are not known by the system. Further, because there are subtle variances between fonts within the real-world images and fonts already known to the system, the accuracy of these systems are deficient in both recognizing fonts and identifying similar fonts. As a result of these and other limitations, such systems rely on significant user interaction and subjectivity and are often inadequate in recognizing and sufficiently comparing fonts in real-world images.
- Embodiments of the present invention relate to font recognition and font similarity identification and learning.
- text images with font labels identifying the font in each of the text images are obtained for training a convolutional neural network (CNN).
- the text images are synthesized by introducing slight variances that replicate slight variances in real-world text images.
- the synthesis process results in the generation of training images that are utilized as input to the CNN.
- the output of the CNN results in a distribution of classified text images over the fonts being classified.
- a test image is normalized in height and squeezed in aspect ratio to generate a plurality of test patches which are utilized as input to the CNN.
- the CNN averages the probability of each test patch belonging to a set of fonts to obtain a font classification.
- feature representations may be extracted from a test image by the CNN and utilized to define font similarity between the test image and one or more fonts.
- the font similarity may be utilized in font suggestion, font browsing, or font recognition applications.
- FIG. 1 is a block diagram showing a font recognition and similarity system for recognizing and suggesting fonts in accordance with an embodiment of the present invention
- FIG. 2 is a flow diagram showing an exemplary convolutional neural network that can be trained and used to facilitate recognizing and suggesting fonts in accordance with an embodiment of the present invention
- FIG. 3 is a diagram illustrating an exemplary ranked list of possible fonts provided for a test image utilizing the font recognition and similarity system in accordance with an embodiment of the present invention
- FIG. 4 is a diagram illustrating an exemplary list of similar fonts provided for a test image utilizing the font recognition and similarity system in accordance with an embodiment of the present invention
- FIG. 5 is a flow diagram showing a method for training a convolutional neural network in accordance with an embodiment of the present invention
- FIG. 6 is a flow diagram showing a method for synthesizing a training image in accordance with an embodiment of the present invention
- FIG. 7 is a flow diagram showing a method for determining font similarity in accordance with an embodiment of the present invention.
- FIG. 8 is a block diagram of an exemplary computing environment suitable for use in implementing embodiments of the present invention.
- Embodiments of the present invention are generally directed to a data-driven font recognition and similarity identification system based on a convolutional neural network (CNN).
- CNN convolutional neural network
- a CNN is trained utilizing text images with font labels during a training phase.
- the text images are synthesized (as described herein) images containing text.
- the font labels or classifications are information that identifies a font corresponding with each text image.
- the training phase which is described in more detail below, comprises two primary steps: data synthesis and preprocessing; and 2) deep CNN network structure design and training. After the CNN is designed and trained, is performed in order to identify a font classification for a test image.
- an input test image is preprocessed in the same way as text images are preprocessed during the training phase (as will be described in further detail below), and the preprocessed test images are fed into the CNN for recognition.
- the output of the testing phase provides a font classification for the test image.
- intermediate layer outputs can be extracted from the CNN as feature representations.
- the feature representations are feature vectors that are output by the CNN and distinguish one font classification from another.
- the feature representations are also able to be utilized to define similarity between different fonts.
- the learned feature representations are not constrained to the set of fonts the CNN is trained on. In other words, font similarity can be generalized to recognize fonts the CNN has never seen.
- Font similarity may then be applied in a number of other applications. For example, a designer may want to find similar fonts to a particular font. The particular font may not be available to the designer because of cost or other reasons. In this way, similar fonts may be identified and suggested to the designer as an alternative to the particular font. In another example, font similarity may be utilized to build a similarity graph for a collection of fonts, which can be browsed by the user to identify similar fonts.
- the font similarity may allow the system to recognize fonts the system has never seen.
- the CNN may be utilized to extract a feature representation.
- training images can be synthesized on the fly and their feature representations can also be extracted.
- a nearest neighbor classifier can then be utilized to classify the input test image into one of the new collection of fonts.
- FIG. 1 a block diagram is provided that illustrates a font recognition and similarity system 100 for recognizing and suggesting fonts in accordance in accordance with an embodiment of the present invention.
- this and other arrangements described herein are set forth only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements may be omitted altogether. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software.
- the font recognition and similarity system 100 may be implemented via any type of computing device, such as computing device 800 described below with reference to FIG. 8 , for example. In various embodiments, the font recognition and similarity system 100 may be implemented via a single device or multiple devices cooperating in a distributed environment.
- the font recognition and similarity system 100 generally operates to recognize fonts and/or identify similarity between a font in a test image and a collection of fonts.
- the font recognition and similarity system 100 includes, among other components not shown, the font recognition and similarity system 100 may include a user device 112 , a font data store 114 , and a font recognition engine 116 .
- the font recognition and similarity system 100 shown in FIG. 1 is an example of one suitable computing system architecture.
- Each of the components shown in FIG. 1 may be implemented via any type of computing device, such as computing device 800 described with reference to FIG. 8 , for example.
- the components may communicate with each other via a network 110 , which may include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs). Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. It should be understood that any number of user devices, font data stores, and font recognition engines may be employed within the font recognition and similarity system 100 within the scope of the present invention. Each may comprise a single device or multiple devices cooperating in a distributed environment. For instance, the font recognition engine 116 may be provided via multiple devices arranged in a distributed environment that collectively provide the functionality described herein. In another instance, the font recognition engine 116 and one or more of the other components that comprise the font recognition and similarity system 100 may be provided via a single device. Additionally, other components not shown may also be included within the network environment.
- LANs local area networks
- WANs wide area networks
- the font recognition and similarity system 100 includes a font data store 114 . While only a single font data store 114 is shown in FIG. 1 , it should be understood that the font recognition and similarity system 100 may employ any number of font data stores.
- the font data store 114 may be utilized by the font recognition engine 116 to synthesize and preprocess training images that can be utilized by the CNN.
- the font data store 114 may also store feature representations for a collection of fonts that can be utilized by the font recognition and similarity system 100 to recognize fonts and identify similar fonts.
- the font recognition engine 116 includes, among other things not shown, a synthesis component 118 , a neural network component 120 , and a recognition component 122 .
- the synthesis component 118 prepares training data (i.e., text images with font labels) for the CNN. Collecting real-world text images with font labels can be extremely difficult.
- the synthesis component 118 with access to a collection of fonts (i.e., such as from font data store 114 ), renders text images.
- the text images are black text on white background.
- real-world text images are much more complicated than rendered text images.
- real-world text images are likely to contain variable character spacing, may be stretched in aspect ratio, and may be corrupted by clutter background, noise, compression artifacts, perspective transformation, low-resolution, and the like. Therefore, in order to build an accurate system, the gap between the rendered text images and real-world text images is minimized by the synthesis component 118 before the rendered text images are utilized to train the CNN (such as the one illustrated in FIG. 2 and described below).
- the synthesis component 118 performs a number of actions to process the rendered text images to generate training images.
- the synthesis component 118 may set the character spacing in each rendered text image to randomly deviate from the default spacing. This helps simulate the variable spacing that is likely to exist in real-world text images.
- a small Gaussian noise may be added by the synthesis component 118 to each rendered text image.
- a Gaussian noise N(0,3) is added in a scale of [0,255].
- the Gaussian noise may simulate distortion in real-world text images that arises during acquisition (e.g., sensor noise caused by poor illumination, high temperature, and/or transmission).
- the background of each rendered text image may be filled by the synthesis component 118 with a gradient to simulate a shading effect. This may simulate variable shading that is likely in some degree in real-world text images.
- the synthesis component 118 randomly perturbs each rendered text image by a small affine transformation. In this way, each rendered text image is slightly altered from its normal state.
- affine transformations include, translation, scaling, homothety, similarity transformation, reflection, rotation, and shear mapping.
- the synthesis component 118 adds a random degree of Joint Photographic Experts Group (JPEG) compression.
- JPEG Joint Photographic Experts Group
- the JPEG compression simulates compression artifacts that are likely to exist in real-world text images. For example, when a real-world text image that has been compressed is reproduced, the result is often diminished quality (i.e., an introduction of artifacts) because the compressor may not be able to reproduce enough data to reproduce the original real-world text image in the original quality or the compression algorithm may not be able to discriminate between distortions that are slight versus those that are noticeable to a viewer, or in this case a CNN.
- JPEG Joint Photographic Experts Group
- the synthesis component 118 may normalize each rendered text image in height and squeeze each rendered text image in width.
- the rendered text image is normalized to 105 pixels in height.
- the rendered text image is squeezed in width by a random ratio uniformly in three different ratios. In some embodiments, the ratios are 1.5, 2.5, and 3.5.
- the synthesis component 118 adds a small amount of Gaussian blur to each rendered text image. This simulates the Gaussian blur that is often used by graphics software to reduce image noise and reduce detail.
- the synthesis component 118 may add the Gaussian blur to each rendered text image by convolving each rendered text image with a Gaussian function.
- the training images can be cropped into image patches that will be provided as input to the CNN.
- the training images are cropped into multiple square image patches. For example, each training image may be cropped into five 105 ⁇ 105 pixel square image patches. Since each training image may be squeezed in width by three different ratios, in some embodiments, fifteen image patches are generated as input to the CNN for each training image.
- Neural network component 120 facilitates designing and training of the CNN.
- an exemplary CNN is illustrated that can be trained and used to facilitate recognizing and suggesting fonts.
- the CNN contains eight layers.
- the input 210 of the CNN is fixed, in some embodiments, as 105 ⁇ 105 pixel gray scale image patches.
- the first five layers of the CNN may be convolutional layers 212 , 218 , 224 , 226 , 228 and the remaining three layers may be fully connected layers 230 , 232 , 234 .
- rectified linear unit (ReLU) non-linearity is applied to the output of each convolutional and fully connected layer.
- ReLU rectified linear unit
- Normalization 214 , 220 and max-pooling 216 , 222 layers may be added, in some embodiments, following the first two convolutional layers 212 , 218 .
- the output of the last fully connected layer 234 is fed, in some embodiments, into an N-way softmax 236 , which produces a distribution over N class labels.
- N represents the number of fonts the CNN is trained on.
- the image patches are divided, in some embodiments, into small mini-batches. Stochastic gradient descent may be used for optimization, following the convention. Additionally or alternatively, the learning rate may be reduced by a factor of two after every epoch.
- the size of the outputs of the first convolutional layer 212 and the first normalization layer 214 is 48 ⁇ 48 ⁇ 64 neurons.
- the first max pooling layer 216 may have a 24 ⁇ 24 ⁇ 64 neuron output.
- the second convolutional layer 218 and the second normalization layer 220 may have 24 ⁇ 24 ⁇ 128 neuron outputs.
- the size of the outputs of the second max pooling layer 222 may be 12 ⁇ 12 ⁇ 128 neurons.
- Each of the third, fourth, and fifth convolutional layers 224 , 226 , and 228 may have 12 ⁇ 12 ⁇ 256 neuron outputs.
- the first and second fully connected layers 230 , 232 may have 4096 dimensional vector outputs while the last fully connected layer may have a 2383 dimensional vector output.
- the output of the last fully connected layer may be fed, in this example, into a 2383-way softmax which produces a distribution of labeled image patches over 2383 classification labels.
- recognition component 122 facilitates testing of the CNN.
- a multi-scale multi-view (MSMV) testing strategy may be utilized to improve the robustness of the system.
- each test image may be normalized to 105 pixels in height and then squeezed in width by three different ratios: 1.5, 2.5, and 3.5.
- For each of the three respective squeezed ratios five 105 ⁇ 105 pixel patches are cropped, in some embodiments, from the image resulting in a total of fifteen test patches which correspond to different aspect ratios and views from the input test image.
- the CNN outputs, in some embodiments, a softmax vector for each cropped image patch indicating a probability of the test image belonging to a set of fonts.
- Each of the fifteen softmax vectors may be averaged to obtain a final classifier for the test image.
- the recognition component 122 may facilitate defining font similarities, which may be accomplished even without a predefined fixed set of fonts. To do so, for the i-th font, a single training image I i is synthesized utilizing any synthesis process described herein. Image patches may then be cropped from the training image and fed into the CNN. In some embodiments, n 105 ⁇ 105 pixel image patches are cropped from the training image. The second to last fully connected layer output for each image patch may then be obtained. In some embodiments, the output is denoted as f 7 , which is a 4096-dimensional vector.
- a max operation is performed by the recognition component 122 along each dimension of f 7 across each of the image patches.
- f 7 j is the second to last fully connected layer output for the j-th image patch.
- F i is also a 4096-dimensional vector.
- an image I j for font j may be synthesized by the synthesis component 118 , as described herein, utilizing the same text that was used for training image I i .
- the feature representation F j may then be extracted by the recognition component 122 from I i .
- multiple training images may be synthesized by the synthesis component 118 using different text sequences for each font.
- the font similarity between two fonts may be defined by the recognition component 122 as
- s k (i,j) is the font similarity based on the k-th synthesized training image.
- the recognition component 122 provides font suggestion or recommendation. For example, a designer may want to use a particular font for a design but the font is either too expensive or unavailable to the designer. The designer may utilize the font recognition similarity system 100 to find similar alternative fonts.
- the recognition component 122 provides font organization and browsing functionality. In this regard, the recognition component 122 may build a similarity graph for a collection of fonts. The similarity graph may allow the designer to browse the collection of fonts.
- the recognition component 122 facilitates the recognition of a font never seen before by the font recognition and similarity system 100 .
- the font may not be stored by the font data store 114 .
- a feature representation may be extracted for a test image that includes the font, as described above.
- training images may be synthesized (e.g., using common words) by the synthesis component 118 and the feature representations may be extracted by the recognition component 122 .
- the synthesis process may be performed while the feature representation is being extracted from the test image.
- a nearest neighbor classifier based on the feature representation, may then be utilized to classify the input test image into a font of the new collection of fonts.
- hashing or other techniques may be utilized to speed up the nearest neighbor classifier. For example, if the feature representation is a vector of dimension 4096, vector quantization can convert the 4096 dimensional vector into a binary representation that approximates the feature representation, but is smaller to store and faster to compute. By determining the nearest neighbor classifier, the input test image is classified as the font of the new collection of fonts that is the most similar to the input test image.
- FIG. 3 An exemplary ranked list of possible fonts that may be provided for a test image utilizing the font recognition and similarity system 100 of FIG. 1 is illustrated in FIG. 3 .
- an input image 310 comprising a font is selected from an image location 312 .
- a designer may have scanned in a particular text image to identify the font used in the text image.
- the designer may have saved the particular text image from another source (e.g., online source) and would like to identify the font used by the source.
- Controls 320 may allow the designer to manipulate (e.g., rotate, stretch, and the like) the font image in accordance with the designer preferences.
- a list 330 of the most similar fonts is populated and ranked. As illustrated, the list 330 includes font names along with sample text that matches the text of the input image. The designer may use the list 330 for font suggestion and browsing.
- FIG. 4 illustrates an exemplary list of similar fonts that may be provided for a test image utilizing the font recognition and similarity system 100 of FIG. 1 .
- FIG. 4 illustrates a query font 410 .
- similar fonts 420 are provided below the query font 410 .
- Each of the fonts 422 , 424 that appear at the bottom of the list have been determined by the font recognition and similarity system 100 as the most similar fonts to the query font 410 .
- FIG. 5 a flow diagram is provided that illustrates a method 500 for training a CNN in accordance with an embodiment of the present invention.
- the method 500 may be employed utilizing the font recognition and similarity system 100 of FIG. 1 .
- one or more text images are received from a collection of known text images.
- each of the one or more text images includes a corresponding font label or classifier that identifies the font in each of the one or more text images.
- the one or more text images are synthesized, at step 512 , to introduce slight variances that approximate variances that may exist in real-world text images.
- the slight variances may be introduced in any number or combination of ways, as described herein.
- the slight variances are intended to mimic the variations that are likely to occur in real-world text images which, in turn, improve the accuracy of the CNN in the testing phase.
- One or more training images are generated, at step 514 , that include the slight variances, which are then cropped into training patches that are utilized as an input to the CNN.
- the training images may be cropped into multiple square gray scale training patches.
- the training patches are 105 ⁇ 105 square image patches.
- FIG. 6 a flow diagram is provided that illustrates a method 600 for synthesizing a training image in accordance with an embodiment of the present invention.
- the method 600 may be employed utilizing the font recognition and similarity system 100 of FIG. 1 .
- each step illustrated in FIG. 6 may be employed individually or in any combination comprising any number of the steps by the font recognition and similarity system 100 of FIG. 1 during the synthesis process.
- the images may be synthesized by setting character spacing in each image to randomly deviate from a default spacing.
- a Gaussian noise may be added, at step 612 , to each image.
- the images may be synthesized by filling each image background of the one or more text images with a gradient to simulate a shading effect.
- Each image may be randomly perturbed, at step 616 , by an affine transformation.
- the images may be synthesized by adding a random degree of Joint Photographic Experts Group (JPEG) compression to each image of the one or more text images.
- JPEG Joint Photographic Experts Group
- Each image may be normalized to a fixed height and squeezed in width by a plurality of different aspect ratios, at step 620 .
- the images may be synthesized, at step 622 , by adding a small amount of random Gaussian blur to each image of the one or more text images.
- the CNN is trained with the training patches.
- the CNN comprises five convolutional layers and three layers fully connected layers.
- the last fully connected layer may be fed into an N-way softmax function.
- N may be based on the number of fonts to model.
- a collection of fonts being labeled comprises, for example, 2383 fonts
- the last fully connected layer is fed into a 2383-way softmax function.
- a distribution of classified text images is produced, at step 518 , over N classification labels. Continuing the example of a collection of 2383 fonts, a distribution of classified text images is produced over 2383 classification labels.
- normalization and max pooling layers are added to the CNN following the first two convolutional layers.
- Rectified linear unit (ReLU) non-linearity an activation function
- any activation function may similarly be applied.
- stochastic gradient descent is utilized for optimization. Stochastic gradient descent works well for large-scale problems, but other training algorithms may be utilized as well.
- the learning rate may be reduced by a factor after every epoch. In some embodiments, the learning rate is reduced by a factor of two after every epoch.
- the CNN is utilized to classify an unknown font image.
- the unknown font image may be normalized in height and squeezed in width in accordance with different aspect ratios. This results in a plurality of test patches corresponding to the different aspect ratios and views from the unknown font image.
- the CNN may output a softmax vector indicating the probabilities of each test patch belonging to a set of fonts. The plurality of softmax vectors may be averaged to obtain a final classification result for the unknown image.
- a flow diagram is provided that illustrates a method 700 for determining font similarity in accordance with an embodiment of the present invention.
- the method 700 may be employed utilizing the font recognition and similarity system 100 of FIG. 1 .
- a test image comprising text is synthesized, by a computing device, to introduce slight variances.
- the text comprises more than one sequence of text resulting in a plurality test images.
- the slight variances include one or more of character spacing, noise, shading, affine transformation, compression artifacts, aspect ratio, or Gaussian blurring.
- a plurality of test image patches are cropped, by the computing device, from the test image.
- Each test image patch is input, by the computing device, to a convolutional neural network (CNN), at step 714 , that has been trained with a plurality of training image patches.
- a test output is obtained by the computing device, at step 716 , from the second to last fully connected layer for each test image patch.
- CNN convolutional neural network
- a single test image I i is synthesized utilizing any synthesis process described herein.
- Image patches may then be cropped from the test image and fed into the CNN.
- n 105 ⁇ 105 pixel image patches are cropped from the test image.
- the second to last fully connected layer output for each image patch may then be obtained.
- the output is denoted as f 7 , which is a 4096-dimensional vector.
- a max operation is performed by the computing device, at step 718 , along each dimension of the test output.
- the result of the max operation is a feature representation for the test image.
- a max operation is performed along each dimension of f 7 across each of the image patches.
- f 7 j is the second to last fully connected layer output for the j-th image patch.
- F i is also a 4096-dimensional vector.
- an image I j for font j may be synthesized, as described herein, utilizing the same text that was used for training image I i .
- the feature representation F j may then be extracted from I i .
- similar fonts are suggested based on the feature representation of the test image.
- feature representations of known fonts are compared to the feature representation of the test image to classify an unknown font of the test image.
- the feature representation of the test image may be compared, by any method described herein or other various methods (e.g., utilizing a font similarity matrix), to the feature representation extracted from other fonts in a collection of fonts.
- a font similarity graph is built for a collection of fonts based on the font similarity.
- FIG. 8 an exemplary operating environment for implementing embodiments of the present invention is shown and designated generally as computing device 800 .
- Computing device 800 is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing device 800 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.
- the invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device.
- program modules including routines, programs, objects, components, data structures, etc., refer to code that perform particular tasks or implement particular abstract data types.
- the invention may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc.
- the invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
- computing device 800 includes a bus 810 that directly or indirectly couples the following devices: memory 812 , one or more processors 814 , one or more presentation components 816 , input/output (I/O) ports 818 , input/output components 1520 , and an illustrative power supply 822 .
- Bus 810 represents what may be one or more busses (such as an address bus, data bus, or combination thereof).
- I/O input/output
- FIG. 8 represents what may be one or more busses (such as an address bus, data bus, or combination thereof).
- FIG. 8 is merely illustrative of an exemplary computing device that can be used in connection with one or more embodiments of the present invention. Distinction is not made between such categories as “workstation,” “server,” “laptop,” “hand-held device,” etc., as all are contemplated within the scope of FIG. 8 and reference to “computing device.”
- Computer-readable media can be any available media that can be accessed by computing device 800 and includes both volatile and nonvolatile media, removable and non-removable media.
- Computer-readable media may comprise computer storage media and communication media.
- Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data.
- Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 800 .
- Computer storage media does not comprise signals per se.
- Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
- modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
- communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.
- Memory 812 includes computer-storage media in the form of volatile and/or nonvolatile memory.
- the memory may be removable, non-removable, or a combination thereof.
- Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc.
- Computing device 800 includes one or more processors that read data from various entities such as memory 812 or I/O components 820 .
- Presentation component(s) 816 present data indications to a user or other device.
- Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.
- I/O ports 818 allow computing device 800 to be logically coupled to other devices including I/O components 820 , some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.
- the I/O components 820 may provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some instance, inputs may be transmitted to an appropriate network element for further processing.
- NUI may implement any combination of speech recognition, touch and stylus recognition, facial recognition, biometric recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with displays on the computing device 800 .
- the computing device 800 may be equipped with depth cameras, such as, stereoscopic camera systems, infrared camera systems, RGB camera systems, and combinations of these for gesture detection and recognition. Additionally, the computing device 800 may be equipped with accelerometers or gyroscopes that enable detection of motion. The output of the accelerometers or gyroscopes may be provided to the display of the computing device 800 to render immersive augmented reality or virtual reality.
- embodiments of the present invention provide for an objective approach for determining the visual similarity between fonts.
- the present invention has been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from its scope.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
Description
- With the advent of computers and digital typography, the number of different fonts has continued to grow. As a result, users often have wide flexibility in choosing fonts in various applications. Given the large number of available fonts, the task of recognizing (i.e., classifying) fonts or identifying similar fonts has become more challenging. In particular, there are a number of scenarios in which it may be desirable to recognize the font or identify the similarity between two fonts among a collection of fonts. For instance, given an image containing text, a user may wish to simply identify the font in the image. In another example, a user may wish to find a font that is similar to the font in the image because use of the font in the image is costly or not available in a particular application.
- Some systems have been developed for recognizing fonts and identifying similar fonts. However, such systems often use a limited amount and type of information that restricts their ability to recognize or identify similar fonts. In particular, these systems were developed with a small scale of font categories. As a result, these systems are unable to recognize fonts or identify similar fonts for fonts that are not known by the system. Further, because there are subtle variances between fonts within the real-world images and fonts already known to the system, the accuracy of these systems are deficient in both recognizing fonts and identifying similar fonts. As a result of these and other limitations, such systems rely on significant user interaction and subjectivity and are often inadequate in recognizing and sufficiently comparing fonts in real-world images.
- This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor should it be used as an aid in determining the scope of the claimed subject matter.
- Embodiments of the present invention relate to font recognition and font similarity identification and learning. Initially, in a training phase, text images with font labels identifying the font in each of the text images are obtained for training a convolutional neural network (CNN). The text images are synthesized by introducing slight variances that replicate slight variances in real-world text images. The synthesis process results in the generation of training images that are utilized as input to the CNN. The output of the CNN results in a distribution of classified text images over the fonts being classified. In a testing phase, a test image is normalized in height and squeezed in aspect ratio to generate a plurality of test patches which are utilized as input to the CNN. The CNN averages the probability of each test patch belonging to a set of fonts to obtain a font classification. In embodiments, feature representations may be extracted from a test image by the CNN and utilized to define font similarity between the test image and one or more fonts. The font similarity may be utilized in font suggestion, font browsing, or font recognition applications. By synthesizing test images, the gap between the rendered test images and the real-world text images is minimized and the accuracy for all applications is greatly increased.
- The present invention is described in detail below with reference to the attached drawing figures, wherein:
-
FIG. 1 is a block diagram showing a font recognition and similarity system for recognizing and suggesting fonts in accordance with an embodiment of the present invention; -
FIG. 2 is a flow diagram showing an exemplary convolutional neural network that can be trained and used to facilitate recognizing and suggesting fonts in accordance with an embodiment of the present invention; -
FIG. 3 is a diagram illustrating an exemplary ranked list of possible fonts provided for a test image utilizing the font recognition and similarity system in accordance with an embodiment of the present invention; -
FIG. 4 is a diagram illustrating an exemplary list of similar fonts provided for a test image utilizing the font recognition and similarity system in accordance with an embodiment of the present invention; -
FIG. 5 is a flow diagram showing a method for training a convolutional neural network in accordance with an embodiment of the present invention; -
FIG. 6 is a flow diagram showing a method for synthesizing a training image in accordance with an embodiment of the present invention; -
FIG. 7 is a flow diagram showing a method for determining font similarity in accordance with an embodiment of the present invention; and -
FIG. 8 is a block diagram of an exemplary computing environment suitable for use in implementing embodiments of the present invention. - The subject matter of the present invention is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
- As noted in the Background, a number of systems have been developed for classifying fonts. However, these systems typically fall short in their usefulness because they are limited to the small scale of font categories they are developed with and are unable to handle the large number of font classes that exist in real-world images. These limitations are due, in large part, to the reliance on significant user labeling and interaction which prevents these systems from being able to handle recognition and similarity identification for a large collection of fonts, or fonts previously unknown to the systems. Additionally, because these systems rely so heavily on user labeling and interaction, application is often subjective, which further limits their ability to recognize or accurately identify similar fonts.
- Embodiments of the present invention are generally directed to a data-driven font recognition and similarity identification system based on a convolutional neural network (CNN). Experimental tests have shown that the system achieves nearly 80% top-five accuracy on a collection of real-world text images. The system has also proven to be effective for font organization, font selection, font browsing, and font suggestion.
- Initially, a CNN is trained utilizing text images with font labels during a training phase. The text images are synthesized (as described herein) images containing text. The font labels or classifications are information that identifies a font corresponding with each text image. The training phase, which is described in more detail below, comprises two primary steps: data synthesis and preprocessing; and 2) deep CNN network structure design and training. After the CNN is designed and trained, is performed in order to identify a font classification for a test image. In the testing phase, an input test image is preprocessed in the same way as text images are preprocessed during the training phase (as will be described in further detail below), and the preprocessed test images are fed into the CNN for recognition. The output of the testing phase provides a font classification for the test image.
- For producing font similarity, intermediate layer outputs can be extracted from the CNN as feature representations. The feature representations are feature vectors that are output by the CNN and distinguish one font classification from another. On the other hand, the feature representations are also able to be utilized to define similarity between different fonts. The learned feature representations are not constrained to the set of fonts the CNN is trained on. In other words, font similarity can be generalized to recognize fonts the CNN has never seen.
- Font similarity may then be applied in a number of other applications. For example, a designer may want to find similar fonts to a particular font. The particular font may not be available to the designer because of cost or other reasons. In this way, similar fonts may be identified and suggested to the designer as an alternative to the particular font. In another example, font similarity may be utilized to build a similarity graph for a collection of fonts, which can be browsed by the user to identify similar fonts.
- As mentioned above, the font similarity may allow the system to recognize fonts the system has never seen. For example, given a test image, the CNN may be utilized to extract a feature representation. For a new collection of fonts the system has never seen, training images can be synthesized on the fly and their feature representations can also be extracted. A nearest neighbor classifier can then be utilized to classify the input test image into one of the new collection of fonts.
- Referring now to
FIG. 1 , a block diagram is provided that illustrates a font recognition andsimilarity system 100 for recognizing and suggesting fonts in accordance in accordance with an embodiment of the present invention. It should be understood that this and other arrangements described herein are set forth only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements may be omitted altogether. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory. The font recognition andsimilarity system 100 may be implemented via any type of computing device, such ascomputing device 800 described below with reference toFIG. 8 , for example. In various embodiments, the font recognition andsimilarity system 100 may be implemented via a single device or multiple devices cooperating in a distributed environment. - The font recognition and
similarity system 100 generally operates to recognize fonts and/or identify similarity between a font in a test image and a collection of fonts. As shown inFIG. 1 , the font recognition andsimilarity system 100 includes, among other components not shown, the font recognition andsimilarity system 100 may include auser device 112, afont data store 114, and afont recognition engine 116. It should be understood that the font recognition andsimilarity system 100 shown inFIG. 1 is an example of one suitable computing system architecture. Each of the components shown inFIG. 1 may be implemented via any type of computing device, such ascomputing device 800 described with reference toFIG. 8 , for example. The components may communicate with each other via anetwork 110, which may include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs). Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. It should be understood that any number of user devices, font data stores, and font recognition engines may be employed within the font recognition andsimilarity system 100 within the scope of the present invention. Each may comprise a single device or multiple devices cooperating in a distributed environment. For instance, thefont recognition engine 116 may be provided via multiple devices arranged in a distributed environment that collectively provide the functionality described herein. In another instance, thefont recognition engine 116 and one or more of the other components that comprise the font recognition andsimilarity system 100 may be provided via a single device. Additionally, other components not shown may also be included within the network environment. - As shown in
FIG. 1 , the font recognition andsimilarity system 100 includes afont data store 114. While only a singlefont data store 114 is shown inFIG. 1 , it should be understood that the font recognition andsimilarity system 100 may employ any number of font data stores. Thefont data store 114 may be utilized by thefont recognition engine 116 to synthesize and preprocess training images that can be utilized by the CNN. Thefont data store 114 may also store feature representations for a collection of fonts that can be utilized by the font recognition andsimilarity system 100 to recognize fonts and identify similar fonts. - As shown in
FIG. 1 , thefont recognition engine 116 includes, among other things not shown, asynthesis component 118, aneural network component 120, and arecognition component 122. Initially, thesynthesis component 118 prepares training data (i.e., text images with font labels) for the CNN. Collecting real-world text images with font labels can be extremely difficult. Thesynthesis component 118, with access to a collection of fonts (i.e., such as from font data store 114), renders text images. In some embodiments, the text images are black text on white background. - In real-world applications, real-world text images are much more complicated than rendered text images. For example, real-world text images are likely to contain variable character spacing, may be stretched in aspect ratio, and may be corrupted by clutter background, noise, compression artifacts, perspective transformation, low-resolution, and the like. Therefore, in order to build an accurate system, the gap between the rendered text images and real-world text images is minimized by the
synthesis component 118 before the rendered text images are utilized to train the CNN (such as the one illustrated inFIG. 2 and described below). - In order to minimize the gap, the
synthesis component 118 performs a number of actions to process the rendered text images to generate training images. Thesynthesis component 118 may set the character spacing in each rendered text image to randomly deviate from the default spacing. This helps simulate the variable spacing that is likely to exist in real-world text images. - A small Gaussian noise may be added by the
synthesis component 118 to each rendered text image. In some embodiments a Gaussian noise N(0,3) is added in a scale of [0,255]. The Gaussian noise may simulate distortion in real-world text images that arises during acquisition (e.g., sensor noise caused by poor illumination, high temperature, and/or transmission). - Additionally or alternatively, the background of each rendered text image may be filled by the
synthesis component 118 with a gradient to simulate a shading effect. This may simulate variable shading that is likely in some degree in real-world text images. - In some embodiments, the
synthesis component 118 randomly perturbs each rendered text image by a small affine transformation. In this way, each rendered text image is slightly altered from its normal state. Examples of affine transformations include, translation, scaling, homothety, similarity transformation, reflection, rotation, and shear mapping. - In some embodiments, the
synthesis component 118 adds a random degree of Joint Photographic Experts Group (JPEG) compression. The JPEG compression simulates compression artifacts that are likely to exist in real-world text images. For example, when a real-world text image that has been compressed is reproduced, the result is often diminished quality (i.e., an introduction of artifacts) because the compressor may not be able to reproduce enough data to reproduce the original real-world text image in the original quality or the compression algorithm may not be able to discriminate between distortions that are slight versus those that are noticeable to a viewer, or in this case a CNN. - The
synthesis component 118 may normalize each rendered text image in height and squeeze each rendered text image in width. In some embodiments, the rendered text image is normalized to 105 pixels in height. In some embodiments, the rendered text image is squeezed in width by a random ratio uniformly in three different ratios. In some embodiments, the ratios are 1.5, 2.5, and 3.5. - In some embodiments, the
synthesis component 118 adds a small amount of Gaussian blur to each rendered text image. This simulates the Gaussian blur that is often used by graphics software to reduce image noise and reduce detail. Thesynthesis component 118 may add the Gaussian blur to each rendered text image by convolving each rendered text image with a Gaussian function. - After the
synthesis component 118 synthesizes each of the rendered text images into training images that simulate real-world text images, the training images can be cropped into image patches that will be provided as input to the CNN. In some embodiments, the training images are cropped into multiple square image patches. For example, each training image may be cropped into five 105×105 pixel square image patches. Since each training image may be squeezed in width by three different ratios, in some embodiments, fifteen image patches are generated as input to the CNN for each training image. -
Neural network component 120 facilitates designing and training of the CNN. In some embodiments, and referring now toFIG. 2 , an exemplary CNN is illustrated that can be trained and used to facilitate recognizing and suggesting fonts. As shown inFIG. 2 , the CNN contains eight layers. Theinput 210 of the CNN is fixed, in some embodiments, as 105×105 pixel gray scale image patches. The first five layers of the CNN may beconvolutional layers layers -
Normalization convolutional layers layer 234 is fed, in some embodiments, into an N-way softmax 236, which produces a distribution over N class labels. As shown inFIG. 2 , the output of the last fully connectedlayer 234 is fed into a 2383-way softmax 236 because the exemplary CNN is being trained on 2383 fonts. As can be appreciated, N represents the number of fonts the CNN is trained on. During training, the image patches are divided, in some embodiments, into small mini-batches. Stochastic gradient descent may be used for optimization, following the convention. Additionally or alternatively, the learning rate may be reduced by a factor of two after every epoch. - In some embodiments the size of the outputs of the first
convolutional layer 212 and thefirst normalization layer 214 is 48×48×64 neurons. The firstmax pooling layer 216 may have a 24×24×64 neuron output. The secondconvolutional layer 218 and thesecond normalization layer 220 may have 24×24×128 neuron outputs. The size of the outputs of the secondmax pooling layer 222 may be 12×12×128 neurons. Each of the third, fourth, and fifthconvolutional layers layers - Referring back to
FIG. 1 ,recognition component 122 facilitates testing of the CNN. A multi-scale multi-view (MSMV) testing strategy may be utilized to improve the robustness of the system. In this way, each test image may be normalized to 105 pixels in height and then squeezed in width by three different ratios: 1.5, 2.5, and 3.5. For each of the three respective squeezed ratios, five 105×105 pixel patches are cropped, in some embodiments, from the image resulting in a total of fifteen test patches which correspond to different aspect ratios and views from the input test image. The CNN outputs, in some embodiments, a softmax vector for each cropped image patch indicating a probability of the test image belonging to a set of fonts. Each of the fifteen softmax vectors may be averaged to obtain a final classifier for the test image. - Additionally or alternatively, the
recognition component 122 may facilitate defining font similarities, which may be accomplished even without a predefined fixed set of fonts. To do so, for the i-th font, a single training image Ii is synthesized utilizing any synthesis process described herein. Image patches may then be cropped from the training image and fed into the CNN. In some embodiments, n 105×105 pixel image patches are cropped from the training image. The second to last fully connected layer output for each image patch may then be obtained. In some embodiments, the output is denoted as f7, which is a 4096-dimensional vector. - In some embodiments, a max operation is performed by the
recognition component 122 along each dimension of f7 across each of the image patches. Thus the feature representation for training image Ii can be defined as Fi=max{f7 j}. As mentioned above, f7 j is the second to last fully connected layer output for the j-th image patch. As a result, Fi is also a 4096-dimensional vector. To define the font similarity between font i and font j, an image Ij for font j may be synthesized by thesynthesis component 118, as described herein, utilizing the same text that was used for training image Ii. The feature representation Fj may then be extracted by therecognition component 122 from Ii. The font similarity between i and j may be defined by therecognition component 122, in one embodiment, as s(i,j)=Fi TFj. In another embodiment, a radial basis function (RBF) kernel, s(i,j)=exp(−∥Fi−Fj∥2 2/σ2) may be utilized by therecognition component 122 to define font similarity. - To improve the font similarity process, in some embodiments, multiple training images may be synthesized by the
synthesis component 118 using different text sequences for each font. Where multiple training images are synthesized, the font similarity between two fonts may be defined by therecognition component 122 as -
- where sk (i,j) is the font similarity based on the k-th synthesized training image.
- In some embodiments, the
recognition component 122 provides font suggestion or recommendation. For example, a designer may want to use a particular font for a design but the font is either too expensive or unavailable to the designer. The designer may utilize the fontrecognition similarity system 100 to find similar alternative fonts. In some embodiments, therecognition component 122 provides font organization and browsing functionality. In this regard, therecognition component 122 may build a similarity graph for a collection of fonts. The similarity graph may allow the designer to browse the collection of fonts. - In some embodiments, the
recognition component 122 facilitates the recognition of a font never seen before by the font recognition andsimilarity system 100. In other words, the font may not be stored by thefont data store 114. A feature representation may be extracted for a test image that includes the font, as described above. For a new collection of fonts that the font recognition andsimilarity system 100 has never seen (e.g., a unique collection of fonts on a designer's desktop system), training images may be synthesized (e.g., using common words) by thesynthesis component 118 and the feature representations may be extracted by therecognition component 122. - In some embodiments, the synthesis process may be performed while the feature representation is being extracted from the test image. A nearest neighbor classifier, based on the feature representation, may then be utilized to classify the input test image into a font of the new collection of fonts. In some embodiments, hashing or other techniques may be utilized to speed up the nearest neighbor classifier. For example, if the feature representation is a vector of dimension 4096, vector quantization can convert the 4096 dimensional vector into a binary representation that approximates the feature representation, but is smaller to store and faster to compute. By determining the nearest neighbor classifier, the input test image is classified as the font of the new collection of fonts that is the most similar to the input test image.
- An exemplary ranked list of possible fonts that may be provided for a test image utilizing the font recognition and
similarity system 100 ofFIG. 1 is illustrated inFIG. 3 . As shown inFIG. 3 , aninput image 310 comprising a font is selected from an image location 312. For example, a designer may have scanned in a particular text image to identify the font used in the text image. Or, in another example, the designer may have saved the particular text image from another source (e.g., online source) and would like to identify the font used by the source.Controls 320 may allow the designer to manipulate (e.g., rotate, stretch, and the like) the font image in accordance with the designer preferences. Once the designer is satisfied and accepts any manipulations, alist 330 of the most similar fonts is populated and ranked. As illustrated, thelist 330 includes font names along with sample text that matches the text of the input image. The designer may use thelist 330 for font suggestion and browsing. - By way of example to illustrate,
FIG. 4 illustrates an exemplary list of similar fonts that may be provided for a test image utilizing the font recognition andsimilarity system 100 ofFIG. 1 . In particular,FIG. 4 illustrates aquery font 410. As shown,similar fonts 420 are provided below thequery font 410. Each of thefonts similarity system 100 as the most similar fonts to thequery font 410. - Turning now to
FIG. 5 , a flow diagram is provided that illustrates amethod 500 for training a CNN in accordance with an embodiment of the present invention. For instance, themethod 500 may be employed utilizing the font recognition andsimilarity system 100 ofFIG. 1 . As shown atstep 510, one or more text images are received from a collection of known text images. Accordingly, each of the one or more text images includes a corresponding font label or classifier that identifies the font in each of the one or more text images. - The one or more text images are synthesized, at
step 512, to introduce slight variances that approximate variances that may exist in real-world text images. The slight variances may be introduced in any number or combination of ways, as described herein. In this regard, the slight variances are intended to mimic the variations that are likely to occur in real-world text images which, in turn, improve the accuracy of the CNN in the testing phase. - One or more training images are generated, at
step 514, that include the slight variances, which are then cropped into training patches that are utilized as an input to the CNN. For example, the training images may be cropped into multiple square gray scale training patches. In some embodiments, the training patches are 105×105 square image patches. - Referring now to
FIG. 6 , a flow diagram is provided that illustrates amethod 600 for synthesizing a training image in accordance with an embodiment of the present invention. For instance, themethod 600 may be employed utilizing the font recognition andsimilarity system 100 ofFIG. 1 . For clarity, each step illustrated inFIG. 6 may be employed individually or in any combination comprising any number of the steps by the font recognition andsimilarity system 100 ofFIG. 1 during the synthesis process. As shown atstep 610, the images may be synthesized by setting character spacing in each image to randomly deviate from a default spacing. A Gaussian noise may be added, atstep 612, to each image. Atstep 614, the images may be synthesized by filling each image background of the one or more text images with a gradient to simulate a shading effect. Each image may be randomly perturbed, atstep 616, by an affine transformation. Atstep 618, the images may be synthesized by adding a random degree of Joint Photographic Experts Group (JPEG) compression to each image of the one or more text images. Each image may be normalized to a fixed height and squeezed in width by a plurality of different aspect ratios, atstep 620. The images may be synthesized, atstep 622, by adding a small amount of random Gaussian blur to each image of the one or more text images. - Referring back to
FIG. 5 , atstep 516, the CNN is trained with the training patches. In some embodiments, the CNN comprises five convolutional layers and three layers fully connected layers. The last fully connected layer may be fed into an N-way softmax function. N may be based on the number of fonts to model. In other words, if a collection of fonts being labeled comprises, for example, 2383 fonts, the last fully connected layer is fed into a 2383-way softmax function. A distribution of classified text images is produced, atstep 518, over N classification labels. Continuing the example of a collection of 2383 fonts, a distribution of classified text images is produced over 2383 classification labels. - In some embodiments, normalization and max pooling layers are added to the CNN following the first two convolutional layers. Rectified linear unit (ReLU) non-linearity, an activation function, may be applied to an output of each convolutional and fully connected layer. As can be appreciated, any activation function may similarly be applied. In some embodiments, stochastic gradient descent is utilized for optimization. Stochastic gradient descent works well for large-scale problems, but other training algorithms may be utilized as well. The learning rate may be reduced by a factor after every epoch. In some embodiments, the learning rate is reduced by a factor of two after every epoch.
- In some embodiments, the CNN is utilized to classify an unknown font image. The unknown font image may be normalized in height and squeezed in width in accordance with different aspect ratios. This results in a plurality of test patches corresponding to the different aspect ratios and views from the unknown font image. For each test patch, the CNN may output a softmax vector indicating the probabilities of each test patch belonging to a set of fonts. The plurality of softmax vectors may be averaged to obtain a final classification result for the unknown image.
- In
FIG. 7 , a flow diagram is provided that illustrates amethod 700 for determining font similarity in accordance with an embodiment of the present invention. For instance, themethod 700 may be employed utilizing the font recognition andsimilarity system 100 ofFIG. 1 . As shown atstep 710, a test image comprising text is synthesized, by a computing device, to introduce slight variances. In some embodiments, the text comprises more than one sequence of text resulting in a plurality test images. In various embodiments, the slight variances include one or more of character spacing, noise, shading, affine transformation, compression artifacts, aspect ratio, or Gaussian blurring. - At
step 712, a plurality of test image patches are cropped, by the computing device, from the test image. Each test image patch is input, by the computing device, to a convolutional neural network (CNN), atstep 714, that has been trained with a plurality of training image patches. A test output is obtained by the computing device, atstep 716, from the second to last fully connected layer for each test image patch. - For example, for the i-th font, a single test image Ii is synthesized utilizing any synthesis process described herein. Image patches may then be cropped from the test image and fed into the CNN. In some embodiments, n 105×105 pixel image patches are cropped from the test image. The second to last fully connected layer output for each image patch may then be obtained. In some embodiments, the output is denoted as f7, which is a 4096-dimensional vector.
- A max operation is performed by the computing device, at
step 718, along each dimension of the test output. The result of the max operation is a feature representation for the test image. For example, a max operation is performed along each dimension of f7 across each of the image patches. The feature representation for test image Ii can be defined as Fi=max{f7 j}. As mentioned above, f7 j is the second to last fully connected layer output for the j-th image patch. As a result, Fi is also a 4096-dimensional vector. - To define the font similarity between font i and font j, an image Ij for font j may be synthesized, as described herein, utilizing the same text that was used for training image Ii. The feature representation Fj may then be extracted from Ii. The font similarity between i and j may be defined, in one embodiment, as s(i,j)=Fi TFj. In another embodiment, a radial basis function (RBF) kernel, s(i,j)=exp(−∥Fi−Fj∥2 2/σ2) may be utilized to define font similarity. Accordingly, a font similarity is determined by the computing device, at
step 720, for the test image. - In some embodiments, similar fonts are suggested based on the feature representation of the test image. In some embodiments, feature representations of known fonts are compared to the feature representation of the test image to classify an unknown font of the test image. For example, the feature representation of the test image may be compared, by any method described herein or other various methods (e.g., utilizing a font similarity matrix), to the feature representation extracted from other fonts in a collection of fonts. In some embodiments, a font similarity graph is built for a collection of fonts based on the font similarity.
- Having described embodiments of the present invention, an exemplary operating environment in which embodiments of the present invention may be implemented is described below in order to provide a general context for various aspects of the present invention. Referring to
FIG. 8 in particular, an exemplary operating environment for implementing embodiments of the present invention is shown and designated generally ascomputing device 800.Computing device 800 is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should thecomputing device 800 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated. - The invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, etc., refer to code that perform particular tasks or implement particular abstract data types. The invention may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. The invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
- With reference to
FIG. 8 ,computing device 800 includes abus 810 that directly or indirectly couples the following devices:memory 812, one ormore processors 814, one ormore presentation components 816, input/output (I/O)ports 818, input/output components 1520, and anillustrative power supply 822.Bus 810 represents what may be one or more busses (such as an address bus, data bus, or combination thereof). Although the various blocks ofFIG. 8 are shown with lines for the sake of clarity, in reality, delineating various components is not so clear, and metaphorically, the lines would more accurately be grey and fuzzy. For example, one may consider a presentation component such as a display device to be an I/O component. Also, processors have memory. The inventors recognize that such is the nature of the art, and reiterate that the diagram ofFIG. 8 is merely illustrative of an exemplary computing device that can be used in connection with one or more embodiments of the present invention. Distinction is not made between such categories as “workstation,” “server,” “laptop,” “hand-held device,” etc., as all are contemplated within the scope ofFIG. 8 and reference to “computing device.” -
Computing device 800 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computingdevice 800 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computingdevice 800. Computer storage media does not comprise signals per se. Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media. -
Memory 812 includes computer-storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc.Computing device 800 includes one or more processors that read data from various entities such asmemory 812 or I/O components 820. Presentation component(s) 816 present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc. - I/
O ports 818 allowcomputing device 800 to be logically coupled to other devices including I/O components 820, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc. The I/O components 820 may provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some instance, inputs may be transmitted to an appropriate network element for further processing. A NUI may implement any combination of speech recognition, touch and stylus recognition, facial recognition, biometric recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with displays on thecomputing device 800. Thecomputing device 800 may be equipped with depth cameras, such as, stereoscopic camera systems, infrared camera systems, RGB camera systems, and combinations of these for gesture detection and recognition. Additionally, thecomputing device 800 may be equipped with accelerometers or gyroscopes that enable detection of motion. The output of the accelerometers or gyroscopes may be provided to the display of thecomputing device 800 to render immersive augmented reality or virtual reality. - As can be understood, embodiments of the present invention provide for an objective approach for determining the visual similarity between fonts. The present invention has been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from its scope.
- From the foregoing, it will be seen that this invention is one well adapted to attain all the ends and objects set forth above, together with other advantages which are obvious and inherent to the system and method. It will be understood that certain features and subcombinations are of utility and may be employed without reference to other features and subcombinations. This is contemplated by and is within the scope of the claims.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/734,466 US9501724B1 (en) | 2015-06-09 | 2015-06-09 | Font recognition and font similarity learning using a deep neural network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/734,466 US9501724B1 (en) | 2015-06-09 | 2015-06-09 | Font recognition and font similarity learning using a deep neural network |
Publications (2)
Publication Number | Publication Date |
---|---|
US9501724B1 US9501724B1 (en) | 2016-11-22 |
US20160364633A1 true US20160364633A1 (en) | 2016-12-15 |
Family
ID=57287781
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/734,466 Active US9501724B1 (en) | 2015-06-09 | 2015-06-09 | Font recognition and font similarity learning using a deep neural network |
Country Status (1)
Country | Link |
---|---|
US (1) | US9501724B1 (en) |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107025284A (en) * | 2017-04-06 | 2017-08-08 | 中南大学 | The recognition methods of network comment text emotion tendency and convolutional neural networks model |
CN107316058A (en) * | 2017-06-15 | 2017-11-03 | 国家新闻出版广电总局广播科学研究院 | Improve the method for target detection performance by improving target classification and positional accuracy |
CN108205699A (en) * | 2016-12-20 | 2018-06-26 | 谷歌有限责任公司 | Generation is used for the output of neural network output layer |
US20180365536A1 (en) * | 2017-06-19 | 2018-12-20 | Adobe Systems Incorporated | Identification of fonts in an application |
CN109492524A (en) * | 2018-09-20 | 2019-03-19 | 中国矿业大学 | Interior structural relationship network for vision tracking |
JP2019087237A (en) * | 2017-11-08 | 2019-06-06 | アドビ インコーポレイテッド | Font recognition improvement using adversarial neural network training |
RU2709661C1 (en) * | 2018-09-19 | 2019-12-19 | Общество с ограниченной ответственностью "Аби Продакшн" | Training neural networks for image processing using synthetic photorealistic containing image signs |
WO2020033641A1 (en) * | 2018-08-09 | 2020-02-13 | Morgan Stanley Services Group Inc. | Optically analyzing domain names |
WO2020093165A1 (en) * | 2018-11-07 | 2020-05-14 | Element Ai Inc. | Removal of sensitive data from documents for use as training sets |
US10658005B1 (en) | 2019-08-19 | 2020-05-19 | Neon Evolution Inc. | Methods and systems for image and voice processing |
US10671838B1 (en) | 2019-08-19 | 2020-06-02 | Neon Evolution Inc. | Methods and systems for image and voice processing |
US10803646B1 (en) | 2019-08-19 | 2020-10-13 | Neon Evolution Inc. | Methods and systems for image and voice processing |
US10949715B1 (en) | 2019-08-19 | 2021-03-16 | Neon Evolution Inc. | Methods and systems for image and voice processing |
CN112926373A (en) * | 2019-12-05 | 2021-06-08 | Sap欧洲公司 | Visually-aware encoding of characters |
WO2021246642A1 (en) * | 2020-06-03 | 2021-12-09 | (주)폰트릭스 | Font recommendation method and device for implementing same |
US11218695B2 (en) | 2017-07-06 | 2022-01-04 | Samsung Electronics Co., Ltd. | Method and device for encoding or decoding image |
US11216658B2 (en) * | 2018-11-14 | 2022-01-04 | Adobe Inc. | Utilizing glyph-based machine learning models to generate matching fonts |
US11308657B1 (en) | 2021-08-11 | 2022-04-19 | Neon Evolution Inc. | Methods and systems for image processing using a learning engine |
Families Citing this family (33)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10074042B2 (en) | 2015-10-06 | 2018-09-11 | Adobe Systems Incorporated | Font recognition using text localization |
US20170132511A1 (en) * | 2015-11-10 | 2017-05-11 | Facebook, Inc. | Systems and methods for utilizing compressed convolutional neural networks to perform media content processing |
US9798612B1 (en) * | 2016-01-14 | 2017-10-24 | Google Inc. | Artifact correction using neural networks |
US10095957B2 (en) * | 2016-03-15 | 2018-10-09 | Tata Consultancy Services Limited | Method and system for unsupervised word image clustering |
US10007868B2 (en) | 2016-09-19 | 2018-06-26 | Adobe Systems Incorporated | Font replacement based on visual similarity |
US10318846B2 (en) * | 2016-12-28 | 2019-06-11 | Ancestry.Com Operations Inc. | Clustering historical images using a convolutional neural net and labeled data bootstrapping |
US10339443B1 (en) * | 2017-02-24 | 2019-07-02 | Gopro, Inc. | Systems and methods for processing convolutional neural network operations using textures |
US10262236B2 (en) | 2017-05-02 | 2019-04-16 | General Electric Company | Neural network training image generation system |
CN108288067B (en) * | 2017-09-12 | 2020-07-24 | 腾讯科技(深圳)有限公司 | Training method of image text matching model, bidirectional search method and related device |
US11003831B2 (en) * | 2017-10-11 | 2021-05-11 | Adobe Inc. | Automatically pairing fonts using asymmetric metric learning |
CN108399409B (en) * | 2018-01-19 | 2019-06-18 | 北京达佳互联信息技术有限公司 | Image classification method, device and terminal |
CN108460772B (en) * | 2018-02-13 | 2022-05-17 | 国家计算机网络与信息安全管理中心 | Advertisement harassment fax image detection system and method based on convolutional neural network |
US10262235B1 (en) | 2018-02-26 | 2019-04-16 | Capital One Services, Llc | Dual stage neural network pipeline systems and methods |
CN108416390B (en) * | 2018-03-16 | 2019-11-01 | 西北工业大学 | Hand-written script recognition methods based on two-dimensional convolution dimensionality reduction |
CN108647723B (en) * | 2018-05-11 | 2020-10-13 | 湖北工业大学 | Image classification method based on deep learning network |
EP3572972A1 (en) * | 2018-05-23 | 2019-11-27 | IDEMIA Identity & Security Germany AG | Extendend convolutional neural network for document analysis |
US11501135B2 (en) * | 2018-05-29 | 2022-11-15 | British Cayman Islands Intelligo Technology Inc. | Smart engine with dynamic profiles |
CN109508628A (en) * | 2018-09-21 | 2019-03-22 | 广州企图腾科技有限公司 | Font type detection method and system in font image based on convolutional neural networks |
CN111259918B (en) * | 2018-11-30 | 2023-06-20 | 重庆小雨点小额贷款有限公司 | Method and device for labeling intention labels, server and storage medium |
CN111339803B (en) * | 2018-12-19 | 2023-10-24 | 新方正控股发展有限责任公司 | Font identification method, apparatus, device and computer readable storage medium |
CN109726657B (en) * | 2018-12-21 | 2023-06-09 | 万达信息股份有限公司 | Deep learning scene text sequence recognition method |
US10915809B2 (en) | 2019-02-04 | 2021-02-09 | Bank Of America Corporation | Neural network image recognition with watermark protection |
US10878298B2 (en) | 2019-03-06 | 2020-12-29 | Adobe Inc. | Tag-based font recognition by utilizing an implicit font classification attention neural network |
US10992902B2 (en) * | 2019-03-21 | 2021-04-27 | Disney Enterprises, Inc. | Aspect ratio conversion with machine learning |
US10803231B1 (en) * | 2019-03-29 | 2020-10-13 | Adobe Inc. | Performing tag-based font retrieval using combined font tag recognition and tag-based font retrieval neural networks |
CN110091331A (en) * | 2019-05-06 | 2019-08-06 | 广东工业大学 | Grasping body method, apparatus, equipment and storage medium based on manipulator |
US10950017B2 (en) | 2019-07-08 | 2021-03-16 | Adobe Inc. | Glyph weight modification |
US11295181B2 (en) | 2019-10-17 | 2022-04-05 | Adobe Inc. | Preserving document design using font synthesis |
US11367254B2 (en) * | 2020-04-21 | 2022-06-21 | Electronic Arts Inc. | Systems and methods for generating a model of a character from one or more images |
US11625932B2 (en) * | 2020-08-31 | 2023-04-11 | Adobe Inc. | Organizing and representing a collection of fonts according to visual similarity utilizing machine learning |
CN113449459B (en) * | 2021-04-09 | 2023-04-07 | 江西高创保安服务技术有限公司 | Universal distributed computing system design method for improving neural network reasoning accuracy and keeping operation speed |
US11947896B2 (en) | 2022-06-24 | 2024-04-02 | Adobe Inc. | Font recommendation |
US11886809B1 (en) | 2022-10-31 | 2024-01-30 | Adobe Inc. | Identifying templates based on fonts |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5542006A (en) * | 1994-06-21 | 1996-07-30 | Eastman Kodak Company | Neural network based character position detector for use in optical character recognition |
US5912986A (en) * | 1994-06-21 | 1999-06-15 | Eastman Kodak Company | Evidential confidence measure and rejection technique for use in a neural network based optical character recognition system |
US7016540B1 (en) * | 1999-11-24 | 2006-03-21 | Nec Corporation | Method and system for segmentation, classification, and summarization of video images |
US7436994B2 (en) * | 2004-06-17 | 2008-10-14 | Destiny Technology Corporation | System of using neural network to distinguish text and picture in images and method thereof |
US7702145B2 (en) * | 2006-06-28 | 2010-04-20 | Microsoft Corporation | Adapting a neural network for individual style |
TWI332640B (en) * | 2006-12-01 | 2010-11-01 | Cyberlink Corp | Method capable of detecting a scoreboard in a program and related system |
-
2015
- 2015-06-09 US US14/734,466 patent/US9501724B1/en active Active
Cited By (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108205699A (en) * | 2016-12-20 | 2018-06-26 | 谷歌有限责任公司 | Generation is used for the output of neural network output layer |
CN107025284A (en) * | 2017-04-06 | 2017-08-08 | 中南大学 | The recognition methods of network comment text emotion tendency and convolutional neural networks model |
CN107316058A (en) * | 2017-06-15 | 2017-11-03 | 国家新闻出版广电总局广播科学研究院 | Improve the method for target detection performance by improving target classification and positional accuracy |
US20180365536A1 (en) * | 2017-06-19 | 2018-12-20 | Adobe Systems Incorporated | Identification of fonts in an application |
US11218695B2 (en) | 2017-07-06 | 2022-01-04 | Samsung Electronics Co., Ltd. | Method and device for encoding or decoding image |
JP2019087237A (en) * | 2017-11-08 | 2019-06-06 | アドビ インコーポレイテッド | Font recognition improvement using adversarial neural network training |
JP7177653B2 (en) | 2017-11-08 | 2022-11-24 | アドビ インコーポレイテッド | Improving font recognition using adversarial neural network training |
WO2020033641A1 (en) * | 2018-08-09 | 2020-02-13 | Morgan Stanley Services Group Inc. | Optically analyzing domain names |
RU2709661C1 (en) * | 2018-09-19 | 2019-12-19 | Общество с ограниченной ответственностью "Аби Продакшн" | Training neural networks for image processing using synthetic photorealistic containing image signs |
CN109492524B (en) * | 2018-09-20 | 2021-11-26 | 中国矿业大学 | Intra-structure relevance network for visual tracking |
CN109492524A (en) * | 2018-09-20 | 2019-03-19 | 中国矿业大学 | Interior structural relationship network for vision tracking |
WO2020093165A1 (en) * | 2018-11-07 | 2020-05-14 | Element Ai Inc. | Removal of sensitive data from documents for use as training sets |
US11216658B2 (en) * | 2018-11-14 | 2022-01-04 | Adobe Inc. | Utilizing glyph-based machine learning models to generate matching fonts |
US11763583B2 (en) | 2018-11-14 | 2023-09-19 | Adobe Inc. | Identifying matching fonts utilizing deep learning |
US10949715B1 (en) | 2019-08-19 | 2021-03-16 | Neon Evolution Inc. | Methods and systems for image and voice processing |
US10803646B1 (en) | 2019-08-19 | 2020-10-13 | Neon Evolution Inc. | Methods and systems for image and voice processing |
US10671838B1 (en) | 2019-08-19 | 2020-06-02 | Neon Evolution Inc. | Methods and systems for image and voice processing |
US10658005B1 (en) | 2019-08-19 | 2020-05-19 | Neon Evolution Inc. | Methods and systems for image and voice processing |
US11670024B2 (en) | 2019-08-19 | 2023-06-06 | Neon Evolution Inc. | Methods and systems for image and voice processing |
CN112926373A (en) * | 2019-12-05 | 2021-06-08 | Sap欧洲公司 | Visually-aware encoding of characters |
WO2021246642A1 (en) * | 2020-06-03 | 2021-12-09 | (주)폰트릭스 | Font recommendation method and device for implementing same |
KR20210150638A (en) * | 2020-06-03 | 2021-12-13 | (주)폰트릭스 | Method of recommending font and device implementing thereof |
KR102357939B1 (en) | 2020-06-03 | 2022-02-04 | (주)폰트릭스 | Method of recommending font and device implementing thereof |
US11308657B1 (en) | 2021-08-11 | 2022-04-19 | Neon Evolution Inc. | Methods and systems for image processing using a learning engine |
Also Published As
Publication number | Publication date |
---|---|
US9501724B1 (en) | 2016-11-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9501724B1 (en) | Font recognition and font similarity learning using a deep neural network | |
US9953425B2 (en) | Learning image categorization using related attributes | |
US9536293B2 (en) | Image assessment using deep convolutional neural networks | |
US11657602B2 (en) | Font identification from imagery | |
Yu et al. | Toward realistic face photo–sketch synthesis via composition-aided GANs | |
Shanmugamani | Deep Learning for Computer Vision: Expert techniques to train advanced neural networks using TensorFlow and Keras | |
US10909459B2 (en) | Content embedding using deep metric learning algorithms | |
Kao et al. | Visual aesthetic quality assessment with a regression model | |
US20200242822A1 (en) | Digital Media Environment for Style-Aware Patching in a Digital Image | |
US20200356842A1 (en) | Decoupling Category-Wise Independence and Relevance with Self-Attention for Multi-Label Image Classification | |
Mungra et al. | PRATIT: a CNN-based emotion recognition system using histogram equalization and data augmentation | |
CN113994384A (en) | Image rendering using machine learning | |
US11620330B2 (en) | Classifying image styles of images based on image style embeddings | |
CN111801680A (en) | Visual feedback of process state | |
Zhu et al. | Learning deep patch representation for probabilistic graphical model-based face sketch synthesis | |
KR20230025906A (en) | Systems and methods for improved facial attribute classification and its use | |
CN112037239B (en) | Text guidance image segmentation method based on multi-level explicit relation selection | |
Takimoto et al. | Image aesthetics assessment based on multi-stream CNN architecture and saliency features | |
Liu et al. | Attentive semantic and perceptual faces completion using self-attention generative adversarial networks | |
CN111445545B (en) | Text transfer mapping method and device, storage medium and electronic equipment | |
Vadakkot et al. | Automatic one-hand gesture (mudra) identification in bharatanatyam using eigenmudra projections and convolutional neural networks | |
Kumar et al. | Pose image generation for video content creation using controlled human pose image generation GAN | |
CN114037644A (en) | Artistic digital image synthesis system and method based on generation countermeasure network | |
CN114692715A (en) | Sample labeling method and device | |
Chaudhary et al. | Application of Transfer Learning & Independent Estimation on Transformed Facial Recognition |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ADOBE SYSTEMS INCORPORATED, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YANG, JIANCHAO;WANG, ZHANGYANG;BRANDT, JONATHAN;AND OTHERS;SIGNING DATES FROM 20150602 TO 20150606;REEL/FRAME:035990/0045 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: ADOBE INC., CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:ADOBE SYSTEMS INCORPORATED;REEL/FRAME:048867/0882 Effective date: 20181008 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |