RU2295154C1 - Method for recognizing text information from graphic file with usage of dictionaries and additional data - Google Patents

Abstract

FIELD: technology for recognizing text information from graphic file.

SUBSTANCE: in accordance to method, set in advance is order of access to additional information, assigned also is estimate of quality for each type of additional information, different variants of division of image of selected rows on fragments are constructed, for each fragment of row linear division graph is built, images of graphic elements are recognized, using a classifier, and an estimate is assigned to each recognition variant, transition from variants of recognition of graphic elements to variants of alphabet symbols is performed, for each chain, connecting starting and ending vertexes, chains are built, appropriate for all variants of recognition of graphical elements and variants of transitions from recognized graphical elements to alphabet symbols, produced variants are ranked in order of decrease of recognition quality estimate, produced variants are processed with usage of information about position of uppercase and lowercase letters, if more than one variant of symbol is available based on results of recognition of graphic element, variants are processed with successive usage of additional information, and/or when necessary simultaneous usage of all types of additional information, quality estimate is assigned to each produced variant, variants of symbols with estimate below predetermined value are discarded, produced variants are sorted using pair-wise comparison, and additional correction of recognition of spaces, erroneously recognized at previous stages, is performed.

EFFECT: increased precision of recognition of text and increased interference resistance of text recognition.

9 cl, 2 dwg

Description

The invention relates to pattern recognition from a graphic image, and in particular to recognition of text in an image of a document in electronic form.

Known methods for pre-processing graphic images, consisting in dividing the image into areas, presumably constituting paragraphs, lines, words, individual characters.

The invention relates to the processing of symbol images, recognition of symbol groups, words, word groups, etc. using additional information.

Terms used.

The recognition option is one of several versions of the results of grapheme interpretation and word composition. One of several versions of the linear division of the image fragment into words, symbols. The result of recognition of fragments (arcs) of a linear division graph (hereinafter GLD) and interpretation of graphemes.

A template is a type of language construction of a word level. Examples: word from Russian letters, number, phone number, URL.

Broadcast - the transition from a graphic image of a symbol (grapheme) to a symbol.

A text fragment is an object for the level of which most of the processing is carried out: building options in accordance with the templates, evaluating them and choosing the best option.

Differential comparator - a mechanism for evaluating fragments recognition options. It is a set of rules that implement both pairwise comparison of options, and an integral assessment.

As additional information, dictionaries of several varieties, document language rules, and topics (literary text, scientific article, fill-in form, etc.) can be used. Attraction of additional information allows to increase the recognition accuracy in the following cases:

- choose the most correct of several options for linear division of a line fragment into characters,

- choose the most correct variant of recognition of the image of a symbol (grapheme) by several classifiers,

- choose the most correct of several letters or ligatures that the selected grapheme can indicate,

- analyze a large section of the recognized text (for example, a string) and, if possible, adjust the recognition results (for example, change the comma to a dot if it is at the end of the sentence or combine the two French words “d” and “Alembert” into one).

A known method of recognizing text characters using additional information.

Patent RU No. 2234734 discloses a method for processing an image of characters and a fragment of text, including several successive steps. They collect additional information (mainly spatial-parametric), which becomes available at each stage, and for its subsequent use in re-analysis of the same fragment.

The specified method uses only a limited number of types of additional information. The method does not use information from additional sources, information based on the features of the language, some other external information. Do not take into account the sequence of access to different types of information.

The technical result consists in increasing the accuracy of text recognition, increasing the noise immunity of text recognition.

The above method, as well as other known methods, does not allow to achieve the required level of recognition accuracy.

The claimed technical result is achieved through the use of language rules and additional rules.

The inventive method consists in the fact that after the operations of dividing the image, presumably containing text, into fragments, presumably containing characters, perform character recognition. As a result of recognition, one or more symbol variants are obtained for each grapheme. After that, the characters are combined into groups, presumably constituting the words. Consider all possible words obtained as combinations of all possible options for splitting the image into characters and recognition options for component fragments. Groups of characters are analyzed, including together with one or more neighboring groups from one or both sides. For words, additional information of several types is applied sequentially from several sources, to the extent necessary for accurate word recognition, by comparing the achieved level of recognition reliability and the amount of computing resources used.

The sequence of analysis is as follows.

Pre-set list and order of attraction of additional information. The list of additional information and the sequence of treatment are as follows:

1. Information on the points of dividing the string into characters.

2. The recognition quality of the graphic element.

3. Dictionary.

4. A dictionary of possible parts of words, for example, trigrams.

5. Rules stipulated by used standard data templates.

6. Rules determined by the location of the word within the runoff and / or paragraph.

7. Rules due to the peculiarities of the document language

8. Rules due to the type of document.

9. Additional rules for handling rare cases.

It is possible to use not all, but only parts of the listed types of additional information.

A quality assessment is also pre-assigned for each type of additional information.

All possible options are determined for dividing image fragments, presumably being lines of text into fragments, presumably related to images of individual words, according to reliably recognized spaces.

For each fragment of the line, presumably being a word, a linear division graph (GLD) is constructed that describes the options for dividing the fragment into graphic elements related to symbol images (graphemes).

Images of graphic elements are recognized using at least one classifier, and a rating is assigned to each graphic recognition option (grapheme).

A transition is made from the recognition options for graphemes that make up the word to the characters of the alphabet, using the options with the highest values for character recognition ratings.

Among all the GLD chains leading from the initial vertex to the final one, one or more chains of recognizable symbols corresponding to the selected recognition options for graphic elements (graphemes) are selected (marked).

All received variants of a group of characters are divided into four varieties according to the following criteria:

- all characters are in capital letters,

- all characters are lowercase,

- the first character is an uppercase letter, the rest is lowercase,

- an option selected based on the assessment of the completed transitions from the recognized grapheme to symbols using the first type of additional information.

If there is more than one variant of a symbol according to the results of recognition of a graphic element, they are processed with successive involvement of subsequent types of additional information, according to a predetermined order.

An additional correction is carried out to recognize gaps erroneously recognized in the previous steps.

If necessary, add new rules and restrictions for data types, moreover, data types are divided into simple and composite, and composite types form as a combination of two or more simple ones or as any combination of simple and complex types.

Limitations (rules) include the use of, including one or more types of templates.

Description of the text recognition process using additional information.

Processing with the use of additional information can be conventionally represented as triple processing of one piece of text.

With the help of character recognition tools, graphic images of lines of text are divided into fragments and processed in turn from left to right. For each fragment of the line, a symbolic linear division graph (GLD) is constructed that describes the options for splitting the fragment into graphemes, and grapheme images are recognized using at least two different types of classifiers.

Next, the first GLD processing is performed using additional information.

The first treatment is as follows:

1. On the GLD, chains of arcs leading from the initial vertex to the final vertex are selected, and in each of the arcs one of the options for its recognition (letter) is selected. Using additional information, several chains of arcs are constructed in descending order of the total recognition quality.

2. The resulting chain of arcs is processed using templates that describe the types of words that may occur in the text. Since a single grapheme can denote several letters, a variant of a word can contain several variants of some letters. Templates to some extent reduce the number of possible options (for example, a word template leaves only variants with language letters, and a number template leaves numbers), but not completely.

3. All generated variants are evaluated by the total value of the quality indicator, and a number of the best quality variants are sorted differentially based on pairwise comparison.

4. From the sorted list, select some of the best options that are passed to the second processing.

As sources for evaluating and comparing options attract:

- The quality of recognition of graphemes of each option (the quality of the path),

- Compliance with the grammar of one of the templates (template quality). The more rigid the grammar and the higher the frequency of application of the template, the more preference is given to it.

- Dictionary check results. The vocabulary score increases with increasing word length and depends on the style and complexity of the word.

- Geometric information. Neighboring letters in the word should be located relative to each other in a given way and be consistent in height.

- Information about linear division points. Certain pairs of letters can touch each other.

- Rules written to handle some special cases.

At the first processing, there is still no information about the text to the right of the fragment being processed (the right fragment) and there may not be enough statistics to calculate the exact line height. However, during the first processing, there is additional information in the form of GLD and grapheme images, which are destroyed after processing the fragment and are not available during the second processing.

Therefore, during the first processing, the final version is not chosen (often there are ambiguities of the “lowercase-capital” type in letters and “letter-number” in identifiers).

The second treatment.

After all fragments of one line are processed, the second processing begins. At this stage, all statistics on the height of letters have already been collected, therefore they put forward the final hypotheses and make final conclusions about the height of the lines. At the same time, the location of the capital letters is finally chosen and the reliability of the broadcast options is evaluated. Fragments of a line are processed in order from last to first.

All variants of characters during the second processing are divided into four groups:

- all uppercase

- all lower case,

- the first capital and

- with the arrangement of lowercase and uppercase characters, selected on the basis of estimates of available broadcasts.

If there are ambiguous broadcasts, they are also concretized according to the following general rule: only those broadcasts that provide the highest possible score are left.

Disambiguation is performed sequentially in the following ways:

1. By geometric characteristics.

2. By the ratio of the height of lowercase and uppercase characters recorded in the template.

3. Given the options for translating the fragment to the left.

4. According to the rules that minimize the number of switchings from letters to numbers in alphanumeric words.

5. If none of the previous methods gives a result, the broadcast option is selected from several available randomly.

After disambiguation, the options are re-evaluated and differentially sorted. In evaluations and sorting, words are taken into account both to the left and to the right of the word being processed. For a word located in the extreme right position, several variants of fragments of words on the right are generated that correspond to different hypotheses about line height. For each word, scoring and sorting is done with several fragments on the right and several best options are left that will serve as fragments on the right for the next word in order.

The third treatment.

After the options for the entire line are selected, they are corrected. Since at this stage there is a whole recognized string, the correction involves “syntactic” information: about the beginning or end of a sentence, about lowercase or uppercase letters of the next or previous word, etc.

на римские "II" и "III", соответственно.At the correction stage, the location of the gaps is refined based on the analysis of the same gap width and the distribution of the width of the gaps, the punctuators (punctuation marks, etc.) and units in numbers are added incorrectly, errors are corrected, such as replacing a point with a comma, correcting words

to Roman "II" and "III", respectively.

Construction of paths on GLD.

The construction of paths on GLD is performed in one of the following ways.

The first method builds the best quality path. It uses the usual algorithm for constructing a better path for a directed acyclic graph.

The second method ("generator") involves enumerating all the paths on the GLD, starting from the best and then in order of quality deterioration. The need for a separate method for constructing the first (best) path is explained by the fact that, on the one hand, the first method builds it much faster than the second ("generator"), and, on the other hand, because often the first (best) path turns out to be obviously the most suitable.

Both methods take into account the many graphemes that are valid in chains of GLD arcs. In addition, the second method (“generator”) can only construct chains of arcs determined by valid trigrams or, for example, by checking the dictionary, which allows increasing the search depth due to the early cutting off of obviously wrong paths.

Variants of recognition results and method of their concretization.

The main result of applying additional information is the word variants. Since the translation of the grapheme into a letter is ambiguous, for each grapheme a set of letter variants is stored. As additional information is attracted (dictionary, line geometry, fragment on the left and right), the number of options gradually decreases. In the first processing, a reduction in the number of options is achieved by applying patterns based on grammar and vocabulary. A further limitation of the number of suitable options is obtained in the second processing.

Recognition of similar graphemes.

There are pairs of very similar graphemes: "r" - "g", "p" - "n", "6" - "b", etc. For their recognition, an additional broadcast is performed. Each grapheme from a pair of similar ones is translated into letters corresponding to both graphemes of the pair, but some of the translations are marked as additional.

For example, graphemes "g" and "r" have the following translations:

"g" → "_G", the main one,

"g" → "_g", the main one,

"g" → "r", optional,

"r" → "_G", optional,

"r" → "_g", optional,

"r" → "r", the main one.

If the dictionary or grammar does not give grounds for an unambiguous choice of translation, choose the main translation.

Generalized graphemes.

For recognition of the divided characters, pair substitutions in GLD are used. Generalized graphemes “||” and “|||” are introduced for two- and three-element graphemes. These graphemes translate into all two- or three-element letters, but all these translations are considered additional. The final selection of broadcasts is carried out with the involvement of additional information.

Assignment of recognition quality assessments when using additional information.

Use integral and differential estimates. The integral assessment consists of a basic quality assessment of the GLD arc chain, template quality and additional quality. Differential assessment is used for pairwise comparison.

Basic assessment of the quality of the path.

A basic path quality estimate is defined as the sum of grapheme recognition quality estimates for all graphemes along a chain of arcs.

Quality tailored templates.

Template quality consists of two components. One component evaluates whether a variant matches the grammar of the template. Another component evaluates the presence of a word in a dictionary.

Extra quality.

Additional quality accrue for an additional list of rules. The main sources of additional corrections are geometric parameters, poor in terms of matching the grapheme or fragment to the left of the word being processed, lowercase-capital letters, language matching in the fragment to the left and right.

SHORT DESCRIPTION OF THE GRAPHIC MATERIAL.

Figure 1 shows the main stages (steps) of recognition of characters, fragments of lines, lines using additional information and means of comparison.

Figure 2 shows the step of selecting the final version of the recognized character string.

DESCRIPTION OF POSITIONS IN FIGURES.

1 - operation (stage) of constructing paths (chains) on GLD by the set (group) of graphemes.

2 - operation (stage) of constructing variants of a group of characters in accordance with the patterns.

3 - operation (stage) of the total assessment of the obtained options.

4 - operation (stage) of the differential sorting of options.

5 - operation (stage) of the final resolution of broadcast ambiguity.

6 - operation (stage) of choosing the chain of the best options for the string.

7 - paths (chains) on GLD.

8 - options.

9 is a sorted list of options.

10 - document language rules.

11 - templates.

12 is a means of comparison.

13 is a list of options.

DESCRIPTION OF THE PRINCIPLE OF WORK.

The essence of the proposed method is presented in figure 1, 2.

After the operations of dividing the image, presumably containing text, into fragments, presumably containing characters, perform character recognition for all options for breaking into fragments. As a result of recognition, one or more symbol variants are obtained for each symbol image (grapheme). After that, the characters are combined into groups, presumably constituting the words.

The essence of the proposed method is that they consider all possible words obtained as a combination of all possible options for splitting the image and recognition of the constituent images of the characters. Groups of characters are analyzed, including together with one or more neighboring groups from one or both sides. Additional words of several varieties are applied to words sequentially from several sources, to the extent necessary for accurate word recognition.

Perform an analysis that includes at least the following steps.

Pre-set list and order of attraction of additional information. The list of additional information and the sequence of treatment are selected from the following list.

1. Information on the points of dividing the string into characters.

2. The recognition quality of the graphic element.

3. Dictionary of complete words.

4. A dictionary of possible parts of words, for example, trigrams.

5. Rules stipulated by the standard data formats used.

6. Rules determined by the location within the runoff and / or paragraph.

7. Rules due to the peculiarities of the document language

8. Rules due to the type of document.

9. Additional rules for handling rare cases.

If the use of a part of the listed types of additional information gives a sufficiently reliable and reliable result, further (next in the list) information is not attracted.

A quality assessment is also pre-assigned for each type of additional information.

All possible options for dividing areas of the image, presumably being lines of text into fragments, presumably related to images of individual words, are determined by reliably recognized spaces.

For each fragment of the line, a GLD is constructed that describes the options for splitting the fragment into graphic elements related to graphemes.

Received images of graphic elements are recognized using one or more classifiers, and each rating option for recognizing a graphic element (grapheme) is assigned a rating.

Carry out the transition from grapheme recognition options to alphabet characters.

Perform the following at least three-step procedure.

First step. For all GLD chains leading from the initial vertex to the final one, one or more chains of recognizable symbols are constructed corresponding to the recognition options for graphic elements (graphemes) and the options for transitions from recognized graphemes to alphabet symbols. The resulting options are ranked in decreasing order of recognition quality.

Second step. All received variants of a group of characters are processed taking into account the rules on the possible arrangement of lowercase and uppercase characters (letters). These rules are divided into four varieties according to the following criteria:

- all characters are in capital letters,

- all characters are lowercase,

- the first character is an uppercase letter, the rest is lowercase,

- an option selected based on the assessment of the completed transitions from the recognized grapheme to symbols, using the first type of additional information.

If there is more than one variant of a symbol based on the recognition of a graphic element, they are processed with sequential involvement of subsequent types of additional information, according to a predetermined order, and / or, if necessary, simultaneously involving all types of additional information. Each received option is assigned a quality rating. Character variations having a score lower than a predetermined one are discarded. The resulting options are sorted using pairwise comparison.

Third step. An additional correction is carried out to recognize gaps erroneously recognized in the previous steps:

- joining elements erroneously separated in the previous steps,

- separation of elements erroneously connected in the previous steps.

The rules due to the peculiarities of the language of the document may include, including phonetic and / or lexical, and / or semantic rules.

When re-evaluating and pairwise sorting for the rightmost word, several variants of the words on the right are generated that correspond to different hypotheses (for example, about the height of the line), and for each word, assessment and sorting is done with several contexts on the right and some of the best options are accepted, which are then used as additional information for the next word in order.

If necessary, add new rules and restrictions and / or edit existing ones.

Means for adding new rules and restrictions may include the introduction of rules for data types, moreover, data types are divided into simple and composite, and composite types form as a combination of two or more simple or any combination of simple and complex.

The data type is specified in the form of at least the following characteristics:

- a list of characters permitted for use in words and / or

- an additional rule restricting the list of characters, and / or

- a list of punctuators authorized for use and / or

- grammar rules for frequently occurring words, or fragments of words.

Limitations include the use of, including one or more of the following types of templates:

- bilingual word,

- bilingual word with numbers,

- dictionary identifier,

- abbreviation

- number

is the Roman number

- number with suffix (ordinal number),

- number with a prefix,

- a word made up of punctuation,

- word + number,

- a word with a number inside,

- a word with brackets,

- telephone number,

- URL pattern

- file name along with full location information,

- regular expression pattern,

- auxiliary template.

A more detailed description of some of the listed varieties of patterns, the meaning of which is not obvious from the name.

Regular expression pattern. A way to describe words that may appear in the text using regular expressions.

Regular expression is a complex formalized data format. Consists of simple data type.

A word is considered preferred if it matches the description given in the regular expression.

Simple data type - a set of words collected in a special dictionary. A special dictionary of limited volume shows which words are more likely in the text and which are unlikely. The word is considered preferable if it is in the dictionary.

Regular expressions are a form of description of "preferred" words in a formal form. For example, the following regular expression:

("cr" | "mon") "ah"

Such a regular expression means that in order for a word to be preferred, the word "kr" or "mon" must be at the beginning of the word, and then "ah" must follow. That is, only two words “crash” and “monk” will be preferred.

(99999) | (999999)

Such a regular expression means that a number having a length of five or six characters is considered preferable.

Other examples.

s? t means "sat" and "set".

s * d means "sad" and "started".

w [io] n means win and won.

[r-t] ight means "right" and "sight".

m [! a] st means "mist" and "most", but not "mast".

t [! a-m] ck means "tock" and "tuck", but not "tack" or "tick".

fe {2} d means "feed" but not "fed".

fe {1,} d means fed and feed.

10 {1,3} means "10" and "100", and "1000".

Helper template. The template used when no other template is suitable. For example, if in the middle of the Russian text the word "Yug12X" is found, then all contextual patterns will not be able to identify this word. In this case, an auxiliary template is used.

Claims (9)

1. A method for recognizing text information from a graphic file, characterized by obtaining a graphic file from a scanning device or otherwise, image segmentation, recognition of text characters; characterized in that the following order of access to additional information is preliminarily specified, including the following types: information about the points of dividing the line into characters, and / or the recognition quality of the graphic element, and / or the dictionary, and / or the dictionary of possible parts of words, and / or rules due to the used typical data patterns or regular expressions and / or rules due to the location of the word within the line and / or paragraph and / or rules due to the peculiarities of the document language and / or rules defined by the type of document, and / or additional rules for handling rare cases; pre-assign a quality assessment for each type of additional information; build various options for splitting the image of selected lines into fragments, presumably containing images of individual words, according to reliably recognized spaces; for each fragment of the line, a linear division graph is constructed that describes the options for dividing the fragment into graphic elements, presumably containing symbol images; recognizing images of graphic elements using one or more classifiers, and each option for recognizing a graphic element is assigned a rating; make the transition from grapheme recognition options to alphabet character variants; perform at least the following steps: first step: for each chain connecting the start and end vertices, construct chains corresponding to all variants of recognition of graphemes and variants of transitions from recognized graphemes to alphabet characters, rank the obtained options in order of decreasing recognition quality assessment, the second step : all obtained variants of a group of symbols are processed using information on the location of uppercase and lowercase letters, if there are more than one variant of the symbol according to the recognition results raficheskogo element, they are processed with the subsequent involvement of the following types of additional information, according to a predetermined order, and / or if necessary, simultaneously involving all these types of additional information, each received option is assigned a quality rating, symbol options having an estimate below a predetermined one are discarded, received options are sorted using pairwise comparisons; third step: make additional correction for recognition of gaps that were erroneously recognized in the previous stages: attachment of elements erroneously separated in the previous steps, separation of elements erroneously attached in the previous steps. 2. The method according to claim 1, characterized in that the rules due to the characteristics of the language of the document include, including phonetic, and / or lexical, and / or semantic. 3. The method according to claim 1, characterized in that in the second step the information on the possible arrangement of uppercase and lowercase letters includes at least four varieties according to the following criteria: all characters are uppercase, all characters are lowercase, the first character is uppercase , the rest are lowercase, the option selected based on the assessment of the completed transitions from the recognized grapheme to symbols using the first type of additional information. 4. The method according to claim 1, characterized in that they use a dictionary of possible fragments of words that exist in a natural language. 5. The method according to claim 4, characterized in that each combination of possible fragments of words is provided with an estimate of the probability of use in the text. 6. The method according to claim 4, characterized in that for evaluating the word, patterns are used that differ in the composition and types of incoming characters: a bilingual word, and / or a bilingual word with numbers, and / or a dictionary identifier, and / or abbreviation, and / or a number and / or a Roman number, and / or a number with a suffix (ordinal number), and / or a number with a prefix, and / or a word from punctuators, and / or a word + number, and / or a word with a number inside, and / or a word with brackets, and / or a phone number, and / or a URL pattern, and / or a file name together with full location information, and / or a regular pattern GOVERNMENTAL expressions, and / or an auxiliary pattern. 7. The method according to claim 1, characterized in that it contains means for adding new rules and restrictions, including the introduction of rules for data types, which are divided into simple and composite. 8. The method according to claim 7, characterized in that the composite data types form as a connection of at least two simple or any combination of simple and composite data types. 9. The method according to claim 7, in which the data type is set in the form of at least the following characteristics: a list of characters allowed for use in words, and / or an additional rule that restricts the list of characters, and / or a list of punctuators allowed for use, and / or grammar rules for frequently occurring words or fragments of words.

Images