US20090132477A1 - Methods of object search and recognition. - Google Patents

Methods of object search and recognition. Download PDF

Info

Publication number
US20090132477A1
US20090132477A1 US11/556,201 US55620106A US2009132477A1 US 20090132477 A1 US20090132477 A1 US 20090132477A1 US 55620106 A US55620106 A US 55620106A US 2009132477 A1 US2009132477 A1 US 2009132477A1
Authority
US
United States
Prior art keywords
searching
element
recited
variants
elements
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/556,201
Inventor
Konstantin Zuev
Diar Tuganbaev
Irina Filimonova
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ABBYY Software Ltd
Original Assignee
Konstantin Zuev
Diar Tuganbaev
Irina Filimonova
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to RU2006101908/09A priority Critical patent/RU2006101908A/en
Priority to RU2006101908A1 priority
Application filed by Konstantin Zuev, Diar Tuganbaev, Irina Filimonova filed Critical Konstantin Zuev
Publication of US20090132477A1 publication Critical patent/US20090132477A1/en
Assigned to ABBYY SOFTWARE LTD reassignment ABBYY SOFTWARE LTD ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FILIMINOVA, IRINA, TUGANBAEV, DIAR, ZUEV, KONSTANTIN
Priority claimed from US12/877,954 external-priority patent/US8571262B2/en
Application status is Abandoned legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • G06F16/9024Graphs; Linked lists

Abstract

The proposed technical solution allows processing of machine-readable forms of unfixed format. An auxiliary brief description may be optionally specified to determine the spatial orientation of the image. A method of searching for elements of a document comprises the following main operations in addition to the operations of preliminary image processing: selecting the varieties of structural description from several available variants, determining the orientation of the image, selecting the text objects, where the text must be recognized, and determining the minimal required volume of recognition, recognizing the text objects, searching for elements of the form. Searching for elements of the form comprises the following actions: selecting a searched element in the structural description, gaining the algorithm of search constraints from the structural description, searching for the element, testing the obtained variants.

Description

  • The present invention relates generally to image recognition and particularly to the recognition of non-text and/or text objects contained in a bit-mapped image of a document.
  • The mentioned methods are also applied for, but not limited to, recognition of data input forms, containing typographical and hand-written texts as well as a set of special text-marks for document navigation. Documents as supposed herein are inquiry lists, questionnaires, bank documents with rigid or arbitrary arrangement of data fields.
  • The mentioned methods may be applied for recognition of predefined form objects contained in an electronic graphical image.
  • PRIOR ART
  • Methods of structure assignment and document element search in an electronic graphical image are known in the art (U.S. Pat. No. 5,416,849 Huang, May 16, 1995).
  • The capability of the known methods to process only fixed forms, not allowing deviations in field arrangement, is the shortcoming of the methods.
  • Anyone of the described methods and the system may be taken as a prototype.
  • The technical result consists in the improvement of searching capabilities as well as the accuracy of identification of obtained image objects, the increase of noise immunity during the process of object search on the image.
  • SUMMARY OF THE INVENTION
  • The declared technical result is achieved by using tools for search and identification of objects on an image; with further assignment of the estimate of correspondence of the search result to the description. Numbers from 0 to 1 are used for the evaluation. The accuracy of evaluation is 10−5 (ten to the power of minus five). The value equal to 1 means the absolute correspondence of the obtained result to the description. If the estimate differs from zero, the application of flexible structural description also comprises the stage of forming block regions, i.e. evaluation of the arrangement of the required fields on the basis of the information about the obtained objects.
  • Structural description comprises the description of spatial and parametric characteristics of document elements, and the logical connections between document elements.
  • The method of preliminary assignment of a document structure consists in setting a description of the document's logical structure in the form of interdependences of spatial and parametric characteristics of elements, algorithms of obtaining the parameters of the search for each element, methods of identifying the obtained elements, methods of decreasing the number of obtained variants of an element, acceleration of the search for the best variant.
  • The method of searching and recognizing the elements (fields or field fragments) of a document on a graphical (bit-mapped) image consists in using of a predefined logical structure of the document in the form of structural description, algorithms of obtaining the parameters of the search for each element, methods of identifying the obtained elements, methods of decreasing the number of obtained variants of an element, acceleration of the search for the best variant.
  • Searching for elements with the help of a flexible structural description is performed sequentially in the order in which they are described in the flexible structural description, top-down through the “tree” (hierarchy) of elements, in accordance with the logical structure of the document description. For each element in the assigned search area, several variants of image objects or sets of image objects corresponding to the description of the element in the structural description may be found. Various obtained variants of objects are considered to be the variants of the position of the element on the image. An estimate of the degree of correspondence of the variant to the element description is assigned to each obtained variant (i.e. the estimate of the quality of the variant).
  • The accuracy of the obtained position of the object determines the accuracy of obtaining the positions of objects described further in the description relative to this object. Searching for the next dependent object is performed separately for each obtained variant of the current object. Therefore, the variants of objects obtained on the image comprise a hierarchical tree, considerable more branched than the hierarchical tree of elements in a structural description.
  • If an element or an object is compound, i.e. composed of several parts, the whole group also represents an element, which requires generating several possible variants, the number of which corresponds to the number of complete chains of group sub-elements (dependent elements of a lower level). The chain is considered complete if all its obtained sub-elements (elements of a lower level) have sufficient quality. The total estimate of the quality of a variant of a compound element is calculated by multiplying the estimates of the quality of element variants forming the compound element. A flexible structural description as a whole also represents a compound element, therefore, the quality of the correspondence of the variant to the flexible structural description is determined by multiplying the quality of its elements.
  • Application of a flexible structural description comprises searching for the best complete branch in the whole tree of variants, i.e. the branch that include all the elements, from first to last. A general solution of such a task implies taking into consideration all the possible combinations of hypotheses for all elements, construction of a total multitude of complete branches and selecting the best among them. However, in practice, such a solution requires too much resources, and is therefore impractical. Moreover, an abrupt increase in the number of variants taken into consideration is possible, caused by an increase in the number of elements and a lack of rigid restrictions on the search area and element parameters.
  • To limit the time required to analyze the variants, one of the several methods of decreasing the volume is used.
  • Each element gets the maximum allowed number of acceptable variants, rated in the order of decreasing quality. These variants will be used in the further search, i.e. when searching for the next element. Any variants beyond this number will be discarded. Usually this number is 5 (five) for simple elements and 1 (one) for compound elements. This means that, if 15 variants are obtained for a simple element in the assigned search area, five variants with the best quality rating will be selected. Other 10 chains of variants will not be complete and will not be taken into consideration. A compound element is identified with a greater quality rating than a simple element, because the quality of identification is determined not only by multiplying the quality ratings of the constituent simple elements, but also by several additional (mainly qualitative) characteristics, such as mutual arrangement, object size, correspondence to the conditions of mutual arrangement several elements, and so on.
  • Since a compound element is identified with a greater quality rating than a simple element, its best variant usually turns out to be accurate.
  • The process of searching for objects almost always includes generating several incomplete chains of variants of obtained objects and, therefore, several directions of further search. Search for the best hypothesis is performed by using an algorithm of “broad searching”, i.e. the search is always directed through the chain of variants which has the best quality rating at the current step, regardless of the length of the chain. For example, if in a flexible structural description of 30 elements 2 chains are obtained during search, one of which consists of 30 elements with the quality rating of 0.89 and the other chain has 2 elements with the quality rating of 0.92, then the second chain will be pursued until its total quality becomes lower than that of the first chain.
  • The following rule of quality optimization is used for compound elements: if an ideal complete chain for this element is obtained, i.e. the quality of the obtained chain equals 1, other variants of sub-elements composition of this compound element are not taken into consideration.
  • Moreover, the maximum number of variants for every element in the entire hypothesis tree is restricted to 1000.
  • DETAILED DESCRIPTION OF THE INVENTION
  • A set of spatial and parametric characteristics sufficient for search for and identification of an element is used to describe elements of a document of anon-fixed format. A structural description consists of a description of spatial and/or parametric characteristics of the element, and a description of its logical connections with other elements.
  • A flexible structural description may also additionally include all or some of the following conditions. The logical structure of a document is represented as a sequence of elements connected mainly by hierarchical dependences; an algorithm of determining the search parameters is set, spatial characteristics for searching for each element are specified, parametric characteristics of the searching for each element are set, the set of parameters for identifying a compound element on the basis of the aggregate of components is set, and an algorithm of estimating the quality of an obtained variant of an element is set.
  • A flexible structural description may also additionally include a separate brief structural description for determining the correct spatial orientation of the image.
  • A flexible structural description may also additionally include a separate brief structural description for determining the document type and selecting the corresponding comprehensive document description from several possible descriptions. A comprehensive description is created for each document type. If a document type does not have a brief description, then the comprehensive description of the document is used for selecting its type.
  • The structure of element connection is mainly realized as a hierarchical structure.
  • The essence of the invention as regards the method of searching (recognizing) elements (fields) on a document form in a bit-mapped image according to the first method consists in the following. A method of searching and identifying (including recognition) the elements of a document with non-fixed format comprises at least the following preliminary actions. Revision of the whole document image. Detection of obtained objects or object fragments. Performing an initial classification of detected objects according to the set of predefined types. Recognition of all or a part of text objects, where each object is recognized partially or entirely. To speed up the processing, recognition of text objects is performed to a degree which is sufficient for identifying the document structure and other elements of the form.
  • A separate structural description is set to detect the spatial orientation of an object. Such a description usually contains a brief set of structural elements which can be easily recognized on a document (form). Orientation is accepted as correct if the elements of the structural description coincide with the elements on the image with the best quality estimate.
  • A corresponding separate brief description is set for quick detection of the type of recognized document and selecting the comprehensive (main) description of the document type from several possible descriptions. A comprehensive description is created for each document type. If any document type does not have a brief description, then the comprehensive description of the document is used for selecting its type, and the selection of the document type is performed by comparing the quality estimates of the used (brief or comprehensive) descriptions of different types.
  • Then the following main actions are performed. Choosing an element for search in the structural description. Obtaining an algorithm of determining the search parameters from the structural description. Searching for the element. Testing the obtained variants.
  • Searching for an element comprises the following operations. Search by using the spatial characteristics of the search area (for example, a half-plane, a rectangle, a circle, a polygon, or any combinations thereof). Searching by using parametric characteristics of an element. Search by using the spatial characteristics of an element. For example, as absolute coordinates and/or coordinates relative to the other elements (located higher in the tree). The coordinates may be specified as exact values or as an interval.
  • Search with the help of the results of preliminary text recognition.
  • Testing the detected elements comprises the following actions. Identification of detected elements. Analysis of the results of testing the hypotheses about the presence of the element, completeness of the element composition, and types of composite parts of the element, analysis of correspondence of the structure of a compound element to the hypothesis.
  • Optimization of the search through element combination variants, in its turn, comprises the following actions. Assigning to each element several variants with the best quality rating, which are kept for further analysis, and discarding all other variants. Searching for the best variant of a compound element, taking into account the best total quality estimate of the composite parts, regardless of their number. The total quality estimate of a compound element is calculated as the product of the quality estimates of the parts thereof. Additionally, other qualitative characteristics are taken into consideration.
  • Initially, the first element in the list is selected.
  • The following spatial characteristics of an element may be also applied: absolute coordinates and/or coordinates relative to the other elements.
  • The coordinates may be specified as exact values or as an interval.
  • The following spatial characteristics of the search area may be used: half-plane, rectangle, circle, polygon.
  • Revision of the element combination variants is considered complete if the total quality estimate of the complete set of elements achieves the quality value of 1.
  • The number of variants of a compound element which have the best quality estimate and are used for further analysis should be in the range from one to three.
  • The number of variants of a simple element which have the best quality estimate and are used for further analysis should be in the range from three to ten.
  • A method of search and recognition (identification) of elements (fields) on a document of non-fixed format according to the second variant comprises at least the following preliminary actions. Revision of the entire document image. Allocation of the detected objects or object fragments. Performing the initial classification of the allocated objects according to the set of predefined types. Recognition of all or a part of text objects, where each object is recognized partially or entirely. Recognition of text objects is performed to a degree which is sufficient for identifying the document structure and other elements of the form.
  • A separate structural description is set to detect the spatial orientation of an object. Such a description usually contains a brief set of structural elements which can be easily recognized on a document (form). Orientation is accepted as correct if the elements of the structural description coincide with the elements on the image with the best quality estimate.
  • A corresponding separate brief description is set for quick detection of the type of a recognized document and selecting the comprehensive (main) description of the document type from several possible descriptions. A comprehensive description is created for each document type. If any document type does not have a brief description, then the comprehensive description of the document is used for selecting its type, and the selection of the document type is performed by comparing the quality estimates of the used (brief or comprehensive) descriptions of different types.
  • Then all or at least a part of the following operations are performed.
  • Choosing the next element in the structural description (starting from the first one).
  • Calculating or getting a predefined algorithm for determining the search parameters.
  • Performing a search for an element, comprising at least the following operations:
  • searching by using the spatial characteristics of the search area such as, for example, half-plane, rectangle, circle, polygon and others;
  • searching by using the parametric characteristics of an element (the type of element);
  • searching by using the spatial characteristics of an element, represented as absolute coordinates and/or coordinates relative to the other elements.
  • The coordinates may be specified as exact values or as an interval.
  • calculating the quality of correspondence of the obtained variant to the description of the required element.
  • Testing the obtained variant of the object comprises the following operations.
  • identifying the obtained element variant;
  • calculating the quality of the identification of the element;
  • analyzing the results of testing the hypotheses about the presence and completeness of the composition of the element and the types of composite parts, analyzing of the correspondence of a compound element to the hypothesis about the type of the element;
  • calculating the total reliability of the obtained variant
  • Optimization of revision of element combination variants comprises
  • assigning to each type of the element several variants with the best quality rating, which are kept for further analysis;
  • searching for the best variant of a compound element, taking into account the best total quality estimate of composite parts, regardless of their number.
  • revision of the quality estimates of the variants which were discarded earlier in order to find any quality estimates which would be higher than the current one.
  • If the total quality estimate is lower than the predefined level, searching for the next variant of the same element and calculating its total quality estimate are performed.
  • If the total quality estimate is higher than the predefined level, searching for the next element is performed.
  • The variant with the maximum total quality estimate is selected.
  • Searching for the best variant of a compound element is performed, taking into account the best total quality estimate of accountable composite parts, regardless of their number.
  • The quality of a variant as supposed herein is the estimation which indicates the degree of correspondence of the obtained variant to the present element (its properties and search constraints). The numerical constituent of the quality of a variant is a number ranging from 0 to 1. The quality of a hypothesis for a compound element is calculated by multiplying the quality estimates of the hypotheses of all the sub-elements thereof.
  • The quality of a variant is a result of multiplication of the quality of the element, assigned at the stage of specification of the structural description during the specification of the element type, and the quality of the element (object), variant is calculated at the stage of the search. The total quality of the variant is calculated as a product of quality ratings of all interdependent composing elements in the chain, from the first element in the structural description to the current element.
  • For optional elements i.e. such elements, which may be missing on a document, a “zero” variant of an element is used, if the element has not been detected. A “zero” variant supposes that the required object is missing in the search area. A “zero” variant is formed, if no objects are detected corresponding to the optional element and having the quality estimate which is greater than the quality of the zero variant. If the zero variant is selected as the most appropriate, the process is started of searching and identifying the next element in the list (including the elements which depend on the element which has not been obtained), or analyzing one of the earlier rejected variants of the same element or another element, simultaneously taking appropriate measures to avoid creating an infinite loop in the process.
  • If no objects are detected corresponding to the optional element, the use of the flexible description is not stopped. Instead, a zero variant is generated which gains the quality value predefined by the user in the description of this optional element.
  • Creation of the flexible structural description uses the following main types of elements conventionally divided into the following: simple element which do not contain other elements: Static Text, Separator, White field, Barcode, Text String, Text Fragment, Set of objects, Date, Phone Number, Currency, and Table, and compound elements—Group, and some other types.
  • Static text, as supposed herein, is an element of structural description describing a text with the known meaning. The text may consist of one word, of several words, or of an entire paragraph. “Several words” differs from “a word” by the presence of at least one blank space or another inter-word separator, depending on the language, for example, a full stop, a comma, a colon, or any other punctuation mark. Several words may take up several text strings.
  • Separator, as supposed herein, is an element representing a vertical or horizontal graphical object between other objects. A separator can be represented, for example, by a solid line or a dotted line.
  • White field, as supposed herein, is an element of description representing a rectangular region of an image which does not contain any objects within it.
  • Barcode, as supposed herein, is an element of flexible description representing a line drawing which codes numerical information.
  • Text string, as supposed herein, is an element representing a sequence of characters located on a single line one after another. Character strings can consist of text objects, for example, words, or of fragments of text objects.
  • Text fragment, as supposed herein, is an element representing an aggregate of text objects.
  • Set of objects (of the specified type), as supposed herein, is an element representing an aggregate of different types of objects on an image, where each object meets the search constraints.
  • Date as supposed herein, is an element representing a date.
  • Telephone number, as supposed herein, is an element representing a telephone number which may be accompanied a by prefix (“tel.”, “home tel.”, etc.) and by a code of the city/region, which is separated from the number by brackets.
  • Currency, as supposed herein, is an element of description representing money sums, where the name of the currency can be used as the prefix.
  • Table, as supposed herein, is an element of flexible description representing data in the form of a table.
  • Compound element (element group), as supposed herein, is an aggregate of several elements (sub-elements). Sub-elements may be simple or compound.
  • Compound elements are used for:
  • joining elements into a group. Each of these compound elements may contain smaller compound elements which search for smaller fragments of the element;
  • providing the logical hierarchy of elements for better navigation through the structural description;
  • reducing the number of possible variants of the element in order to speed up the search for the resulting variant. Joining elements into a compound element allows to analyze this set of sub-elements as a single entity which has its own complete variant (consisting of the variants of the sub-elements) and a total estimate of reliability of the entire group. Revision of possible combinations of variants of the sub-elements is performed within the group, and only a predefined number of the best variants in the group take part in the further analysis and search for the next elements. The number of the best variants of a compound element which take part in further searching is usually 1;
  • specifying restrictions of the search area which are common for all the sub-elements. The search area of a certain sub-element in this case is calculated as the intersection of the search area set for the sub-element itself and the search area of the group which contains this sub-element.

Claims (26)

1. A method of searching for element of a document, comprising at least the following operations of preliminary image processing:
searching for objects on the image;
allocating the obtained objects;
selecting the text objects, where the text must be recognized, and determining the minimal required volume of recognition.
recognizing the text objects;
then the program searches for elements of the form. The search comprises at least the following actions:
selecting a searched element in the structural description;
gaining the algorithm of obtaining the search constraints from the structural description;
searching for the element;
testing the obtained variants;
optimizing the analysis of variants of combinations of elements,
and searching for an element comprises the following operations:
searching with the use of the spatial characteristics of the search area;
searching with the use of the parametric characteristics of the element;
searching with use of the absolute and/or relative spatial characteristics of the element—represented as exact values and/or as intervals;
searching with the use of the results of preliminary text recognition,
and testing of the obtained elements comprises the following actions:
identifying the obtained variant of the element;
estimating the quality of the identification of the element;
analyzing the results of testing the hypotheses about the presence, completeness of composition, and types of composite parts, analyzing their correspondence to the hypothesis about the type in the case of a compound element;
estimating the total reliability of the obtained variant,
optimization of the analysis of variants of combinations of elements, comprising:
assigning a number of variants with the best quality ratings which will be kept for further analysis to each type of the element;
searching for the best variant of the compound element, taking into account the best total quality of its accountable composite parts, regardless of their number,
analyzing the quality estimates of earlier rejected variants in order to obtain quality estimates higher than the current estimate.
and optimization of the analysis of variants of combinations of elements comprises the following actions:
assigning a number of variants with the best quality ratings which will be kept for further analysis to each type of the element;
discarding the other variants;
searching for the best variant of the compound element, taking into account the best total quality of its accountable composite parts, regardless of their number,
analyzing the quality estimates of earlier rejected variants in order to obtain quality estimates higher than the current estimate.
2. A method of searching, as recited in claim 1, where the orientation of the image is determined.
3. A method of searching, as recited in claim 2, where all or a part of elements of the structural description are used to determine the correct orientation of the image.
4. A method of searching, as recited in claim 3, where an auxiliary description in brief form is optionally assigned to determine the spatial orientation of the image.
5. A method of searching, as recited in claim 3, where the orientation of the image whose objects of the description coincide with the objects on the image with the maximal quality rating is accepted as the correct orientation of the image.
6. A method of searching, as recited in claim 1, where the type of a document is selected from several available types.
7. A method of searching, as recited in claim 6, where a separate brief structural description is optionally included for determining the document type and selecting the corresponding comprehensive document description from several available descriptions.
8. A method of searching, as recited in claim 7, where the type of the document which corresponds to the current image is selected on the basis of comparing the quality estimates of the used descriptions.
9. A method of searching for an element, comprising at least the following operations of preliminary image processing:
searching for the object on the image;
allocating the obtained objects
classifying the objects according to their types;
selecting the text objects, where the text must be recognized, and determining the minimal required volume of recognition;
recognizing the text objects;
then the program searches for elements of the form. This search comprises at least the following actions:
selecting a searched element in the structural description;
gaining the algorithm of obtaining the search constraints;
searching for the element;
testing the obtained variants;
and searching for an element comprises the following operations:
searching with the use of the spatial characteristics of the search area;
searching with the use of the parametric characteristics of the element;
searching with the use of the spatial characteristics of the element
and testing the obtained elements comprises the following actions:
identifying the obtained elements;
analyzing the results of testing the hypotheses about the presence and completeness of composition of the elements, and the types of the composite parts, analyzing the correspondence to the hypothesis about the composition of the compound element.
optimization of the analysis of variants of combinations of elements comprising:
assigning a number of variants with the best quality ratings which will be kept for further analysis to each type of the element;
searching for the best variant of the compound element, taking into account the best total quality of its accountable composite parts, regardless of their number,
analyzing the quality estimates of earlier rejected variants in order to obtain quality estimates higher than the current estimate.
10. A method of searching, as recited in claims 1 or 9, where initially the first element in the list is selected.
11. A method of searching, as recited in claims 1 or 9, where the spatial characteristics of an element including its absolute coordinates and/or relative coordinates are used.
12. A method of searching, as recited in claims 1 or 9, where the exact and/or interval characteristics of an element are used.
13. A method of searching, as recited in claims 1 or 9, where the following spatial characteristics of the search area are also used: a half plane, a rectangle, a circle, a polygon, or a combination thereof.
14. A method of searching, as recited in claims 1 or 9, where the quality estimates of all earlier rejected variants are analyzed after the estimation of the total quality rating of the variant to select a variant with the best quality rating.
15. A method of searching, as recited in claims 1 or 9, where searching for the next object is performed if there are no variants for the current element or the total quality rating is lower than the predefined level.
16. A method of searching, as recited in claims 1 or 9, where revision of variants of combinations of the elements is considered complete if the total quality estimate of the complete set of elements achieves the quality value of 1.
17. A method of searching, as recited in claims 1 or 9, where one to three variants of a compound element which have the best quality estimate are used for further analysis.
18. A method of searching, as recited in claims 1 or 9, where three to ten variants of a simple element which have the best quality estimate are used for further analysis.
19. A method of searching, as recited in claims 1 or 9, where the program starts searching for an object corresponding to the next element of the structural description, if no objects are detected in the region of the image which is specified for the current element.
20. A method of searching, as recited in claim 9, where the orientation of the image is determined.
21. A method of searching, as recited in claim 20, where all or a part of elements of the structural description are used to determine the correct orientation of the image.
22. A method of searching, as recited in claim 21, where an auxiliary brief description is optionally specified to determine the spatial orientation of the image.
23. A method of searching, as recited in claim 21, where the orientation of the image whose objects of the description coincide with the objects on the image with the maximal quality rating is accepted as the correct orientation of the image.
24. A method of searching, as recited in claims 1 or 9, where the type of the document is selected from several available types.
25. A method of searching, as recited in claim 24, where a separate brief structural description is optionally included for determining the document type and selecting the corresponding comprehensive document description from several available descriptions.
26. A method of searching, as recited in claim 25, where the type of the document which corresponds to the current image is selected on the basis of comparing the quality estimates of the used descriptions.
US11/556,201 2006-01-25 2006-11-03 Methods of object search and recognition. Abandoned US20090132477A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
RU2006101908/09A RU2006101908A (en) 2006-01-25 2006-01-25 Structural description of the document, method for description of the structure of graphic objects and methods of their recognition (options)
RU2006101908A1 2006-01-25

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12/877,954 US8571262B2 (en) 2006-01-25 2010-09-08 Methods of object search and recognition
US13/963,616 US8750571B2 (en) 2006-01-25 2013-08-09 Methods of object search and recognition

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/877,954 Continuation-In-Part US8571262B2 (en) 2006-01-25 2010-09-08 Methods of object search and recognition

Publications (1)

Publication Number Publication Date
US20090132477A1 true US20090132477A1 (en) 2009-05-21

Family

ID=39360877

Family Applications (3)

Application Number Title Priority Date Filing Date
US11/556,201 Abandoned US20090132477A1 (en) 2006-01-25 2006-11-03 Methods of object search and recognition.
US11/556,204 Abandoned US20080263021A1 (en) 2006-01-25 2006-11-03 Methods of object search and recognition
US11/556,196 Active 2029-12-28 US8171391B2 (en) 2006-01-25 2006-11-03 Method of describing the structure of graphical objects

Family Applications After (2)

Application Number Title Priority Date Filing Date
US11/556,204 Abandoned US20080263021A1 (en) 2006-01-25 2006-11-03 Methods of object search and recognition
US11/556,196 Active 2029-12-28 US8171391B2 (en) 2006-01-25 2006-11-03 Method of describing the structure of graphical objects

Country Status (2)

Country Link
US (3) US20090132477A1 (en)
RU (1) RU2006101908A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110188759A1 (en) * 2003-06-26 2011-08-04 Irina Filimonova Method and System of Pre-Analysis and Automated Classification of Documents
US20120011152A1 (en) * 2010-07-12 2012-01-12 Microsoft Corporation Generating Programs Based on Input-Output Examples Using Converter Modules
US20120032986A1 (en) * 2007-05-29 2012-02-09 Research In Motion Limited System and method for resizing images prior to upload
US20140307959A1 (en) * 2003-03-28 2014-10-16 Abbyy Development Llc Method and system of pre-analysis and automated classification of documents
US8972930B2 (en) 2010-06-04 2015-03-03 Microsoft Corporation Generating text manipulation programs using input-output examples
US20160307067A1 (en) * 2003-06-26 2016-10-20 Abbyy Development Llc Method and apparatus for determining a document type of a digital document
US9552335B2 (en) 2012-06-04 2017-01-24 Microsoft Technology Licensing, Llc Expedited techniques for generating string manipulation programs

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2006101908A (en) * 2006-01-25 2010-04-27 Аби Софтвер Лтд. (Cy) Structural description of the document, method for description of the structure of graphic objects and methods of their recognition (options)
US8571262B2 (en) 2006-01-25 2013-10-29 Abbyy Development Llc Methods of object search and recognition
GB0706788D0 (en) * 2007-04-05 2007-05-16 Dymo Nv Tape printing apparatus
US9424167B2 (en) 2014-05-21 2016-08-23 Cgi Technologies And Solutions Inc. Automated testing of an application system

Citations (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5251273A (en) * 1992-04-15 1993-10-05 International Business Machines Corporation Data processing system and method for sequentially repairing character recognition errors for scanned images of document forms
US5317646A (en) * 1992-03-24 1994-05-31 Xerox Corporation Automated method for creating templates in a forms recognition and processing system
US5416849A (en) * 1992-10-21 1995-05-16 International Business Machines Corporation Data processing system and method for field extraction of scanned images of document forms
US5434962A (en) * 1990-09-07 1995-07-18 Fuji Xerox Co., Ltd. Method and system for automatically generating logical structures of electronic documents
US5822454A (en) * 1995-04-10 1998-10-13 Rebus Technology, Inc. System and method for automatic page registration and automatic zone detection during forms processing
US5864629A (en) * 1990-09-28 1999-01-26 Wustmann; Gerhard K. Character recognition methods and apparatus for locating and extracting predetermined data from a document
US6004845A (en) * 1998-03-11 1999-12-21 United Microelectronics Corp. Method for fabricating a crown-shaped capacitor
US20020059265A1 (en) * 2000-04-07 2002-05-16 Valorose Joseph James Method and apparatus for rendering electronic documents
US6507671B1 (en) * 1998-12-11 2003-01-14 International Business Machines Corporation Method and system for dropping template from a filled in image
US20040006467A1 (en) * 2002-07-07 2004-01-08 Konstantin Anisimovich Method of automatic language identification for multi-lingual text recognition
US6687404B1 (en) * 1997-06-20 2004-02-03 Xerox Corporation Automatic training of layout parameters in a 2D image model
US6694053B1 (en) * 1999-12-02 2004-02-17 Hewlett-Packard Development, L.P. Method and apparatus for performing document structure analysis
US20040047508A1 (en) * 2002-09-09 2004-03-11 Konstantin Anisimovich Text recognition method using a trainable classifier
US20040114802A1 (en) * 2002-12-17 2004-06-17 Konstantin Anisimovich Bit-mapped image multi-stage analysis method
US20040117738A1 (en) * 2002-12-17 2004-06-17 Konstantin Anisimovich System of automated document processing
US20040190790A1 (en) * 2003-03-28 2004-09-30 Konstantin Zuev Method of image pre-analyzing of a machine-readable form of non-fixed layout
US20040223197A1 (en) * 2003-02-13 2004-11-11 Canon Kabushiki Kaisha Image processing method
US20040264774A1 (en) * 2003-06-24 2004-12-30 Konstantin Anisimovich Method of graphical objects recognition using the integrity principle
US20060104511A1 (en) * 2002-08-20 2006-05-18 Guo Jinhong K Method, system and apparatus for generating structured document files
US20060217956A1 (en) * 2005-03-25 2006-09-28 Fuji Xerox Co., Ltd. Translation processing method, document translation device, and programs
US7149347B1 (en) * 2000-03-02 2006-12-12 Science Applications International Corporation Machine learning of document templates for data extraction
US7171615B2 (en) * 2002-03-26 2007-01-30 Aatrix Software, Inc. Method and apparatus for creating and filing forms
US7225197B2 (en) * 2002-10-31 2007-05-29 Elecdecom, Inc. Data entry, cross reference database and search systems and methods thereof
US20070172130A1 (en) * 2006-01-25 2007-07-26 Konstantin Zuev Structural description of a document, a method of describing the structure of graphical objects and methods of object recognition.
US7310635B2 (en) * 2004-05-17 2007-12-18 Knowitall, Llc. Record management and retrieval computer program and method
US7346215B2 (en) * 2001-12-31 2008-03-18 Transpacific Ip, Ltd. Apparatus and method for capturing a document
US20080195968A1 (en) * 2005-07-08 2008-08-14 Johannes Schacht Method, System and Computer Program Product For Transmitting Data From a Document Application to a Data Application
US20080263021A1 (en) * 2006-01-25 2008-10-23 Konstantin Zuev Methods of object search and recognition

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE511242C2 (en) 1997-04-01 1999-08-30 Readsoft Ab Method and device for the automatic data capture of the form
US6400845B1 (en) * 1999-04-23 2002-06-04 Computer Services, Inc. System and method for data extraction from digital images
US7149367B2 (en) * 2002-06-28 2006-12-12 Microsoft Corp. User interface for a system and method for head size equalization in 360 degree panoramic images
WO2007117334A2 (en) * 2006-01-03 2007-10-18 Kyos Systems Inc. Document analysis system for integration of paper records into a searchable electronic database
US8351706B2 (en) * 2007-07-24 2013-01-08 Sharp Kabushiki Kaisha Document extracting method and document extracting apparatus

Patent Citations (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5434962A (en) * 1990-09-07 1995-07-18 Fuji Xerox Co., Ltd. Method and system for automatically generating logical structures of electronic documents
US5864629A (en) * 1990-09-28 1999-01-26 Wustmann; Gerhard K. Character recognition methods and apparatus for locating and extracting predetermined data from a document
US5317646A (en) * 1992-03-24 1994-05-31 Xerox Corporation Automated method for creating templates in a forms recognition and processing system
US5251273A (en) * 1992-04-15 1993-10-05 International Business Machines Corporation Data processing system and method for sequentially repairing character recognition errors for scanned images of document forms
US5416849A (en) * 1992-10-21 1995-05-16 International Business Machines Corporation Data processing system and method for field extraction of scanned images of document forms
US5822454A (en) * 1995-04-10 1998-10-13 Rebus Technology, Inc. System and method for automatic page registration and automatic zone detection during forms processing
US6687404B1 (en) * 1997-06-20 2004-02-03 Xerox Corporation Automatic training of layout parameters in a 2D image model
US6004845A (en) * 1998-03-11 1999-12-21 United Microelectronics Corp. Method for fabricating a crown-shaped capacitor
US6507671B1 (en) * 1998-12-11 2003-01-14 International Business Machines Corporation Method and system for dropping template from a filled in image
US6694053B1 (en) * 1999-12-02 2004-02-17 Hewlett-Packard Development, L.P. Method and apparatus for performing document structure analysis
US7149347B1 (en) * 2000-03-02 2006-12-12 Science Applications International Corporation Machine learning of document templates for data extraction
US20020059265A1 (en) * 2000-04-07 2002-05-16 Valorose Joseph James Method and apparatus for rendering electronic documents
US7346215B2 (en) * 2001-12-31 2008-03-18 Transpacific Ip, Ltd. Apparatus and method for capturing a document
US7171615B2 (en) * 2002-03-26 2007-01-30 Aatrix Software, Inc. Method and apparatus for creating and filing forms
US20040006467A1 (en) * 2002-07-07 2004-01-08 Konstantin Anisimovich Method of automatic language identification for multi-lingual text recognition
US20060104511A1 (en) * 2002-08-20 2006-05-18 Guo Jinhong K Method, system and apparatus for generating structured document files
US20040047508A1 (en) * 2002-09-09 2004-03-11 Konstantin Anisimovich Text recognition method using a trainable classifier
US7225197B2 (en) * 2002-10-31 2007-05-29 Elecdecom, Inc. Data entry, cross reference database and search systems and methods thereof
US20040114802A1 (en) * 2002-12-17 2004-06-17 Konstantin Anisimovich Bit-mapped image multi-stage analysis method
US20040117738A1 (en) * 2002-12-17 2004-06-17 Konstantin Anisimovich System of automated document processing
US20040223197A1 (en) * 2003-02-13 2004-11-11 Canon Kabushiki Kaisha Image processing method
US20040190790A1 (en) * 2003-03-28 2004-09-30 Konstantin Zuev Method of image pre-analyzing of a machine-readable form of non-fixed layout
US20040264774A1 (en) * 2003-06-24 2004-12-30 Konstantin Anisimovich Method of graphical objects recognition using the integrity principle
US7310635B2 (en) * 2004-05-17 2007-12-18 Knowitall, Llc. Record management and retrieval computer program and method
US20060217956A1 (en) * 2005-03-25 2006-09-28 Fuji Xerox Co., Ltd. Translation processing method, document translation device, and programs
US20080195968A1 (en) * 2005-07-08 2008-08-14 Johannes Schacht Method, System and Computer Program Product For Transmitting Data From a Document Application to a Data Application
US20070172130A1 (en) * 2006-01-25 2007-07-26 Konstantin Zuev Structural description of a document, a method of describing the structure of graphical objects and methods of object recognition.
US20080263021A1 (en) * 2006-01-25 2008-10-23 Konstantin Zuev Methods of object search and recognition

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140307959A1 (en) * 2003-03-28 2014-10-16 Abbyy Development Llc Method and system of pre-analysis and automated classification of documents
US9633257B2 (en) * 2003-03-28 2017-04-25 Abbyy Development Llc Method and system of pre-analysis and automated classification of documents
US20110188759A1 (en) * 2003-06-26 2011-08-04 Irina Filimonova Method and System of Pre-Analysis and Automated Classification of Documents
US10152648B2 (en) * 2003-06-26 2018-12-11 Abbyy Development Llc Method and apparatus for determining a document type of a digital document
US20160307067A1 (en) * 2003-06-26 2016-10-20 Abbyy Development Llc Method and apparatus for determining a document type of a digital document
US20120032986A1 (en) * 2007-05-29 2012-02-09 Research In Motion Limited System and method for resizing images prior to upload
US8873885B2 (en) * 2007-05-29 2014-10-28 Blackberry Limited System and method for resizing images prior to upload
US8972930B2 (en) 2010-06-04 2015-03-03 Microsoft Corporation Generating text manipulation programs using input-output examples
US20120011152A1 (en) * 2010-07-12 2012-01-12 Microsoft Corporation Generating Programs Based on Input-Output Examples Using Converter Modules
US9613115B2 (en) * 2010-07-12 2017-04-04 Microsoft Technology Licensing, Llc Generating programs based on input-output examples using converter modules
US9552335B2 (en) 2012-06-04 2017-01-24 Microsoft Technology Licensing, Llc Expedited techniques for generating string manipulation programs

Also Published As

Publication number Publication date
RU2006101908A (en) 2010-04-27
US20080263021A1 (en) 2008-10-23
US20080109403A1 (en) 2008-05-08
US8171391B2 (en) 2012-05-01

Similar Documents

Publication Publication Date Title
JP3653141B2 (en) An automatic method for selecting a key phrase for a processor from a machine-readable document
TWI524193B (en) Computer-readable media and computer-implemented method for semantic table of contents for search results
KR100411697B1 (en) Character reading method and the reading method Address
KR101145986B1 (en) Segmentation-based recognition
JP2004139553A (en) Document retrieval system and question answering system
US7584188B2 (en) System and method for searching and matching data having ideogrammatic content
US20070019864A1 (en) Image search system, image search method, and storage medium
EP1617340A2 (en) A method and apparatus for providing proper or partial proper name recognition
DE60204005T2 (en) Method and device for recognizing a handwritten pattern
US7756335B2 (en) Handwriting recognition using a graph of segmentation candidates and dictionary search
CN101611417B (en) Method for character recognition
US7373291B2 (en) Linguistic support for a recognizer of mathematical expressions
KR20120011010A (en) Handwriting recognition method and device
CN101419661B (en) Method for displaying image based on text in image and system
JPH11316800A (en) Method and device for character recognition
US5917941A (en) Character segmentation technique with integrated word search for handwriting recognition
CN101925904A (en) Document verification using dynamic document identification framework
Tomai et al. Transcript mapping for historic handwritten document images
JP4504702B2 (en) Document processing apparatus, document processing method, and document processing program
CN101925905A (en) Identification and verification of unknown document according to eigen image process
JP2003228581A (en) Method of retrieving similarity by compatibility feedback
JPH09134372A (en) Method and device for retrieving document
CN1237742A (en) Address reader, division machine such as mail thing and character string recognition method
CA2438187A1 (en) Holistic-analytical recognition of handwritten text
DE10342594A1 (en) Method and system for collecting data from a plurality of machine readable documents

Legal Events

Date Code Title Description
AS Assignment

Owner name: ABBYY SOFTWARE LTD, CYPRUS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZUEV, KONSTANTIN;TUGANBAEV, DIAR;FILIMINOVA, IRINA;REEL/FRAME:023528/0133

Effective date: 20091113

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION