JP2002537604A - Document similarity search - Google Patents

Abstract

(57) [Summary] Decompose query document 1 into elements of different data types (22) Search for data sources such as data bases that represent the contents on the walt-white web, including the steps, to find documents that are similar to the query document Method. Thereafter, for one or more elements of the first data type, a similarity search of the first data type is performed (23), and for one or more elements of the first data type, Returns the matching result from the data space of For one or more elements of the second data type, a similarity search of the second data type is performed, and one or more elements of the second data type are matched from the data surface. Returns the result. The matching result from the first data type is combined with an appropriate weight to provide a matching result of the query document. The data type can include the layout of the text, the features, the graphics, and the entire document.

Description

DETAILED DESCRIPTION OF THE INVENTION TECHNICAL FIELD OF THE INVENTION

The present invention relates to a method and apparatus for performing a search to find similar documents in response to a query, and more particularly to obtaining similar documents using a single document as a query for the search.

[Prior art]

Similarity search in a database of electronically stored documents
ity searching) is an important area in practical use. Such searches are well known for text. Typically, the input for such a search is a text string, and the search engine searches the database for a matching entry for the text string and returns an entry with an acceptable similarity threshold. A similar search can be used for images.
One example is the IBM Corporation's Query by Image Content (QBIC) package, which is described on the site http://wwwqbic.almaden.ibm.com/, and is available from the site. it can. In particular, the German Research Center for Artificial Intelligence GmbH (DFK
I) (German Artificial Intelligence Research Center) has been studying the use of structural analysis of documents when searching with systems such as Office Maid and SALT. These systems are described in more detail at the site http://www.dfki.uni-kl.de. If the query consists essentially of one data type (i.e., only text strings or only images), existing techniques will be effective. However, in general, an electronic document consists of a combination of data types, and a typical document is one or more text passages, one or more images, and line art ( Or line drawings). The text sections can also be easily broken down into different types such as headings, legends, and bulk text (ie, a group of texts). Using the existing technology as described above, the similarity search has the steps of extracting one element in a specific data type, and then performing a similarity search suitable for the data type. .

[Problems to be solved by the invention]

One example of such a series of approaches is described in US Pat. No. 6,002,798. This provides an initial structural analysis of the document into different types of regions (ie, not just images + text, but multiple regions with different functional meanings (eg, titles, headings, text blocks)). Is what you do. This structural information is then used to allow the user to search and text indexing on selected functional elements of the document. This mechanism is particularly useful for making the problem of text search in complex documents more manageable. However, it is not effective to enable searching for documents that are similar to the query document as a whole. It would be desirable to provide a method of similarity search that allows the features of a document that correctly represent a complete document to be properly used in the search.

[Means for Solving the Problems]

Therefore, in a first aspect of the present invention, a method for searching a database to find documents similar to a query document, comprising: decomposing the query document into elements of different data types; Performing a similarity search of a first data type for one or more of the elements, and matching results from a database for one or more elements of the first data type; Returns one of the elements in the second data type
Performing a similarity search of a second data type for one or more elements and returning a match result from a database for one or more elements of a second data type; A method is provided having the steps of combining matching results from a data type similarity search and the second data type similarity search to provide a matching result of a query document. Advantageously, combining each matching result of the query document allows the query to be refined using any data type, alone or in further combinations. In a second aspect of the present invention, a method for searching a database to find documents similar to a query document, comprising decomposing the query document into elements of different data types and determining a spatial arrangement of the elements in the document. A method is provided that includes determining a layout element in a layout data type, performing a layout similarity search on the layout element, and returning a matching result from a database for the layout element.

BEST MODE FOR CARRYING OUT THE INVENTION

Specific embodiments for the purpose of illustration of the invention are described below with reference to the drawings. A typical document contains multiple data types. The most basic data types are text and image. The document 1 shown in FIG. The text block 12 is data of the first data type. Document 1 also contains two different types of images. One type, image block 13, is a photographic image, which is typically composed of an array of pixels, each having one color value. The other type, the lineart block 11, is also an image, but a "drawn" image that can be easily represented as a combination of geometric or mathematical elements, and is generally easily scaled. It is something that can be done.
Photographic and line art images (hereinafter "pictures" and "graphics") exhibit different responses to different image processing and analysis techniques and are the ones most effectively processed as different data types. In addition, since pictures and graphics are generally used for different purposes in a document, processing pictures and graphics separately is also practical for similarity search purposes. Each step involved in the similarity search for the document of FIG. 1 according to the first embodiment of the present invention is shown in FIG. First, at step 21, document 1 is selected. In the case of an electronic document, this can be achieved by any suitable application capable of supporting one or more file types of the document. Also, in the case of a physical document, this can be achieved by scanning the document with a scanner. Second, at step 22, the document is broken down into discrete elements. That is, in the case of document 1, such elements are graphic block 11, text block 1
2 and picture block 13. In the case of text block 12, optical character recognition is performed at this point, and the resulting text block element is
It should be composed of ASCII text. Decomposition of the document is achieved by an analysis and recognition process, through which different parts of the document are recognized as text, pictures or graphics. Decomposition of documents into such distinct data types is known, for example, FM
"Block Segmentation and Text Extracti" by Wahl, KY Wong, and RG Casey
on in Mixed Text / Image Documents "(Computer Graphics and Image Processi
ng, Vol. 20 (1982)) (another example is disclosed in US Pat. No. 6,002,798). Software configured for use with proprietary scanners to decompose scanned page elements into separate data types (to optimize the scanning process for each data type) Hewlett-Packard Company offers the HP PrecisionScan. The output of this HP PrecisionScan is
A set of elements, each element having a single data type, each of which can be selected for further processing. The result of the decomposition is a set of elements, each of which has a single data type. All text is then determined to be part of a single element, depending on how the decomposition was performed for a particular data type, such as text, or
Physically distinct areas of text are considered separate elements. In one embodiment of the invention, all elements of the document are used in a similarity search, and in another embodiment,
One or more elements are selected for use in the similarity search (or the user is given the opportunity to select a portion of one element for such further processing). The separate elements are then used in a similarity search 23,24 against a database (eg, a database representing content available on the World Wide Web). If all elements consist of one data type, the problem is reduced to the problem of conventional similarity search that can be handled by a single search engine for related data types. However, if each element is of a different data type,
A separate search engine is used for each data type. Suitable search engines for similarity search for different data types are known. For example, for text, Teragram Corporation (http://www.teragram.com) and Inx
From ight Software, Inc. (http://www.inxight.com), select the appropriate language match (l
Inguistic matching) toolkits are available. In any case, it is desirable to perform a suitable pre-processing step 23 before the matching step 24, as will be described briefly below in relation to the main data types. For example, “Inxight Summarizer” is a software component technology that extracts a key sentence from a document to summarize the document. This is the pre-processing step 23. Then, such summaries can be matched against each other in a matching step 24. The Inxight Summarizer generates an indicative summary from the document that includes key sentence elements. Stemming and text normalization techniques extract the essence of the text and provide a concise and normalized summary of the text
(concise and canonical synopsis) are obtained. Here, "stemming" refers to replacing a word with its root and part of speech (for example, "I had wanted" or "to want" or "to want"). first per
son (first person) / pluperfect (past completed) ", and" normalization "involves replacement of one of several forms with" concept "(for example, , “2/3/99”, “Feb 3 ^nd , 1999” and “3 ^nd February” are all alternative forms of the same concept). A matching step 24 is then performed on the stemmed and normalized result of pre-processing step 23 with the confidence that exactly the same text content will be matched without the adverse consequences of considering the undesirable grammar. Can be performed. One example of an image search tool is the IBM QBIC package, as described above.
QBIC is further described at http://wwwqbic.almaden.ibm.com. The package is configured to preprocess the image by analyzing a number of different criteria, such as the percentage of colors occurring in the image, the layout of the colors, and the texture. The combination of the criteria is then used in matching step 24. From robot vision (robots searching for parts in containers) to traffic monitoring systems (automatic detection of car license plates), "exploring""new" images of known objects "and many other known uses Exists. This matching problem is essentially the exact opposite of these known problems. It can also be appreciated that the sequential method can be used effectively. For example, first, a "straight edge" histogram is used to allow the distinction between natural and artificial scenes, and then an "edge length" histogram (if there are few long edges) Is likely to indicate a natural scene), tests on a wide blue tonal region (indicating an outdoor scene) at the top of the image, and tests for significant elements of flesh tones. Note that the color tone of the skin color indicates that there is an image including a human expression, and thereby, the same face can be found by face matching analysis. Obviously, a combination of sequential and parallel steps can be employed. The result of the similarity search is a set consisting of a series of matching scores for documents in the database (such as a set that exists for each element searched).
It is. Each of these search scores needs to be normalized 25 with respect to combination 26 to obtain combined search results 27. The normalization step 25 is to ensure that the results of the different search steps 24 are in correct equilibrium. This involves weighting each element of the document equally, or weighting each element of the document according to the perceived importance in the document or according to a user's assessment of the relative importance of the different elements of the document. It is possible to use any one of the following. The preferred solution can involve both automatic and manual weighting. A particularly effective method is to generate a set of text search criteria using outline generation techniques for text parts, and to provide a set of possible criteria based on non-text parts. These criteria are then provided to the user for verification. Such a user-based method is easy to use (it is also easy for the user to tell when the method is invalid). For example, regardless of whether the user desires “this person”, “this scene”, “picture including this object”. A question can be given as to whether one wants to search for a match with the summary or to search for images and drawing parts. The combined result 27 is, for a conventional similarity search, a series of match scores (typically expressed as percentages) that list the documents in the database from the best match to the worst match. In general, by using the results of a first query as a basis for constructing the next query, the most effective user queries will be achieved, in which case the user will have a continuous refinement of the user queries (refinement), and indeed, the combined result 27 is frequently fed back to subsequent selection steps, thus enabling an effective iterative search. Become. Information derived from page decomposition in the similarity search can be further used.
In addition to the separate elements obtained from the page decomposition (graphics 11, text blocks 12, and pictures 13), further information is provided in the composition of the different elements in the document. As shown in FIG. 3, a further output resulting from the page separation is a data type plan 31 that represents the document as linear art blocks, text blocks, and image blocks arranged sequentially in the vertical direction. Layout decomposition is described in U.S. Patent No. 6,002,798. However, the inventor has recognized that the data type plan itself can be used as a layout data type. This allows yet another element, the layout data type element, to be used in the search 32 of the database (if layout information is available in the database entry or can be derived from the database entry). The results of a similarity search for such layout elements can be combined with similarity searches for other elements as shown in FIG. In this case, the layout data type 31 results from the decomposition step 22 and is then used in a search step 32 which is equivalent and parallel to the search steps 23, 24 (then after the normalization step is performed, the step 26) In combination with results from other data types). In the second embodiment of the present invention, the similarity search is performed using only the layout data type. Each subsequent step is essentially similar to a conventional similarity search.
This is shown in FIG. 4 and elements common to the first embodiment of the present invention are shown in FIG.
The same reference numerals are used. Whether the layout similarity search is used on its own or as one of multiple elements in a combined search as described in the first embodiment of the present invention, Will be more powerful if a large number of different data types are used for and for all document types. Using a rule-based approach, especially in the case of formal workflow documents, different text blocks and entire documents
Specific functions can be assigned with relatively high reliability. For example, it is well known that a block of text at the top of a page and a handwritten text at the bottom are indicative of a letter, so that different spatial areas of the document can be represented by appropriate functional fields (address , Letter text, etc.). Similarly, tables and currency totals in a document can be identified as separate uses, and their presence causes the document to be in a different group (
Billing, price list, or invoice). Thus, the layout search can include matching with templates representing different workflow document types (this facilitates matching documents determined to be letters to other letters). A suitable mechanism is to normalize the layout for size, orientation, and skew, and then perform an "exclusive or" operation on the query elements and the layout records in the database. This is effective when all the records included have a common common format. The difficulty of this problem depends on the nature and type of documents considered for matching. If the "universe" of the document is well-established, tools can be used that can do the right job, such as classification and labeling, within that area (eg OfficeMaid by DFKI)
). What is needed in this case is a classification according to a set of conventions established for all types of documents available for consideration. Here, an "arrangement" is essentially a rule that does not need to be strictly followed. Therefore, a suitable approach to this problem is rule-based (most preferably using fuzzy rules). Neural network training is also an effective method to adopt. One skilled in the art will recognize how to adapt conventional fuzzy rules or neural network approaches for use in solving this problem. It will also be appreciated by those skilled in the art that modifications of the embodiments described above may be readily made without departing from the scope of the invention, which is defined in the claims.

[Brief description of the drawings]

FIG. 1 is an illustration showing a typical document page containing different data types.

FIG. 2 is a flowchart illustrating a method for performing a similarity search on the document illustrated in FIG. 1 according to a first embodiment of the present invention;

FIG. 3 is an illustration showing the document shown in FIG. 1 as a layout of a plurality of data types and showing search steps that can be used in a further embodiment of the method of the invention.

FIG. 4 is a flowchart illustrating a method for performing a similarity search on layout information according to a second embodiment of the present invention;

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Burns, Roland, John 94404, California, Foster City, Rock Harbor Lane 62 F-term (reference) 5B075 ND03 ND06 QM05 5L096 BA17 BA20 EA45 JA03 JA09

Claims (17)

[Claims] 1. A method for searching a database for documents similar to a query document, comprising: decomposing the query document into elements of a plurality of different data types; Performing a similarity search of a first data type for one or more elements and returning a match from a database for the one or more elements of the first data type; Performing a second data type similarity search on one or more of the elements of the second data type to determine the one or more elements of the second data type Returns a match result from the database for the similarity search of the first data type and the similarity search of the second data type. How to combine fruit, it provides a butt results of the query document comprises the steps of, searching the database to find a document similar to the query document. 2. The method of claim 1, wherein one of the plurality of data types represents text.
The method described in. 3. The method of claim 2, wherein the plurality of data types represent text, and wherein distinct ones of the plurality of data types represent different functional blocks of text. 4. The method of claim 1, wherein one of the plurality of data types represents a picture image.
A method according to any one of claims 1 to 3. 5. The method according to claim 1, wherein one of the plurality of data types is representative of a radical image. 6. The method of claim 1, wherein one of the plurality of data types is indicative of an arrangement of another data type in the document. 7. The method according to claim 1, wherein the step of performing the similarity search and returning the matching result is performed separately for a plurality of elements each having three or more data types. The method described in the section. 8. The method according to claim 1, wherein all features of a common data type in the document are treated as one element. 9. The method according to claim 1, wherein spatially different features of a common data type in the document are treated as separate elements. 10. The method according to claim 1, wherein an element is selectable or non-selectable by a user for the step of performing a similarity search. 11. The method according to claim 1, wherein the results of the similarity search for different elements are weighted and then combined. 12. The method of claim 11, wherein said weighting is selected by a user. 13. The method of claim 11, wherein the weighting is performed according to a determined meaning of each of the related elements in the document. 14. A method of searching a database to find documents similar to a query document, comprising: decomposing the query document into elements of different data types; Determining a layout element of a layout data type, performing a layout similarity search on the layout element, and returning a match result from the database for the layout element. How to explore the database to find. 15. The method of claim 14, wherein the layout similarity search comprises a search for templates representing different document types. 16. The method of claim 14, wherein the elements include elements of different data types representing different functional blocks of text. 17. The method according to claim 14, wherein the elements include elements of a data type representing an image.