JPH06348758A - Device and method for retrieving document information - Google Patents

Abstract

PURPOSE:To provide a device and method for retrieving document information to retrieve a desired document with information expressing the external appearance of the document as a retrieval key. CONSTITUTION:The specified part (such as a background part, character part, ruled line part of a table, drawing part or photograph part) of inputted image data is extracted (2), the feature (such as a size, position, kind or color) is discriminated, information expressing the external appearance of the document (such as the color of paper, paper quality, spot, color of characters, character type, character kind, writer, writing tool, font, amount of writing in margins, longitudinal and lateral lengths of documents, size/density of characters, kind of paper or positional relation of drawings or photographs) is extracted (6), and this information expressing the external appearance of the document is stored in a data storage part (4) taking correspondence with a document picture or one of recognition result of it. At the time of retrieval, the desired document is retrieved by using this information expressing the external appearance of the document.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document information retrieving apparatus and method for retrieving a desired document in an apparatus for filing a document image or a result of recognition processing of the document image.

[0002]

2. Description of the Related Art Conventionally, a filing apparatus has been provided which electronically realizes an alternative to a file of a paper document by reading a document image with a scanner and storing the image information. However, when a huge amount of documents are accumulated, it is good to file them, but I forget the keywords attached to the documents, and it takes a lot of effort to find the desired one from the huge documents. Therefore, such a filing device is very difficult to use.

Paper documents can be searched relatively easily from the external characteristics, for example, based on human memory such as "a document with coffee stains". Since such information is discarded as extra information at the time of processing, the conventional electronic filing device cannot search for a desired document by a natural way of human beings to remember, as in the case of paper.

Particularly, in an apparatus for filing a document image that has undergone processing such as document structure analysis and character recognition,
During the recognition process, noise such as spots is removed to increase the recognition rate. Then, the document displayed at the time of the search is after the recognition process is performed, and is different in appearance from the original document image that remains in human memory. Can not determine whether the document is.

[0005]

As described above, in the conventional electronic filing apparatus, it is difficult to search the filed document information depending on the appearance characteristics of the document, which is likely to remain in human memory. There was a point.

The present invention has been made in view of this point,
An object of the present invention is to provide a document information search device and method capable of searching a desired document by using the external characteristics of the document as a search key.

[0007]

SUMMARY OF THE INVENTION A document information retrieval apparatus and method according to the present invention recognizes the input document image or information indicating the external characteristics of a document extracted from the input document image. When the information indicating the external characteristics of the document is entered as a search key and stored in association with the processed one, the entered search key and the stored information are matched, and the matching result is obtained. According to the above, the corresponding document image or the recognition-processed document image is output.

The search using the information showing the external characteristic of the document can be used together with the search for designating the document name or the keyword. Information representing the appearance characteristics of a document is roughly classified into the following three types. (1) Information indicating the characteristics of the medium on which the document image has been fixed (for example, paper color, paper quality, paper type, etc.), (2) Information indicating the characteristics of the substance fixed on the medium as the document image information ( For example, the type of writing instrument, the presence or absence of stains, etc., and (3) information indicating the characteristics of the image of the document image information displayed on the medium (for example, margin amount, type of character, writer, character density, layout, etc.). Is.

[0009]

According to the present invention, information representing the external characteristics of a document is extracted from the input document image and stored in association with the document, so that the external appearance of the document that is likely to remain in the impression of a human is You can search using various features as search keys. Furthermore, since the information that represents the external characteristics of a document, which is the key to realizing such a natural search, is automatically extracted from the input document image, no special sensor is required, and the user is not required to do so. There is no extra burden.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic configuration diagram of the apparatus of this embodiment. A document, which is a document to be stored as a file, is input as image data from the image input unit 1 (for example, a scanner). Next, the specific portion extraction unit 2 based on the input image data, the components of the document (for example, the background portion of the document, the character portion, the ruled line portion of the table, the photograph portion, the illustration portion, the graph portion, etc. There are types).

On the other hand, the storage information generation unit 3 stores the input image data in the file as it is, stores the image data in the data storage unit 4 as it is, and performs some processing on the input image data. The image data is converted into a data format stored in a file and stored in the data storage unit 4. The image data may be subjected to document structure analysis or character recognition processing and stored in the data storage unit 4. The data stored in this way is called stored data.

The specific portion feature identifying unit 5 identifies the features (eg, form, position, size, shape, color, type, etc.) of the constituent elements extracted by the specific portion extracting unit 2. This feature identification may be performed on the entire input image in parallel with the extraction of the specific portion.

The derived information extraction unit 6 uses the analysis result of the specific portion extraction or the specific portion feature identification as the derived information, such as paper type, paper quality, stain, paper color, writing instrument type, writing ratio, document type. And so on.

Then, the constituent elements obtained by the specific portion extracting section 2 and the characteristics (called attribute information of the constituent elements) obtained by the specific portion characteristic identifying section 5 are added to the stored data together with other auxiliary information. Then, all of them are stored as a file in the data storage unit 4. Alternatively, the derivative information extraction unit 6
The derivation information and its attribute information obtained in step 1 are added to the stored data together with other attached information, and the whole is stored as a file in the data storage unit 4. In this way, the component information and derivative information added to the stored data are information representing the external characteristics of the document.

At the time of file search, the operator inputs information representing the external characteristics of the document as search data through the search data input unit 7. The search unit 8 compares and collates the component information and derivative information added to the stored data with the input search data, and outputs the stored data that matches these to the output unit 9 as a search result.

When the stored information generation unit 3 performs recognition processing on image data, noise removal or spot removal may be performed as preprocessing in order to improve recognition accuracy. The spot is the specific part feature identification section 5
Since it is the same as the noise or stain identified by, the preprocessing portion may be shared by both parties.

Further, the input image is color data,
Either gray (multi-value) data or binary data may be used.
Information about the component that is commensurate with each piece of data will be selected and its characteristics will be identified and stored.

The specific portion extracting unit 2, the specific portion feature identifying unit 5, and the derivative information extracting unit 6 will be described in detail below. First, the case of extracting the background portion will be described with reference to FIG. It is assumed that the input image data is represented by color information.

First, the input image data is color-separated by the color separation section 11 and the image data separated and extracted for each color is stored in each color image buffer 12. Only a specific color may be stored in the color image buffer. Color separation is based on the three primary colors of RGB or the three of lightness, saturation, and hue.
However, at the stage of storing in the color image buffer, each representative color (red, blue, yellow, green, purple, orange, indigo, white,
In addition to black, the colors used for documents may be light blue, pink, yellow green, etc.).
In principle, this color separation analyzes the color components included in each dot of the image data, determines which representative color the dot color belongs to, and determines the color image corresponding to the determined representative color. This is done by storing the dot information in the buffer.

Next, the dominant color among them is determined as the background color. That is, the total (total area) of the number of dots stored in each color image buffer is calculated by the calculating unit 13, and the color having the maximum total area is determined by the background color determining unit 14. Based on the color determined here, the background extraction unit 15 distinguishes and specifies the background in the input image from the other parts, and the background color is mainly focused on the information in the color image buffer. Extract the copy information from the color image buffer. At this time, other types of components obtained from the input image data are specified (extraction of character portions described later) in advance or at the same time, and the total area is maximized except for the parts identified as other types of components. If the portion of is extracted, the background portion can be extracted more accurately.

In addition, the background color is not determined based on the total area, but the input image data is encoded into a run expression for each color, and the background color is determined from the run length of each color portion and its distribution. You can also There is also a method of obtaining a connected region for each color and specifying the background portion from the size of the connected region, the average value of the areas of the connected regions, and their distribution.

The background color determined here is information representing the "paper color" of the document, and this information is
Through the “paper color” extraction unit 21 (derivative information extraction unit 6),
It is stored in the data storage unit 4 together with the stored data of the document. Further, it is also possible to extract the information indicating the “margin amount” of the document from the size of the extracted background portion and handle it in the same manner as above.

With respect to the background portion extracted here, the noise detecting portion 17 counts the number of points of another very small color included in the background,
By calculating the density of this per unit area, the degree of noise in the background color is obtained. This is information indicating the quality of the paper (“paper quality”), and this information is stored in the data storage unit 4 together with the stored data of the document via the “paper quality” extraction unit 19 (derivative information extraction unit 6). To be done. The “paper quality” may be stored as a numerical value of noise, but may be stored after being converted into information such as “plain paper”, “recycled paper”, “straw paper” by comparing with a predetermined threshold value. . The definition of “paper quality” may be determined by comprehensively considering the color, density, noise amount, etc. of the paper. It is also possible to provide the scanner with a mechanism for detecting the "paper thickness" and handle the information obtained from it in the same manner as described above.

Further, the background may be composed of a plurality of colors. At this time, for example, when there is a white color background part covering the entire area and a background part of a smaller part of another color, the background part other than white is another constituent element that is a "spot part". Is extracted by the "spot" portion extraction unit 16. "Blemish" is the part of the shape (outline) that is not linear and is not white.
You may decide as a part. When such a "spot" portion exists, information indicating the presence or absence of the "spot" is stored in the data storage unit 4 via the "spot" information extraction unit 20 (derivative information extraction unit 6). Store with. At this time, the size (total area) and position (center point or representative point) of the "spot" portion are determined by the size / position detection unit 18
Alternatively, the color may be detected by, and the color of the "spot" portion may be stored together with the "spot" information. These "blemishes" and "paper quality" can be extracted even if the input image is gray.

Next, the case of extracting the character portion will be described with reference to FIG. Input image data can be color or gray 2
Value can be used. First, the connected area extracting unit 31 extracts a connected area in which black pixels are solidified to some extent (in most cases, one character forms one connected area) from the input image data. Then, in the analysis unit 32, regarding the connected regions, the connected regions that are complicated, or the regions formed by merging the connected regions that are very close to each other, whether the arrangement is linear or not, their sizes are uniform. Is determined, whether the arrangement pitch is substantially constant, or whether a certain degree of certainty is obtained as a result of performing character recognition for them, and the like, and the character part extraction unit 33 It is determined whether or not the extracted connected area is specified as a character portion.

The characteristics of the character portion extracted here are identified as follows. First, when a color image is input, the character color detection unit 35 determines the color of the character portion. When a plurality of character parts of different colors are present in one document, the color is detected for each character part. Then, the information representing the detected color of the character is stored in the data storage unit 4 together with the stored data of the document. When there are a plurality of colors, information in which the position of the character part and the color are associated is stored.

The character type, which is one of the characteristics of the character portion, includes handwriting, printed characters, stamps, dot printing, font type, and the like. These types are the size of the character,
The result is stored in the data storage 4 after being identified by the color, the arrangement of the character string, the shape of the frame surrounding the character string, and the like. For example, attention is paid to the fact that most of the documents are memorized by human beings, whether most of the documents are written by hand or printed or printed out. At least one of the handwriting dictionary 40 and the type dictionary 41 is used to identify whether it is a handwriting type. For example, if character recognition is performed using the print dictionary 41, most of the documents will have low certainty if they are handwritten, and if they are print, almost certain confidence can be obtained. If the sum of all the characters of the certainty factor is higher than a predetermined threshold value, it is determined to be a print type, and if the total is low, it is determined to be a handwriting. This handwritten type information is stored in the data storage unit 4 together with the stored data of the document as character type information. It should be noted that whether the handwritten type is a handwritten type is determined without using a dictionary in the vertical and horizontal arrangements of the connected areas if the deviation is very small, and if it is different, it is determined to be a handwritten type. You can also do it.

If it is determined here that the handwriting is performed, the following processing can be further performed. That is, if the apparatus is used for personal use, it is important whether it is written by oneself or written by another person. Therefore, using the dictionary 45 having the handwritten character feature pattern of the owner of the apparatus, the writer identifying unit 44 determines whether or not the writer is the owner, based on the degree of certainty. If the dictionary 45 is held for a plurality of persons, the name of the writer of the document can be estimated. This writer information is stored in the data storage unit 4 together with the stored data of the document.

In the case of handwriting, it is also possible to specify the type of writing instrument. That is, the writing tool identification unit 46 writes with a pencil based on the blurring and density of the character line (image data needs to be gray), the line width detected by the character line thickness detection unit 36, and the like. It is determined whether the object is a ballpoint pen, a felt-tip pen, or the like. This writing instrument information is stored in the data storage unit 4 together with the stored data of the document. The thickness of the character line may be stored as it is as the writing instrument information. In addition, a detector for reflected light is separately prepared in the image input unit 1, a means for analyzing reflectance and spectral characteristics is added, and whether the substance fixed on the paper is a pencil lead or a ballpoint pen ink. It is also possible to specify the type of writing instrument or the distinction between the copy and the base paper by identifying the ink of the felt-tip pen, the toner of the copy, the ribbon of the printer, or the like.

When it is determined that the character is a print type, the font (Mincho, brush, Gothic, italic, etc.) may be further determined by the font identification unit 47 and stored in the data storage unit 4. It suffices that the information representing these characteristics is extracted for the characters that occupy most of the document.

When it is determined that the handwritten characters and the printed characters are mixed, the mixture ratio detecting unit 48 estimates the number of handwritten characters or the estimated total number of handwritten characters with respect to the total number of characters. It is also effective to calculate the ratio and store it in the data storage unit 4 as information indicating the amount of characters handwritten in the document. This information can also be obtained by calculating the ratio of the total area of the area where handwritten characters are present and the area where typed characters are present.

Character types that are another one of the characteristics of the character portion include numbers, letters, kana, and kanji. Here, paying attention to whether the document was written in English or Japanese as a feature that is likely to remain in human memory, and in the character type determination unit 38, the English dictionary 42 or the Kana-Kanji dictionary 43 is used. At least one of the above will be used to identify whether it is English or Japanese. For example, English letters (alphabet)
If character recognition is performed using the dictionary 42, many characters can be obtained with low confidence when the document is Japanese, and almost appropriate confidence is obtained when the document is English. If the total is higher than a predetermined threshold, it is determined to be English, and if it is low, it is determined to be Japanese. This language type information is stored in the data storage unit 4 together with the stored data of the document.
Further, it is possible to perform the same process for numbers and add information indicating that the document is a document in which numbers are listed, such as a form, to the information of the language type.

In addition, the pitch detection unit 39 detects a character pitch or a line pitch for the extracted character portion, and based on the result, the vertical / horizontal identification unit 49 determines whether the document is in vertical writing or in horizontal writing. Identify if there is. There can be four states. The first is, for example, A4 paper placed vertically and horizontally, the second is the paper horizontally placed and vertically written, and the third is the paper vertically placed. Vertical writing and the fourth is horizontal writing with the paper placed horizontally. Therefore, if the pitch in the horizontal direction is smaller than the pitch in the vertical direction, it is determined to be 1 or 2, and if it is the opposite, it is determined to be 3 or 4. Furthermore, character recognition assuming that the characters are written so that they can be read as they are placed on the paper and character recognition assuming that the characters are written so that they can be read when the orientation of the paper is changed to a right angle direction By doing both, and comparing the results,
A distinction is made between 1 and 2, or a distinction between 3 and 4. The information indicating which of the four states the data storage unit 4 has.
Is stored with the stored data of the document.

The character size / density detecting section 50 can also determine the character size and density. Also in this case, the numerical values indicating the size and the density are directly stored in the data storage unit 4.
Instead of storing it in, the information may be stored after being converted into information such as “fine characters / closely packed”, “large characters / slightly shaded”.

In the above example, the process of extracting the "paper quality" or "spot" part from the background part has been described, but when performing character recognition on the character part, as a pre-process in order to increase the normal recognition rate. By performing recognition on the data as it is without performing normalization, etc., a portion where the recognition rate is aggregated and is worse than a predetermined threshold value is specified as a "spot" portion, or the recognition rate is one. If it is bad for the entire paper, it can be specified as "poor quality paper".

The case of extracting other specific parts (components) will be described below with reference to FIG. In the extraction of the ruled line portion of the table, the straight line / curve detecting unit 61 detects many straight lines, the intersection detecting unit 62 detects many intersections where the straight lines are orthogonal to each other, and the analyzing unit 63 detects the position of each straight line. This is performed by extracting the area determined to have the same length as the ruled line portion of the table.

After that, when the paper type determining unit 71 determines that straight lines arranged at a constant pitch exist on the entire surface of the paper, it is determined that the paper type is ruled report paper or notepaper. Further, it is possible to determine whether the paper has vertical ruled lines or horizontal ruled lines based on the linear arrangement. In addition, the line type, color, and mark (entering the company name, etc.) of frequently used paper sheets are registered in the paper type dictionary 72, and matching is made with the extracted straight line groups, etc. It can be specified as in “report form” “part A section submission form”. The information indicating the type of the sheet and the information indicating the position / size of the ruled line portion of the table are stored in the data storage unit 4 together with the storage data of the document.

Extraction of the drawing portion is performed by detecting many straight lines and curves, detecting many intersections thereof, and extracting areas that are not regarded as ruled line portions of the table.

The photograph portion can be extracted by using an image area separation technique known as an image processing technique. The photograph portion includes a gravure photograph portion in which the gradation of the image changes smoothly, and a halftone photograph portion in which black dots whose size varies depending on the image portion are arranged. Further, by analyzing the color of the photograph portion, it is possible to determine whether the photograph is a color photograph or a monochrome photograph.

The extraction of the graph portion is realized by using techniques such as circle extraction, rectangle extraction, line segment extraction and the like which are usually used in drawing recognition. It is possible to specify whether the drawing is a graph or another drawing by performing these extraction processes only on the drawing portion extracted as described above. The graph section includes a bar graph, a pie graph, and a line graph.

The background thus extracted,
Components such as letters, drawings, photographs, and graphs, including their position and size, attribute information such as type, and derivative information,
It is stored in the data storage unit 4 together with the stored data of the document. At this time, instead of storing the position and size itself, a positional relationship such as “the photograph part is in the upper right and the graph part is in the lower left” is stored via the positional relationship / ratio detection unit 73 of each component. It is also effective to convert and store the information such as the above information or information having a ratio such as "drawings occupy 60% of the whole" and "drawings and characters exist in a ratio of 1: 2".

As another component, a mark or color existing in a predesignated place can be considered. That is, it is detected whether or not a specific mark or color exists at the designated place, and this information is stored in the data storage unit 4 together with the stored data of the document. For example, if you want to check a document that the user thinks is important with a red pen in the upper right corner, detect whether a red color exists in the upper right corner of the input image,
It is effective to add it to the stored data as information indicating whether it is an important document. Instead of specifying the location, the entire screen may be searched to find the specific mark or color.

FIG. 5 shows how the above-described information is stored. Numerical data or codes corresponding to names are assigned to each component, derivative information, and their attribute information. As for the constituent elements, as shown in the left half of the figure, an attribute name and an attribute value, which is the value thereof, are paired, and this is called an attribute set. The attribute set (plurality) and the component are paired and stored in, for example, a tabular format. As for the derivative information, as shown in the right half of the figure, the derivative information and its attribute value are stored as a set. Even if only one of these is stored, the effect of the present invention can be obtained. Also, FIG. 5 is an example,
It is not necessary to store all this information.

Tabular information about the above components,
The information represented by the combination of the derived information and its attribute value may be stored in a place different from the stored data (the document image and the recognition result), for example, in the directory part, or added to the header part of the stored data. And store it. It is better to store it separately
When searching using these pieces of information, only the directory part is searched and the stored data is read out only for the matched data, so the search speed is increased.

Further, the types of attribute names, the types of derivation information, and the types of attribute values that can be defined by the derivation information, which are included in each component, are set in advance. That is, for example, in the case of the drawing part, there are three types of attribute names, that is, color, size, and position. Then, which attribute name of which component is to be assigned to which (memory address) in the table is determined in advance. Then, the attribute value obtained by the specific portion extraction unit 2 or the specific portion feature identification unit 5 is written in the corresponding attribute name. If extraction or identification fails, or if the scanner is not a color and a color cannot be obtained, NULL is written at the attribute name that could not be obtained.

Regarding the derivative information, it is determined in advance which derivative information should be assigned to which storage position in the memory. The attribute values that can be defined for each piece of derived information are also determined in advance, for example, three types of large / medium / small for margins and two types of handwriting / printing for character types. If the attribute value defined for each piece of derived information is stored in the form of a table, it may be convenient for a search described later. The attribute value calculated by the derivative information extraction unit 6 is selected from the predefined ones, and the calculated attribute value is written in the corresponding derivative information. N for derived information that could not be obtained
Write ULL.

An example of the operation at the time of retrieval is shown in FIG. 6 (a).
Input from the search data input unit 7 in natural language, for example, "a document written by myself on a pink paper and having a stain on coffee". Then, the search information extraction unit 8
1 is called "pink", "myself", or "spot" from the above search data, using the information in the storage unit 82 that stores a table in which each derivative information is associated with the attribute values defined in advance. The word that is the basis of the search information is extracted, and three search information of “paper color: pink”, “writer: myself”, and “stain: present” are obtained. Then, the search information comparing and collating unit 83
Is an attribute set including the obtained search information items (such as "paper color") (a pair of derived information and attribute values as shown in the right half of Fig. 5)
From among the data stored in the data storage unit 4 in association with each stored data (document), compare and collate this attribute value with that of the search information (“pink”, etc.), and find the documents that match these. It is selected and output to the document presentation unit 85.

In determining whether or not they match, the similarity between the attribute values is defined (eg 100% similarity for perfect matching, 80% similarity for pink and red, 0 similarity for white and black). %, If the stored attribute value is NULL, the degree of similarity is 50% (indicating that it cannot be determined), etc.), even if all of the search information obtained by the search information extraction unit 81 is not a complete match, each search The document may be selected if the value obtained by summing the similarities as a result of comparison and collation of information in all search information is larger than a predetermined threshold value. Also,
Some of the derived information is such that it is possible to extract attribute values with a certain degree of certainty, such as "paper color," and ambiguous in determining whether you are yourself or another person, such as "scribe." There are some things that often remain. Therefore, a weight is set in advance for each piece of derived information, and when the similarity is summed, the degree of similarity for certain derived information is emphasized, and the degree of similarity for ambiguous derived information is set to a reference level. A weighted sum may be performed. This weight may not be set in advance but may be determined by also obtaining the certainty factor at the time of extraction by the derivative information extraction unit 6.

It is also effective to use a keyword search in an ordinary filing apparatus together. In other words, keywords that represent the contents of the document are added to the document data automatically or manually at the time of file, and if you can remember this keyword at the time of search, first narrow down the number of searches by keyword and then search using the derived information as described above. I do.
In this way, when adding a keyword, it is not necessary to pay attention to whether or not the keyword is unique, and the burden on the user can be reduced. In addition, the information "about what month was filed" is taken out by the clock function provided in the filing device and added to the document data, and the information related to this time and the derived information as described above are used in combination to search. Can improve the search accuracy.

Although the search using the derivative information has been described above, the search using the component and its attribute information can be performed as follows. In this case, since the attribute value is stored closer to the raw data than in the case of the derived information, the storage unit 82 stores each component, the attribute name of the component, and the information name represented by the set. deep. Then, for example, if the search data "the paper color is pink, the stain is large, the stain size is large, and the stain position is in the upper right corner" is input, the information name in the storage unit 82 is matched. While extracting, extract the items of search information such as “paper color”, “size of stains” and “position of stains”, and enter the search data “pink”, “large” and “upper right” written immediately after each item. Extract and
Search information is set as a set.

Furthermore, using the information in the storage unit 82, the information name "paper color" is changed to "background color".
After conversion into the component and attribute name, the search information is compared and collated with the set of component and attribute information as shown in the left half of the data storage unit 4 in FIG.
At this time, first, the data storage unit 4 is searched for an attribute set including the item of the search information "background color", and this attribute value and that of the search information ("pink") are searched.
Etc.) and collate, and select the documents that match these. In the search information, a rough expression such as "large" is used, but for example, for "size of stain", the numerical value "1
"10" is "small", "11-20" is "medium", "21-3"
By storing a correspondence such as “0” is “large” in advance, if it is “large”, it is converted into a numerical value of “21 to 30” and compared and collated with the attribute value of the data storage unit 4. . In this case, since the numerical values are compared and collated, the similarity calculation can be easily performed.

It should be noted that it is possible to perform a search using both the derived information and the attribute information of the constituent elements. In particular, the derived information contains "Stain: Yes"
If there is detailed information such as the color and size of "spot" in the attribute information of the component,
First, looking at the derived information added to the document data from the search data "spotted", select the documents (spots: yes), and then select the attribute name of the corresponding component "spot". The user inputs the continuation of the search data, such as "color" and "size", and the user inputs "brown" for "color" and "large" for "size", and narrows down the attribute information of the constituent elements. . Also, in the derivative information, if the "character type" is "printing", such as "character type" and "font" or "writer", there may be derivative information such as "font". If so, there is a hierarchical structure such that there can be derived information called "writer", and the interactive narrowing down described above is also possible here.

FIG. 6B shows another example of the search operation. First, the derivation information item display unit 86 displays derivation information that can be extracted by the derivation information extraction unit 6 together with attribute values (plurality) defined in advance for the derivation information, as indicated by 100 in the figure. The user looks at this and instructs the derived data by the search data input unit 7, remembering that the "paper color" of the desired document was "pink" and the "margin" was medium. I will do it. If you can't remember it, you do not need to enter that item, because the comparison and collation will be performed later, excluding that item. With respect to the search data input in this way, the search information comparing and collating unit 83
As in the case of FIG. 6A, the matching document is selected and presented.

The above is the embodiment in which the attribute information and the derivative information of the constituent elements are extracted from the document image to be filed, and the retrieval is performed by using the extracted information, but it is also effective to flip these. In other words, in a system that allows a user to select a desired document by presenting stored data (documents) while flipping through, the attribute information and derivative information of the constituent elements that are stored in association with the presented document are developed into an image. . For example, the image information of "spots" is superimposed and displayed on the image of the document according to the stored color and size information of "spots". This makes it easy for the user to see at first glance because there is no familiar noise in the document presented at the time of flipping, especially when the input data is stored with the noise removed from the input document image. It is possible to improve usability by not being unable to determine whether or not the document is a desired document.

Also, the extracted derivative information and attribute information (displayed as, for example, paper color: pink) are extracted together with the original document image or the stored data, and the user can modify and add the derivative information and attribute information. You may. in this case,
If the user corrects the color of the paper: white, adds the information that the writing amount is large, and deletes the information that the stain is present, the derived information corrected in this way is associated with the relevant document. The data is stored in the data storage unit 4.

[0056]

As described in detail above, according to the present invention, information representing the appearance of a document automatically extracted from an input image (paper color, paper quality, stain, character color, character type, character type, The desired document can be searched using the writer, writing instrument, font, amount of writing in the margins, vertical and horizontal dimensions of the document, character size and density, paper type, positional relationship between drawings and photos, etc. Even if the content or keyword of the document is not clearly remembered, the retrieval can be realized by remembering the peripheral information of the document.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of a device of this embodiment.

FIG. 2 is a diagram showing a processing example when a background portion is extracted by the device of this embodiment.

FIG. 3 is a diagram showing a processing example when a character portion is extracted by the device of this embodiment.

FIG. 4 is a diagram showing a processing example when a ruled line portion, a drawing portion, a photograph portion, and a graph portion of a table are extracted by the device of this embodiment.

FIG. 5 is a diagram showing a format example of component information and derivative information stored in a data storage unit 4.

FIG. 6 is a diagram showing a configuration for searching in the device of this embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image input part, 2 ... Specific part extraction part, 3 ... Storage information generation part, 4 ... Data storage part, 5 ... Specific part feature identification part, 6
Derived information extraction unit, 7 ... Search data input unit, 8 ... Search unit, 9 ... Output unit, 11 ... Color separation unit, 12 ... Color image buffer, 13 ... Total area calculation unit, 14 ... Background color determination unit, 15 ... Background extraction unit, 16 ...
"Blemish" portion extraction unit, 17 ... Noise detection unit, 18 ... Size / position detection unit, 19 ... "Paper quality" extraction unit, 20 ... "Blemish" information extraction unit, 21 ... "Paper color" extraction unit, 31 ... connected area extraction unit, 32 ... analysis unit, 33 ... character portion extraction unit, 34
Image buffer, 35 ... Character color detection unit, 36 ... Character line thickness detection unit, 37 ... Character type determination unit, 38 ... Character type determination unit, 39 ... Pitch detection unit, 40 ... Handwriting dictionary, 41 ...
Type dictionary, 42 ... English dictionary, 43 ... Kana-Kanji dictionary, 44
... writer identification section, 45 ... owner handwriting dictionary, 46 ... writing instrument identification section, 47 ... font identification section, 48 ... mixture rate detection section,
49 ... Vertical / horizontal identification section, 50 ... Character size / density detection section,
61 ... Straight line / curve detection unit, 62 ... Intersection detection unit, 63 ...
65 ... Analysis part, 64 ... Table ruled line part extraction part, 66 ... Drawing part extraction part, 67 ... Image area separation part, 68 ... Photo part extraction part, 69
... Circle / rectangle / line segment extraction unit, 70 ... Graph part extraction unit, 71
... Paper type determination unit, 72 ... Paper type dictionary, 73 ... Positional relationship / ratio detection unit of each component, 81 ... Search information extraction unit,
82 ... Derivation information / attribute value correspondence table storage unit, 83 ... Search information comparison / collation unit, 84 ... Similarity / weight storage unit, 85 ... Document presentation unit, 86 ... Derivation information item display unit

Claims (5)

[Claims] 1. A storage unit for storing an input document image or a result of processing the document image as a document, a designation unit for designating the stored document by a document name or a keyword, and a designated document. In a document information search device comprising an output means for outputting, a means for extracting information representing an external characteristic of a document from a document image input by the input means, and the information extracted by this means in the storage means. Means for storing the document in association with the stored document, and, in addition to the designation of the document by the designation means or in addition to the designation of the document by the designation means, information indicating an external characteristic of a desired document. A means for determining a document to be output by inputting as a search key, collating the input search key with the information stored in association with the document, and determining the document to be output Document information retrieval apparatus characterized by. 2. A step of storing, as a document, an input document image or a result of processing the document image, and information indicating the external characteristics of the document extracted from the input document image, in the document. The step of storing in association with each other, the step of inputting information indicating the external characteristics of a desired document as a search key, the step of matching the input search key with the stored information, and the matching result And outputting a corresponding document from the documents stored according to the document information retrieval method. 3. A means for inputting a document image fixed on a medium, a means for storing the input document image or a result obtained by processing the document image as a document, and characteristics of the medium based on the input document image. Means for extracting the information representing the information and storing this information in association with the document, and means for inputting the information indicating on which medium the desired document image has been fixed as a search key, A document information search apparatus comprising: a means for matching a search key with the stored information; and a means for outputting a corresponding document from the stored documents according to the matching result. 4. A means for inputting a document image fixed on a medium, a means for storing the input document image or a result of processing the document image as a document, and fixing the input document image on the medium. Extracting information representing the characteristics of the substance, storing the information in association with the document, and inputting, as a search key, information indicating which substance fixed the desired document image on the medium. Means for collating the inputted search key with the stored information, and means for outputting a corresponding document from the documents stored according to the collation result. Document information retrieval device. 5. A means for inputting a document image fixed on a medium, a means for storing the input document image or a result obtained by processing the document image as a document, and the input document image on the medium. A means for extracting information representing characteristics of the represented information as an image, storing the information in association with the document, and information indicating what kind of image the desired document image is represented on the medium. Means for inputting as a search key, means for collating the input search key with the stored information, and means for outputting the corresponding document from the stored documents according to the collation result. A document information retrieval device characterized by the above.