JP3142986B2 - Document information retrieval device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document information retrieving apparatus and method for retrieving a desired document in a device for filing a document image or the result of recognizing the document image.

[0002]

2. Description of the Related Art Conventionally, there has been provided a filing apparatus which reads a document image by a scanner and accumulates the image information, thereby electronically realizing an alternative to a paper document file. However, when an enormous amount of documents is accumulated, it is good to file, but forgets the keywords attached to the documents, and a great deal of effort is required to find the desired one from the vast amount of documents. However, such a filing apparatus is very difficult to use.

[0003] A paper document can be found relatively easily on the basis of human characteristics such as a "document with a coffee stain" on the basis of its appearance characteristics. In such a case, such information is discarded as redundant information, so that a conventional electronic filing apparatus cannot search for a desired document in a natural way of remembering a person as in the case of paper.

[0004] In particular, in an apparatus for filing a document image that has been subjected to processing such as document structure analysis and character recognition,
At the time of the recognition processing, noise such as blots for increasing the recognition rate is removed. Then, the document displayed at the time of the search is the one after the recognition process is performed, and the appearance of the document image is different from the original document image remaining in the memory of the human. Document cannot be determined.

[0005]

As described above, the conventional electronic filing apparatus has a problem that filed document information cannot be searched depending on the appearance characteristics of the document, which are likely to remain in human memory. There was a point.

[0006] The present invention has been made in view of this point,
It is an object of the present invention to provide a document information search apparatus and a method capable of searching for a desired document by using the appearance characteristics of the document as a search key.

[0007]

SUMMARY OF THE INVENTION According to the present invention, there is provided a document information retrieval apparatus and method according to the present invention, in which information representing appearance characteristics of a document extracted from an input document image is input or the document image is recognized. When information indicating the appearance characteristics of the document is input as a search key, the input search key is compared with the stored information, and the matching result is stored. And outputs the corresponding document image or the one subjected to the recognition processing.

[0008] The search using the information representing the appearance characteristics of the document can be used together with the search for specifying a document name or a keyword. Information representing the appearance characteristics of a document is roughly classified into the following three types. (1) information indicating the characteristics of the medium on which the document image has been fixed (for example, paper color, paper quality, paper type, etc.); and (2) information indicating the characteristics of the substance fixed on the medium as the document image information ( (For example, the type of writing implement, the presence or absence of a stain), (3) information representing the image characteristics of the document image information represented on the medium (for example, the amount of blank space, type of character, scribe, character density, layout, etc.) It is.

[0009]

According to the present invention, information representing the appearance characteristics of a document is extracted from an input document image, and the extracted information is stored in association with the document. Search can be performed using various features as search keys. Furthermore, since information representing the appearance characteristics of a document, which is the key to realizing such a natural search, is automatically extracted from an input document image, a special sensor is not required, and the user is not required. There is no extra burden.

[0010]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic configuration diagram of the apparatus of the present embodiment. A document of a document to be stored as a file is input as image data from the image input unit 1 (for example, a scanner). Next, the specific part extracting unit 2 determines the components of the document (for example, the background portion of the document, the character portion, the ruled portion of the table, the photograph portion, the illustration portion, the graph portion, etc.) based on the input image data. Type).

On the other hand, when storing the input image data in a file as it is, the storage information generating unit 3 stores the image data in the data storage unit 4 as it is and performs some processing on the input image data. , The image data is converted into a data format stored in a file and stored in the data storage unit 4. The image data may be subjected to document structure analysis or character recognition processing and stored in the data storage unit 4. The data stored in this way is called stored data.

The specific part feature identifying section 5 identifies the features (for example, form, position, size, shape, color, type, etc.) of the components extracted by the specific part extracting section 2. This feature identification may be performed in parallel with the specific portion extraction for the entire input image.

The derived information extraction unit 6 analyzes the results of the specific part extraction or the specific part feature identification as derived information such as paper type, paper quality, stain, paper color, writing instrument type, writing ratio, and document type. Etc. are determined.

The components obtained by the specific part extracting unit 2 and the features (called attribute information of the components) obtained by the specific part feature identifying unit 5 are added to the stored data together with other attached information. All of them are stored in the data storage unit 4 as files. Alternatively, the derived information extracting unit 6
The derived information and the attribute information obtained in the above are added to the storage data together with other attached information, and the whole is stored in the data storage unit 4 as a file. The component information and derivative information added to the stored data as described above are information indicating the appearance characteristics of the document.

At the time of file search, information representing the appearance characteristics of the document is input as search data by the operator via the search data input unit 7. The search unit 8 performs comparison and collation of the information on the constituent elements and the derivative information added to the stored data with the input search data, and outputs the stored data that matches these to the output unit 9 as a search result.

When performing recognition processing on image data in the storage information generating unit 3, noise removal or stain removal may be performed as preprocessing to improve recognition accuracy. The stain is a specific partial feature identification unit 5
Since this is the same as the noise or the stain identified by, this preprocessing part may be shared by both.

The input image includes color (color) data,
Either gray (multi-valued) data or binary data may be used.
The information on the components corresponding to each data is selected, and its characteristics are identified and stored.

Hereinafter, the specific part extracting unit 2, the specific part feature identifying unit 5, and the derived information extracting unit 6 will be described in detail. First, a case of extracting a background portion will be described with reference to FIG. It is assumed that input image data is represented by color information.

First, color separation is performed on input image data by a color separation unit 11, and image data separated and extracted for each color is stored in each color image buffer 12. Only a specific color may be stored in the color image buffer. Separation of colors is performed by three primary colors of RGB or three of lightness, saturation, and hue.
It is performed using elements, but at the stage of storing in the color image buffer, each representative color (red, blue, yellow, green, purple, orange, indigo, white,
In addition to black, light blue, pink, yellow green, and the like may be set as colors used for documents.)
This color separation, in principle, analyzes the color components contained in each dot of the image data, determines which representative color the color of the dot belongs to, and determines the color image corresponding to the determined representative color. This is performed by storing the dot information in a buffer.

Next, the dominant color among them is determined as the background color. That is, the total (total area) of the number of dots stored in each color image buffer is calculated by the calculation unit 13, and the color having the maximum total area is determined by the background color determination unit 14. Based on the color determined here, the background part extraction unit 15 distinguishes and identifies the background part in the input image from other parts, and focuses on the background in the color image buffer of the background color. The unit information is extracted from the color image buffer. At this time, the other types of components obtained from the input image data are specified (extraction of a character portion or the like described later) in advance or at the same time, and the total area other than the portion specified as the other types of components is maximized. If the portion is extracted, the background portion can be more accurately extracted.

Further, instead of determining the background color based on the total area, the input image data is encoded into a run expression for each color, and the background color is determined from the run length of each color portion and its distribution. Can also. There is also a method in which a connected region for each color is obtained, and the background portion is specified from the size of the connected region, the average value of the area of the connected region, and their distribution.

The background color determined here is information representing the "paper color" of the document.
Via the “paper color” extraction unit 21 (derived information extraction unit 6),
The data is stored in the data storage unit 4 together with the storage data of the document. Further, information representing the “margin amount” of the document can be extracted from the extracted size of the background portion, and can be handled in the same manner as described above.

Further, with respect to the background portion extracted here, the noise detection unit 17 counts the number of very small dots of another color included in the background,
By calculating its density per unit area, the degree of noise in the background color is obtained. This is information representing the quality of the paper (“paper quality”), and this information is stored in the data storage unit 4 via the “paper quality” extraction unit 19 (derived information extraction unit 6) together with the storage data of the document. Is done. “Paper quality” may be stored as a numerical value of noise, or may be stored by being converted to information such as “plain paper”, “recycled paper”, and “straw half paper” by comparing with a predetermined threshold value. . The definition of “paper quality” may be determined by integrating the color, density, noise amount, and the like of the paper. Further, a mechanism for detecting “paper thickness” may be provided in the scanner, and information obtained from the mechanism may be handled in the same manner as described above.

Further, the background may be composed of a plurality of colors. At this time, for example, if there is a white background portion that covers the entire area and a smaller background portion of another color, the background component other than white is another component called a “stain portion”. Is extracted by the “stain” portion extraction unit 16. Colors other than white with non-linear shapes (contours)
It may be determined as a part. When such a “stain” portion exists, information indicating the presence or absence of the “stain” is stored in the data storage unit 4 via the “stain” information extraction unit 20 (derived information extraction unit 6). Store with At this time, the size (total area) and position (center point or representative point) of the “stain” portion are determined by the size / position detection unit 18.
And may be stored together with the color of the “stain” portion in the “stain” information. The extraction of “stain” and “paper quality” can be performed even when the input image is gray.

Next, a case of extracting a character portion will be described with reference to FIG. Input image data can be color or gray 2
It can be a value. First, the connected area extracting unit 31 extracts, from the input image data, a connected area in which a certain amount of black pixels are present (in most cases, one character constitutes one connected area). Then, the analysis unit 32 determines whether or not the arrangement of the connected regions, the intricate connected regions, or the regions formed by merging the close connected regions is linear, whether or not the arrangement is linear. Is determined, whether or not the arrangement pitch is substantially constant, or whether or not a reasonable degree of certainty is obtained as a result of performing character recognition on them, and the like. It is determined whether or not the extracted connected region is specified as a character portion.

The features of the character portion extracted here are identified as follows. First, when a color image is input, the color of the character portion is determined by the character color detection unit 35. When a plurality of character portions of different colors exist in one document, the color is detected for each character portion. Then, the information indicating the color of the detected character is stored in the data storage unit 4 together with the storage data of the document. When there are a plurality of colors, information that associates the position of the character portion with the color is stored.

The character type, which is one of the features of the character portion, includes handwriting, printed characters, stamps, dot printing, font types, and the like. These types include the size of the characters,
The result is identified by the color, the arrangement of the character strings, the shape of the frame surrounding the character strings, and the like, and the result is stored in the data storage 4. For example, attention is paid to the fact that most of the documents are written in handwritten characters or printed or printed in printed characters as features that are likely to remain in human memory. It is assumed that at least one of the handwritten dictionary 40 and the type dictionary 41 is used to determine whether the type is handwritten. For example, if character recognition is performed using the type dictionary 41, if the majority of the document is handwritten, many characters can only have low confidence, and if it is type, almost reasonable confidence can be obtained. If the sum of all the characters of the certainty factor is higher than a predetermined threshold value, it is determined that the character is a print type. The information of handwritten type is stored in the data storage unit 4 as character type information together with the storage data of the document. It should be noted that whether a handwritten type is used without using a dictionary, if the deviation is extremely small in the vertical and horizontal directions of the connected area, it is determined to be a typeface, and if it is not the same, it is determined to be a handwritten type. You can also.

If it is determined that the handwriting is performed, further processing can be performed. That is, if this apparatus is used personally, it is important whether it is written by oneself or another person. Thus, using the dictionary 45 having the characteristic pattern of the handwritten character of the owner of the apparatus, the writer identification unit 44 determines whether or not the writer is the owner based on the degree of certainty. If the dictionary 45 is provided for a plurality of persons, the name of the writer of the document can be estimated. The writer information is stored in the data storage unit 4 together with the storage data of the document.

In the case of handwriting, the type of writing implement can be specified. In other words, the writing implement identification unit 46 uses a pencil based on the blurring and density of the character line (the image data must be gray), the line width detected by the character line thickness detection unit 36, and the like. It is determined whether the object has been written with a ballpoint pen or a felt-tip pen. The writing instrument information is stored in the data storage unit 4 together with the storage data of the document. The thickness of the character line may be directly stored as writing instrument information. It should be noted that a separate detector for reflected light is prepared in the image input unit 1 and a means for analyzing the reflectance and spectral characteristics is added, so that the substance fixed on the paper is a pencil lead or a ballpoint pen ink. By distinguishing between ink of a felt-tip pen, toner of a copy, and ribbon of a printer, it is also possible to specify the type of writing implement or the distinction between copy and base paper.

If it is determined that the font is a print type, the font (Mincho, writing brush, Gothic, italic, etc.) may be further determined by the font identification unit 47 and stored in the data storage unit 4. It is sufficient that information representing these characteristics be extracted for characters that occupy the majority of the document.

If it is determined that the handwritten characters and the printed characters are mixed, the mixture ratio detecting unit 48 detects the estimated number of handwritten characters or the estimated number of handwritten characters with respect to the estimated total number of characters. It is also effective to calculate the ratio and store it in the data storage unit 4 as information indicating the amount of characters written by hand in the document. This information can also be obtained by calculating the ratio of the total area of the area where the handwritten character exists and the area where the printed character exists.

The character type which is another one of the features of the character portion includes numerals, alphabets, kana, kanji and the like. Focusing on whether the document is written in English or written in Japanese as a feature that easily remains in human memory, the character type determination unit 38 determines whether the document is written in English or in the Kana-Kanji dictionary 43. At least one is used to distinguish between English and Japanese. For example, English letters (alphabet)
When character recognition is performed using the dictionary 42, many characters can be obtained only with low confidence if the document is Japanese, and almost reasonable confidence can be obtained if the document is English. If the total is higher than a predetermined threshold, it is determined that the language is English, and if the total is lower, the language is determined to be Japanese. The information of the language type is stored in the data storage unit 4 together with the storage data of the document.
Further, similar processing can be performed for numbers, and information indicating that the document is a list of numbers, such as a form, can be added to the information of the language type.

In addition, the pitch detection unit 39 detects the character pitch and the line pitch of the extracted character part, and based on the result, the vertical / horizontal identification unit 49 determines whether the document is written vertically or horizontally. Identify if there is. There can be four states, the first one is for example A4 paper placed vertically and horizontally, the second is paper horizontally placed and vertically written, the third is paper placed vertically The fourth one is written horizontally with the paper placed sideways. Therefore, if the pitch in the horizontal direction is smaller than the pitch in the vertical direction, it is determined that the above is 1 or 2, and if it is the opposite, it is determined that it is 3 or 4. Furthermore, character recognition assuming that characters are written so that they can be read as they are in the direction where the paper is placed, and character recognition assuming that characters are written so that they can be read when the paper direction is replaced with a right angle direction , And comparing the results,
A distinction between 1 and 2 or a distinction between 3 and 4 is made. The information indicating which of these four states is stored in the data storage unit 4
Is stored together with the storage data of the document.

Further, the character size / density detecting section 50 can determine the character size and density. Also in this case, the numerical values indicating the size and density are stored in the data storage unit 4 as they are.
Instead of storing the information in a small character, the information may be converted into information such as “fine characters / closely” or “large characters / watermark” and stored.

In the above example, the process of extracting the "paper quality" and "stain" portions from the background portion has been described. By performing recognition on the data as it is without performing normalization, etc., a portion where the recognition rate is collectively worse than a predetermined threshold is specified as a “stain” portion, or the recognition rate is one. If the whole paper is bad, it can be specified as "poor quality paper".

Hereinafter, a case of extracting another specific portion (component) will be described with reference to FIG. The ruled line portion of the table is extracted by detecting a large number of straight lines by the straight line / curve detecting unit 61, detecting a large number of intersections where the straight lines are perpendicular to each other by the intersection detecting unit 62, and analyzing the position of each straight line by the analyzing unit 63. This is performed by extracting areas determined to have the same length as ruled line portions of the table.

Thereafter, when the paper type determination section 71 determines that straight lines arranged at a constant pitch exist over the entire surface of the paper, it determines that the paper type is ruled report paper or stationery. Furthermore, it is possible to determine whether the paper is a vertical ruled line or a horizontal ruled line based on the arrangement of straight lines. In addition, the arrangement, color, and marks (including the company name) of the ruled lines of frequently used paper are registered in the paper type dictionary 72, and matching is performed with the extracted straight line group, etc. It can be specified as "report paper" or "entry paper for submitting to Section A". The information indicating the type of paper and the information indicating the position and size of the ruled line portion of the table are stored in the data storage unit 4 together with the storage data of the document.

The extraction of the drawing portion is performed by detecting a large number of straight lines and curves, detecting a large number of intersections thereof, and extracting an area where they are not regarded as ruled line portions of the table.

The photographic part can be extracted using an image area separation technique known as an image processing technique. The photographic section includes a gravure photographic section in which the shading of the image smoothly changes, and a shaded photographic section characterized by arranging black spots whose size varies according to the image portion. Further, by analyzing the color of the photographic portion, it can be determined whether the photograph is a color photograph or a monochrome photograph.

The extraction of the graph portion is realized by using techniques such as circle extraction, rectangle extraction, line segment extraction and the like which are usually used in drawing recognition. These extraction processes may be performed only on the drawing part extracted as described above to specify whether the drawing is a graph or another drawing. The graph section includes a bar graph, a pie graph, and a line graph.

The background thus extracted,
Components such as characters, drawings, photographs, graphs, etc., include their position and size, attribute information such as type, derivative information,
The data is stored in the data storage unit 4 together with the storage data of the document. At this time, instead of storing the position and the size itself, the positional relationship such as “a photograph portion exists at the upper right and a graph portion exists at the lower left” via the positional relationship / ratio detector 73 of each component. It is also effective to convert the information into information having a ratio such as “drawings occupy 60% of the whole” or “drawings and characters exist at a ratio of 1: 2” and store them.

As another component, a mark or a color existing at a place designated in advance can be considered. That is, it detects whether or not a specific mark or color exists at the designated place, and stores this information in the data storage unit 4 together with the storage data of the document. For example, if the user considers important documents to be checked with a red pen in the upper right corner, it detects whether there is a red color in the upper right corner of the input image,
It is effective to add to the stored data as information indicating whether or not the document is important. Instead of specifying the location, the entire screen may be searched to find the specific mark or color.

FIG. 5 shows how the information described above is stored. Numerical data or codes corresponding to the names are assigned to each component, derivative information, and their attribute information. As for the constituent elements, as shown in the left half of the figure, an attribute name and an attribute value which is a value thereof are paired, and this is called an attribute set. The attribute set (plurality) and the constituent elements are paired and stored, for example, in a table format. As shown in the right half of the figure, the derived information is stored as a set of the derived information and its attribute value. Even if only one of these is stored, the effect of the present invention can be obtained. FIG. 5 is an example,
It is not necessary to store all of this information.

Tabular information on the above components,
Information represented by a set of derived information and its attribute value may be stored in a location different from the storage data (the document image and its recognition result), for example, in a directory portion, or added to a header portion of the storage data. May be stored. If you store it separately,
When a search is performed using these pieces of information, only the directory portion is searched, and only the matching data is read out, so that the search speed is increased.

The type of the attribute name, the type of the derived information, and the type of the attribute value that can be defined by the derived information included in each component are determined in advance. That is, for example, in the case of a drawing section, the attribute names for the drawing section are like three types of color, size, and position. Then, it is determined in advance which part of the table (memory address) is assigned which attribute name of which component. Then, the attribute value obtained by the specific part extracting unit 2 or the specific part feature identifying unit 5 is written in the corresponding attribute name. If the extraction or identification fails or the scanner is not color and the color cannot be determined, NULL is written in the attribute name that could not be determined.

Regarding derivative information, it is determined in advance which derivative information is to be allocated to which storage location in the memory. The attribute values that can be defined for each derivative information are also predetermined, such as, for example, three types of large, medium, and small for margins, and two types of handwriting and type for character types. If the attribute values defined for each of the derivative information are stored in the form of a table, it may be convenient for a search described later. The attribute value obtained by the derived information extracting unit 6 is selected from predefined ones, and the obtained attribute value is written in the corresponding derived information. N for the derivative information that could not be obtained
Write ULL.

FIG. 6A shows an example of the operation at the time of retrieval.
From the search data input unit 7, an input is made in a natural language, for example, "a document written by yourself on pink paper and stained with coffee". Then, the search information extraction unit 8
Reference numeral 1 designates “pink”, “self”, and “stain” from the above search data using information in the storage unit 82 that stores a table in which each derivative information is associated with a predetermined attribute value. A word that is the source of the search information is extracted, and three search information items “paper color: pink”, “writer: myself”, and “stain: yes” are obtained. Then, the search information comparison / collation unit 83
Is an attribute set including the obtained search information items (such as "paper color") (a set of derived information and attribute values as shown in the right half of FIG. 5).
From the data stored in the data storage unit 4 along with each stored data (document), and compares this attribute value with that of the search information (eg, “pink”) to find a document that matches these. Select and output to the document presentation unit 85.

In determining whether or not the attribute values match, the similarity between the attribute values is defined (for example, a perfect match is 100% similarity, pink and red have a similarity of 80%, and white and black have a similarity of 0%). %, If the stored attribute value is NULL, the similarity is 50% (indicating that it is not possible to judge, etc.), and even if not all of the search information obtained by the search information extraction unit 81 is a perfect match, each search If the sum of the similarities as a result of the comparison and collation of the information in all the search information is larger than a predetermined threshold value, the document may be selected. Also,
Some of the derived information has the property that attribute values can be extracted with a certain degree of certainty, such as "paper color", and the ambiguity in determining whether oneself or another person, such as "writer" There are those of the nature that often remains. Therefore, a weight is determined in advance for each derivative information, and in the case of the sum of the similarities, the similarity for the likely derivative information is emphasized, and the similarity for the ambiguous derivative information is set to the reference level. A weighted sum may be performed. This weight may be determined by obtaining a certainty factor at the time of extraction by the derivative information extraction unit 6 instead of being determined in advance.

It is also effective to use a keyword search in an ordinary filing apparatus. In other words, a keyword representing the content of the document is automatically or manually added to the document data at the time of the file, and if the keyword can be remembered at the time of the search, the number of searches is first narrowed down by the keyword, and then the search using the derived information as described above I do.
By doing so, it is not necessary to pay attention to whether or not the keyword is unique when adding the keyword, and the burden on the user can be reduced. In addition, the information of "what month was filed" is extracted by the clock function provided in the filing device, added to the document data, and searched using a combination of the information on the time and the derivative information as described above. Can also improve search accuracy.

Although the search using the derived information has been described above, the search using the component and its attribute information can be performed as follows. In this case, since the attribute value is closer to the raw data than in the case of the derived information, the storage unit 82 stores each component, the attribute name of the component, and the information name represented by the set. deep. Then, for example, if search data such as “document in which the color of the paper is pink, stained, the size of the stain is large, and the location of the stain is near the upper right” is input, the matching is performed with the information name in the storage unit 82. And extract the search information items "paper color", "stain size", and "stain position", and search for "pink", "large", and "top right", which are the search data written immediately after each item. Extract,
Each is set as search information.

Further, using the information in the storage unit 82, the information name “paper color” is changed to “background color”.
After that, the search information is compared with a set of the component and attribute information as shown in the left half of FIG. 5 of the data storage unit 4 which is stored in association with each document.
At this time, first, an attribute set including an item of the search information “color of background” is searched from the data storage unit 4, and this attribute value and that of the search information (“pink”) are searched.
, Etc.), and select a document that matches these. In the search information, a rough expression such as "large" is used.
"10" is "small", "11-20" is "medium", "21-3"
By storing in advance correspondence such as “0” as “large”, if “large”, it is converted to a numerical value of “21 to 30” and compared with the attribute value of the data storage unit 4. . In this case, since the numerical value is compared and compared, the similarity calculation can be easily performed.

Note that a search using both the derived information and the attribute information of the constituent elements can be performed. In particular, the derivative information contains "stain: yes"
If the attribute information of the component contains detailed information such as the color and size of “stain”,
First, the document (plural) having "stain: present" is selected by looking at the derived information added to the document data from the search data "stained", and the attribute name of the corresponding component "stain" is displayed. The color and size are presented, and the user inputs the continuation of the search data such as "brown" for "color" and "large" for "size", and narrows down by the attribute information of the component. . Also, in the derived information, if the "character type" is "print type", such as "character type" and "font" or "scriber", there may be derived information called "font", while "handwritten" Then, there is a thing having a hierarchical structure such that there may be derived information called "scriber", and the above-mentioned interactive narrowing down is also possible here.

FIG. 6B shows another example of the operation at the time of retrieval. First, the derivative information item display unit 86 displays the derivative information that can be extracted by the derivative information extraction unit 6 along with attribute values (plurality) defined in advance for the derivative information as shown in FIG. The user sees this and instructs each derivative information through the search data input unit 7 while remembering that the “paper color” of the desired document was “pink” and the “margin” was medium. I will do it. If it cannot be remembered, the comparison and collation will be performed except for the item, so that the item need not be entered. With respect to the search data input in this way, the search information comparison / collation unit 83
As in the case of FIG. 6A, a matching document is selected and presented.

The above is an embodiment in which attribute information and derivative information of constituent elements are extracted from a document image to be filed, and a search is performed using the extracted information. However, it is also effective to use these for flipping. In other words, in a system in which a user selects a desired document by presenting stored data (documents) while flipping over, the attribute information and derivative information of constituent elements stored along with the presented document are developed into an image. . For example, the image information of “stain” is superimposed and displayed on the image of the document in accordance with the stored information on the color and size of “stain”. Accordingly, especially when the input document image is stored as storage data in which noise has been removed, since the document presented at the time of flipping has no recognizable noise, the user may at first glance It is not impossible to determine whether the document is the desired document, and the usability can be improved.

Further, extracted derivative information and attribute information (for example, displayed as a paper color: pink) are displayed together with the original document image or the stored data so that the user can modify, change, and add the derivative information and attribute information. You may. in this case,
For example, if the user corrects the color of the paper to white, adds the information of the writing amount: many, and deletes the information of the stain: present, the derivative information and the like corrected in this way are associated with the corresponding document. The data is stored in the data storage unit 4.

[0056]

As described in detail above, according to the present invention, information representing the appearance of a document automatically extracted from an input image (paper color, paper quality, stain, character color, character type, character type, The desired document can be searched using the writer, writing implements, fonts, the amount of writing in the margins, the length and width of the document, the size and density of characters, the type of paper, the positional relationship between drawings and photos, etc. Even when the user does not clearly remember the contents and keywords of a document, a search can be realized by remembering peripheral information of the document.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a schematic configuration of an apparatus according to an embodiment.

FIG. 2 is a diagram illustrating a processing example when a background part is extracted by the apparatus according to the embodiment.

FIG. 3 is a diagram illustrating a processing example when a character portion is extracted by the apparatus according to the embodiment.

FIG. 4 is a diagram showing an example of processing when a ruled line portion, a drawing portion, a photograph portion, and a graph portion of a table are extracted by the apparatus according to the embodiment.

FIG. 5 is a diagram showing an example of the format of information on constituent elements and derivative information stored in a data storage unit 4;

FIG. 6 is a diagram showing a configuration for searching in the apparatus according to the embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image input part, 2 ... Specific part extraction part, 3 ... Storage information generation part, 4 ... Data storage part, 5 ... Specific part characteristic identification part, 6
... Derived information extraction unit, 7 ... Search data input unit, 8 ... Search unit, 9 ... Output unit, 11 ... Color separation unit, 12 ... Color image buffer, 13 ... Total area calculation unit, 14 ... Background color determination unit, 15 ... background part extraction part, 16 ...
"Stain" part extraction unit, 17 ... Noise detection unit, 18 ... Size / position detection unit, 19 ... "Paper quality" extraction unit, 20 ... "Stain" information extraction unit, 21 ... "Paper color" extraction unit, 31 ... Connected area extraction unit, 32. Analysis unit, 33.
... Image buffer, 35 ... Character color detector, 36 ... Character line thickness detector, 37 ... Character type determiner, 38 ... Character type determiner, 39 ... Pitch detector, 40 ... Handwriting dictionary, 41 ...
Type dictionary, 42: English dictionary, 43: Kana-kanji dictionary, 44
.., A writer identification unit, 45, an owner handwriting dictionary, 46, a writing implement identification unit, 47, a font identification unit, 48, a mixture rate detection unit,
49: vertical / horizontal identification section, 50: character size / density detection section,
61: straight line / curve detecting unit, 62: intersection detecting unit, 63
65: analysis unit, 64: table ruled line part extraction unit, 66: drawing part extraction unit, 67: image area separation unit, 68: photograph part extraction unit, 69
... Circle / rectangle / line segment extraction unit, 70 ... Graph part extraction unit, 71
... Paper type determination unit, 72 ... Paper type dictionary, 73 ... Position relation / ratio detection unit of each component, 81 ... Search information extraction unit,
82: Derived information / attribute value correspondence table storage unit, 83: Search information comparison and collation unit, 84: Similarity / weight storage unit, 85: Document presentation unit, 86: Derived information item display unit

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-2-41566 (JP, A) JP-A-1-229373 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 17/30 G06T 1/00 JICST file (JOIS)

Claims (3)

(57) [Claims] An image input unit for inputting a document image fixed on a medium; a color separation process for the document image input by the image input unit; and a background of the document image based on a distribution of each separated color. Background color extraction means for extracting a color; background part extraction means for extracting a background part of the document image based on the background color extracted by the background color extraction means and a component of the document image; Noise detection means for detecting noise included in the background portion extracted by the background part extraction means; and paper quality extraction means for extracting paper quality information from the noise extracted by the noise detection means and the background color Storage means for storing the paper quality information extracted by the paper quality extraction means in association with the input document image; and a desired sentence to be searched. Search key input means for inputting information on the paper quality of an image as a search key; and search for collating the search key input by the search key input means with the paper quality information stored in the storage means. Means for outputting a corresponding document image from the stored document images in accordance with a result of the comparison by the search means. 2. An image input means for inputting a document image fixed on a medium, a color separation process of the document image input by the image input means, and a background of the document image from a distribution of each separated color. Background color extraction means for extracting a color; background part extraction means for extracting a background part of the document image based on the background color extracted by the background color extraction means and a component of the document image; In the background portion extracted by the background portion extraction means,
A stain portion extracting means for extracting a portion having a color different from the background color and having a non-linear shape or contour as a stain, and information on the stain portion extracted by the stain portion extracting means in association with the input document image. Storage means for storing; search key input means for inputting information on a stain portion of a desired document image to be searched as a search key; search key input by the search key input means; and storage in the storage means Search means for performing a comparison with the information on the stained portion, and output means for outputting a corresponding document image from the stored document images in accordance with a result of the comparison by the search means. A document information search device characterized by the following: 3. An image input means for inputting a document image fixed on a medium, a character part extracting means for extracting a character part from the document image input by the image input means, and a character part extracting means. Based on the thickness of the character portion extracted by the writing instrument identification means for identifying the writing implement used in the document image, information of the writing implement identified by the writing implement identification means, Storage means for storing in association with a document image, search key input means for inputting, as a search key, information on a writing implement of a desired document image to be searched, and a search key input by the search key input means Searching means for comparing the information of the writing implement stored in the storage means with the sentence information stored in the storage means. Document information retrieval apparatus characterized by comprising an output means for outputting the corresponding document image from the image.