JP2009009307A - Document image processor and processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the convenience of a user in high level document processing such as retrieval by implementing document image analysis processing to an input document image, executing the detection of a character region, the discrimination of the attributes (for example, titles, captions and texts or the like) of the region, and the recognition processing of a character itself, and detecting accuracy to the result, and calculating the accuracy of word units, clause units, sentence units, and a whole document. <P>SOLUTION: This document image processor is configured of: an image input part such as a scanner; a pre-processing part; a document image structure analyzing part; a character recognition part; an accuracy calculation part; and an electronic document generation part. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a document image processing apparatus and method.

  In recent years, computerization of information has progressed, and there is an increasing demand for converting paper documents into electronic documents. When a paper document is digitized, it is not limited to simply photoelectrically converting the paper surface with a scanner or the like to convert it into image data, and the text, symbols, photos, Divided into areas with different properties such as tables, character code information for figures, figures and lines, table frames for vector data, photographs for image data, etc. Yes.

  Such coding of paper documents currently has technical problems in recognition technology, such as block selection for recognizing the description content, recognition technology for encoding characters from the character area of the description content, diagrams, lines, etc. It is in an incomplete state in all technical ranges such as vectorization technology. However, on the other hand, digitization is progressing rapidly as described above, and there is a reality that digitization of paper documents is expanding. Then, coding technology is used to search for paper documents in an incomplete state. In this case, inability to search due to misrecognition of character recognition is divided, has a plurality of candidate characters at the time of character recognition, considers the combination at the time of search, or manually corrects after character recognition processing. For this reason, in advanced document processing techniques such as sentence summarization and classification of digitized paper documents, there are rare cases where results that cannot be put to practical use are generated due to erroneous recognition of characters.

For example, Patent Document 1 and Patent Document 2 can be cited as conventional examples.
Japanese Patent Laid-Open No. 04-364593 Japanese Unexamined Patent Publication No. 08-305805

  In this way, with the digitization of information, the demand for advanced document processing such as search technology, summarization, classification, etc. by electronic documents is expanding and becoming practical, but electronic documents by digitizing paper documents are becoming practical Inexperienced computerization techniques such as analysis and recognition impose inconvenience on users. Therefore, the present invention provides improved convenience for users by adding regular misrecognition accompanying the digitization of paper documents to the technical field developed from search technology to advanced document processing technology. It is an invention. For example, in the case of a paper document search, avoiding a search defect due to erroneous recognition. Usually, the search results depend on the result of character recognition, and since the accuracy of the extracted results is not mentioned even in the search with candidate characters, the judgment of the credibility of the results is left to the user. However, it is necessary to grasp the contents regardless of misrecognition. Furthermore, since no consideration is given to misrecognition regarding the extraction of important sentences in the summary, in many cases, a summary result with misrecognition is output. Furthermore, when the purpose is to create a new document including a table of contents automatic generation by summarizing and synthesizing from a plurality of document contents, the output itself may be meaningless due to misrecognition. In addition, the classification result may deteriorate due to the deterioration of accuracy due to misrecognition.

  In light of these current problems, the present invention encourages users to share added information and improve reusability by encouraging users to add value to paper documents without compromising electronic technology. An easy-to-use document image processing apparatus is provided.

  According to the present invention, when a search query is thrown to a database, when the search query is a word, and in a system that accesses a character recognition result of a paper document, individual characters included in the character recognition result This is characterized in that the search is executed by adding weighting based on the accuracy in units of words calculated from the accuracy (reliability) of the recognition accuracy. That is, according to the present invention, even if the search target word does not match the word query, if the accuracy of the search target word is low, it is output as a search target document, and when the accuracy is high Is excluded from the target document.

  According to the present invention, when summary document processing is performed using digitized data in which a character of a paper document is recognized, the important sentence in the sentence is ranked and extracted by the important sentence extraction algorithm. In the case where the important sentence is a paper electronic document as a character recognition result, the ranking of the important sentence is corrected by adding weighting based on the sentence accuracy calculated based on the character recognition result. In other words, according to the present invention, the ranking of the important sentence taking into account the character recognition result is used as a summary output by lowering the ranking when the accuracy of the recognition result of each character in the important sentence is low. It is characterized by providing easy-to-read summary documents.

  Furthermore, according to the present invention, document structure analysis and important sentence extraction are performed from a plurality of documents, automatic extraction of headings obtained from the document structure analysis, and the automatic extraction of a table of contents by collecting the headings from the automatically extracted headings. In the creation system (apparatus), based on the accuracy of the headline sentence calculated based on the character recognition result of the document, if the accuracy is low, the font style of the headline character portion of the table of contents sentence is changed. It is characterized by prompting the user with a warning. Furthermore, when creating a new document summarized from the plurality of documents, the heading sentence changes the output style according to the accuracy from the accuracy of the headline sentence and the accuracy of the important sentence based on the character recognition result, and the summary sentence has the accuracy. It is characterized by being composed of high sentences.

  Further, according to the present invention, when classified document processing is performed using digitized data in which a paper document is character-recognized, the calculation for classification is performed based on a concept vector of words, phrases, sentences, etc. of the character-recognized electronic data. In addition, the concept vector is weighted based on the accuracy of character recognition. In other words, the strength of the concept vector is the strongest when the recognition accuracy is high and the concept elements match, and even if the recognition accuracy is low, the concept elements are similar in character classification. It is characterized in that the recognition accuracy is high, the accuracy is high, and when the concept elements are similar in character classification, it becomes the lowest so that the influence of classification by character recognition can be suppressed.

  As described above, according to the present invention, by performing the document image analysis process on the input document image, the detection of the character region, the determination of the attribute of the region (for example, heading, caption, text, etc.) and the character itself By executing the recognition processing and detecting the accuracy of the result, the accuracy of the word unit, the phrase unit, the sentence unit, and the whole document is calculated, thereby improving the convenience of the user for advanced document processing such as retrieval. Have achieved that. That is, when using digitized data of a paper document for searching or the like, the result of character recognition affects the search hit rate. In other words, if a condition is included in a plurality of words in order to avoid hits due to misrecognition, hits, or reduce the hit rate of misrecognition, the user may be confused about the reason for the hit.

  Therefore, the accuracy of character recognition according to the present invention is calculated, and advanced document processing such as search is executed in consideration of the calculated accuracy, thereby reducing the decrease in the unhit rate due to misrecognition to the user. In addition, by providing the accuracy to the user, the user can easily be informed of the reason for the hit. The accuracy providing means for the user can be changed by changing the color means or the like to the hit character or word, and can be taken into consideration when ranking.

  FIG. 1 is a block diagram of a document image processing apparatus according to this embodiment. The document image input from the image input unit 10 such as a scanner is subjected to subsequent processing by the preprocessing unit 11 by tilt detection / correction, document orientation detection / correction, noise removal, and in some cases binarization processing. Image processing is performed to ensure that it is performed. Further, the document image structure analysis unit 12 analyzes the attribute determination of the document image image and the layout structure of the attribute. According to the present invention, the attribute is intended to detect image attributes such as a character image area, a background image, a photographic image, and a graphic image. The analysis of the layout structure is aimed at detecting the character string attributes such as the heading (including title), body, and caption of the area in the case of the physical arrangement of the image attributes and the character image area. Analysis results of these layout structures are output to an electronic document 15 (described later). On the other hand, the character image area enters the character recognition 13 and is character-coded in accordance with a general character recognition process. Further, the accuracy calculation 14 indicates that the character-coded character recognition result indicates the accuracy of the recognition result. Desired. The accuracy is assumed to include one character unit to a set of a plurality of characters by using the technique of the present invention. Further, these character recognition results, accuracy, and layout results are input to the electronic document 15, the electronic document maintains the layout of the paper document, and the character image area is reconstructed with the character recognition results having accuracy information. Is. Further, the electronic document can be switched between display and non-display of the character recognition result. If not, the image of the character image area itself is displayed.

FIG. 2 is a flowchart of the process of extracting the document accuracy used according to this embodiment. The image input 20 corresponds to 10 and 11 in FIG. 1, and document image input and preprocessing are executed. The document image portion extraction 11 extracts a character image portion related to the present invention in the document image structure analysis portion 12 of FIG. 1, and the character image analysis 22 is a portion of the document image structure analysis portion 12 of FIG. Thus, it is a part for detecting character string attributes such as headlines (including titles), texts, captions, etc. of character areas related to the present invention. Therefore, image data such as a headline character region, a body character region, and a caption character region is generated by the document image portion extraction 11 and the character image analysis 22, and the character region attributes are input to the electronic document 15 and will be described below. Wait for character recognition result and accuracy information data. Here, the attribute extraction of the headline, the text, and the caption is not mentioned here because it is disclosed by the invention of the prior application that it is not the object of the present invention. On the other hand, the character image area output by the character image analysis 22 is input to the character recognition 23. The character accuracy is input to the character recognition 23, coded one by one by a general character recognition processing means, and character accuracy indicating the accuracy of recognition of each coded character is obtained by the character accuracy calculation 24. This indicates the reliability of recognition of individual characters, and since various inventions have been proposed, they will not be mentioned here either. Therefore, the character code of each character unit and the accuracy of the character code are output from the character recognition 23 and the character accuracy calculation 24. In particular, the recognized character code string is input to the word extraction 25, and word-by-word extraction is executed from the character code string. In this method, a word dictionary-based word is detected using a word dictionary stored in the system in advance. Therefore, the unknown word is not detected unless it is defined by the user, etc., but the recognition result is set in order to improve the detection rate in a wide range of words considering misrecognition. Extraction in units of words is performed with reference to the candidate character group obtained when obtaining and the similar character group of the recognition result character code not related to the recognition result. Further, the detected character code string in units of words is input to the word accuracy calculation 26, and the accuracy in units of words is calculated from the previously calculated character accuracy. An example of the calculation means is shown below. For example, to extract the accuracy of the character code “literature”
Sentence (90%) Studies (90%) 0.9x0.9-> 0.81
Thus, it can be obtained by the product of each character.

Accordingly, the word and the result of the accuracy of the word are obtained from the word extraction 25 and the word extraction calculation 26. Further, based on the result, particularly in the phrase extraction 27, a phrase is extracted from the extracted word and the subsequent characters, and the accuracy of the extracted phrase is calculated by the phrase accuracy calculation 28. The calculation method is as follows:
(Word accuracy) x (Subsequent character accuracy)
It ’s also good to consider the weighting of word and letter accuracy,
λ (word accuracy) + (1-λ) (subsequent character accuracy) − However, λ may be a weighting coefficient.

Further, in the sentence extraction 29, a sentence is extracted from the phrase extracted by the phrase extraction 27 and the unextracted characters. The obtained sentence is characterized in that the sentence accuracy is calculated from the already calculated clause and character accuracy values by the sentence accuracy calculation 30. Regarding the calculation method of the sentence, the product of already calculated accuracy values, that is,
(Sentence accuracy) x (Accuracy of other characters)
Alternatively, the accuracy of the sentence may be calculated according to the above-described weighted phrase accuracy calculation means in consideration of weighting the accuracy calculated by the above-described processing. Since the accuracy calculation means is different from the gist of the present invention, details are not described.

  Further, here, a sentence block is represented as an electronic document, and is generated together with character string attributes such as a heading (including title), text, and caption of a character area, which is a result of document structure analysis obtained by character image analysis in the electronic document 31. It is characterized by being. Furthermore, according to the present invention, the accuracy of the electronic document is calculated by the document accuracy calculation 32 from the obtained electronic document.

FIG. 3 is a flowchart showing the process of extracting the document accuracy used according to this embodiment. The image input 33 corresponds to 10 and 11 in FIG. 1, and document image input and preprocessing are executed. The document image part extraction 11 extracts a character image part related to the present invention in the document image structure analysis part 12 of FIG. 1, and the character image analysis 35 is a part of the document image structure analysis part 12 of FIG. Thus, it is a part for detecting character string attributes such as headlines (including titles), texts, captions, etc. of character areas related to the present invention. Therefore, the document image part extraction 11 and the character image analysis 35 generate image data such as a headline character region, a body character region, and a caption character region, and character region attributes are input to the electronic document 44, which will be described below. Wait for character recognition result and accuracy information data. Here, the attribute extraction of the headline, the text, and the caption is not mentioned here because it is disclosed by the invention of the prior application that it is not the object of the present invention. On the other hand, the character image area output by the character image analysis 35 is input to the character recognition 36. The character accuracy is input to the character recognition 36, encoded one by one by general character recognition processing means, and character accuracy indicating the accuracy of recognition of the encoded individual characters is obtained by the character accuracy calculation 37. This indicates the reliability of recognition of individual characters, and since various inventions have been proposed, they will not be mentioned here either. Therefore, the character code of each character unit and the accuracy of the character code are output from the character recognition 36 and the character accuracy calculation 37. Further, in the present invention, the character code and the character accuracy obtained by the processing are input to a word extraction 38, a phrase extraction 39, and a sentence extraction 40 which will be described later. In the above-described processing, the recognized character code string is input to the word extraction 38, and word unit extraction is performed from the character code string. In this method, a word dictionary-based word is detected using a word dictionary stored in the system in advance. Therefore, the unknown word is not detected unless it is defined by the user, etc., but the recognition result is set in order to improve the detection rate in a wide range of words considering misrecognition. Extraction in units of words is performed with reference to the candidate character group obtained when obtaining and the similar character group of the recognition result character code not related to the recognition result. Further, the detected character code string in units of words is input to the word accuracy calculation 41, and the accuracy in units of words is calculated from the previously calculated character accuracy. An example of the calculation means is shown below. For example, to extract the accuracy of the character code “literature”
Sentence (90%) Studies (90%) 0.9x0.9-> 0.81
Thus, it can be obtained by the product of each character.

Accordingly, the word and the result of the accuracy of the word are obtained from the word extraction 38 and the word extraction calculation 41. Here, the calculated word accuracy is input to the electronic document 44 which is the final output system. Further, based on the result of word extraction, particularly in the phrase extraction 39, a phrase is extracted from the extracted word and the subsequent characters, and the accuracy of the extracted phrase is calculated by the phrase accuracy calculation 42. The calculation method is calculated based on the accuracy for each recognized character code. The calculation method is as follows:
(Character accuracy) x (Character accuracy) x ---- x (Character accuracy)
It ’s also good to consider the accuracy weighting of individual characters,
Σλ (character accuracy) − where λ is a weighting coefficient and Σλ = 1.
It is good.

Through this processing, the obtained clause and clause accuracy are input to the electronic document 44 as the final output. Further, in the sentence extraction 40, a sentence is extracted from the phrase extracted by the phrase extraction 39 and unextracted characters. The sentence is characterized in that the sentence accuracy calculation 43 calculates the accuracy of the sentence from the already calculated character accuracy value. Regarding the calculation method of the sentence, the product of already calculated accuracy values, that is,
(Character accuracy) x (Character accuracy) x ---- x (Character accuracy)
Alternatively, the accuracy of the sentence may be calculated according to the above-described weighted phrase accuracy calculation means in consideration of weighting the accuracy calculated by the above-described processing. Since the accuracy calculation means is different from the gist of the present invention, details are not described.

  Further, here, a sentence block is represented as an electronic document, and character string attributes such as headings (including titles), texts, captions, and the like of character regions, which are document structure analysis results obtained by character image analysis in the electronic document 44, words It is generated together with unit accuracy, phrase unit accuracy, and sentence unit accuracy. Further, according to the present invention, the accuracy of the electronic document is calculated from the value of the word accuracy, the phrase accuracy, and the sentence accuracy by the document accuracy calculation 32 according to the present invention.

FIG. 4 is a process flowchart for extracting the document accuracy used according to this embodiment. The image input 46 corresponds to 10 and 11 in FIG. 1, and document image input and preprocessing are executed. The document image portion extraction 11 extracts a character image portion related to the present invention in the document image structure analysis portion 12 of FIG. 1, and the character image analysis 48 is a portion of the document image structure analysis portion 12 of FIG. Thus, it is a part for detecting character string attributes such as headlines (including titles), texts, captions, etc. of character areas related to the present invention. Therefore, image data such as a headline character region, a body character region, and a caption character region are generated by the document image portion extraction 11 and the character image analysis 48, and character region attributes are input to the electronic document 57, which will be described later. Wait for character recognition result and accuracy information data. Here, the attribute extraction of the headline, the text, and the caption is not mentioned here because it is disclosed by the invention of the prior application that it is not the object of the present invention. On the other hand, the character image area output by the character image analysis 48 is input to the character recognition 49. The character accuracy is input to the character recognition 49, coded one by one by general character recognition processing means, and character accuracy indicating the accuracy of recognition of the coded individual characters is obtained by the character accuracy calculation 50. This indicates the reliability of recognition of individual characters, and since various inventions have been proposed, they will not be mentioned here either. Therefore, the character code of each character unit and the accuracy of the character code are output from the character recognition 49 and the character accuracy calculation 50. Further, in the present invention, the character code and the character accuracy obtained by the processing are input to the word extraction. In the above-described processing, the recognized character code string is input to the word extraction 51, and word unit extraction is executed from the character code string. In this method, a word dictionary-based word is detected using a word dictionary stored in the system in advance. Therefore, for unknown words, in the present invention, in order to avoid undetection, a wide range of candidate character groups obtained when obtaining recognition results for the purpose of improving the detection rate in terms of words in consideration of misrecognition, Then, word-wise extraction is performed with reference to the similar character group of the recognition result character code not related to the recognition result. Furthermore, word detection as an unknown word is executed for characters that are not required by this processing. In particular, assuming that particles included between words are also words, the process continues. Furthermore, the detected character code string in units of words is input to the word accuracy calculation 54, and the accuracy in units of words is calculated from the previously calculated character accuracy. An example of the calculation means is shown below. For example, to extract the accuracy of the character code “literature”
Sentence (90%) Studies (90%) 0.9x0.9-> 0.81
Thus, it can be obtained by the product of each character.

Therefore, the word and the result of the accuracy of the word are obtained from the word extraction 51 and the word extraction calculation 54. Here, the calculated word accuracy is input to the electronic document 44 which is the final output system. Further, based on the result of word extraction, particularly in the phrase extraction 52, a phrase is extracted from the extracted word and the word obtained as an unknown word, and the accuracy of the extracted phrase is calculated by the phrase accuracy calculation 55. Is done. The calculation method is calculated by the recognized accuracy obtained by the word accuracy. However, if the accuracy by word is not required, the accuracy is used as an unknown word. The calculation method is as follows:
(Word accuracy) x (word accuracy) x ---- x (word accuracy)
It ’s also good to consider the accuracy weighting of individual characters,
Σλ (word accuracy) − where λ is a weighting factor and Σλ = 1.
It is good.

By this processing, the obtained clause and clause accuracy are input to the electronic document 57 which is the final output. Further, in the sentence extraction 53, a sentence is extracted from the phrase extracted by the phrase extraction 52. The obtained sentence is characterized in that the sentence accuracy is calculated from the already calculated phrase accuracy value by the sentence accuracy calculation 56. Regarding the calculation method of the sentence, the product of already calculated accuracy values, that is,
(Phrase accuracy) x (Phrase accuracy) x ---- x (Phrase accuracy)
Alternatively, the accuracy of the sentence may be calculated according to the above-described weighted phrase accuracy calculation means in consideration of weighting the accuracy calculated by the above-described processing. Since the accuracy calculation means is different from the gist of the present invention, details are not described.

  Further, here, a sentence lump is represented as an electronic document, and character string attributes such as headings (including titles), texts, captions, and the like of character areas, which are document structure analysis results obtained by character image analysis in the electronic document 57, words It is generated together with unit accuracy, phrase unit accuracy, and sentence unit accuracy. Further, according to the present invention, the obtained electronic document is calculated by the document accuracy calculation 58, and the accuracy of the electronic document is calculated from the value of the word unit accuracy, the phrase unit accuracy, and the sentence unit accuracy.

  In this example, three examples are shown as examples. In the example 1 (FIG. 2), means for obtaining the accuracy of the electronic document as the final output is shown. In Example 2 (FIG. 3), in order to obtain the accuracy of the electronic document as the final output as a document, the word accuracy, the phrase accuracy, and the sentence accuracy are obtained from the character accuracy, and the accuracy of the document is calculated from each result. The above-mentioned words, phrases, and sentence accuracy values are stored in the final electronic document. In the third embodiment (FIG. 4), the word accuracy is obtained from the character accuracy, the word accuracy is obtained from the word accuracy, the sentence accuracy is obtained from the phrase accuracy, and the document accuracy is obtained from each result in order to obtain the accuracy of the electronic document as the final output. It was shown that the above-mentioned word, phrase, and sentence accuracy values were stored in the final output electronic document.

  FIG. 5 shows an example in which the present invention is used for search processing. First, as shown in FIG. 5A, this is an example of full-text search in a conventional document image processing apparatus. In this example, the device is instructed to search for the word “document” by query. As a search result, the detected document image is misrecognized at the time of character recognition, and the document is written in a cross. Normally, when a full-text search is performed using a document query, a copy that is a misrecognized portion is not hit. In this example, the word “document” was correctly answered in the same document, so it was a hit, but this is rare. Therefore, an example using the present invention is shown in FIG. This is an example in which the word accuracy according to the present invention is used as data. In this case, the recognized document naturally hits, and in the conventional example, the book that has not been searched also hits. However, this indicates that the document itself is a hit as a search result, indicating the possibility of the document from similar characters from the viewpoint that the accuracy of the recognition error is high.

FIG. 1 is a block diagram of a document image processing apparatus according to the present embodiment (until creation of an electronic document). Processing flowchart of document accuracy extraction of this embodiment Processing flowchart of document accuracy extraction of this embodiment Processing flowchart of document accuracy extraction of this embodiment (a) is a diagram for explaining a conventional example search (full text), and (b) is a diagram for explaining a case in which this embodiment is used for a search.

Claims (5)

The document image processing apparatus has means for inputting a document image, means for digitizing the inputted document image, means for storing the digitized document,
In the apparatus, the digitized electronic document has a character image area extracted by means for extracting a character image, and the non-character image area has means for character encoding in character recognition processing.
In the device, a character string that is composed of at least one character coded character has accuracy calculated in character recognition,
Means for calculating the accuracy of the word from the accuracy of the calculated character string;
Means for calculating the sentence accuracy from the calculated word accuracy;
Means for calculating the accuracy of a plurality of sentences from the accuracy of the calculated sentences;
In the apparatus, a document image processing apparatus that calculates the accuracy of a sentence.   2. The document image processing apparatus according to claim 1, wherein the accuracy of a sentence is obtained by calculating the accuracy of a phrase and a phrase (word + character).   2. The document image processing apparatus according to claim 1, wherein the plurality of sentences are composed of at least two consecutive sentences.   4. The document image processing apparatus according to claim 3, wherein the accuracy of a plurality of sentences includes an accuracy in a paragraph (paragraph) unit.   4. The document image processing apparatus according to claim 3, wherein the accuracy of a plurality of sentences includes the accuracy of a section and a chapter.

Images