JP2006031129A - Document processing method and document processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that it is difficult to take countermeasures to the increase of a processing time due to the reading of a document image on a whole surface, or the error of the character line extraction of an OCR due to the mixture of a document/graphic plate/ruled line or the deterioration of reading precision due to the difficulty of the arrangement of general-purpose notation knowledge suitable for a document image since a paper document group or a document image group used to be collectively converted into a text in batch processing by using an OCR device, and a document processing task used to be operated to this in a conventional manner. <P>SOLUTION: A character string block is extracted from a document image being an object to be processed, and the two-dimensional arrangement structure is analyzed so that the described contents category of the character string block can be estimated. When a reading object region is designated by the interactive operation of a user, a recognition processing is performed only for a character string block group in the neighborhood of the region, so that it is possible to reduce a processing time. Also, character string recognition is performed by using notation knowledge corresponding to the described contents of the character string block so that it is possible to improve the recognition precision. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an interactive document processing technique using character recognition and a recording medium on which a document processing program is recorded.

  Even today, with the spread of digital information technology using computers, paper documents are still widely used as a medium for information transmission. For this reason, there is a strong social demand for efficient document processing for a large number of paper documents, but paper documents have a problem in that they require more time for reference / search / modification processing than digital documents. In particular, in document processing work related to authorization and inspection, it is important to be able to quickly search for business know-how related to the contents of application forms and to quickly convert only necessary information into digital data in terms of improving work efficiency. In order to solve these problems, various methods for paper document processing have been proposed.

  As a means for realizing document processing equivalent to a digital document with a paper document, a batch operation of recognizing a paper document using an OCR (optical reading device) and converting all the written contents of the paper document into digital data Processing is common. If all paper documents are converted to digital data (text) by OCR in advance, the subsequent processing can be performed efficiently by searching for related business know-how using text or by encoding text. Can perform well. However, since the text converted by OCR generally includes an error, there may be a case that cannot be dealt with only by a simple batch reading process.

  One of the countermeasures for the above problem is improvement of recognition accuracy, particularly improvement of accuracy using notation knowledge of character strings. In general, character string notation analysis processing is widely used to compensate for character cutout and character recognition uncertainty in character string recognition and to convert a character string image into character string text. As the algorithm, those using morphological analysis, those using RTN matching (recursive transition network matching), and ascending parsing algorithm are generally used.

  For example, Japanese Patent Laying-Open No. 05-108891 (Patent Document 1) describes a method of applying morphological analysis to an OCR recognition result as means for improving OCR reading accuracy. Although it is possible to correct misreading by performing knowledge processing such as morphological analysis, the dictionary used in normal morphological analysis is intended for general sentences such as newspapers, and proofreads documents for special business use with high accuracy. To do this, it is necessary to define additional special dictionaries suitable for the field. Therefore, problems remain in terms of maintainability and computational complexity. Furthermore, since a wide range of notation knowledge called morphological analysis is targeted, it takes time to analyze notation knowledge, and there is a problem that enormous storage capacity is required for notation analysis.

  Japanese Patent Laid-Open No. 2002-117374 (Patent Document 2) proposes a character string notation analysis process using ascending syntax analysis for a handwritten digit string. In general, it is said that ascending parsing can reduce the amount of calculation compared to descending parsing, and it is often applied to what can be expressed by a simple rule such as a numeric string. However, there is a limit to the robustness against problems such as misreading of characters and noise mixing that can occur in character string recognition because the algorithm does not guarantee optimality. It is also possible to manually correct a text code converted by OCR and perform a search on the correction result. However, it is difficult to say that correction with human intervention is practical in terms of processing speed and cost. Even if manual correction is omitted, converting millions of paper documents into texts by OCR is a process that requires enormous processing time and system construction costs.

  On the other hand, in document processing in which character recognition is performed interactively, a character string that is a target to be recognized by a user of the document processing system is partially specified and converted into text, thereby reducing processing time. Therefore, it is not necessary to construct a large-scale system. The technique proposed in the present invention is a document processing technique based on a process of performing character recognition interactively, and by analyzing the arrangement structure of the character strings described in the document, the character strings designated as recognition targets are analyzed. The category of description contents (date, date, monetary number, business word, etc.) is estimated, and the recognition accuracy is improved by reading the character string using notation knowledge that matches this category.

Japanese Patent Laid-Open No. 05-108891

JP 2002-117374 A Japanese Patent Application Laid-Open No. 09-319824 Japanese Patent Laid-Open No. 2000-251012 JP 2001-014411 A

  It is an object of the present invention to perform character recognition based on a user's interactive operation in a document processing apparatus that displays and processes a document image on a screen, and related items from a business database based on the result. Is to provide a document processing system that retrieves and displays the document, a document processing system that converts the document into data, a device thereof, and a recording medium on which the document processing program is recorded.

  In the conventional method, a batch processing using an OCR device converts a document image group into text and performs document processing for this. However, the processing time is increased by reading the entire document image. It was difficult to cope with a reading accuracy drop due to an error in extracting a character line of OCR due to a mixed document-illustration-ruled line, or difficulty in preparing general-purpose notation knowledge suitable for the document image. . An object of the present invention is to propose a method for avoiding an increase in processing time due to OCR reading and an adverse effect of OCR reading errors on a document processing operation.

  In order to achieve the above object, the present invention starts character recognition processing at the time of request in response to a user's interactive operation in a document display / operation device, and immediately recognizes only necessary portions. In addition, a system is provided that performs information retrieval from a business database using the recognition result and partial data conversion of a processing target document. Also, by extracting a character string block from the document image to be processed and analyzing its two-dimensional arrangement structure, the description content category (year / month / day, monetary number, business word, etc.) of the character string block is analyzed. , Etc.), a character string block to be recognized is selected by a user's interactive operation, and the character string is read by switching the notation knowledge according to the description content category of the estimated character string block. Thus, a mechanism for improving the reading accuracy is provided.

  In the conventional technique, the paper document group and the document image group are collectively converted into text by batch processing using an OCR device, and the document processing work is performed on this, but by reading the document image over the entire surface. To cope with a decrease in reading accuracy due to an increase in processing time, an OCR character line extraction error due to mixed document-illustration-ruled lines, and difficulty in preparing general-purpose notation knowledge suitable for the document image It was difficult. On the other hand, according to the present invention, the character recognition process is interactively started in the document processing operation, and the text is limited to only the part necessary for the document process, so that the processing time can be reduced. In addition, by analyzing the two-dimensional arrangement structure of the character string described in the document, the category of the description content of the character string designated as the recognition target (year, month, monetary number, business word, etc.) ) And the character string recognition using notation knowledge that matches this is possible to improve the recognition accuracy.

  First, an outline of a document processing flow using click recognition will be described with reference to FIG. In a document processing apparatus according to an embodiment of the present invention, a paper document is imaged using an OCR apparatus, a scanner apparatus, a document camera, and the like, and a document image converted into electronic image data is handled. First, data of a document image to be processed is read from an external device through an external storage device or a communication line (0101). Next, document structure analysis such as ruled line extraction from the document image data, frame structure analysis, and position estimation of the reading target frame is performed (0102). For the recognition processing used at this time, known techniques (Japanese Patent Laid-Open No. 09-319824 (Patent Document 3), Japanese Patent Laid-Open No. 2000-2521012 (Patent Document 4), etc.) are used. In general, in order to perform document structure analysis, a document structure dictionary storing frame layout information of a target document is used (0108). This document structure dictionary is generally stored in an external storage device. In document structure analysis, document image data and a document structure dictionary recorded in a memory or an external storage device are input, and ruled line position information, frame layout information, and estimated frame attribute information are paired. , Output to memory or external storage device.

Next, in response to the result of the document structure analysis, a character string block that is a recognition target candidate is extracted (0103). In general, a character string block means a block of characters that has no breaks such as white space and is considered to represent the same unit in meaning. For example, a block of words corresponds to a character block. A character line is composed of a plurality of character string blocks. Next, by analyzing the two-dimensional arrangement structure from the extracted character string block information, it is estimated what attributes the character string block has. The two-dimensional arrangement structure includes the position and size of character string blocks in the document image, information on ruled lines existing in the upper, lower, left, and right directions, and information on the arrangement relation between character string blocks. This is referred to as arrangement structure analysis (0104), and the attribute of the character string block is also referred to as a description content category. This description content category includes, for example, a classification such as a date character string, an amount character string, an ID numeric character string, and a general word character string. Generally, in the arrangement structure analysis, arrangement structure definition information stored in an external storage device is used (0109). In the arrangement structure definition information, for each document type, information on a two-dimensional arrangement structure and a description content category of a character string block having the arrangement structure are stored in association with each other. The arrangement structure information includes at least information such as coordinate information, size information, attribute information, information representing adjacent relationships such as up and down, left and right, and description content category information.
The processing described so far is a so-called pre-processing of the recognition processing based on the region / recognition mode selected by the user in the following procedure, and is performed non-selectively on the entire region that the user may select. Is called.

  After the document structure analysis, character string block extraction, and arrangement structure analysis described above are completed, the document image is displayed to the user of the document processing system through the display device (0110). The user performs business processing (search, inspection, data registration) on the document through an information input device such as a keyboard (0111), a mouse (0112), an electronic pen (0113), or a touch pad (0110). The document display / operation control unit (0105) performs interactive processing with humans and mediates between document processing and character recognition processing. In the document display / operation control unit, in accordance with an event (user action) such as mouse click, pen drag, cursor movement, etc., a character recognition of a necessary part of the document (0106), a document such as business database search and result display, etc. Processing (0107) is performed. The character recognition unit (0106) will be described with reference to FIG. 2, and the document processing unit (0107) will be described with reference to FIG.

  FIG. 2 is a diagram showing an internal flow of the character recognition unit. A document display / operation control unit (0105) is provided above the character recognition unit, from which a document image, arrangement structure information, and recognition request information are output. The recognition request information is data describing in what mode the recognition is performed. Next, an area (character string block group) to be recognized is determined by the image area selection unit based on these input data (0201). Next, character extraction is performed on the character line in the recognized recognition area (0202). Further, each extracted character pattern is identified (0203). As a result, a candidate character network as described later with reference to FIGS. 8 and 9 is obtained. The candidate character network represents a character line image to be recognized as a directed graph having a character pattern and its identification result as an edge and a cut point of the character pattern as a node. The character identification unit generally identifies character patterns using a character identification dictionary (0206) stored in an external storage device or memory. Next, a notation analysis is performed on the candidate character network obtained as a result of the processing of the character cutout unit (0202) and the character identification unit (0203) (0204). In notation analysis, in general, a notation knowledge dictionary (0207) stored in an external storage device or memory is used to check what word the character string is composed of, whether the word is arranged, and the like. Determine a text string from the character network.

  At this time, the notation knowledge dictionary (0207) is switched using the description content category information of the character string block to be recognized, which is included in the arrangement structure information input from the upper document display / operation control unit (0105). Do. The notation knowledge dictionary is prepared corresponding to each description content category. Thereby, the character string recognition accuracy of the character string block is improved. There may be multiple category candidates in the description content category. In that case, character string recognition is performed using the notation knowledge dictionary corresponding to each description content category candidate, and multiple results are output in order of likelihood. become. Finally, the recognition result group obtained by the processing of the recognition result integration unit is determined by using the text code obtained as a result of the character identification and the arrangement structure information (0205). The above is the flow of the character recognition unit, and the processing result is returned to the upper processing (0105) as the reading result.

  FIG. 3 is a diagram showing an internal flow of the document processing unit. A document display / operation control unit (0105) is provided above the document processing unit. The document display / operation control unit displays a document image to a user through a display device, and performs processing of the document, encoding of image data, document search, business data browsing, and the like through a control input device. Among them, when character recognition is required, information on the region to be recognized is output to the character recognition unit (0106, details are described in 0201 to 0205), and input of recognition results is received. Further, when document processing such as search or business data browsing is performed in response to the recognition result, necessary data is input to the document processing unit (0107) and processing is entrusted. In the document processing unit, the document image, the arrangement structure information, the recognition result, and the document processing request information are input from the upper level. The document image may be an ID number that uniquely identifies the image, instead of the data as it is. Further, the document processing request information includes information necessary and sufficient to specify what kind of document processing is desired to be performed using the recognition result and the user's request. In response to the above input, the business-related process control unit operates (0301).

  The document processing control unit performs a search from a document database, a search of a business database, or the like according to a user request. For example, when it is desired to search for a document including a recognized word, first, an ambiguity handling process (0302) that absorbs uncertainty and instability of the recognition result is performed, and then data stored in the document database (0304). The requested document is searched from the list (0303). In addition, if you want to search the data that accompanies the recognized word (for example, if you want to know the names of drugs that should not be used at the same time in the case of medical literature), the recognition results are uncertain or unstable as before. After performing the ambiguity handling process (0305) to absorb the necessary information, the necessary information is searched from the business database (0307) based on the recognized keyword (0306). The above document search and information search results are returned to the higher-level document processing control unit, and from there to the higher-level document display / operation control unit (0105) as a document search result or information search result.

  FIG. 4 shows a specific configuration example of the document processing apparatus described above. 4, the paper document is converted into electronic data by the image input device (0401), stored in the external storage device (0404) and the memory (0405), and read by the central processing unit (0406). I do. The central processing unit (0406) performs the processes from 0101 to 0109, the processes from 0201 to 0207, and the processes from 0301 to 0307 among the processes shown in FIGS. As shown in FIG. 4, when the document processing apparatus exists outside, the central processing unit (0406) performs document image related processing from 0101 to 0105, or performs only display / operation related processing of 0105, and the others. It is possible to provide a high-speed document processing function by sharing the above process by the external document processing apparatus. The definition of the document format is stored in the external storage device (0404), and the stored definition is referred to for the document structure analysis. These processes can be operated by a human through the operation terminal device (0402), and the processing results and the like are displayed through the display terminal device (0403).

  Event information such as processing results and recognition activation is stored in an external storage device as needed, or data is sent to an external device through a communication device (0407). The user browses the document image and performs business processing through the display terminal device (0403) and the operation terminal device (0402). When a character recognition result is required during business processing, the character recognition program is activated by designating the corresponding part by clicking the mouse. The character recognition program is stored in the external storage device (0404) or the memory (0405), and is operated by a user's interactive input action. As a result, a large amount of calculation time required for batch processing can be reduced, and document image processing using the character recognition function can be realized with a small system configuration. The above devices are connected by an internal bus (0408).

  The business processing apparatus in the lower part of FIG. 4 performs document retrieval / business database retrieval using the recognition result output from the document display apparatus. The business processing device receives the recognition result from the communication device (0413) and the external storage device (0410), loads the result into the memory (0411), and searches the document or performs business processing by the central processing unit (0412). The necessary data is retrieved, and the result is notified to the document display device through the communication device (0413) and the external communication line (0409). The processing performed by the central processing unit (0412) corresponds to the processing from 0201 to 0207 and the processing from 0301 to 0307 among the processing shown in FIGS. When the central processing unit (0406) exclusively performs the display / operation related processing of 0105, the central processing unit (0412) performs the processing of 0101 to 0104 and 0106 to 0109 in addition to the above. These devices are connected by an internal bus (0414). In this example, the document display device that performs interactive operations and the business processing device that searches the business database are described separately, but a device in which these are integrated may be used.

  FIG. 5 is a diagram describing the concept of click recognition. First, there is a document image (a, 0501). Here, medical-related literature is taken as an example. First, a character string block is extracted from this document image (b). 0502 is an extracted ruled line, and 0503 is a circumscribed rectangle representing a character string block. Next, the arrangement structure analysis is performed using the arrangement information of the character string block (c). This medical literature is roughly classified into four categories by arrangement structure analysis. Reference numeral 0504 represents one of the categories. This analysis is generally performed using the arrangement structure definition. When the user clicks a specific location with the mouse (0505) in the user action, it is click recognition that uses the layout structure analysis information to recognize the text block near the click using the notation knowledge that matches the category. It is. If layout structure analysis cannot be used, identify the character string to be recognized using the character string block, and perform character string recognition using general notation knowledge or not using notation knowledge. become. In general, when the arrangement structure analysis is used, notation knowledge limited to a business category can be used, so that it is possible to reduce misreading of character string recognition.

  FIG. 6 is a diagram showing various actions (user interface) of the user for activating recognition. The recognition designation interface 1 (a) indicates that recognition is performed by clicking a mouse at a location indicated by 0601 or a tap operation of an electronic pen. In the recognition designation interface 2 (b), by dragging or moving the mouse in the horizontal direction (direction shown by 0602) or sliding the electronic pen in the horizontal direction, the width specified by the amount of movement in the horizontal direction is set. The operation | movement which recognizes the character string (character string block) of is shown. At that time, as an operation feedback, an underline indicating a designated range as indicated by 0603 is displayed on the GUI. In the recognition designation interface 3 (c), similarly, a process of recognizing a character string (or character string block) belonging to a line within a designated vertical range by dragging, moving, and sliding the mouse or the electronic pen downward. Show.

  The recognition designation interface 4 (d) is processing for recognizing a character string / character string block in a rectangular area by dragging and sliding a mouse or an electronic pen in an oblique direction. In 0605, the enclosed rectangular area is displayed with a GUI. The recognition designation interface 5 (e) shows processing for recognizing a character string block in or near the area by designating the area with a mouse or an electronic pen in a circle. Reference numeral 0606 denotes a display line when the enclosed area is displayed using a GUI. The recognition designation interface 6 (f) shows a method for designating a recognition area by a gesture. For example, when a check mark as shown in 0607 recognizes all the character string blocks in the vertical direction in which the mark is written, the entire area as shown in 0608 is a recognition target. This can be used in the case of recognizing all the specific fields in the form recognition, and can save time and effort for enclosing the entire area. This is an interface that is made possible by analyzing the arrangement information in advance.

  The recognition result of the object specified by the interface is fed back to the user of the document processing system by means such as displaying it in a pop-up window on the spot or displaying related information retrieved from the business database. If there are multiple recognition results, for example, depending on whether the mouse or electronic pen is stopped (stopped), the recognition results are displayed differently at different times, and the correct recognition results are selected from among them, or the business database A feedback method such as displaying related information retrieved from a plurality of recognition results side by side is conceivable.

  FIG. 7 is a diagram showing that the same action leads to different recognition results depending on the recognition mode using the arrangement structure information. Here, it is assumed that there are four types shown as 0701 as the arrangement structure information (a). When region designation by click recognition is performed on an image that has already undergone arrangement structure analysis (b), the recognition result changes by changing the recognition mode designation. For example, in area selection 1 (b1), “Get (Column (x), Row (x))” is designated as the recognition mode as indicated by 0707. X in this specification sentence indicates the clicked location (location 0706). Column is a column of the location, Row is a row of the location, and constitutes a command statement for acquiring and recognizing the location specified as an argument by Get. In this case, the shaded portion in the figure is selected and subjected to recognition.

  In the area selection 2 (b2), as indicated by 0708, the recognition mode is designated as “Get (ABCD, Row (x))”. This is a designation of selecting categories A, B, C, and D as columns, and selecting the row at the clicked location as a row. Categories A, B, C, and D are arrangement structure information in 0701, and correspond to vertical series shown in 0702, 0703, 0704, and 0705, respectively. Therefore, in (b2), the area shown by shading is selected, and recognition is performed for each block.

In area selection 3 (b3), as shown in 0709, the column of the clicked part and all the rows are selected as the recognition mode. Therefore, in this case, all of the B series (the portion 0703) shown in the halftone screen is selected, indicating that these are to be recognized. Similarly, the same operation occurs not only by specifying an area by clicking but also by specifying an area by dragging and sliding (c). In 0710, two blocks are designated by line designation. When the recognition mode is as shown in 0711, it indicates that the designated row group and column group are selected as the recognition target area (c1). Such a recognition mode can be selected in advance according to the business mode of document processing. For example, if the vertical measurement of a specific item is desired, if 0709 is selected as the recognition mode, the vertical direction of items desired by the user can be collectively recognized with a simple operation.
The character string notation analysis process and the character string hypothesis are outlined in FIGS. FIG. 8 is a diagram for explaining the flow of character string recognition using the character string hypothesis and notation knowledge. FIG. 9 is a diagram showing the concept of the character string hypothesis and details of the data.

  FIG. 8 will be described. A character string hypothesis (b) is a character pattern hypothesis (b) in which a portion estimated to be a character pattern is cut out from a character line (a) to be read to create a character pattern candidate and each character pattern candidate is identified. It is assumed that the character string hypothesis has at least character pattern candidates, ranked identification character code groups obtained as a result of character identification, and information on connection relations between character pattern candidates in the character string hypothesis. In this way, the character string hypothesis is represented by a representation in a graph format, and is therefore also referred to as a candidate character network. Next, the character string path knowledge (c) is used to calculate the character string path (d) from the character string hypothesis (candidate character network). The character string path means a character code string (text) uniquely determined and a character pattern corresponding to each character code. In this example, knowledge of character string notation is expressed by arranging words with an OR symbol (|). That is, it means that a word group sandwiched between symbols | is designated as notation knowledge. In addition to this expression, there is a method using a try, a context free grammar, etc. (described in Japanese Patent Application Laid-Open No. 2001-014411 (Patent Document 5)).

  Details of the character string hypothesis (candidate character network) are shown in FIG. The character string hypothesis is expressed as a directed graph in which a character pattern candidate is an arc (0901) and a character pattern boundary is a node (0902). Each character pattern includes boundary ID numbers representing left and right (upper and lower if vertical writing) (pattern boundaries), information on character identification candidates (0903), and identification similarity (0904). Knowledge processing is processing for finding words and pattern strings that can be included in the character string hypothesis using the character string hypothesis and knowledge of character string notation as inputs. For example, the word “blood chemistry test” in the character string notation knowledge can be found by following the character code and character pattern (0905) indicated by a circle in the character string hypothesis of FIG. When the notation of the character string written in the field is determined in advance, the character string code is determined by performing this process.

  By the processing described above, the character recognition processing is interactively activated in the document processing operation, and the text is limited to only the portion necessary for the document processing, so that the processing time can be reduced. In addition, by analyzing the two-dimensional arrangement structure of the character string described in the document, the category of the description content of the character string designated as the recognition target (year, month, monetary number, business word, etc.) ) And the character string recognition using notation knowledge that matches this is possible to improve the recognition accuracy.

The document processing flowchart using click recognition. The processing flow figure of a character recognition part. The processing flow figure of a document processing part. 2 is a configuration example of a document display device and a document processing device. The conceptual diagram of the click recognition using arrangement | positioning structure analysis. Click recognition interface. Click recognition range mode specification. The conceptual diagram of the notation knowledge process using a character string hypothesis. The conceptual diagram of a character string hypothesis.

Explanation of symbols

0101: Image input unit, 0102 ... Document structure analysis unit, 0103 ... Character line / character block extraction unit, 0104 ... Arrangement structure analysis unit, 0105 ... Document display / operation control unit, 0106 ... Character recognition unit, 0107 ... Document processing unit , 0108 ... Document structure dictionary, 0109 ... Arrangement structure definition, 0110 ... Display device, 0111 ... Keyboard, 0112 ... Mouse, 0113 ... Electronic pen,
0201 ... Recognition target area (character string block group) selection unit, 0202 ... Character extraction unit, 0203 ... Character identification unit, 0204 ... Notation analysis unit, 0205 ... Recognition result integration unit, 0206 ... Character identification dictionary, 0207 ... Notation knowledge dictionary,
0301 ... Business related processing control unit, 0302 ... Ambiguity correspondence unit, 0303 ... Document search unit, 0304 ... Document database, 0305 ... Ambiguity correspondence unit, 0306 ... Information search unit, 0307 ... Business database,
0401: Image input device in document display device, 0402: Operation terminal device in document display device, 0403 ... Display terminal device in document display device, 0404 ... External storage device in document display device, 0405 ... Memory in document display device, 0406 ... Central processing unit in document display device, 0407 ... Communication device in document display device, 0408 ... Internal bus in document display device, 0409 ... Data communication line, 0410 ... External storage device in business processing device, 0411 ... Memory in business processing device, 0412 ... Central processing unit in the business processing device, 0413 ... Communication device in the business processing device, 0414 ... Internal bus in the business processing device,
0501 ... Example of document image to be processed, 0502 ... Ruled lines extracted from document image, 0503 ... Character string block extracted from document image, 0504 ... Character string block with tag as a result of arrangement structure analysis, 0505 ... Cursor such as mouse and electronic pen, 0506 ... Click recognition result,
0601: Cursor such as mouse / electronic pen, 0602: Arrow representing cursor movement, 0603: Horizontal line displaying selection area with GUI, 0604 ... Vertical line displaying selection area with GUI, 0605 ... circumscribing selection area with GUI Enclosed line displayed as a rectangle, 0606... Enclosed line with a selection area displayed as a circle in the GUI, 0607... Movement line when a gesture is performed, 0608... Enclosed line indicating an area selected as a result of the gesture.
0701 ... Arrangement structure information, 0702 ... A category of arrangement structure information, 0703 ... B category of arrangement structure information, 0704 ... C category of arrangement structure information, 0705 ... D category of arrangement structure information, 0706 ... Movement of the cursor (corresponding location) , 0707... Region selection 1 by recognition mode designation and cursor click position 1, 0708 ... region selection by recognition mode designation and cursor click position 2, 0709 ... region selection by recognition mode designation and cursor click position 3, 0710 ... Cursor movement (drag line at the location), 0711 ... Selection of region by specifying recognition mode and cursor drag line position,
0901: Cutout character pattern and identification candidate (graph arc), 0902: Character cutout boundary (graph node), 0903 ... Character identification candidate group, 0904 ... Identification similarity group corresponding to character identification candidate, 0905 ... Knowledge Character identification candidates selected as a result of processing.

Claims (5)

  An image input device that accepts input of a document image in which characters are written, a central processing unit, a storage device that holds notation knowledge prepared in correspondence with an arrangement structure definition and a description content category, a display device, and a user input device An interactive document processing device comprising an operation terminal device having a communication device and a communication device, wherein the central processing unit extracts a plurality of character string blocks from an input document image, By analyzing a two-dimensional arrangement structure on a document and referring to the arrangement structure definition based on the arrangement structure, a description content category representing the description contents of the character string block is estimated, and the operation terminal device When the input unit receives an input of an operation for instructing activation of recognition processing from a user of the interactive document processing apparatus, the character string block to be specified is selected and the sentence is selected. A character extraction candidate is extracted from the column block, the character extraction candidate is further identified, and the character identification result and the description corresponding to the character string block selected above for the character extraction candidate A document processing apparatus that recognizes the character string block with reference to notation knowledge related to a category.   In the central processing unit, a cursor is moved and clicked through a user input device such as a mouse, an electronic pen, and a touch pad, and a display operation terminal device including a display device such as a cathode ray tube display, a liquid crystal display, and a portable display terminal. Event such as drag, gesture, etc. is accepted as an instruction to start recognition processing, and the selection method of the character string block to be recognized is switched according to the recognition mode designation status stored in the storage device. The notation knowledge corresponding to the character string block is switched according to the description content category of the character string block to recognize the character string, and the output format of the reading result by the display device is switched according to the recognition mode. The document processing apparatus according to claim 1.   The central processing unit further lists a plurality of description content category candidates by estimating the description content of the character string block, calculates the likelihood of the description content category, extracts the character extraction candidates, and determines the character identification Is used to calculate the character identification result and the similarity of the identification result, the likelihood of the description content category of the estimated character string block, the similarity of the character identification result, and the notation corresponding to the description content category The text that is the reading result of the character string block obtained by applying the knowledge is also used as the reading result of the character string block, the reading results of the character string block are arranged in order of likelihood, and a plurality of reading results are obtained. The document processing apparatus according to claim 1, wherein the document processing apparatus stores the reading result in a data storage device or sends the read result to another document processing device through the communication device.   2. The document processing apparatus according to claim 1, comprising an operation terminal device, a storage device, a central processing unit, and a communication device, and a business processing device including the central processing unit, the storage device, and the communication device. The central processing unit of the interactive document processing device transmits a reading result of a character string block group designated as a recognition target to the business processing device through the communication device, and the reading processing system is configured. The central processing unit of the business processing device retrieves information related to the result from the business database stored in the storage device, and transmits information obtained from the business database from the communication device of the business processing device, Information is received by the interactive document processing apparatus, and the received information is transmitted as data related to the character string block designated as a recognition target through the display operation terminal apparatus. Document processing system is presented to the user, and performs an interactive document processing.   A program executed by a computer including an operation terminal device, a storage device, and a central processing unit, the step of receiving input of an image in which characters are described in the central processing unit, and a character string from the image Extracting a block; estimating a description content category representing a description content of the character string block from a two-dimensional arrangement structure of the character string block with reference to an arrangement structure definition stored in the storage device; A step of displaying the image on a display screen of the operation terminal device and receiving a user operation input from an input unit of the operation terminal device; a step of determining a recognition start by a user operation; and a recognition start Receiving a character string block to be recognized from the image and extracting a character extraction candidate from the character string block; and The candidate is character-identified, and as a result of the character identification, the description content category corresponding to the character string block and the data including the character extraction candidate match the description content category held in the storage device. A step of performing character string recognition using notation knowledge, a step of retrieving related information from a business database in response to the reading result, and a step of displaying the related information on a display terminal Program to do.

Images