JP2008003740A - Input correction method, postscript information processing method, postscript information processor, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately specify hnadwritten input information to be corrected and correct efficiently, quickly and accurately in a system for performing automatic data processing on the basis of the input information. <P>SOLUTION: An input correction method obtains not only a feature quantity related to reliability of recognition processing itself of postscript information of data processing object but also feature quantity related to reliability of a variety of processing relating to the recognition processing in a preceding step of the recognition processing (S300), and calculates sub-reliability of each processing, based on each feature quantity (S320). The method specifies final reliability of notable postscript information on the basis of each sub-reliability (S304). The method specifies difficult-to-recognize information of which the reliability of the recognition processing is lower than a prescribed level by determining whether or not the final reliability is lower than the prescribed level (S306), and prompts to correct the difficult-to-recognize information by presenting recognition performance information to enhance the recognition performance of the specified difficult-to-recognize information (S308, S310). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an input correction method, a write-once information processing method, a write-once information processing apparatus, and a program to which the input correction method is applied. More specifically, for example, it is used when additional information (also referred to as additional information or annotation) additionally written in a document by handwriting is separated from the document body and used for various information processing related to the document body. This relates to a mechanism for correcting input additional information.

  In today's remarkable progress in information processing technology, there is a mechanism for automatically processing various processes related to documents. For example, additional additional information is additionally written by hand on a document original in which predetermined information is written in advance, and the additional document with the additional information added is processed, based on the additional information added by handwriting. There is a mechanism for automatically executing predetermined data processing.

  For example, there is an automatic form processing mechanism (see, for example, Patent Documents 1 and 2), a date and a schedule entry field in which information is entered by handwriting on a standard form (so-called form) and the entered information is processed. There is also a mechanism for personal information management (particularly referred to as schedule management) (see Patent Document 3) in which a schedule is written by hand in a prepared notebook, memo or the like, and the entered schedule is converted into electronic data.

  Also, in schools, school cram schools, etc., automatic scoring and summarization processing is performed on papers (educational teaching materials) that result from scoring on answer sheets filled with answers by students and examinees. There is also a mechanism (see Patent Document 4).

JP-A-5-342239 JP-A-6-274156 JP-A-5-216932 Japanese Patent Laid-Open No. 11-31046

  For example, in Patent Document 1, an identification mark for designating the type of table, the position in the row direction and the position in the column direction of each data in the table is added, and the table data is printed as a data table form with an identification mark. By reading the recognition mark attached to the data table form with the mark reading unit, the form is identified from the identification data and the corresponding table data is displayed on the display unit, and the corresponding part is corrected. A mechanism has been proposed in which data can be corrected and collated efficiently with a small amount of work by displaying in a possible state.

  Japanese Patent Application Laid-Open No. 2004-228561 converts an output document into image data, displays the converted image data, indicates form specified position coordinates on the displayed image, and based on the instructed form specified position coordinates. Form graphic data can be derived and the form can be overlaid on the image data displayed on the display unit based on the derived form graphic data so that a form for a desired output document can be created easily. A mechanism has been proposed.

  Further, in Patent Document 3, as a mechanism for managing personal information, paper such as a written notebook is input from a scanner, characters and frame lines are detected by a known method, and the detected characters and frame lines are detected. There has been proposed a mechanism for collating the layout with knowledge of a notebook format stored in advance and generating data representing the attributes of each description item.

  Information handwritten in a notebook or memo (in this example, the written description of a schedule entered in the notebook in handwriting) is automatically converted to electronic data, and each written item is switched in correspondence with the generated attribute. When the user requests output of a certain format, the contents of the database are printed at a location corresponding to each content on the paper by referring to the output format stored in advance. It can be useful.

  Further, in Patent Document 4, a handwritten entry field on an answer sheet is defined as an input area on the tablet, and format data that defines the input area to be aggregated is stored in a storage device, and the answer is stored in the tablet. Key entry after scoring, etc., by entering the information entered in the handwritten entry field as written data entered in the tablet input area and recognizing characters with the sheets superimposed A mechanism has been proposed to save labor in data input by.

  On the other hand, when executing data processing based on additional information input by handwriting, how to recognize the input handwritten information (characters and figures), or automatically recognize if the automatic recognition result is unreliable The problem is how to efficiently correct the processed recognition result, and various mechanisms for automatic recognition / correction of such handwritten information are considered (for example, see Patent Document 5). reference).

JP 2004-152115 A

  Here, in the mechanism described in Patent Document 5, the certainty factor of content is calculated for each item for a plurality of items included in the acquired data, and the presentation method is dynamically changed using the calculated certainty factor. I am doing so. By using the certainty of each item included in the input data and dynamically changing the correction method of the data, the contents of highly related items such as correcting the address from the zip code are corrected or always suspicious. In other words, it is possible to preferentially correct the contents of items with low confidence, and to make it possible to efficiently and promptly and accurately perform input correction work involving the operator regardless of the data input method. I have to.

  However, since the mechanism described in Patent Document 5 only calculates the “certainty factor” by using the similarity of pattern recognition when recognizing each item, it occurs in various pre-processes before the recognition process. The event that affects the recognition process is not considered. Therefore, for example, the presence or absence of local noise affects the “confidence”, and the accuracy of the “confidence” becomes a problem. This is because if there is a problem at the pre-processing stage, the input to the recognition process has already been altered, and there is a possibility that a recognition result of “high reliability but incorrect” may be output.

  The present invention has been made in view of the above circumstances, and it is possible to perform correction work efficiently and quickly with respect to additional information provided for automatic data processing, and to add additional information that requires correction with high accuracy. The purpose is to provide a mechanism that can be specified.

  In the mechanism according to the present invention, for the additional information of interest, the feature amount related to the reliability of the recognition process is acquired, and the sub-reliability related to the recognition process of each process is calculated based on each feature amount. Based on the sub-reliability, the final reliability of the additional information to be noticed is identified, and the reliability of the recognition process is lower than the fixed level by determining whether or not the final reliability is lower than the fixed level. The recognition performance information for improving the recognition performance of the identified difficulty recognition information is presented.

  Thereafter, the corrected additional information entered corresponding to the presentation of the recognition performance information is reflected in the data processing. As a method of reflecting the correction, for example, a technique realized by directly changing data stored after recognition can be adopted. For example, the correction result may be reflected by changing the data directly by database operation after recognition.

  Here, the “feature value related to the reliability of the recognition process” includes not only the feature quantity related to the reliability in the recognition process itself but also the feature quantity related to the reliability in various processes related to the recognition process preceding the recognition process. It is a waste.

  Identify difficult-to-recognize information whose recognition process reliability is lower than a certain level, taking into account not only the recognition process itself, but also the events that affect the recognition process that can occur in the various pre-processes before the recognition process. To do.

  The mechanism according to the present invention can be realized by software using an electronic computer (computer), and a program for this purpose and a recording medium storing this program can also be extracted as an invention. The program may be provided by being stored in a computer-readable storage medium, or may be provided by distribution via wired or wireless communication means.

  According to the present invention, not only the feature quantity related to the reliability in the recognition process itself but also the feature quantity related to the reliability in various processes related to the recognition process preceding the recognition process is acquired, and the reliability of the recognition process is constant. The difficulty recognition information lower than the standard was specified.

  As a result, it is possible to consider events that may affect the recognition process that may occur in various pre-processes prior to the recognition process.Therefore, it is not necessary to check all additional information and cause erroneous recognition. In addition to being able to efficiently and quickly correct the postscript information that can be corrected, it is possible to identify difficult recognition information with high accuracy. It becomes possible to specify the postscript information to be corrected with high accuracy and to efficiently and quickly correct it.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Example of processing target document>
FIG. 1 is a diagram showing an example of a document to be processed in an information processing system including an additional information processing apparatus according to the present invention.

  The document shown in FIG. 1 is an educational material 80 used as a paper medium in an educational setting, and typically has a question sentence 82 and an answer column 84 (part shown in parentheses in this example). FIG. 1 (A) shows the original document 8A, and FIG. 1 (B) schematically shows a state in which additional information has been entered (added document 8B). An example of registration information in the database is shown in (C).

  Specifically, the educational material 80 corresponds to a paper test or a practice question sheet used in an educational institution. The educational material 80 only needs to have at least an answer column 84. For example, an answer may be written in the answer column 84 for a question read by the grader, and the question sentence 82 is not necessarily described. Not necessary.

  As shown in FIG. 1 (A), the educational material 80 includes not only a question sentence 82 and an answer field 84 in which an answer, which is an example of the first type of additional information, but also a scoring field 83 (by question). An item score field 83a, a partial score field, and a total score field 83b), and an identification information field 85 for entering information for identifying and specifying the educational material 80 as an example of the first type of additional information; , Answerer information which is an example of the first type of additional information, and an answerer information column 86 in which information related to the answer entry person in the answer column 84 is entered. The answer field 84, the identification information field 85, and the answerer information field 86 are all examples of entry fields for entering the first type of additional information.

  In the identification information column 85, for example, the subject, title, or applicable grade of the educational material 80 is described in advance. However, in addition to these descriptions or separately from these descriptions, code information for identifying the educational material 80 may be embedded.

  The embedding of the code information may be realized by using a known technique. As one specific example, for example, a line screen or a dot screen as a gradation expression such as a so-called “iTone (registered trademark)” is used. It is conceivable to use a technique of embedding digital information in a halftone image by changing the form (position, shape, etc.) of the pixels constituting the image. On the other hand, in the answerer information column 86, the class 86a, attendance number 86b, name 86c, etc. of the answer entry person can be entered.

  In the scoring field 83 (particularly, the item scoring field 83a), scoring information for each answer field 84 is entered. Scoring information is information for identifying the number of points assigned to the answer column 84 at each position for each answer column 84 in the educational material 80. The points may be different for each answer column 84 or may be uniform.

  Such educational teaching material 80 can be obtained by printing out with a printing apparatus based on the electronic data of the corresponding original (teaching material original). The electronic data of the original teaching material can be generated using application software such as word processing software using an electronic computer such as a personal computer, and is stored in a predetermined database or the like in advance.

  The electronic data of the original teaching material can specify the layout of the answer column 84 and the identification information column 85 in the educational teaching material 80 and can be stored and accumulated in a predetermined database. Any data format is acceptable. For example, the application document data created by the document creation software is not limited to image data.

  At the educational site, educational teaching material 80, which is an example of the original document 8A shown in FIG. 1A, is distributed to students, examinees, etc. First, the names and answer fields 84 to the answerer information field 86 are assigned by the students. The first type additional information such as the answer to is entered in a predetermined column and collected. After this, a scoring symbol (correct / incorrect determination) for the answer entered in each answer field 84 by a scoring officer such as a teacher, as in the additional information filled teaching material 81 which is an example of the added document 8B shown in FIG. The second type of additional information such as a comment 88 indicated by a graphic) 87 or other graphic or text related to the scoring symbol 87 is entered. Note that since the additional information processing apparatus 10 performs automatic scoring processing, there is no entry in the scoring field 83C by the scoring officer at this time.

  In this case, the pen color usually used for the entry of the first type of additional information by the student is different from the pen color used for the entry of the second type of additional information by the grader or the like. The color used is different from the color pre-filled as the educational material 80.

  In addition, as the scoring symbol 87, for example, “O” indicating a correct answer and other figures (for example, an elliptical figure), “X” indicating an incorrect answer and other figures (for example, a check mark such as “R”), There are “△” indicating a correct answer and other figures.

  The comment 88 is information that does not have a direct relationship with the first data processing based on the scoring symbol 87, but supports (reinforces) the result of the first data processing or has no relationship at all. It is used for data processing.

  In the learning material automatic scoring system, predetermined data processing is performed for the additional information filled learning material 81 which is an example of the added document 8B. At this time, in addition to the second type of additional information (in this example, scoring symbol 87 and comment 88), individual data processing corresponding to each is performed. In this example, automatic scoring processing based on the scoring symbol 87 is performed as the first data processing. As this automatic scoring processing result, for example, scoring information to be entered in the scoring field 83C is acquired. Further, the second data processing different from the automatic scoring processing result based on the additionally written contents of the comment 88 is performed.

  As a second data process different from the automatic scoring process, for example, a process for classifying the additional contents of the comment 88 based on a predetermined condition, a classified comment (not only all the additional contents but also a part of the additional data) The content may be registered) and stored in the student guidance database in association with the scoring processing result for the corresponding answer column 84.

  In this way, as a use form of the second data processing different from the automatic scoring process, information can be taken out from the student guidance database and used for subsequent student guidance. For example, if all the scoring results and the corresponding comments are displayed, the scoring results can be confirmed together with the subcategories of evaluation such as excellent, excellent, average, inferior. In addition, if only comments with incorrect answers are displayed, it can be used to analyze the ability of students who have many mistakes in reading questions, many mistakes in writing answers, or many calculation errors. Alternatively, it may be used for student guidance based on a cautionary note or mistake correction indicated by the comment 88.

  In addition, when registering the educational material 80 (original image) for executing data processing for the scoring symbol 87 and the comment 88 in the database, it is usually necessary when processing the data for the scoring symbol 87. The entry field position area information 38 indicating the position information, the question number, the scoring information, etc. of the answer field 84 and the classification reference information necessary for data processing for the comment 88 are registered. It should be noted that the information of the educational material 80 itself is text data or document file data corresponding to the text data so that a character part (for example, a question sentence including a question number or a score) in the educational material 80 can be referred to as text information. It is preferable to preserve it.

  For example, as shown in FIG. 1C, the answer field position area information, which is an example of the entry field position area information 38, is replaced with the number of the problem existing on the educational material 80 (an example of first attribute information). , Points assigned to the answer of the question (an example of second attribute information), the xy coordinates of a predetermined point (for example, the upper left vertex) of the area treated as the answer field 84 for entering the answer of the question, and the circumscribed rectangle Information consisting of a width (W) and a height (h) is stored and accumulated in a predetermined storage area in a table format that associates them with each other.

  When the original teaching material 80 corresponding to the additional information filled teaching material 81 is not registered in the document management server, the blank teaching material 80 is read by the document input device, and the position of the question sentence and the answer column 84 are read. This is dealt with by specifying points and points.

  In addition, when executing automatic data processing for additional information, if additional information is classified into a plurality of types and individual data processing is to be performed for each classification, each additional information is assigned to the other It is necessary to identify and identify them separately. If the separation and recognition of both are not properly performed, the respective data processing cannot be properly performed.

  For example, in the case of the additional information-added teaching material 81 shown in FIG. 1, there are an automatic scoring process based on the scoring symbol 87 and a student guidance database construction process based on the comment 88, and such individual processing is realized. In this case, it is necessary to distinguish between the grading symbol 87 and the comment 88 and perform a recognition process or the like before performing a final data process.

  On the other hand, in scoring the answer, a comment 88 other than the scoring symbol 87 may be described with the same pen as that of the scoring symbol 87. For example, the pen color corresponds to the pen color through color component recognition processing on the extraction result in the difference extraction unit 132. By simply extracting the predetermined color component, it is not possible to recognize the two appropriately and specify the description. The scoring symbol 87 and the comment 88 cannot be properly separated, which adversely affects the automatic scoring process. In addition to the scoring symbol 87 for automatic scoring processing, the postscript information 81 after returning the additional information filled teaching material 81 to the student, although there is a comment 88 that can be used for student guidance etc. , It will be a situation where students can not use it other than confirming the description.

  In order to avoid such a situation, for example, a plurality of additional information existing in the additionally written document 8B is based on various features of additional information such as a description position, an image feature amount, or reliability during recognition processing. Thus, it is preferable to classify (separate) into a plurality of types. At this time, it is preferable to perform more accurate separation recognition by referring to a plurality of features as well as based on a single feature of the additional information. In addition, detailed description is omitted about these.

<System configuration>
FIG. 2 is a diagram illustrating a configuration example of an embodiment of an information processing system including the additional information processing apparatus according to the present invention. Note that this information processing system is shown as an application example to a teaching material automatic scoring system that targets teaching materials 80 such as answer sheets.

  As shown in the figure, the automatic teaching material scoring system 1 includes a write-once information processing device 10 that forms the center of the system, and a document input device that digitizes and inputs additional information-filled learning material 81 that is a processing target document into the write-once information processing device 10. 20, a document management server 30 that stores electronic data of the template 6 corresponding to the additional information filled teaching material 81 that is the processing target document and the educational teaching material 80 (specifically, the original image), and information processing (automatic in this example). A processing result storage server 40 that stores results of scoring processing and the like is connected to the network using wired or wireless communication.

  The document input device 20 enters an answer in the answer field 84 in the educational material 80, fills in the name in the answerer information field 86, and a scoring symbol 87 by the grader for the answer entered in the answer field 84 (specifically, For example, the additional information filled teaching material 81 in which “○” or “×” is entered) is read using a known optical image reading technique, and the additional information filled teaching material 81 is read. Image data is obtained from the image data.

  The document input device 20 only needs to have a function of converting the additional information-filled teaching material 81, which is a processing target document, into electronic data. For example, a copying machine, a multifunction device, or It can be realized by using a scanner device. In that case, if an automatic document feeder (ADF) is attached, it is convenient that images can be read continuously for a plurality of educational materials.

  Note that the test using the educational material 80 is not limited to using a paper medium. For example, in the case where a test is performed using a tablet PC and the scoring is performed, the teaching material 81 in which the additional information has been entered can be obtained from the beginning in the form of electronic data. In this case, the document input device 20 is included in the system configuration. Is no longer necessary.

  The document management server 30 obtains the original image of the educational material 80 corresponding to the additional information-filled educational material 81, and identification information and identification codes such as subjects, titles, and applicable grades for specifying the original image. In association therewith, it is registered as a document original information database DB2 in a predetermined storage medium such as a hard disk device or an optical disk device.

  Further, the document management server 30 stores the original educational teaching material 80 (original image) corresponding to the additional information-added educational material 81, and also issues the problem number and the stipulation necessary for data processing for the grading symbol 87 and the comment 88. As shown in FIG. 1C, entry column position area information 38 indicating information and the like is held and accumulated as a document original information database DB2 in a predetermined storage area in a table format.

  The processing result storage server 40 may be any server that is connected to the appending information processing apparatus 10 through a network and can manage the automatic scoring and counting results for the additional information filled teaching material 81. For example, the processing result database apparatus or the processing Applicable to result file server devices.

  The postscript information processing apparatus 10 that forms the center of the teaching material automatic scoring system 1 includes a read image processing unit 110 that performs predetermined signal processing based on image data of the additional information filled teaching material 81 input from the document input device 20, and a reading A document original specifying unit 120 that identifies and specifies the original document 8A that is the basis of the read image input from the document input device 20 based on processing by the image processing unit 110 is provided. In the learning material automatic scoring system 1, the document original specifying unit 120 functions as a learning material specifying unit 122 that specifies an educational material 80 which is an example of the document original 8A.

  Further, the additional recording information processing apparatus 10 includes an additional recording information extraction unit 130 that extracts additional recording information (annotation) from the image data processed by the read image processing unit 110, and a data processing target extracted by the additional recording information extraction unit 130. The data processing target additional information specifying processing unit 150 for recognizing and specifying the description content and entry position based on the additional information, and data processing based on the description content of the additional information specified by the data processing target additional information specifying processing unit 150 And a data processing unit 170 to perform.

  Further, the postscript information processing apparatus 10 is specified based on information (specifically, recognition rate) indicating the recognition performance of handwritten input information such as handwritten characters and figures as a component unique to the present embodiment. A recognition performance information presentation processing unit 190 that presents the user with recognition performance information for improving the recognition performance of the difficult recognition information with respect to difficult recognition information whose reliability of recognition processing is lower than a certain level.

  The recognition performance information presentation processing unit 190 identifies difficult recognition information whose reliability of recognition processing is lower than a certain level from among the scoring symbols 87 and comments 88 that are additionally written, and determines the recognition performance of the identified difficult recognition information. The user terminal 171 presents recognition performance information for improvement. The data processing unit 170 performs automatic scoring processing, automatic comment classification processing, and the like using the corrected scoring symbols 87 and comments 88 entered by the user in response to presentation of the recognition performance information by the recognition performance information presentation processing unit 190. Perform data processing.

  Here, the recognition performance information presentation processing unit 190 of the present embodiment recognizes each of the plurality of processes in each functional unit of the additional information processing apparatus 10 to enable efficient correction of additional information to be recognized. By collecting information related to the rate and automatically recognizing the contents of the entry, we extract particularly difficult recognition information that is less reliable, that is, difficult to recognize than a certain level. It is characterized in that recognition performance information about information is presented to prompt correction. By eliminating the need to check whether or not all additional information needs to be corrected, the correction work for erroneously recognized additional information is made more efficient.

  For example, in order to obtain the effect of “no need to check all figures”, it is necessary that the warning based on the reliability be highly accurate. This is because the warning cannot be trusted if there is an omission. If there is a problem at the pre-processing stage, the input to the recognition process has already changed, and there is a possibility that a recognition result of “high reliability but wrong” will be output.

  Therefore, by collecting information not only from recognition processing but also from multiple processes, high-precision reliability is calculated, and high-precision warnings are issued from that, for the first time, “Efficiency without having to check all figures” The effect of "is to be obtained." Details of this point will be described later.

  Although not shown, the read image processing unit 110 performs well-known images such as layout analysis, character / graphic separation, character recognition, code information recognition, graphic processing, and color component recognition on the image data input from the document input device 20. An image data analysis unit that performs analysis processing using a processing technique (each detailed description is omitted), an inclination of image data input from the document input device 20, a scaling factor in the main scanning direction or the sub-scanning direction, and the like And a distortion correction unit that corrects image distortion. The distortion correction unit compares and collates the image data input from the document input device 20 with the corresponding original image in the document management server 30 to be compared, and corrects the image distortion (tilt, scaling, etc.). May be.

  The teaching material specifying unit 122 omits the illustration, but, for example, the identification that analyzes the identification information such as the subject, the title, or the applied grade that is entered in the identification information column 85 based on the data analysis result by the image data analysis unit An information analysis unit and a code information analysis unit that analyzes code information that identifies the educational material 80 embedded in the identification information field 85 are also included.

  The teaching material specifying unit 122 is identified based on the analysis result in the image data analysis unit, for example, identification information and identification codes such as subjects, titles, applied grades, and the like, and the teaching material 80 stored and stored in the document management server 30. If the corresponding original image is not stored and stored in the document management server 30 with reference to the information of the original image (for example, identification information or identification code such as subject, title, applied grade, etc.), it is obtained by the document input device 20. It is determined that the electronic data to be compared with the image data cannot be specified, and an identification specifying error signal is output.

  Note that the learning material specifying unit 122 only needs to be able to identify and specify the original educational material 80 corresponding to the image data input from the document input device 20 (corresponding to the additional information-filled learning material 81). The analysis unit only needs to be provided with an appropriate one corresponding to the format of the identification information described or embedded in the identification information column 85 of the additional information filled-in teaching material 81, and is not necessarily provided with both. .

  The postscript information extraction unit 130 includes image data whose image distortion has been corrected by the distortion correction unit, and image data specified by the teaching material specifying unit 122 and input from the document input device 20 (corresponding to the additional information filled teaching material 81). ) To the original image (corresponding to the educational material 80 for education) using a known image processing technique, and a difference extracting unit 132 for extracting a difference between them.

  Further, the additional information extraction unit 130 extracts answerer information (an example of the first type of additional information) in the additional information filled teaching material 81 to be read by the document input device 20 based on the extraction result by the difference extraction unit 132. The answerer extraction unit 134 that performs the same processing as that of the graded symbol 87 and the comment 88 in the additional information-filled teaching material 81 to be read by the document input device 20 based on the extraction result by the difference extraction unit 132. And a data processing additional information extraction processing unit 140 for extracting two types of additional information.

  Based on the extraction result by the difference extraction unit 132, the answerer extraction unit 134 obtains handwritten numbers and character images of answerers written in the class 88a, attendance number 88b, and name 88c fields of the answerer information column 86. Handwritten information extraction unit 136 that extracts the text information as it is, and handwritten information in the answerer information column 86 based on the extraction result by the difference extraction unit 132 (preferably about the handwritten information extracted by the handwriting information extraction unit 136) And a character recognition processing (OCR: Optical Character Reader) unit 138 that converts information into character data that can be processed and edited on the additional information processing apparatus 10.

  The answerer extraction unit 134 only needs to be able to extract the answerer information entered in the answerer information column 86 in the additional information-filled teaching material 81 to be read by the document input device 20. Any one of the character recognition processing units 138 may be provided. In addition, regarding the part of the comment 88 in which the character recognition processing unit 138 is not provided or the character recognition processing unit 138 cannot recognize the character, the extracted answerer information is handled as an image as it is.

  The additional write information extraction processing unit 140 for data processing refers to the additional write color information of interest and, based on the extraction result by the difference extraction unit 132, of the additional write information extracted by the additional write information extraction unit 130, the document input device 20. The data processing target additional information extracting unit 142 for extracting the data processing target additional information 9a (the scoring symbol 87 and the comment 88 in this example) in the additional written document 8B (in this example, the additional information-added teaching material 81) that has been read in (1). And an additional write information shaping unit 146 that shapes the additional write information of the data processing target extracted by the data processing target additional write information extraction unit 142 so as to withstand the data processing. In the present embodiment, the data processing target additional information extraction unit 142 includes functions of a scoring symbol extraction unit that extracts a scoring symbol 87 and a comment extraction unit that extracts a comment 88.

  The data processing target additional information extracting unit 142 only needs to be able to extract the data processing target additional information 9a, and may further classify the additional information using the color as an index. For example, through a color component recognition process on the extraction result in the difference extraction unit 132, the grader may extract a predetermined color component corresponding to the pen color used for entering the scoring symbol 87 and the comment 88. For example, the scoring symbol 87 and the comment 88 in the additional information-filled teaching material 81 are generally entered with a red pen (a red writing instrument). In this case, it is only necessary to perform an extraction focusing on the red component.

  However, even though the red pen is used, there are similar colors such as pink to orange, and the red pen is not always used for entering the scoring symbol 87 and the comment 88. The scoring symbol 87 and the comment 88 are not necessarily used. May be entered in a different pen color, so that the pen color information used for entering the scoring symbol 87 and the comment 88 is added to the data processing target additional information extracting unit 142 functioning as a scoring symbol extracting unit and a comment extracting unit. It is preferable to improve the extraction performance by configuring it to be settable.

  For this reason, the write-on color actually used is specified, and extraction with attention to the write-on color is performed with reference to the information on the specified write-on color. If the pen color actually used is specified, the data processing target additional information extracting unit 142 can narrow the extraction allowable range. Thereby, the grading symbol 87 and the comment 88 can be extracted with high accuracy and distinguished from other additional information.

  The additional write information shaping unit 146 performs correction processing on the additional write information of the data processing target extracted by the data processing target additional write information extracting unit 142 so that the line segments are connected to each other and the interruption between the extracted line segments is eliminated. It is preferable that the extraction line segment break correction unit 148 be included.

  In general, figure entry on the added document 8B, figure entry such as "double line", "wavy line", "Hanamaru", "arrow", etc., and comment sentences, etc., are repeated on the completed entry. Sometimes. For example, in the case of the additional information filled-in teaching material 81, the grades such as “○” and “x” are superimposed on the frame for identifying each question sentence 82 and each answer column 84 and the answer entry contents in each answer column 84. The symbol 87 may be entered, or the comment 88 may be additionally written with graphics or characters. For this reason, the extraction result of the predetermined color component by the data processing target additional information extracting unit 142 may result in the discontinuity of the graphic or character as a result of removing the overlapping portion.

  From this, the extraction line segmentation correction unit 148 performs thinning processing on the extraction result that should be a graphic or a character such as “◯”, “×”, “line”, and other marks (marks). End point extraction processing, end point connection processing (so-called connection processing), or linear approximation of a line figure is appropriately executed. Note that the thinning process, end point extraction process, end point connection process, straight line approximation of line figures, etc. performed at this time may be performed using known techniques, and therefore detailed description thereof is omitted here (for example, , "Image processing and recognition", Takeshi Aoiin, published by Shosodo).

  The data processing target additional information specifying processing unit 150 adds the additional information based on the difference extraction result by the difference extracting unit 132, specifically, based on the data processing target additional information 9a corrected by the extraction line segmentation correcting unit 148. The first data processing additional information recognition unit 154 for recognizing the entry contents of the first data processing additional information in the completed document 8B separately from the second data processing additional information, and the second data processing additional information. There is a second data processing additional information recognition unit 164 that recognizes the entry content of the additional information separately from the additional information for the first data processing.

  As the data processing target additional information specifying processing unit 150, the first data processing additional information recognition unit 154 recognizes the content of the scoring symbol 87 in the additional information-added teaching material 81 separately from the comment 88, and The two-data processing additional information recognition unit 164 recognizes the content of the comment 88 in the additional information-added teaching material 81 separately from the scoring symbol 87.

  Each of the data processing additional information recognition units 154 and 164 performs the shape recognition process on the data processing additional information that has been corrected by the extraction line segmentation correction unit, so that the data processing additional information is entered. Recognize the entry contents of the additional information for data processing by performing character recognition processing on the entry contents of the additional information for data processing corrected by the figure shape recognition sections 156 and 166 and the extraction line segmentation correction section. On the original document 8A (additional document 8B) of the entry contents of the additional information for data processing recognized by the character recognition units 157 and 167 and the figure shape recognition units 156 and 166 and the character recognition processing units 157 and 167. There are entry position recognition units 158 and 168 for recognizing the entry position. The figure shape recognition units 156 and 166 and the character recognition processing units 157 and 167 constitute a separation recognition processing unit 155 that recognizes the scoring symbol 87 and the comment 88 separately.

  In the case where the character recognition processing units 157 and 167 are not provided or the data processing additional information portion that cannot be recognized by the character recognition processing units 157 and 167, the extracted additional information for data processing is handled as an image as it is. To do.

  As shown in the figure, each of the figure shape recognizing units 156 and 166, the character recognition processing units 157 and 167, and the entry position recognizing units 158 and 168 may be configured such that one function unit realizes both functions. However, they may be provided independently as individual functional units.

  For example, in the configuration in which the additional information filled-in teaching material 81 is a processing target, the first data processing additional information recognition unit 154 uses the scoring symbol 87 as additional information for the first data processing. It functions as a part. In this case, the figure shape recognizing unit 156 for the scoring symbol 87 determines whether the entry content of the scoring symbol 87 is “correct answer (◯)”, “incorrect answer (×)”, or “partially correct answer (Δ)”. It is only necessary to be able to recognize from the side of the figure. For example, shape recognition may be performed by pattern matching with the figure shapes of “◯”, “×”, and “Δ”. Alternatively, the feature amount of the recognition target figure may be calculated and the shape may be recognized from the feature amount. As the feature amount, for example, the number of holes, the area ratio of the target figure occupying the circumscribed rectangle, or the like can be used.

  The character recognition processing unit 157 for the scoring symbol 87 determines whether the contents of the scoring symbol 87 are “correct answer (◯)”, “incorrect answer (×)”, or “partially correct answer (Δ)”. It only needs to be recognized from the side. If the scoring symbol 87 is only a figure, the character recognition processing unit 157 can be omitted.

  Further, the entry position recognition unit 158 for the scoring symbol 87 may recognize the entry position of the entry contents of the scoring symbol 87 on the additional information-added teaching material 81 by, for example, coordinate analysis on the educational teaching material 80.

  Here, in the configuration of the present embodiment, a recognition performance information presentation processing unit 190 is provided, and information related to the recognition rate is collected from a plurality of processes in each functional unit of the additional recording information processing apparatus 10, and the contents to be entered are displayed. In addition to the automatically recognized additional information, only the difficult-to-recognize difficult-to-recognize information is presented to facilitate correction, so it is not necessary to check whether the additional information needs to be corrected. The postscript information correction work can be efficiently executed. Of course, as a result, the recognition rate in each of the data processing additional information recognition units 154 and 164 can be improved.

  Note that the figure shape recognizing unit 156, when recognizing the shape related to the scoring symbol 87, treats the continuous pixel group constituting the figure indicating the scoring symbol 87 such as “◯” and “x” collectively as one. In order to give an identifier to the continuous pixel group, a labeling process which is a general image processing technique is performed. Therefore, even when the position recognition by the entry position recognition unit 158 is performed, the result of the labeling process is used to select one continuous pixel group constituting the graphic indicating the scoring symbol 87 such as “◯” or “×”. Treat as a group of two.

  In addition, since the entry position recognition unit 158 generally recognizes the entry position of the scoring symbol 87, a plurality of scoring symbols 87 are generally entered on the additional information filled teaching material 81. For each of the symbols 87, the positions of the scoring symbols 87 detected in the predetermined scanning order are sequentially recognized.

  Position recognition regarding each scoring symbol 87 is performed, for example, by calculating circumscribing rectangle information of a figure (or character) indicating the scoring symbol 87 such as “◯” or “×”, and further calculating the center coordinates of the circumscribing rectangle. It is possible. Specifically, a circumscribed rectangle is extracted for a figure or character (a group of continuous pixels) to be recognized, and the xy coordinates of a predetermined point (for example, the upper left vertex) of the circumscribed rectangle, and the width of the circumscribed rectangle ( W) and height (h) are calculated. Then, from these calculation results, the center x coordinate = x + w / 2 and the center y coordinate = y + h / 2 are calculated, and the calculation result is used as the recognition result of the position of the continuous pixel group, that is, the entry position of the scoring symbol 87.

  On the other hand, the second data processing additional information recognition unit 164 functions as a comment recognition unit that uses the comment 88 as the second data processing additional information. In this case, the graphic shape recognition unit 166 for the comment 88 only needs to be able to recognize the additional content of the comment 88 from the side surface of the graphic. For example, “single line”, “double line”, “(single or double) The shape of the line may be recognized by pattern matching with a graphic shape indicating a line such as a “double wavy line”. Alternatively, the feature amount of the recognition target figure may be calculated, and the line shape may be recognized from the feature amount. As the feature amount, for example, the number of lines, the pixel density for the circumscribed rectangle, or the like can be used.

  In addition, the character recognition processing unit 167 only needs to be able to recognize the content of the comment 88 in the additional information filled teaching material 81 from the side of the character. In addition, the entry position recognition unit 168 for the comment 88 may recognize the entry position of the content to be added to the comment 88 on the additional information-added teaching material 81 by, for example, coordinate analysis on the educational material 80.

  Note that, when the shape recognition unit 166 recognizes the shape of the comment 88, the continuous shape pixel group constituting the graphic indicating the comment 88 such as “double line” and “flower circle” is handled as one. In addition, a labeling process, which is a general image processing technique, is performed to give an identifier to the continuous pixel group. Therefore, even when the position is recognized by the entry position recognizing unit 168, a group of continuous pixels constituting a figure indicating the comment 88 such as “double line” or “flower circle” is used by using the result of the labeling process. Are treated as one group.

  The entry position recognition unit 168 generally has a plurality of comments 88 written on the additional information filled teaching material 81 in the process of recognizing the entry position of the comment 88. The positions of the comments 88 detected sequentially in a predetermined scanning order are sequentially recognized.

  Position recognition for each comment 88 is performed by, for example, calculating circumscribing rectangle information of a comment sentence, characters and figures of the comment 88 such as “double line” and “Hanamaru”, and further calculating the center coordinates of the circumscribing rectangle. It is possible to do it. Specifically, a circumscribed rectangle is extracted from a character or figure (continuous pixel group) to be recognized, and the xy coordinates of a predetermined point (for example, the upper left vertex) of the circumscribed rectangle, and the width of the circumscribed rectangle ( W) and height (h) are calculated. Then, from these calculation results, the center x coordinate = x + w / 2 and the center y coordinate = y + h / 2 are calculated, and the calculation result is set as the recognition result of the position of the continuous pixel group, that is, the comment 88 entry position.

  Also, in this position recognition, each comment 88 is often written in the vicinity thereof in correspondence with the scoring mark 87 for the answer column 84 at a certain position. The process should be performed in cooperation with the position recognition for 87. By doing so, it becomes easy to associate the position information of each position information with each answer field 84, and as a result, to associate the scoring symbol 87 with the comment 88.

  The data processing unit 170 executes the first data processing related to the first data processing target additional information entered in the additional document 8B for the image data of the additional document 8B input from the document input device 20. 1 data processing part 170_1 and 2nd data processing part 170_2 which performs the 2nd data processing regarding 2nd data processing object additional record information.

  Each of the data processing units 170_1 and 170_2 sets the entry position of the additional information to be processed as data processing apparatuses (the document management server 30 functioning as the template information database DB1 and the document original information database DB2) that stores the position information of the entry field. ) To access the entry field position area information 38, entry field position area information 68, or template association information 69, and execute data processing while associating the entry position of the additional information with the additional information. To do.

  The first data processing unit 170_1 performs scoring and summarization on the image data of the additional information filled teaching material 81 which is an example of the added document 8B based on the scoring symbol 87 written in the additional information filled teaching material 81. Unit 172, and a totaling result output unit 174 that outputs the result of scoring totaling by scoring totaling unit 172 in association with the answerer information extracted by answerer extracting unit 134. A processing result in a state in which the scoring result and the answerer information are associated is particularly referred to as a scoring recognition result.

  The scoring tabulation unit 172 recognizes the result of the additional writing of the scoring symbol 87 by the graphic shape recognition unit 156 from the side of the graphic and the recognition result of the additional information of the scoring symbol 87 by the character recognition processing unit 157 from the side of the character information. The result of recognition of the entry position of the scoring symbol 87 by the entry position recognition unit 158 and the electronic data (original image) of the educational material 80 corresponding to the additional information filled educational material 81 held and accumulated in the document input device 20 are included. The additional information-added teaching material 81 read by the document input device 20 is added based on the scoring information defined in the scoring column 83 for each answer column 84 of the teaching material 80 (additional information-added teaching material 81). A scoring process and a summing process (collectively referred to as scoring summarization) regarding the scoring symbols 87 entered in the information-filled teaching material 81 are performed.

  The counting result output unit 174 associates the score totaling result totaled by the scoring totaling unit 172 with the answerer information extracted by the answerer extracting unit 134, and stores the processing result storage server 40 (processing result database device or processing result file server device). Etc.). Alternatively, the score of the scoring result is entered in the totaling field 83b of the additional information-added teaching material 81 and output on the paper as a return answer 81b so that it can be returned to the student.

  In addition, the second data processing unit 170_2 performs a classification process on the image data of the additional information filled teaching material 81, which is an example of the added document 8B, based on the comment 88 entered in the additional information filled teaching material 81. A classification processing unit 176 and a comment processing result output unit 178 that outputs the classification result by the comment classification processing unit 176 in association with the scoring recognition result output by the total result output unit 174 and each answer are provided.

  The comment classification processing unit 176 inputs the recognition result from the side of the figure of the additional content of the comment 88 by the graphic shape recognition unit 166 and the recognition result from the side of the character information of the additional content of the comment 88 by the character recognition processing unit 167. The additional information filled teaching material 81 read by the document input device 20 based on the recognition result of the position where the comment 88 is entered by the position recognizing unit 168 and the classification information defined in advance so as to correspond to the additional content of the comment 88. The comment 88 entered in the additional information filled teaching material 81 is classified.

  The comment processing result output unit 178 associates the classification result by the comment classification processing unit 176 with the scoring recognition result output by each answer column or the total result output unit 174, and stores the processing result storage server 40 (processing result database device or processing result file). To the server device).

  Note that the scoring symbol 87 is generally entered corresponding to each of the plurality of answer fields 84 on the educational material 80, and the comment 88 is entered in conjunction with the scoring symbol 87 entry. Since the scoring symbol 87 and the comment 88 are handwritten by a scoring officer such as a teacher, the entry position in each answer column 84 is not necessarily uniquely determined.

  On the other hand, when scoring the scoring symbols 87, it is necessary to clarify the correspondence between the answer columns 84 and the entry positions of the scoring symbols 87. In the scoring for the scoring symbol 87, after clarifying the entry result of the scoring symbol 87 corresponding to each answer column 84, the contents of the scoring symbol 87 (whether the answer is correct, incorrect or partially correct) and each answer column 84 It is because it is performed based on the scoring about. Similarly, in the classification process for the comment 88, it is necessary to clarify the correspondence between each answer column 84 (that is, the scoring symbol 87) and the entry position of the comment 88. This is because the classification process related to the comment 88 is performed based on the content of the comment 88 after clarifying the entry result of the comment 88 corresponding to each answer column 84.

  From this, the scoring totaling unit 172 and the comment classification processing unit 176 perform scoring totaling of the scoring symbols 87 and classification processing of the comments 88 in the following procedure. For example, the scoring totaling unit 172 and the comment classification processing unit 176 indicate that the circumscribed rectangles of the scoring symbols 87 and comments 88 specified by the entry position recognition units 158 and 168 are the additional information filled teaching materials. It is determined whether or not there is an overlap with the area that becomes the answer column 84 on 81, the overlapping answer column 84 and the scoring symbol 87 and the comment 88 are associated with each other, and the scoring symbol 87 and the comment 88 are associated with the answer column 84. It is possible to employ the first matching method that uses the determination result of the scoring symbol 87 and the comment 88 entered for.

  However, when one scoring symbol 87 or comment 88 overlaps the areas of the plurality of answer columns 84, it is not possible to specify which one should be associated with, so determination about association by the first association method Is determined to be impossible. In addition, since the circumscribed rectangle of the scoring symbol 87 or the comment 88 of interest does not overlap with any answer field 84, it cannot be specified which one should be associated with, so the first association method It is determined that the determination regarding the association by cannot be made.

  In addition, the scoring totaling unit 172 and the comment classification processing unit 176 are a circumscribed rectangle of the scoring symbol 87 and the comment 88 specified by the entry position recognition units 158 and 168, and an area that becomes the answer column 84 on the additional information filled teaching material 81. The scoring symbol 87 and the comment 88 and the answer column 84 having the largest area (the same as the area ratio to the circumscribed rectangle) are associated with each other, and the scoring symbol 87 and the comment 88 are stored in the answer column 84. On the other hand, it is possible to employ a second matching method that makes the determination result of the scoring symbols 87 and comments 88 entered.

  When this second association method is adopted, even if one scoring symbol 87 or comment 88 overlaps the areas of the plurality of answer columns 84, even if the association cannot be specified by the first association method, the overlapping is performed. Based on the size of the area, it can be determined which one should be handled. However, when the ratio of the overlapping area to the circumscribed rectangle is less than the predetermined threshold, it is determined that the determination regarding the association is impossible because the overlapping portion is small.

  Alternatively, the scoring totaling unit 172 and the comment classification processing unit 176 obtain the distance between the center coordinate position of each scoring symbol 87 and comment 88 specified by the entry position recognition units 158 and 168 and the center coordinate position of each answer column 84, The scoring symbol 87 or comment 88 with the smallest distance is associated with the answer column 84, and the scoring symbol 87 or comment 88 is used as the determination result of the scoring symbol 87 or comment 88 entered in the answer column 84. A third association technique can be employed.

  When this third matching method is adopted, the first matching method specifies the correspondence when the circumscribed rectangle of the scoring symbol 87 or the comment 88 of interest does not overlap the area of any answer column 84. If the scoring symbol 87 or the comment 88 is written out of the answer column 84 and the overlapping portion is small and the ratio of the overlapping area to the circumscribed rectangle is less than a predetermined threshold, the second association method associates. Even if it is not possible to identify the user, it is possible to determine which one should be handled. However, if the difference between the distances from the answer columns 84 is less than the predetermined threshold, it is determined that the determination about the association is impossible because the distance difference is small.

  Then, after associating each scoring symbol 87 with the answer column 84, if the scoring symbol 87 is “◯”, the score specified from the scoring information for the corresponding answer column 84 is added. If the scoring symbol 87 is “x”, the scoring is not added to the answer field 84 corresponding to this, and such scoring is performed for all the answer fields 84 on the additional information filled teaching material 81.

  In addition, the area that becomes the answer column 84 on the additional information filled teaching material 81 is assigned to the document management server 30 corresponding to the additional information filled teaching material 81 as the score information for each answer column 84 or in the same manner as the score information. The entry field position area information 38 included in the registered original image is specified.

  In addition, after associating each comment 88 with the answer column 84, the classification destination identified from the classification criteria corresponding to the comment 88 is identified, and such comment classification processing is performed on the additional information filled-in teaching material 81. This is done for all the comments 88.

  Note that the scoring and summarizing process in the scoring and summarizing unit 172 and the classification process in the comment classification processing unit 176 may be complete automatic processing, but the CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) of the user terminal 171 may be used. ) Or the like may be displayed on the display unit configured so that the operator can appropriately correct the processing step or the processing result via an instruction input unit such as a keyboard or a mouse.

  The recognition performance information presentation processing unit 190, which is a characteristic part of the present embodiment, collects the history of handwritten input information and stores and stores it in the processing result storage server 40 as a database. A difficult recognition additional record information extracting unit 194 for extracting difficult recognition information with particularly low reliability that is difficult to recognize than a certain degree, and a recognition performance for presenting recognition performance information on the user terminal 171 in accordance with a predetermined display mode And an information presentation unit 198.

  The additional information recognition history holding unit 192 collects the history of handwritten input information by adopting a history collection and storage method of any one or any combination of first to third history collection and storage methods described later, and collects this history The history of the handwritten input information is stored in the processing result storage server 40.

  When collecting the history of handwritten input information and storing it in the database, various history collection and storage techniques can be adopted depending on at what point the entry is made as a history. For example, it is conceivable to adopt a first history collection and storage method in which handwritten input information data recognized in the process of normal automatic data processing is accumulated and stored. When adopting the first history collection and storage method, it is conceivable that only handwritten input information is stored separately from the original document original 8A, or together with a predetermined range of image information including handwritten input information. In other words, it may be possible to save each surrounding image.

  If the first history collection and storage method is adopted, the history can be collected and stored in a normal processing process, so that data such as the shape, position, and correction status can be used. Further, it is possible to check the entry mode such as the scoring method without taking any special effort, and also to reflect the correctness of recognition. However, since the history is small at the beginning and improvement of the input mode by the user cannot be expected, the recognition rate is low, and there is a possibility that it takes time and effort for correction.

  Also, it is not a normal automatic data processing process, but before the processing is started, the input is made to the practice document original 8A such as a practice answer, and the input information data is accumulated and stored. It is possible to adopt a history collection and storage method. If this second history collection and storage method is adopted, entry information for practice can be designated, so that correction need not be considered. This is because, in the process of adding to the prepared practice answer and performing the normal automatic scoring process, the entry form is also specified. In addition, there is an advantage that a figure shape, a character shape, and a position can be determined, and the result can be presented including correct or incorrect recognition.

  In addition, you can practice writing methods such as scoring methods in advance, so you can confirm the correct shape and position, you can expect entry with good recognition performance, and as a result, during actual automatic data processing The advantage is that no modification is required. Further, since it is possible to fill in a blank portion, it is only necessary to save handwritten input information for the designated entry information, so that the storage capacity can be reduced.

  Further, it is conceivable to adopt a third history collection and storage method for accumulating and storing input information data in a past additionally written document 8B (for example, a past scoring answer) that has not been used for automatic data processing. Since automatic data processing is not performed, the difference is extracted by adding and reading the blank document original 8A, and only the handwritten input information is extracted. In addition, it is only necessary to perform shape recognition and character recognition without presenting the entire process of automatic data processing and present the result.

  By adopting this third history collection and storage method, past appended documents can be used, so even when automatic data processing is used for the first time, a correction method for improving recognition performance with respect to handwritten input information can be presented. In the actual automatic data processing, there is an advantage that less correction is required. Further, it is possible to save the trouble of creating the original document 8A for practice. However, since the correctness of the result is unknown, determination based on correctness cannot be made.

  The difficult recognition additional record information extraction unit 194 employs a difficult recognition additional record information extraction method of any of the various methods described later or any combination thereof, and the reliability of the various handwritten input information is lower than a certain level. Extract difficult recognition information that was difficult to recognize automatically.

  The recognition performance information presentation unit 198 employs any of the various recognition performance information presentation methods described later, and presents the recognition performance information to the user on the user terminal 171.

<Overall procedure>
3 and 4 are diagrams for explaining the procedure of the processing operation of the teaching material processing method in the teaching material automatic scoring system 1 which is an embodiment of the information processing system. Here, FIG. 3 shows the overall outline in association with the system configuration diagram, and FIG. 4 is a flowchart showing the teaching material processing procedure.

  First, the educational material 80 is completed and registered in the original document information database DB2 (S104). In addition, when using the template 6 in the form in which no scoring is set in each entry field 6 a, scoring information is also set for the entry field 6 a set as the answer field 84.

  Thereafter, when the test is executed, the educational material 80 is read from the original document information database DB2, printed, and distributed to students and examinees (S106). After the test, the grader adds a scoring symbol 87 and a comment 88 to the student's answer (S108).

  In the case of using the postscript information processing apparatus 10 (particularly corresponding to the teaching material processing apparatus in the teaching material automatic scoring system 1), first, a student enters a name in the answerer information column 86 and enters the answering column 84. Answer entry, i.e., the entry of the first type of additional information by the student, and a second such as a scoring symbol 87 such as “○” or “x” and a comment 88 for the answer entered in each answer column 84 by the teacher or the like With respect to the additional information filled teaching material 81 in which the additional information has been entered, the document input device 20 reads the additional information filled learning material 81 (S110) and adds image data representing the additional information filled learning material 81 as additional information. It inputs into the processing apparatus 10 (S112). The document input device 20 temporarily stores image data obtained by image reading by the document input device 20 in a memory or the like used as a work area.

  At this time, if the document input device 20 uses an ADF device, for example, a plurality of additional information-filled teaching materials 81 to be processed in one group such as the same class are collectively read (batch scan). The image data corresponding to each additional information filled teaching material 81 can be continuously input to the additional information processing apparatus 10.

  The additional recording information processing device 10 sequentially adds the following additional information extraction / separation processing, additional information identification processing, and automatic scoring processing to the image data of each additional information filled teaching material 81 captured from the document input device 20. In addition, additional information end processing such as automatic comment processing is executed.

  For example, prior to automatic scoring processing and automatic comment processing in the data processing unit 170, the image data analysis unit of the read image processing unit 110 performs analysis processing on image data obtained from a certain additional information-added teaching material 81. (S122) The learning material specifying unit 122 identifies and specifies the original educational material 80 corresponding to the additional information-added learning material 81 based on the result of the analysis processing (S124).

  This identification specification (S124) is, for example, a title analysis by an identification information analysis unit such as “science”, “5 years”, “1. change in weather and temperature” or a code information analysis unit for code information embedded in the identification information column 85 This can be done through code analysis. Through this identification and specification, the teaching material specifying unit 122 specifies the electronic data (original image) of the teaching material 80 to be compared with the image data of the additional information filled teaching material 81 obtained by the document input device 20. can do.

  This identification and identification may be performed sequentially for each of the plurality of additional information-filled teaching materials 81 from which the document input device 20 has read an image. However, in general, additional information that has been processed in one group is already entered. Since all the teaching materials 81 are the same, it is only necessary to perform the additional information-filled teaching material 81 that is processed first during the batch processing.

  When the teaching material specifying unit 122 completes the specification of the educational material 80 corresponding to each additional information-added teaching material 81, the document management server 30 follows the specified result and selects the corresponding educational material 80 from the stored and stored. The original image (electronic data) is taken out and transferred to the difference extraction unit 132 (S126).

  Further, the distortion correction unit corrects the distortion of the image data obtained from a certain additional information-added teaching material 81 (S128). This image distortion correction is performed in order to correct image distortion that may occur when the document input device 20 reads an image. After that, the difference extraction unit 132 performs comparison with the original image or difference extraction. This is to improve accuracy.

  The difference extraction unit 132 receives the original image (educational teaching material 80) delivered from the document management server 30 and the image data (additional information) input from the document input device 20 and corrected for image distortion by the distortion correction unit. The completed teaching material 81) is compared with each other, and the difference is extracted (S130). The difference extraction unit 132 passes the extracted difference information 9 to the answerer extraction unit 134 and the additional information extraction processing unit 140 for data processing.

  By the difference extraction by the difference extraction unit 132, for example, as shown in the center part in FIG. 3, the content of the first type additional information entered by the answerer in the answerer information column 86 and each answer column 84, and each answer column Difference information 9 represented only by the content of the second type additional information entered by the grader such as a grading symbol 87 and a comment 88 for 84 is extracted.

  The answerer extracting unit 134 extracts answerer information in the additional information-added teaching material 81 to be read by the document input device 20 through character recognition processing by the character recognition processing unit for the difference information 9 (S132). As a result, the class, attendance number, name, and the like of the answer entry person who wrote the answer in one additional information-filled teaching material 81 can be specified.

  In the data processing additional information extraction processing unit 140, the additional recording member specifying unit 141 first specifies an additional recording color that is a pen color used for entering the additional information to be processed (S141), and performs data processing. The target additional information extraction unit extracts additional information for data processing from the difference extraction result by the difference extraction unit 132 based on the additional recording color specified by the additional member specifying unit 141 (S142).

  In the case of this example, in order to extract the additional contents of the grading symbol 87 and the comment 88 in each answer column 84, the information about the predetermined color component, specifically, for example, the red component is extracted from the difference information 9 To do. For example, if the difference extraction result is composed of pixel data, the predetermined color component can be extracted by paying attention to the color component data constituting the pixel data.

  The extracted line segmentation correction unit appropriately executes additional information shaping processing such as thinning processing, end point extraction processing, end point connection processing, or linear approximation of line figures, on the extraction result of the data processing target additional information extraction unit. (S146). The extracted line segment break correction unit passes the extraction result of the scoring symbol 87 that has been subjected to the break correction process to the first data processing additional information recognition unit 154 that functions as the scoring symbol recognition unit, and also extracts the comment 88 that has undergone the break correction process. The result is passed to the second data processing additional information recognition unit 164 that functions as a comment recognition unit.

  The data processing target additional information specifying processing unit 150 is first stored in the document management server 30 in cooperation with the figure shape recognition units 156 and 166 and the character recognition processing units 157 and 167 constituting the separation recognition processing unit 155. The scoring symbol 87 and the comment 88 are separated by referring to the position information in the answer field 84 (S162), and thereafter, the specified content processing and the entry position are identified separately from the separated scoring symbol 87 and the comment 88. Execute the process.

  The graphic shape recognition unit 156 and the character recognition processing unit 157, regarding the scoring symbol 87 separated from the comment 88 (S163-scoring symbol), the content of the scoring symbol 87 by shape recognition or character recognition for the content of the scoring symbol 87. The scoring result of the scorer indicated by the scoring symbol 87 is specified, such as whether the answer is “correct” or “incorrect” (S164). At this time, a correction instruction from the user is accepted (S165). Subsequently, the entry position recognition unit 158 recognizes the entry position on the additional information entry teaching material 81 with respect to the entry contents of the scoring symbol 87 (S166).

  After the entry position recognizing unit 158 recognizes the entry position of the scoring symbol 87 in this way, the scoring totaling unit 172 recognizes the entry contents of the scoring symbol 87 by the graphic shape recognition unit 156 and the character recognition processing unit 157. And the recognition result of the entry position of the scoring symbol 87 by the entry position recognizing unit 158 and the education included in the original image (educational instruction material 80) corresponding to the additional information entry instruction material 81 held and accumulated in the document management server 30. Based on the scoring information for each answer field 84 of the teaching material 80, scoring and counting are performed (S168).

  The total result output unit 174 stores the scoring / total results in the processing result storage server 40 (S169). Alternatively, the score of the scoring result is entered in the totaling field 83b of the additional information filled teaching material 81 and returned to the student as a return answer 81b.

  As the file format of the scoring result (scoring result by problem) for each additional information filled teaching material 81, for example, as shown in FIG. 3, the number of the problem existing on the additional information filled teaching material 81 and the problem It is a table format that associates these with each other with information consisting of correct / incorrect determination on the answer and a score based on the correct / incorrect determination. In addition, as a file format of the counting result, for example, as shown in FIG. 3, information including an attendance number and answerer information and score information (item points and total points written in the counting field 83b) Is a table format for associating each other.

  The result of scoring for correct / incorrect judgment entered on each additional information-added teaching material 81 is output as a question-based scoring result, and the result of scoring-by-problem scoring is output as a file. In the server 40, it is possible to manage or use the score totaling result for the additional information filled teaching material 81 in a list form, for example.

  The graphic shape recognizing unit 166 and the character recognizing processing unit 167, regarding the comment 88 separated from the scoring symbol 87 (S163-comment), the comment 88 added by the grading officer by shape recognition or character recognition with respect to the entered content of the comment 88. The entry contents are specified (S170). At this time, a correction instruction from the user is accepted (S171). Subsequently, the entry position recognizing unit 168 recognizes the entry position on the additional information entry teaching material 81 with respect to the entry contents of the comment 88 (S172).

  After the entry position recognizing unit 168 recognizes the entry position of the comment 88 in this way, the comment classification processing unit 176 displays the recognition result of the entry contents of the comment 88 by the figure shape recognition unit 166 and the character recognition processing unit 167. The comment 88 is classified based on the recognition result of the entry position of the comment 88 by the entry position recognition unit 168 and the classification reference information 39 stored and accumulated in the document management server 30 (S174).

  The comment processing result output unit 178 associates each comment 88 classified by the comment classification processing unit 176 with the scoring result of the scoring symbol 87 having a close position (S178), and stores it in the processing result storage server 40 (S179). As the file format of the classification result for each comment 88, for example, as shown in FIG. 3, it is a table format that associates each comment with information including a scoring symbol 87 in the vicinity. In addition, it is not limited to actually storing in the table format, and each comment 88 may be stored in association with association information that associates both the comment 88 and the scoring result corresponding to the comment 88.

  Since the score summation result of the scoring symbol 87 written on each additional information filled teaching material 81 and the comment 88 are stored in correspondence with each other, the processing result storage server 40 scores the additional information filled teaching material 81. The counting results and the comments 88 can be managed or used, for example, in a list format, and the comments 88 can be used for student ability analysis and student guidance.

  As described above, in the teaching material automatic scoring system 1 shown as an embodiment of the information processing system, the image data read from the supplementary information-added teaching material 81 in which the scoring symbol 87 and the comment 88 are entered, and the supplementary information are already entered. The original educational material 80 for the educational material 81, that is, the first type of additional information by the student such as entering the answer in the answer column 84, and the second type of additional information such as a scoring symbol 87 and a comment 88 by the grader for the answer. Are compared with the data for those not filled in, and the written contents of the grading symbol 87 and the comment 88 are separated from each other's difference to identify the filled content, and the totaling and comment 88 for the grading symbol 87 are identified. The classification process is executed.

  Therefore, regarding the data processing related to the scoring symbol 87, the entry content of the comment 88 is excluded from the difference result extracted by the difference extraction unit 132, and only the scoring symbol 87 is separated to specify the entry content. So, even if both are added with the same pen, automatic scoring will not be adversely affected.

  In addition, since the scoring results can be automatically totaled, the scoring process for the additional information filled teaching material 81 is saved as a result. In the case where the additional information filled teaching material 81 is obtained as a paper medium, it may be based on image data obtained by reading the additional information filled teaching material 81 with the document input device 20, for example, a copying machine, a multifunction peripheral, a scanner device, or the like. System configuration can be easily realized with a scanning function realized by the computer and an information storage processing function, an image processing function, and an arithmetic processing function of a computer device such as a personal computer (PC). Is not required.

  Furthermore, in order to compare the image data of the additional information filled teaching material 81 with the electronic data held by the document management server 30, the electronic data of various educational teaching materials 80 can be held and accumulated in the document management server 30. Thus, the versatility of the additional information filled teaching material 81 that can be handled can be sufficiently secured. Furthermore, by storing and storing electronic data in the document management server 30 in advance, when comparing with image data captured from the document management server 30, it is possible to input the electronic data to be compared. As a result, a quick scoring process can be realized.

  In addition, regarding the data processing related to the comment 88, the entry content of the scoring symbol 87 is excluded from the difference result extracted by the difference extraction unit 132, and only the comment 88 is separated to specify the entry content. Even if both are added with the same pen, the classification process related to the comment 88 is not adversely affected. In addition, since each comment 88 is stored in the processing result storage server 40 in association with each scoring symbol 87, the scoring officer himself / herself is used for ability analysis and student guidance in addition to the purpose of confirming the comment 88 by the student. It can be utilized.

<< Recognition performance information presentation process >>
5 to 16 show details of the processing of the recognition performance information presentation processing unit 190, that is, a technique for taking a history of handwritten input information such as characters and figures entered by handwriting, and recognition performance information for improving recognition performance. It is a figure explaining the method of showing to a user.

  In this embodiment, it is determined whether or not the handwritten input information entered in the original document 8A by handwriting is suitable for automatic recognition. Support information (recognition performance information) is presented so that the user can determine whether or not to improve.

  Here, the recognition performance information may be any information that allows the user himself / herself to specifically and easily determine a correction method for improving the recognition performance, and can take various presentation modes. For example, it is possible to adopt a mode of presenting information that suggests where and how to correct it. By doing so, it is possible to easily determine how the description improves the recognition performance.

  Or the aspect which presents the recognition reliability of the past description can be taken. By doing so, it is possible to easily determine how much the recognition performance is to be described, and based on this, the user can make a description that improves the recognition performance.

  Whichever presentation mode is adopted, entry of handwritten input information suitable for recognition can be promoted, and as a result, the recognition rate can be improved.

  In particular, in the present embodiment, the difficult-to-recognize additional write information extraction unit 194 collects information related to the recognition rate from a plurality of processes in each functional unit of the additional write information processing apparatus 10 and automatically adds the information to be written. In particular, only the difficult recognition information with low reliability is specified, and the recognition performance information presentation unit 198 determines the recognition performance of the difficult recognition information for the difficult recognition information specified by the difficult recognition additional record information extraction unit 194. Presents recognition performance information for improvement and prompts correction.

  In relation to automatic recognition performance, the check frequency is reduced by presenting only difficult-to-recognize information with particularly poor recognition performance and low reliability. Thereby, it is not necessary to check the necessity of correction for all the additional write information, and the correction work of the erroneous additional write information can be performed efficiently.

  In addition, not only the feature quantity related to the reliability level in the recognition process itself, but also the feature quantity related to the reliability level in various processes related to the recognition process preceding the recognition process is acquired and the recognition information is specified, so correction is required. Additional write information (difficulty recognition information) can be specified with high accuracy.

  In order to present the recognition performance information of handwritten input information to the user, first of all, a history of handwritten input information is saved, and the degree of improvement is requested in order to improve the recognition performance with reference to the reliability at the time of recognition. Present recognition performance information for strong ones. In other words, the recognition performance information is presented only for handwritten input information (difficult recognition information) that is more difficult to recognize than a certain degree. In this way, the recognition performance can be made higher than a certain level. The recognition performance only needs to be higher than a certain level, and it is not always necessary that the handwritten input information completely matches the ideal reference information.

  Since the handwritten input information is almost certainly different from the reference information, if the recognition performance information was presented for all handwritten input information entered in the past, the information more than necessary due to the recognition performance. However, it is possible to prevent unnecessary information presentation by narrowing the presentation range to the difficult recognition information range. By doing so, the correction work can be performed efficiently, and the possibility of correcting more than necessary can be prevented.

  In general, since the entry habits differ depending on the user, the improvement mode to be presented also differs for each user. Therefore, when taking a history of handwritten input information, it is preferable to keep a history for each user.

<Extracting difficult recognition information based on reliability information>
In addition, in the difficult recognition additional record information extraction unit 194, when presenting the recognition performance information only for the difficult recognition information that is difficult to recognize than a certain degree, the separation recognition processing unit 155 performs normal recognition processing, The difficult recognition additional record information extraction unit 194 extracts the difficult recognition information based on the reliability at the time of the recognition process, in other words, the similarity to the candidate information (reference information).

  Further, in the present embodiment, the difficult recognition additional record information extraction unit 194 performs various processes related to the recognition process including the recognition process itself in the data processing target additional information specification processing unit 150 (for example, a difference extraction unit) with respect to the additional information to be noted. 132 to each processing unit of each functional unit from the data processing target additional information specifying processing unit 150), the feature quantity related to the reliability of the recognition process is acquired, and the sub-process related to the recognition process of each process is acquired based on each feature quantity. Calculate the reliability, specify the final reliability for the additional information of interest based on the calculated sub-reliability, and determine whether the final reliability is lower than a certain level. To identify difficult recognition information.

  That is, not only the reliability at the time of recognition processing in the separation recognition processing unit 155 and the entry position recognition units 158 and 168 but also information (features) related to the recognition rate and reliability at the time of each processing in each functional unit in the preceding stage. Are also collected, sub-reliability for each process is calculated based on each collected feature amount, and final reliability is obtained based on the calculated sub-reliability for each process. And the difficult recognition information which needs correction based on this final reliability is specified.

  In the mechanism described in Japanese Patent Application Laid-Open No. 2004-152115, a certainty factor is calculated for each item for a plurality of items included in acquired data, and a correction presentation method is dynamically used using the calculated certainty factor. However, in this case, since only the “certainty” is calculated using the similarity of pattern recognition when recognizing each item, it occurs in various pre-processing before the recognition processing. Events that affect the recognition process, such as the presence or absence of local noise, cannot be considered.

  On the other hand, in the difficult recognition additional record information extraction unit 194 of the present embodiment, the feature amount related to the recognition process in each functional unit before the data processing target additional information specification processing unit 150 is also acquired and comprehensively acquired. Since the difficulty recognition information is specified by calculating the similarity, rather than the case where the difficulty recognition information is specified based on the recognition similarity in the data processing target additional information specifying processing unit 150 or the certainty based on the similarity, It is possible to specify difficult recognition information with high accuracy.

  In other words, not only the similarity of the recognition process in the data processing target additional information specifying processing unit 150 but also the characteristics related to the reliability of the recognition process in various processes including the processing preceding the data processing target additional information specifying processing unit 150 Each amount is acquired, the sub-reliability related to the recognition processing of each process is calculated based on each feature amount, and the final reliability regarding the additional information to be noticed is specified based on the calculated sub-reliability Since the difficulty recognition information is specified by determining whether or not the final reliability is lower than a certain level, the preceding stage of the recognition processing in the data processing target additional information specifying processing unit 150 is performed. Therefore, it is possible to specify the difficult recognition information with high accuracy in consideration of the presence or absence of local noise that may occur in various types of preprocessing.

<History collection processing>
When calculating the sub-reliability for each process in each function unit, the recognition at the time of each process in each function unit in the previous stage is performed from all the processes related to the database registration of the recognition process result in the separation recognition process unit 155. Collect features related to rate and reliability.

  In addition, the tendency of the recognition rate and reliability may differ depending on the type (whether it is a character or a graphic or a line) and the shape of the postscript information itself, and is also related to the state of the postscripted document 8B to be processed. In view of these points, the past statistical information is saved so that the final reliability can be acquired, and the reliability is calculated based on the sub-reliability and the past statistical information.

  For example, in the past automatic data processing, scoring is generally such that the recognition rate (reliability) of “◯” was 0.95 and the recognition rate (reliability) of “x” was 0.9. In addition to the symbol 87, the recognition rate (reliability) may be different.

  Further, the recognition rate (reliability) is affected by whether or not the additionally-written document 8B includes a photograph or a figure (clip art). In particular, it is easy to make an error in a part that comes into contact with a photograph or a figure (clip art), and a recognition rate (reliability) can be greatly different between a part that comes into contact with a photograph or a figure and other parts. For this reason, the additional information such as the grading symbol 87 and the comment 88 is entered in the additional information filled teaching material 81 (educational teaching material 80) as the added document 8B (original document 8A). Can be distinguished.

  For example, FIG. 5 is a diagram illustrating an example of a technique for collecting feature amounts related to the recognition rate and reliability during each process. In the example shown in FIG. 5, questions (question 4) using photographs are mixed, and the answer column 84 is arranged close to the photograph portion. The degree is “0.7”, and the reliability of the scoring symbol 87 in the other answer column 84 is “0.9 to 0.95”, for example, for each question in the added document 8B (additional information filled teaching material 81) Keep a history of the recognition rate (reliability).

<Reliability information collection processing and reliability integration processing>
In addition, the difficulty recognition additional record information extraction unit 194 obtains the final reliability based on the sub-reliability for each process, that is, when integrating the sub-reliabilities for each process, for example, a note to be noted Regarding information (scoring symbol 87 and comment 88), it is conceivable to adopt a first integration method in which the one with the lowest reliability among the processes is the final reliability. By repeating the first integration method for all the additional write information, additional write information having a sub-reliability value lower than a certain level is extracted. Note that a method for calculating the sub-reliability for each process will be described in a third integration method described later.

  The recognition performance information presentation unit 198 presents the recognition performance information to the writer so as to urge correction of the additional information whose sub-reliability value extracted by the difficult recognition additional record information extraction unit 194 is lower than a certain level. Only the additional information whose recognition performance is below a certain level is urged to be corrected by the entrant, so there is no need to determine (check) the necessity of correction for all additional information, and correction of additional information that may cause misrecognition Work can be made more efficient.

  If this 1st integration method is taken, it can respond to the case where the mistake which has a serious influence on a recognition result by one process is committed. For example, if the lower half of the triangle shape overlaps with the photo, and the lower half cannot be extracted by the difference extraction process, it will look like a cross shape depending on how you fill in. There is a possibility of judging. However, if this first integration method is adopted, the sub-reliability in the difference extraction process can be used, and thus such an error can be avoided.

  In addition, the difficult recognition additional record information extraction unit 194 adds the sum of the noticeable additional information (scoring symbol 87 and comment 88) to the sub-reliabilities (multiple processing results) for each process and adds the final reliability. It is also possible to adopt a second integration method for calculating the degree. By repeating this second integration method for all the additional write information, the additional write information whose final reliability value is lower than a certain level is extracted. Note that a method of calculating the sub-reliability for each process in consideration of “weighting” will be described in a third integrated method described later.

  As an example of weighting, a predetermined value can be set at random regardless of the processing order, weighting is increased as it goes upstream, or weighting is increased as it goes downstream. The weighting value can be gradually changed according to the processing order.

  In the method in which weighting is increased as it goes upstream, the initial process is emphasized, and therefore, there is an effect that can cope with the case where it is assumed that the upstream stage of the process as shown in FIG. In the method of increasing the weight as it goes to, since processing after correction such as distortion is emphasized, there is an effect that it is possible to deal with the case where the shape of the original entry figure is the main cause (such as a rounded triangle).

  As another example of weighting, when there is a data group of feature amounts and correct recognition results, an optimal weighting can be calculated for the data group by multiple regression analysis. This method has an advantage of high accuracy when the data group and the additional information to be input are similar (the writer and the original document are the same).

  The recognition performance information presentation unit 198 displays the recognition performance information so as to prompt correction of additional information whose final reliability value extracted by the difficult recognition additional record information extraction unit 194 in consideration of weighting is lower than a certain level. Present to the writer. Only the additional information whose recognition performance is below a certain level is urged to be corrected by the entrant, so there is no need to determine (check) the necessity of correction for all additional information, and correction of additional information that may cause misrecognition Work can be made more efficient.

  If this second integration method is adopted, weighting can be freely changed, so that there is an effect that the accuracy can be adjusted so as to increase in accordance with the characteristics of the writer or the original document.

  Further, the difficult recognition additional record information extraction unit 194 sets in advance a type of misrecognition (type of error) with low reliability, and among those corresponding to the set “type of misrecognition” during data processing. It is also conceivable to adopt a third integration method in which the lowest reliability type is the final reliability. At this time, a combination and weighting of reliability used at the time of determination are set for each type of erroneous recognition. Then, based on the estimated misrecognition type, a combination of processing and feature amount is selected, and weighted addition is performed to calculate the reliability. Then, based on the addition processing result, the type of erroneous recognition whose reliability is lower than or lower than a predetermined level is selected. Thereby, the difficult recognition additional record information extraction unit 194 can specify difficult recognition information having the selected type of erroneous recognition and having a reliability of recognition processing lower than a certain level.

  Here, “selecting a combination of processing and feature values based on the estimated misrecognition type and calculating the reliability by weighting them” includes a combination of feature values that appear peculiar to a certain type ( (Feature of feature quantity) is defined in advance, and it is examined whether or not it is applicable. Simply, in each process, 1 is added if the characteristic amount is specific to a certain type, and 0 is added if not, to see whether a certain level is satisfied.

  6 to 10 are diagrams illustrating the recognition performance information presentation processing unit 190 and explaining the types of misrecognition related to each processing. As the determination of “type of misrecognition”, for example, a scan error related to the document input device 20, an extraction error related to the difference extraction unit 132 or the data processing target additional information extraction unit 142, and an extraction line segmentation correction unit Closely related to the processing itself of each functional unit such as a missing interpolation error related to 148, a shape deformation related to the deformation processing units 156a, 166a, 157a, 167a, and a positional shift related to the entry position recognition units 158, 168 It is preferable to determine what has been added, what is added due to multiple writing, correction, additional information such as pen color / thickness / fading, and what is attributed to the added document 8B itself such as paper stains.

  For example, the misrecognition type of “extraction mistake” is likely to occur when the reliability of shape recognition or character recognition is low and the amount of overlap with a dark background is large. This may be affected by the presence / absence of contact between the postscript information and the original image due to the positional relationship between the images of the postscript added document 8B and the original document 8A during the differential extraction process in the differential extraction unit 132. Moreover, it is also possible to be affected when the additional information is overlapped with a dark background such as a photograph or a figure. In addition, when extracting additional information for data processing in the data processing target additional information extraction unit 142, when trying to extract a pen color (for example, red pen) component used for additional writing, the target color and actual This is because it can be affected by the difference from the pen color, and can also be affected by the presence or absence of contact with the original image due to the positional relationship between the red extracted image and the original image.

  For example, as shown in FIG. 6A, when the target image of the additional information extraction process has a scoring symbol 87 written on a dark background, the difference extraction unit 132 and the data processing target additional information extraction unit 142 As shown in FIG. 6 (B), the additional record information extraction result is extracted with the overlapping portion missing. The extraction line segment break correction unit 148 performs correction so that the missing portion is connected.

  That is, the extraction line segmentation correction unit 148 thins the extraction result of the predetermined color component by the data processing target additional information extraction unit 142, that is, the extraction result that should be a graphic such as “◯” or “x”. The process is executed, and the end point extraction process is further executed. As a result, when there is an interrupted portion in the graphic such as “◯” or “×”, the end point at the interrupted portion is extracted. When the end points are extracted, a predetermined one of all the extracted end points is connected.

  Thereby, for example, when the figure shown in FIG. 6B is extracted, even if the end points B and C exist within a predetermined distance from the end point A, the nearest end point B is selected. By connecting with the end point A, as shown in FIG. 6C, the discontinuity portion in the “◯” image is corrected. As a result, although the writer originally wrote “O” in general, the image of “O” in a distorted state in which the portion overlapping the dark background is omitted is input to the separation recognition processing unit 155. Will be.

  As a result, the separation recognition processing unit 155 should identify “◯” by graphic recognition processing or character recognition processing based on the distorted “◯” image. As a result, the feature of the figure changes drastically, so that the recognition rate (reliability) decreases and the possibility of a recognition error increases.

  For this reason, as shown in FIG. 6D, in the difficult-to-recognize additional record information extracting unit 194, the type of misrecognition of such “extraction mistake” can be specified. In the specific color component extraction process in the difference extraction process or the data processing target additional information extraction unit 142, the quality information J10, J11 at the time of extraction obtained from the extraction amount or the like, or the dark background of the scoring symbol 87 and the comment 88 The amount of overlap J12, the contact amount J13 between the original image and the red extracted image obtained from the positional relationship between the red extracted image and the original image, etc. are obtained as feature amounts related to the recognition rate and reliability, and these information are difficult to recognize. The postscript information extraction unit 194 is notified.

  Based on the notified quality information J10, J11, the overlap amount J12, and the contact amount J13, the difficult recognition additional record information extraction unit 194 adds, for example, as 1 if the criterion is satisfied and 0 if not satisfied, thereby extracting the difference. The sub-reliability T132 of the process and the sub-reliability T142 of the specific color component extraction process are calculated.

  Further, the type of erroneous recognition of “missing interpolation error” is likely to occur when the amount of contact with the original image is large and a plurality of associations are made. This is because the extraction line segment interruption correction unit 148 can be affected by the distance between the end points during the end point connection process during the end point connection process. In addition, as a case where a plurality of associations are made, a case where one image is originally divided by an overlapping portion with the original image and a case where two images originally are connected by correcting the discontinuity of the overlapping portion with the original image are connected. There are cases.

  As a case where one image is originally divided by an overlapping portion with the original image, for example, there is a case shown in FIG. As shown in FIG. 7A, when the target image of the additional information extraction process is marked with a scoring symbol 87 overlaid on a relatively small dark background such as a line segment, the difference extraction unit 132 and the data processing target additional recording are added. As shown in FIG. 7B, the extraction result of the postscript information in the information extraction unit 142 is extracted with the overlapping portion missing. The extraction line segment break correction unit 148 performs correction so that the missing portion is connected.

  That is, the extraction line segmentation correction unit 148 thins the extraction result of the predetermined color component by the data processing target additional information extraction unit 142, that is, the extraction result that should be a graphic such as “◯” or “x”. The process is executed, and the end point extraction process is further executed. As a result, when there is an interrupted portion in the graphic such as “◯” or “×”, the end point at the interrupted portion is extracted. When the end points are extracted, a predetermined one of all the extracted end points is connected.

  For example, when the graphic shown in FIG. 7B is extracted, the image of “◯” is divided at an overlapping portion of a relatively small dark background such as a line segment, and four end points A to D appear. The extraction line segment break correction unit 148 tries to connect these end points based on a predetermined condition. In this case, even if the closest end points are connected to each other as a basic condition, the end points may not be connected as such for some reason. This phenomenon is particularly likely to occur when a plurality of associations are made by dividing an image showing characters and figures by overlapping portions. This is because it is difficult to discriminate whether one image is originally divided or different from another, and end points are tried to be connected for each divided image.

  Therefore, for example, as shown in FIG. 7 (C1), while the divided upper half end point A and the end point C are connected, the divided lower half end point B and the end point D are not connected, As shown in FIG. 7 (C2), the divided upper half end point A and end point C are connected, while the divided lower half end point B and end point D are also connected in some cases.

  As a result, in any case, although the writer originally wrote “O” in general, the distorted state (substantially close to a semicircle) in which a portion overlapping with the dark background is omitted "Is input to the separation recognition processing unit 155.

  As a result, the separation recognition processing unit 155 should identify “◯” by graphic recognition processing or character recognition processing based on the distorted “◯” image. Since the feature of the figure is significantly changed due to the omission of the overlapped portion, the recognition rate (reliability) is lowered, and the possibility of a recognition error increases.

  Moreover, as a case where two images are originally connected by correcting the discontinuity of the overlapping portion with the original image, for example, there is a case shown in FIG. As shown in FIG. 8A, if the target image of the additional information extraction process is overlaid with a dark background such as a frame of the answer column 84 and a plurality of scoring symbols 87 are entered, the difference extraction unit 132 and the data process As shown in FIG. 8B, the extracted result of the additional write information in the target additional write information extraction unit 142 is extracted with each scoring symbol 87 lacking the overlapping portion. The extraction line segment break correction unit 148 performs correction so that the missing portion is connected.

  That is, the extraction line segmentation correction unit 148 thins the extraction result of the predetermined color component by the data processing target additional information extraction unit 142, that is, the extraction result that should be a graphic such as “◯” or “x”. The process is executed, and the end point extraction process is further executed. As a result, when there is an interrupted portion in the graphic such as “◯” or “×”, the end point at the interrupted portion is extracted. When the end points are extracted, a predetermined one of all the extracted end points is connected.

  For example, when the graphic shown in FIG. 8B is extracted, two “◯” images are divided by overlapping portions of relatively small dark backgrounds such as frame lines, and four end points A1 to D1 respectively. , A2 to D2 appear. The extraction line segment break correction unit 148 tries to connect these end points based on a predetermined condition.

  Here, one image was originally divided based on whether or not the distance between end points generated in a plurality of image components (whether or not they are divided) is equal to or less than a predetermined value. If the basic condition is that the nearest end points are connected to each other, the extraction part of each “○” image is As shown in FIG. 8C, by connecting the nearest end points, the original “◯” is almost reproduced.

  However, for the two “◯” image components, the end points may be further connected for some reason. This phenomenon tends to occur when a plurality of images are originally associated with each other by being divided by the same background. Even if there are a plurality of images originally, it is difficult to distinguish whether one image was originally divided or originally, and it is difficult to separate the endpoints from the divided images. This is because trying to connect occurs.

  Thus, for example, as shown in FIG. 8C, the two “◯” image components on the right and left sides are connected by a line segment to correct the overlapping portion with the frame, so that one image as a whole is connected. turn into.

  As a result, although the entry person originally entered each item with a distinction of “○”, the portions that overlap the dark background are connected by line segments, and the distorted “○” (two The images (connected to one) are input to the separation recognition processing unit 155.

  As a result, the separation recognition processing unit 155 should identify and distinguish each from “◯” by graphic recognition processing or character recognition processing based on the distorted “◯” image. Since the feature of the figure is greatly changed by the line segment for correcting the overlapping portion with the original image, the recognition rate (reliability) is lowered, and the possibility of occurrence of a recognition error increases.

  Therefore, as shown in FIG. 7D and FIG. 8D, the difficult recognition additional recording information extraction unit 194 can identify the type of such erroneous recognition of “missing interpolation error”. In the interruption correction processing in the extraction line segment interruption correction unit 148, the interpolation length information J20 indicating the interpolation length at the time of the interruption correction processing, the number of end points J21 as connection candidates, and the like are related to the recognition rate and reliability. It calculates | requires as quantity and notifies the difficult recognition additional record information extraction part 194 of interpolation length information J20.

  Based on the notified interpolation length information J20 and the like, the difficult recognition additional record information extraction unit 194 calculates the sub-reliability T148 of the interruption correction process by adding, for example, as 1 if the standard is satisfied and 0 if not satisfied, respectively. To do.

  In addition, the type of misrecognition of “multiple addition” is when the reliability of shape recognition or character recognition is low, and there is a correspondence between a plurality of scoring symbols 87 and comments 88 because the distance to other adjacent entry fields is short. Easy to happen. This is because the presence or absence of contact between a plurality of additional information is the reliability of recognition during the graphic recognition processing in the graphic shape recognition units 156 and 166 of the separation recognition processing unit 155 and the character recognition processing in the character recognition processing units 157 and 167. It is based on influencing.

  For example, as shown in FIG. 9A, when two answer columns 84 are described in two lines and there is not enough distance between them, each student answer entered in each answer column 84 is assigned a scoring symbol. When 87 is entered by a grader, there is typically not enough space between two scoring symbols 87 to contact.

  For this reason, as shown in FIG. 9B, the separation recognition processing unit 155 can identify the type of misrecognition of such “multiple addition” in the difficult recognition additional recording information extraction unit 194. First, contact information J30 indicating the presence / absence of contact between the scoring symbols 87 and contact information J31 indicating the presence / absence of contact between the comments 88 are obtained. Further, in the graphic recognition processing by the graphic shape recognition unit 156, the scoring symbol 87 is obtained. And the reliability information J32 and J33 of the recognition result of the graphic recognition process for the comment 88 are obtained as feature quantities related to the graphic recognition rate and the graphic reliability for the scoring symbol 87 and the comment 88. In the character recognition processing in the character recognition processing units 157 and 167, the reliability information J34 and J35 of recognition of the character recognition processing result for the scoring symbol 87 and the comment 88 is used as the character recognition processing for the scoring symbol 87 and the comment 88. It is obtained as a feature quantity related to the rate and the reliability of the characters, and these pieces of information are notified to the difficult-to-recognize additional information extracting unit 194.

  The difficult recognition additional record information extraction unit 194 adds, for example, as 1 if the criteria are satisfied and 0 if not, based on the contact information J30, reliability information J32, and reliability information J34 regarding the notified scoring symbol 87. Thus, the sub-reliability T156 of the graphic recognition process and the sub-reliability T157 of the character recognition process are calculated. Further, based on the contact information J31, reliability information J33, and reliability information J35 for the comment 88, for example, 1 is added if the standard is satisfied, and 0 is added if the standard is not satisfied, thereby sub-reliability T168 of the graphic recognition process. And the sub-reliability T167 of the character recognition process are calculated.

  Furthermore, the type of misrecognition of “additional correction” is highly reliable in shape recognition and character recognition, but is far from the entry column and has a plurality of scoring symbols 87 and comments 88 for one entry column. It is likely to occur when there is a correspondence. This is to identify which entry field corresponds to the additional information to be noticed in the data processing unit 170 based on the result of specifying the entry position specified by the entry position recognition units 158 and 168. Thus, when data processing is executed, whether or not one additional information relates to a plurality of entry positions affects the data processing result.

  For example, as shown in FIGS. 10A and 10B, when two answer columns 84 are described in two lines and the distance between them is widened to some extent, the student answers entered in each answer column 84 When the scoring symbol 87 is entered by the grader for each of the above, since there is a wide interval, one scoring symbol 87 for either one is typically entered between the two answer fields 84. is there.

  Here, as shown in FIG. 10 (A), in the case where the additional information is large and is written across the upper and lower answer fields 84, the circumscribed rectangle of the additional information and the areas to be the respective answer fields 84 Since the overlapping area is substantially the same, and the distance differences between the answer columns 84 are also substantially the same, the additional information entered between the two answer columns 84 should correspond to any answer column 84. It becomes difficult to specify. In other words, there is a possibility that a plurality of scoring symbols 87 are associated with one answer field 84.

  In addition, as shown in FIG. 10B, in the case where the additional information is small and is entered between the upper and lower answer columns 84, the circumscribed rectangle of the additional information overlaps with the area serving as each answer column 84. Since the area is approximately zero and the distance difference between each answer field 84 is substantially the same, the additional information entered between the two answer fields 84 should correspond to which answer field 84. It becomes difficult to specify. In other words, there is a possibility that a plurality of scoring symbols 87 are associated with one answer field 84.

  For this reason, as shown in FIG. 10 (C), the entry recognition unit 158, 158, so as to be able to identify the type of misrecognition of such “additional correction” by the difficult recognition additional recording information extraction unit 194. In the entry position recognition process at 168, a plurality of information indicating the possibility of associating one additional information with a plurality of entry fields or the possibility of associating a plurality of additional information with respect to one entry field. Additional information J40, J41, distance information J42, J43 indicating the distance between each additional information and each entry field is obtained as a feature quantity related to the recognition rate and reliability, and these pieces of information are recognized as difficult recognition additional information extraction unit 194. Notify

  Based on the notified information J40 and J42, the difficult recognition additional record information extraction unit 194 adds 1 as, for example, 0 if the criteria are not satisfied, and adds 0 as otherwise, for the scoring symbol 87 in the entry position recognition unit 158. The sub-reliability T158 of the entry position recognition process is calculated, and based on the notified information J41 and J43, for example, 1 is added if the standard is satisfied, and 0 is added if the reference is not satisfied. The sub-reliability T168 of the entry position recognition process for the comment 88 is calculated.

  The difficult-to-recognize additional information extraction unit 194 performs misrecognition on the additional information to be noticed (scoring symbol 87 and comment 88) with a sub-reliability value lower than a certain level within the sub-reliabilities of each process obtained. Extract type. At this time, only the type of misrecognition with the lowest sub-reliability may be extracted. By repeating this third integration method for all additional write information, additional write information having a type of erroneous recognition whose sub-reliability value is lower than a certain level is extracted.

  The recognition performance information presentation unit 198 fills the recognition performance information so as to prompt correction of the additional information having the type of erroneous recognition whose sub-reliability value extracted by the difficult recognition additional information extraction unit 194 is lower than a certain level. To present.

  Only the additional information with the type of misrecognition whose recognition performance is less than a certain level is prompted by the writer, so there is no need to judge (check) the necessity of correction for all the additional information and cause misrecognition. It is possible to improve the efficiency of the correction of the additional information that can be performed. When a misrecognition type having a reliability lower than a predetermined level is presented, the writer can improve the current entry state from the aspect of the misrecognition type. When two types of misrecognition with a reliability lower than a predetermined level are presented for one additional information, the writer can improve the current entry state from the aspect of each misrecognition type. When only the type of misrecognition with the lowest reliability is presented, the writer can improve the current entry state from the aspect of the type of misrecognition with the lowest reliability.

  By adopting such a third integration method, a specific improvement method can be prepared for each type, so that the writer can easily pay attention to the entry method and the improvement effect is improved.

<Recognition performance information presentation method>
FIGS. 11-16 is a figure explaining the method in which the recognition performance information presentation part 198 presents the recognition performance information for improving recognition performance to a user.

  When the recognition performance information presentation unit 198 presents the recognition performance information to the user (entry person), it is preferable to present the recognition performance information corresponding to each of the entry persons. As a presentation method at that time, for example, as shown in FIG. 11, the recognition performance information presentation unit 198 changes the method of presenting the recognition performance information depending on the reliability for those whose reliability is below a certain level. It is conceivable to adopt the first presentation method to be presented. Here, “reliability” means the final reliability.

  Here, the first presentation method of “changing the display method according to the reliability” is intended to make it possible to easily distinguish the final reliability level and the type of misrecognition visually. Yes, as long as it can be distinguished by various presentation methods. For example, it can be displayed with different gradients depending on the final reliability and the type of misrecognition. Further, for example, it is possible to adopt a technique in which a frame is attached to those whose reliability is below a certain level, and the frame is thicker as the reliability is lower.

  Depending on the display method, the viewer (entrant) can determine which of the additional information recognized and specified is the one whose reliability is lower than a certain level. Therefore, even when this first presentation method is adopted, the recognition performance information is presented to the writer so as to urge correction only for the additional information having a reliability level lower than a certain level. The writer can improve the entry mode for additional information whose reliability value is lower than a certain level.

  If this 1st presentation method is taken, a serious oversight can be prevented by making the thing which needs correction easy to see. In addition, by showing the difference in reliability, it is possible to compare and learn what figure is not suitable for recognition, which has the effect of improving the entry method.

  In addition, the recognition performance information presentation unit 198 presents the recognition performance information for improving the recognition performance for only the difficult recognition information whose reliability is lower than a certain level for all the additional information subjected to the recognition process. It is conceivable to adopt a method. For example, the difficulty recognition additional record information extraction unit 194 specifies the final reliability specified based on the first and second integration methods based on the additional information that is lower than a predetermined level or the third integration method. The recognition performance information is presented only for the additional information having the type of erroneous recognition whose reliability is lower than the predetermined level or lowest.

  Here, “presenting recognition performance information for improving recognition performance only with respect to difficult recognition information whose reliability is lower than a certain level” means that for all additional information subjected to recognition processing, “reliability is This means that the recognition performance information is presented so that the “difficult recognition information with low reliability” can be reliably identified by separately presenting “low difficulty recognition information” and “other additional information”. For example, information other than “difficult recognition information with low reliability” may not be displayed.

  The recognition performance information can be presented to the writer so as to prompt correction only for difficult recognition information whose reliability level is lower than a certain level, and the writer has difficulty recognition whose reliability level is lower than a certain level. It is possible to improve the information entry mode.

  If this second presentation method is adopted, recognition performance information for improving recognition performance is presented only for difficult recognition information whose reliability is lower than a certain level, so that only information that needs to be corrected can be quickly confirmed. This has the effect of making correction work efficiently.

  In the case of adopting the second presentation method, it is possible to recognize at a glance where the additional information is extracted as the additional information extracted as the target of the recognition performance information because the recognition performance is poor and needs to be corrected. It is better to highlight in such a state. By highlighting, it is possible to distinguish and present difficult-to-recognize information whose reliability is lower than a certain level and other information.

  For example, as shown in FIGS. 12A and 12B, the postscript information is surrounded by a frame (FIG. 12A), an arrow (or other mark) is attached to the postscript information (FIG. 12B), and the postscript information is electronic. The sticky note is attached (FIG. 13), or the illustration is omitted, but the thickness of the postscript information is changed depending on the reliability, and the color of the postscript information is changed depending on the reliability. It is good to distinguish and display difficult recognition information lower than a certain level and other information.

  Since the range is easy to specify in the display mode surrounded by the frame, there is an effect that it is easy to visually recognize especially when a plurality of figures are connected as shown in FIG. In addition, in the display mode in which an arrow or other mark is added, the size can be set freely, so that there is a low possibility that the document will be covered with the description of the original document. In the display mode for attaching an electronic sticky note, even when a plurality of documents are overlapped, the sticky note other than the topmost one can be visually recognized, so that there is an effect that it is easy to confirm the entire amount and position of the postscript information with low reliability.

  Alternatively, as shown in FIG. 14, a list of links is created and displayed for additional information extracted as a target of presentation of recognition performance information because the recognition performance is poor and needs to be corrected, and one of the lists is clicked. When it is done, the additional information filled teaching material 81 may be displayed and the additional information of the corresponding part may be displayed. At this time, it is preferable to display the additional information of the corresponding part by adopting the highlighting shown in FIG.

  In the list of links, only information that can reach difficult recognition information is presented, such as a question sentence corresponding to an answer column 84 in which a grading symbol 87 or a comment 88 with a reliability level of a certain level or less is entered. Based on this list of links, the necessary difficulty recognition information can be displayed before starting the correction work.

  On the additional information filled-in teaching material 81, not only difficult recognition information but all additional information is presented, but the list of links presents only information that can reach difficult recognition information. An aspect is also included in “presenting recognition performance information for improving recognition performance only for difficult recognition information whose reliability is lower than a certain level”.

  Alternatively, as shown in FIG. 15, with respect to the additional information extracted as the recognition performance information presentation target because the recognition performance is poor and needs to be corrected, the additional information filled teaching material 81 is displayed and the next jump button is clicked , By using a predetermined gesture such as clicking or double-clicking the “send to next” button, or a predetermined key operation (such as an arrow key), the recognition performance is poor and multiple corrections are required. The additional information may be displayed so as to be able to jump one after another within the same additional information filled learning material 81 or to another additional information filled learning material 81. At this time, it is preferable to display the additional information of the corresponding part (jump destination) by using the highlighting shown in FIG.

  Further, as shown in FIG. 16, the recognition performance information presenting unit 198 recognizes the determination reason so determined together with the reliability for additional information (difficult recognition information) that requires correction whose reliability is less than a certain level. It is conceivable to adopt a third presentation method presented as information.

  Here, “reliability” means the final reliability. In addition, the “determination reason” may be, for example, to specify a process having a low sub-reliability as a determination reason by using various sub-reliabilities notified to the difficult-to-recognize additional write information extraction unit 194.

  By adopting this third presentation method, not only can additional information be reported with a reliability value lower than a certain level, but also the reason for such determination can be presented to the writer as recognition performance information. The person can improve the additional information according to the presented reason for the additional information whose reliability value is lower than a certain level. Since it is possible to know in detail which points should be noted, it can be used to learn how to fill in. It is possible to reliably grasp an inappropriate part such as the shape of handwritten input information and to learn an accurate figure or character shape.

  In any of the first to third presentation methods described above, information related to the recognition rate in various processing processes is collected, and additional information whose reliability value is lower than a certain level based on the collected information. Identify and present recognition performance information for improving the recognition performance of handwritten input information, that is, recognition performance information that encourages correction of the current additional information for additional information that has a reliability value lower than a certain level. Therefore, it is not necessary to check all the additional write information, and the work of correcting the description of the additional write information that causes erroneous recognition can be performed efficiently.

  In addition, during the display of the additional information filled-in teaching material 81, the recognition performance information that prompts the user to correct the current additional information (difficult recognition information) whose recognition reliability is lower than the matching level is presented. Unlike the mechanism described in Japanese Patent Application No. -105796, there is an advantage that it is not necessary to create a screen dedicated to correction.

<Reliability information collection and presentation processing; specific example>
FIGS. 17 and 18 are diagrams showing specific reliability information collection and presentation processing for an educational material 80 which is an example of the original document 8A. Here, FIG. 17 shows the overall outline in association with the system configuration diagram, and FIG. 18 is a flowchart showing the reliability information collection and presentation processing procedure.

  Here, an example is shown in which an answer as an educational material 80 is created, and an automatic scoring process based on a scoring symbol 87 and a comment classification process based on a comment 88 are performed after a test using the answer.

  When the test is executed, the educational material 80 is read from the original document information database DB2, printed by a printer, and distributed to students and examinees (S206). After the test, the grader adds a scoring symbol 87 and a comment 88 to the student's answer (S208).

  At the time of automatic data processing, first, the document input device 20 reads the additional information filled teaching material 81 of each answerer in which the scoring symbol 87 and the comment 88 are entered by the grader (S210), and the additional information filled teaching material 81 is read. The represented image data is input to the additional recording information processing apparatus 10 (S212). The document input device 20 temporarily stores image data obtained by image reading by the document input device 20 in a memory or the like used as a work area.

  The additional recording information processing apparatus 10 (teaching material processing apparatus) receives the read image data of the additional information filled teaching material 81, specifies the original document specifying code embedded in the halftone image (S224), and the corresponding teaching teaching material. Original information such as 80 original images and entry field position area information 38 is obtained from the document management server 30 functioning as the document original information database DB2 (S225).

  Then, as automatic data processing, the scoring symbol 87 and the comment 88 entered in the additional information filled teaching material 81 are extracted by the difference extraction unit 132 and the data processing additional information extraction processing unit 140 (S242), and the separation recognition processing is performed. The entry contents and entry positions are specified by the section 155 and the entry position recognition sections 158 and 168 (S262), and then the data processing section 170 performs automatic scoring and summarizing processing for the scoring symbols 87 and automatic comment classification processing for the comments 88. Is executed (S266).

  At this time, the separation recognition processing unit 155, the entry position recognition units 158 and 168, or the data processing unit 170 receives a correction instruction from the user (S310). Thereafter, when the automatic data processing is completed (S264-YES), the data processing unit 170 registers the result of scoring / counting and the comment classification result in the processing result storage server 40 (S268).

  Here, in such a process, the difficult recognition additional record information extraction unit 194 of the recognition performance information presentation processing unit 190 collects each feature quantity related to the recognition rate in the processing from each functional unit (S300). Based on each collected feature amount, a sub-reliability for each process is calculated (S302).

  For example, the difference extraction unit 132 includes the quality information J10 in the difference extraction process between the educational teaching material 80, which is an example of the original document 8A, and the additional information filled learning material 81, which is an example of the added document 8B, and a scoring symbol. The overlap amount J12 to the dark background of 87 or the comment 88 is extracted and notified to the difficult-to-recognize additional write information extracting unit 194. Further, the data processing target additional information extraction unit 142 of the data processing additional information extraction processing unit 140 extracts a grading symbol 87 and a comment 88 as additional data processing target additional information of the difference information 9 extracted by the difference extraction unit 132. The contact amount J13 between the original image and the red extracted image obtained from the quality information J11 and the positional relationship between the red extracted image and the original image is extracted and notified to the difficult recognition additional record information extracting unit 194. Based on the notified quality information J10, J11, the overlap amount J12, and the contact amount J13, the difficult recognition additional record information extraction unit 194 calculates the sub reliability T132 of the difference extraction process and the sub reliability T142 of the specific color component extraction process. calculate.

  Further, the extracted line segment break correction unit 148 extracts the interpolation length information J20 indicating the interpolation length at the time of the break correction process, the number of end points J21 as connection candidates, and notifies the difficult recognition additional record information extraction unit 194. The difficult recognition additional record information extraction unit 194 calculates the sub-reliability T148 of the interruption correction process (missing interpolation) based on the notified interpolation length information J20 or the like.

  The separation recognition processing unit 155 first extracts contact information J30 indicating the presence / absence of contact between the scoring symbols 87 and contact information J31 indicating the presence / absence of contact between the comments 88, and notifies the difficult recognition additional record information extraction unit 194. To do.

  Further, the graphic shape recognition unit 156 of the separation recognition processing unit 155 extracts the reliability information J32 of the recognition of the graphic recognition processing result for the scoring symbol 87, notifies the difficult recognition additional record information extraction unit 194, and performs graphic shape recognition. The unit 166 extracts the recognition reliability information J33 of the graphic recognition processing result for the comment 88, and notifies the difficult recognition additional record information extraction unit 194 of it.

  In addition, the character recognition processing unit 157 of the separation recognition processing unit 155 extracts the recognition reliability information J34 of the character recognition processing result for the scoring symbol 87, notifies the difficult recognition additional record information extraction unit 194, and performs character recognition processing. The unit 167 extracts the recognition reliability information J35 of the recognition result of the character recognition process for the comment 88 and notifies the difficult recognition additional record information extraction unit 194.

  The difficult recognition additional record information extraction unit 194, based on the contact information J30, reliability information J32, and reliability information J34 for the notified scoring symbol 87, sub-reliability T156 for graphic recognition processing and sub-reliability for character recognition processing. T157 is calculated, and the sub-reliability T168 of the graphic recognition process and the sub-reliability T167 of the character recognition process are calculated based on the contact information J31, the reliability information J33, and the reliability information J35 for the comment 88. .

  Further, the entry position recognition unit 158 extracts a plurality of additional writing information J40 and distance information J42 for the scoring symbol 87 and notifies the difficult recognition additional writing information extraction unit 194, and the entry position recognition unit 168 The multiple additional writing information J41 and the distance information J43 are extracted and notified to the difficult recognition additional writing information extraction unit 194.

  The difficult recognition additional record information extraction unit 194 calculates the sub-reliability T158 of the entry position recognition process for the scoring symbol 87 based on the notified information J40, J42, and the comment 88 based on the notified information J41, J43. The sub-reliability T168 of the entry position recognition process for is calculated.

  The difficult-to-recognize additional recording information extraction unit 194 further assigns weights for each process based on the sub-reliability of each process and the past statistical information obtained as described above, for example, as shown in the following equation (1). The final reliability Tfinal is calculated by changing α (S304).

  The recognition performance information presentation unit 198 specifies low-reliability additional recording information whose reliability Tfinal is equal to or lower than a predetermined threshold on the automatic scoring result correction software incorporated in the software for automatic scoring processing (S306). ) On the user terminal 171, the recognition performance information that prompts the user to correct the specified current additional write information as the recognition performance information for improving the recognition performance of the handwritten input information regarding the additional write information whose reliability value is lower than the threshold value. (S308). In this case, as described above, for example, the additional writing information is highlighted or the reason for determination is indicated, so that the additional writing information that needs to be corrected and the correction method can be easily determined. It is better to display the status.

  The writer of the scoring symbol 87 and the comment 88 confirms the recognition performance information presented on the user terminal 171 while checking the recognition performance information, the character representing the scoring symbol 87 (which represents a misrecognition with a reliability value lower than the threshold value, The correction process is executed on the user terminal 171 only for the graphic) and the comment 88 (characters and graphics representing the graphic) (S310).

  When all the correction work is completed (S264-YES), the corrected additional writing information is reflected in the data processing. For example, final data for scoring and counting results and final data for comment classification results are registered in the processing result storage server 40 (S264). The correction result by the user is received and reflected by changing the data directly by database operation after recognition (S268). In other words, the data stored in the database may be changed directly.

  In the processing procedure of this example, correctness of recognition is reflected in the process of normal automatic data processing, and correction is required only for recognized handwritten input information that has poor recognition performance. Make corrections while checking the entry form such as scoring symbols 87 and comments 88 without any special effort, and correct the results of automatic scoring and automatic comment classification using the postscript information after correction. Can be changed.

<Additional information processing device; computer configuration>
FIG. 19 is a block diagram illustrating another configuration example of the additional recording information processing apparatus 10. Here, a more realistic hardware configuration is shown that is constructed from a microprocessor or the like that executes additional write information processing by software using an electronic computer such as a personal computer.

  That is, in this embodiment, the mechanism for executing the data processing related to the additional information is not limited to the hardware processing circuit, and is software-based using a computer (computer) based on the program code that realizes the function. It can also be realized.

  Therefore, a program suitable for realizing the mechanism according to the present invention by software using an electronic computer (computer) or a computer-readable storage medium storing this program can be extracted as an invention. By adopting a mechanism that is executed by software, it is possible to enjoy the advantage that the processing procedure and the like can be easily changed without changing hardware.

  When the data processing function related to additional information is executed by software in an electronic computer, a program (such as a built-in microcomputer) in which a program constituting the software is incorporated in dedicated hardware or a CPU (Central Processing) Unit), logic circuit, storage device, etc. on a single chip to realize the desired system SOC (System On a Chip) or various programs by installing various programs It is installed from a recording medium in a general-purpose personal computer or the like capable of executing functions.

  The recording medium causes a state change of energy such as magnetism, light, electricity, etc. according to the description contents of the program to the reading device provided in the hardware resource of the computer, and in the form of a signal corresponding to the change. The program description can be transmitted to the reader.

  For example, a magnetic disk (including a flexible disk FD), an optical disk (CD-ROM (Compact Disc-Read Only Memory)), a DVD on which a program is recorded, which is distributed to provide a program to a user separately from a computer. (Including Digital Versatile Disc), magneto-optical disc (including MD (Mini Disc)), or package media (portable storage media) made of semiconductor memory, etc. It may be configured by a ROM, a hard disk, or the like in which a program is recorded, which is provided to the user in a state of being recorded.

  The program constituting the software is not limited to being provided via the recording medium without using the recording medium, and may be provided via a wired or wireless communication network.

  For example, a storage medium in which a program code of software that realizes a data processing function related to additional information is supplied to a system or apparatus, and a program code stored in the storage medium by a computer (or CPU or MPU) of the system or apparatus The same effect as in the case where the hardware processing circuit is used is also achieved by reading and executing. In this case, the program code itself read from the storage medium realizes a data processing function related to additional write information.

  In addition, by executing the program code read by the computer, not only a data processing function related to additional information is realized, but also an OS (Operating Systems; basic software) running on the computer based on an instruction of the program code Or the like may perform part or all of the actual processing, and the data processing function regarding the additional information may be realized by the processing.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. There may be a case where a CPU or the like provided in the card or the function expansion unit performs part or all of the actual processing, and the data processing function regarding the additional information is realized by the processing.

  Note that the program is provided as a file describing a program code that realizes a data processing function related to additional information, but in this case, the program is not limited to being provided as a batch program file, and the hardware of a system configured by a computer Depending on the configuration, it may be provided as an individual program module.

  For example, the computer system 900 reads and records data from a controller unit 901 and a predetermined storage medium such as a hard disk device, a flexible disk (FD) drive, a CD-ROM (Compact Disk ROM) drive, or a semiconductor memory controller. And a recording / reading control unit 902.

  The controller unit 901 includes a CPU (Central Processing Unit) 912, a ROM (Read Only Memory) 913 which is a read-only storage unit, and a RAM (Random Access) which can be written and read at any time and is an example of a volatile storage unit. Memory) 915 and RAM (described as NVRAM) 916 which is an example of a nonvolatile storage unit.

  In the above description, the “volatile storage unit” means a storage unit in which the stored contents are lost when the power of the apparatus is turned off. On the other hand, the “nonvolatile storage unit” means a storage unit in a form that keeps stored contents even when the main power supply of the apparatus is turned off. Any memory device can be used as long as it can retain the stored contents. The semiconductor memory device itself is not limited to a nonvolatile memory device, and a backup power supply is provided to make a volatile memory device “nonvolatile”. You may comprise as follows.

  Further, the present invention is not limited to a semiconductor memory element, and may be configured using a medium such as a magnetic disk or an optical disk. For example, a hard disk device can be used as a nonvolatile storage unit. In addition, it is possible to use as a nonvolatile storage unit by adopting a configuration for reading information from a recording medium such as a CD-ROM.

  Further, the computer system 900 includes an instruction input unit 903 as a functional unit that forms a user interface, a display output unit 904 that presents a user with predetermined information such as a guidance screen and a processing result, and each functional unit. And an interface unit (IF unit) 909 that performs an interface function between them.

  Note that an image forming unit 906 that outputs the processing result to a predetermined output medium (for example, printing paper) may be provided so that the data processing result is printed out and presented to the user.

  As the instruction input unit 903, for example, the operation key unit 985b of the user interface unit 985 can be used. Alternatively, a keyboard or mouse can be used.

  The display output unit 904 includes a display control unit 919 and a display device. As the display device, for example, the operation panel unit 985a of the user interface unit 985 can be used. Alternatively, other display units such as CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) can be used.

  For example, the display control unit 919 displays the guidance information, the entire image captured by the document input device 20, and the like on the operation panel unit 985a and the display unit. It is also used as a display device for notifying the user of various information. Note that an instruction input unit 903 for inputting predetermined information with a fingertip, a pen, or the like can be configured by using a display unit having a touch panel on the display surface.

  The interface unit 909 includes a system bus 991 that is a transfer path for processing data (including image data) and control data, a printer IF unit 996 that functions as an interface with the image forming unit 906 and other printers, and the like. A communication IF unit 999 is provided to mediate communication data exchange with the network.

  In such a configuration, the CPU 912 controls the entire system via the system bus 991. The ROM 913 stores a control program for the CPU 912 and the like. The RAM 915 is configured by SRAM (Static Random Access Memory) or the like, and stores program control variables, data for various processes, and the like. The RAM 915 includes an area for temporarily storing data obtained by calculation according to a predetermined application program, data obtained from the outside, and the like.

  For example, a program that causes a computer to execute a data processing function related to additional recording information is distributed through a recording medium such as a CD-ROM. Alternatively, this program may be stored in the FD instead of the CD-ROM. In addition, an MO drive may be provided to store the program in the MO, or the program may be stored in another recording medium such as a nonvolatile semiconductor memory card such as a flash memory. Furthermore, the program may be downloaded from another server or the like via a network such as the Internet, or may be updated or updated.

  As a recording medium for providing the program, in addition to FD and CD-ROM, optical recording medium such as DVD, magnetic recording medium such as MD, magneto-optical recording medium such as PD, tape medium, magnetic recording A semiconductor memory such as a medium, an IC card, or a miniature card can be used. A part or all of functions for realizing a data processing function related to additional write information can be stored in an FD or CD-ROM as an example of a recording medium.

  Further, the hard disk device includes an area for storing data for various processes by the control program, and temporarily storing a large amount of data acquired by the device itself or data acquired from the outside.

  With such a configuration, additional information is written to the RAM 915 from a readable recording medium such as a CD-ROM in which a program for executing the additional recording information processing method is stored in accordance with an instruction from the operator via the operation key unit 985b. The processing program is installed, and the additional information processing program is started by an instruction or automatic processing by the operator via the operation key unit 985b. For example, if the teaching material automatic scoring system 1 is to be realized, the teaching material processing program recognizes a difference extraction result of a predetermined color component, specifically, for example, a red component as the contents of the marking symbol 87 and the comment 88, In addition, processing steps for separating both are described, and the program is started.

  The CPU 912 performs calculation processing according to the additional information processing method according to the additional information processing program, stores the processing result in a storage device such as the RAM 915 or a hard disk, and displays the operation panel unit 985a or a display such as a CRT or LCD as necessary. Output to the device. By using a recording medium recorded with a program for executing the additional recording information processing method, the additional recording information processing system can be constructed universally without changing the existing system.

  Not only the configuration using such a computer, but also a write-once information processing apparatus 10 that performs a data processing function related to write-once information by a combination of dedicated hardware that performs processing of each functional unit shown in FIG. It can also be configured.

  In addition, for example, a processing circuit 908 may be provided in which not all processing of each functional part for the data processing function related to the additional information is performed by software, but part of these functional parts is performed by dedicated hardware. . Although the mechanism performed by software can flexibly cope with parallel processing and continuous processing, the processing time becomes longer as the processing becomes complicated, so that a reduction in processing speed becomes a problem.

  On the other hand, it is possible to construct an accelerator system with a higher speed by using a hardware processing circuit. Even if the processing is complicated, the accelerator system can prevent a reduction in processing speed and can obtain a high throughput.

  For example, in the case of realizing a data processing function related to additional information, the processing circuit 908 includes a read image processing unit 908 a corresponding to the read image processing unit 110 shown in FIG. 2 and a document corresponding to the document original specifying unit 120. Data corresponding to the original data specifying unit 908b, the additional information extracting unit 908c corresponding to the additional information extracting unit 130, the data processing target additional information specifying unit 908d corresponding to the data processing target additional information specifying unit 150, or the data processing unit 170. The processing unit 908e and the like may be configured by hardware.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various changes or improvements can be added to the above-described embodiment without departing from the gist of the invention, and embodiments to which such changes or improvements are added are also included in the technical scope of the present invention.

  Further, the above embodiments do not limit the invention according to the claims (claims), and all combinations of features described in the embodiments are not necessarily essential to the solution means of the invention. Absent. The embodiments described above include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. Even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, as long as an effect is obtained, a configuration from which these some constituent requirements are deleted can be extracted as an invention.

  For example, a distortion correction process and a break correction process are not necessarily essential.

  In addition, as an example of the additional document 8B to be processed, the additional information filled teaching material 81 used for the automatic scoring processing mechanism has been described as an example. However, this is merely an example, and the handwritten data to be processed is a handwritten data. As long as additional information is included, the type of the document is not limited. For example, a completed form (see Japanese Patent Laid-Open Nos. 5-342239 and 6-274156) used for an automatic form processing mechanism and an addition used for a mechanism for personal information management (especially also called schedule management) An information-filled notebook (see JP-A-5-216932) may be used.

  Further, as described above, the post-recorded document 8B to be processed is not limited to using a paper medium, and even if the post-recorded document 8B is obtained in the form of electronic data from the beginning, The above-described mechanism can be similarly applied as long as additional information to be processed is included.

It is a figure which shows the educational teaching material which is an example of the document made into the process target in the postscript information processing which concerns on this invention. It is a figure which shows the structural example of one Embodiment of the information processing system provided with the postscript information processing apparatus which concerns on this invention. It is the figure which showed the whole outline | summary of the teaching material processing method in a teaching material automatic scoring system matched with the system block diagram. It is a flowchart which shows the whole process sequence in a teaching material automatic scoring system. It is a process of a recognition performance information presentation process part, Comprising: It is a figure which shows an example of the method of collecting the feature-value regarding the recognition rate at the time of each process, and reliability. It is a process of a recognition performance information presentation process part, Comprising: It is a figure explaining the type of misrecognition of "extraction mistake". It is a process (1) explaining the type of misrecognition of "missing interpolation error", which is a process of the recognition performance information presentation processing unit. It is a process (2) explaining the type of misrecognition of “missing interpolation error”, which is a process of the recognition performance information presentation processing unit. It is a process of a recognition performance information presentation process part, Comprising: It is a figure explaining the type of misrecognition of "multiple addition". It is a process of a recognition performance information presentation process part, Comprising: It is a figure explaining the type of misrecognition of "additional correction". It is a figure explaining the 1st presentation method in which recognition performance information for improving recognition performance in a recognition performance information presentation part is shown to a user. It is the figure (the 1) explaining the 2nd presentation method in which the recognition performance information presentation part presents the recognition performance information for improving recognition performance to a user. It is FIG. (2) explaining the 2nd presentation method in which the recognition performance information presentation part presents the recognition performance information for improving recognition performance to a user. It is the figure (the 3) explaining the 2nd presentation method in which recognition performance information for improving recognition performance in a recognition performance information presentation part is shown to a user. It is FIG. (4) explaining the 2nd presentation method in which recognition performance information for improving recognition performance in a recognition performance information presentation part is shown to a user. It is a figure explaining the 3rd presentation method in which recognition performance information for improving recognition performance in a recognition performance information presentation part is shown to a user. It is the figure which showed the concrete reliability information collection presentation process which makes educational materials as a process target matched with a system block diagram. It is a flowchart which shows a reliability information collection presentation process procedure. It is the figure which showed an example of the hardware constitutions in the case of constituting a postscript information processor using an electronic computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Teaching material automatic scoring system, 8A ... Original document, 8B ... Additional recorded document, 9 ... Difference information, 10 ... Additional information processing apparatus, 20 ... Document input device, 30 ... Document management server, 40 ... Processing result storage server, 80 ... Educational teaching materials 81 ... Additional information filled teaching materials 87 ... Scoring symbols 88 ... Comments 110 ... Reading image processing unit 120 ... Original document specifying unit 122 ... Teaching material specifying unit 130 ... Additional information extracting unit 132 ... difference extraction unit, 134 ... answerer extraction unit, 136 ... handwritten information extraction unit, 138 ... character recognition processing unit, 140 ... data processing additional information extraction processing unit, 142 ... data processing target additional information extraction unit, 146 ... Additional writing information shaping unit, 148... Extraction line segmentation correction unit, 150... Data processing target additional writing information specifying processing unit, 154... First data processing additional writing information recognition unit, 155. 6 ... Graphic shape recognition unit, 157 ... Character recognition processing unit, 158 ... Entry position recognition unit, 164 ... Additional data recognition unit for second data processing, 166 ... Graphic shape recognition unit, 167 ... Character recognition processing unit, 156a, 166a 157a, 167a ... deformation processing unit, 168 ... entry position recognition unit, 170 ... data processing unit, 170_1 ... first data processing unit, 170_2 ... second data processing unit, 171 ... user terminal, 172 ... scoring totaling unit, 174 ... Aggregation result output unit, 176 ... Comment classification processing unit, 178 ... Comment processing result output, 190 ... Recognition performance information presentation processing unit, 192 ... Additional information recognition history holding unit, 194 ... Difficulty recognition additional information extraction unit, 198 ... Recognition Performance information presentation section

Claims (10)

An input correction method for correcting additional information that is additionally written and used for recognition processing,
With regard to the additional information to be noticed, the feature amount related to the reliability of the recognition process in each of the various processes related to the recognition process including the recognition process itself is acquired, and the recognition process of each process is obtained based on each feature amount. Sub-reliability related to each is calculated, a final reliability for the additional information of interest is specified based on the calculated sub-reliability, and whether or not the final reliability is lower than a certain level. By identifying the difficulty recognition information whose recognition processing reliability is lower than a certain level, the recognition performance information for improving the recognition performance of the identified difficulty recognition information is presented. How to fix. This is an additional information processing method for recognizing the content of the additional information added to the document with additional information added to the original document as a processing target, and performing predetermined data processing based on the recognized information. And
With regard to the additional information to be noticed, the feature amount related to the reliability of the recognition process in each of the various processes related to the recognition process including the recognition process itself is acquired, and the recognition process of each process is obtained based on each feature amount. Sub-reliability related to each is calculated, a final reliability for the additional information of interest is specified based on the calculated sub-reliability, and whether or not the final reliability is lower than a certain level. Identifying difficult recognition information whose recognition processing reliability is lower than a certain level by determining whether or not, presenting recognition performance information for improving the recognition performance of the identified difficult recognition information,
A postscript information processing method, wherein the corrected additional information entered corresponding to the presentation of the recognition performance information is reflected in data processing. This is an additional information processing apparatus for recognizing the content of the additional information added to the original document with additional information added to the original document, and performing predetermined data processing based on the recognized information. And
With regard to the additional information to be noticed, the feature amount related to the reliability of the recognition process in each of the various processes related to the recognition process including the recognition process itself is acquired, and the recognition process of each process is obtained based on each feature amount. Sub-reliability related to each is calculated, a final reliability for the additional information of interest is specified based on the calculated sub-reliability, and whether or not the final reliability is lower than a certain level. In order to improve the recognition performance of the difficult recognition information identified by the difficult recognition additional record information extracting unit and the difficult recognition additional record information extracting unit that identifies difficult recognition information whose reliability of recognition processing is lower than a certain level by determining A recognition performance information presentation unit for presenting recognition performance information of
A postscript information processing apparatus, comprising: a data processing unit that reflects, in data processing, the corrected additional information entered corresponding to the presentation of the recognition performance information by the recognition performance information presenting unit. 4. The postscript information processing according to claim 3, wherein the difficult recognition additional record information extraction unit identifies, as the final reliability, the one with the lowest reliability among the sub-reliabilities of each process. apparatus. The write-once information processing according to claim 3, wherein the difficult-to-recognize additional information extraction unit identifies the final reliability by an addition process in which the sub-reliability of each process is assigned a predetermined weight. apparatus. The difficult recognition additional recording information extraction unit defines in advance the type of misrecognition with low reliability, and selects the type of misrecognition based on the result of addition processing of the sub-reliability of each process, The postscript information processing apparatus according to claim 3, wherein the difficulty recognition information is specified. The additional information processing apparatus according to claim 3, wherein the recognition performance information presentation unit changes a method of presenting the recognition performance information according to reliability. The recognition performance information presenting section distinguishes and presents difficult-to-recognize information, which is additional information having a reliability lower than a certain level, and other additional information for all the additional information subjected to recognition processing. Item 7. The postscript information processing device according to any one of items 3 to 6. The additional performance information processing apparatus according to any one of claims 3 to 6, wherein the recognition performance information presenting unit presents the determination reason so determined together with the reliability as the recognition performance information. . In order to perform processing of recognizing the content of the additional information added to the document with additional information added to the original document as a processing target, and to perform predetermined data processing based on the recognized information using a computer The program of
The computer,
With regard to the additional information to be noticed, the feature amount related to the reliability of the recognition process in each of the various processes related to the recognition process including the recognition process itself is acquired, and the recognition process of each process is obtained based on each feature amount. Sub-reliability related to each is calculated, a final reliability for the additional information of interest is specified based on the calculated sub-reliability, and whether or not the final reliability is lower than a certain level. In order to improve the recognition performance of the difficult recognition information identified by the difficult recognition additional record information extracting unit and the difficult recognition additional record information extracting unit that identifies difficult recognition information whose reliability of recognition processing is lower than a certain level by determining A recognition performance information presentation unit for presenting recognition performance information of
A program that functions as a data processing unit that reflects corrected additional information entered in response to presentation of the recognition performance information by the recognition performance information presentation unit in data processing.

Images