JP2020053891A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

To provide an information processing apparatus, or the like, configured to properly manage documents.SOLUTION: An information processing apparatus 1 include: an acquisition unit which acquires a sheet image obtained by scanning a filled sheet having a content printed thereon and a handwritten object written by hand; a content specifying unit which specifies the content from the sheet image; an extraction unit which extracts the handwritten object from the sheet image; and a generation unit which specifies a masking area by excluding the handwritten object from a content area, which is a peripheral area including the contents, and generates the sheet image processed to obscure the masking area.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing device, an information processing method, and a program.

  When performing document management, it is common practice to manage a document after making a portion of the document to be concealed invisible by masking or the like. For example, Patent Document 1 discloses a document management system in which a scanned image of a document is masked and then stored, and a disclosure level is set for each area in the document, and a plurality of areas are masked in different areas according to the disclosure level. A document management system for generating an image is disclosed.

JP 2007-140958 A

  However, the invention according to Patent Document 1 masks a predetermined area in a document. Therefore, when a character or the like is written by hand in a document, depending on the position where the character or the like is written, even a handwritten character or the like may be masked.

  An object of one aspect is to provide an information processing apparatus or the like that can appropriately perform document management.

  In one aspect, an information processing apparatus specifies an acquisition unit that acquires a paper image obtained by scanning an entry sheet having pre-printed content and a handwritten object entered by hand, and the content from the paper image. A content specifying unit, an extracting unit that extracts the handwritten object from the paper image, and a masking area that excludes the handwritten object from a content area that is a peripheral area including the content so that the masking area cannot be visually recognized. And a generation unit that generates the paper image subjected to image processing.

  In one aspect, document management can be performed appropriately.

FIG. 1 is a schematic diagram illustrating a configuration example of a document management system. FIG. 2 is a block diagram illustrating a configuration example of a PC. FIG. 9 is an explanatory diagram regarding a process of specifying a content area. It is explanatory drawing regarding the specific processing of a handwriting entry area. It is explanatory drawing regarding a mask process. 9 is a flowchart illustrating an example of a processing procedure executed by a PC. FIG. 14 is an explanatory diagram relating to a content area specifying process according to the second embodiment; 13 is a flowchart illustrating an example of a processing procedure executed by the PC according to the second embodiment. It is a functional block diagram showing operation of PC of the above-mentioned form.

Hereinafter, the present invention will be described in detail with reference to the drawings showing the embodiments.
(Embodiment 1)
FIG. 1 is a schematic diagram illustrating a configuration example of a document management system. In the present embodiment, as an example of a document to be managed, a test sheet (entry sheet) in which a respondent (student) manually writes an answer to a question in a school test is assumed, and a sheet image obtained by scanning the test sheet is assumed. A document management system that performs masking and then saves the document will be described. The document management system includes an information processing device 1, a server 2, and a scanner 3. The information processing device 1 and the server 2 are communicatively connected via a network N such as the Internet.

  The information processing device 1 is an information processing device capable of performing various types of information processing and information transmission / reception, and is, for example, a personal computer, a multifunction peripheral, a multifunction terminal, or the like. In the present embodiment, it is assumed that the information processing apparatus 1 is a personal computer, and in the following description, it is replaced with PC1 for simplicity. The PC 1 according to the present embodiment is a terminal device operated by a school teacher, and is connected to a scanner 3 that optically reads a paper medium. The PC 1 uploads the image of the test paper scanned by the scanner 3 to the server 2 and stores it. In this case, the PC 1 performs masking on a content area in which content such as text, a picture, and a photograph, which is printed in advance on a test sheet, and which may correspond to a literary work, is masked and masked. Upload the image.

  The server 2 is a so-called cloud server, and stores an image of a test sheet uploaded from the PC 1 in a database. Although the detailed description is omitted because it is irrelevant to the present embodiment, the server 2 performs the scoring, totaling, and analysis of the answers written on the test sheet, notifies the analysis result to the student and the teacher, and notifies each student. Provide suitable teaching materials.

  The storage location of the image is not limited to the server 2 on the cloud, and the PC 1 may store the image locally, for example.

  In the following description, it is assumed that the PC 1, which is the local terminal, performs the masking process, but the server 2, which is the upload destination of the image, may perform the masking process.

  When the test paper is scanned and the image is stored as described above, if the test paper contains a copyrighted work, if the scanned image is stored as it is, it may correspond to an illegal copy of the copyrighted work. In order to avoid such a situation, it is conceivable to mask a portion that may be a copyrighted work.

  On the other hand, for example, when a respondent fills in an answer with a question sentence over, there is a possibility that the answer cannot be determined by masking depending on the position where the answer is written (see FIGS. 4 and 5). Therefore, in the present embodiment, the PC 1 identifies a region in which a handwritten answer (object) is written from an image, and excludes the identified region from a target of masking, thereby preventing a situation in which the answer cannot be determined.

FIG. 2 is a block diagram illustrating a configuration example of the PC 1. The PC 1 includes a control unit 11, a main storage unit 12, a communication unit 13, a display unit 14, an input unit 15, and an auxiliary storage unit 16.
The control unit 11 has one or more arithmetic processing units such as a CPU (Central Processing Unit) and an MPU (Micro-Processing Unit), and reads and executes the program P stored in the auxiliary storage unit 16, Various information processing, control processing, and the like related to the PC 1 are performed. The main storage unit 12 is a temporary storage area such as a RAM, and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module, and transmits and receives information to and from the outside. The display unit 14 is a display device such as a liquid crystal display and an organic EL (Electro Luminescence) display, and displays an image given from the control unit 11. The input unit 15 is an operation interface such as a keyboard and a mouse, and inputs operation contents to the control unit 11. The auxiliary storage unit 16 is a non-volatile storage area such as a hard disk or a large-capacity memory, and stores a program P and other data necessary for the control unit 11 to execute a process.

  FIG. 3 is an explanatory diagram relating to the process of specifying the content area. FIG. 3 conceptually illustrates how a region where the content is printed is specified from an image obtained by scanning a test sheet. Hereinafter, an outline of the present embodiment will be described.

  The PC 1 is a paper image obtained by scanning a test sheet (answer sheet) on which a respondent writes a handwritten answer (handwritten object) from the scanner 3 and on which a content that may be a copyrighted work is printed. To get. The test sheet is an entry sheet for writing an answer to the question, and, for example, as shown in FIG. 3, a question sentence, a question content, an answer column, and the like are printed with characters, symbols, and the like. The print contents (contents) of the test paper may include, for example, figures, photographs, pictures, and the like in addition to characters and symbols, and the contents are not particularly limited.

  The respondent writes the answer by hand on a test sheet using characters or the like. The contents to be entered may be strokes, symbols, figures, and the like in addition to the characters, and may be any object that is entered by hand.

  The PC 1 specifies a portion corresponding to a question sentence, a question content, an answer column, and the like, that is, a content, from the paper image, and specifies a content region that is a peripheral region including the content. In FIG. 3, a hatched portion corresponds to a content area. For example, the PC 1 specifies a content area from a sheet image of a test sheet for which no answer has been entered. The PC 1 specifies a content area that may correspond to a literary work, and masks the area.

  Various methods are conceivable for specifying the content area. FIG. 3 conceptually illustrates three methods as examples.

  As a first method, the PC 1 performs character recognition on a paper image to extract a text printed on a test paper, specifies the extracted text as content, and specifies an area surrounding the text as a content area. . The upper right part of FIG. 3 illustrates how character recognition is performed to specify a content area. The PC 1 performs character recognition on the paper image and extracts individual texts printed on the test paper. The PC 1 specifies each text as a content, and sets (specifies) a rectangular area indicated by hatching as a content area so that each text is covered.

  As a second method, the PC 1 specifies a content area according to the distribution of pixel values (pixel density) in a paper image. In the center on the right side of FIG. 3, a state in which a content area is specified based on a distribution of pixel values is illustrated. The PC 1 refers to the pixel value of each pixel in the paper image, determines the distribution state of the pixel value, and determines a portion different from the background of the test paper, such as characters, symbols, strokes, etc., printed on the test paper. To identify. For example, when focusing on a certain pixel in the paper image, the PC 1 counts the number of pixels in the pixel and its surrounding pixels (for example, 3 × 3 pixels) whose difference in RGB value from the background color is equal to or larger than a threshold value. When the number of counted pixels is equal to or larger than a predetermined number (for example, three pixels), the PC 1 specifies a portion including the pixel as the content.

  The PC 1 sets (specifies) a content area so as to cover the specified portion. This makes it possible to mask even symbols and strokes that are difficult to identify by character recognition.

  Note that the pixel value referred to by the PC1 may be a value related to color such as an RGB value or a value related to brightness such as luminance.

  As a third method, the PC 1 specifies a content from a paper image using a classifier (learned model) constructed in advance by machine learning such as deep learning. The lower side of FIG. 3 illustrates a state in which a paper image is input to the classifier and the result of the identification of the content area is obtained as an output. For example, the PC 1 is a classifier related to a CNN (Convolution Neural Network) or the like, and based on teacher data of a paper image to which a correct answer value (correct coordinate range) of a content area is labeled, the PC 1 A classifier with learned features is used. The method of machine learning is not limited to CNN, but may be another neural network, an SVM (Support Vector Machine), a Bayesian network, a decision tree, or the like. The PC 1 inputs the paper image to the discriminator, extracts the image feature amount, and obtains a result of discriminating the content as an output value. The PC 1 sets (specifies) a content area according to the identification result output from the identifier.

  The PC 1 specifies the content area by using any of the methods exemplified above. Alternatively, the PC 1 may specify a content area by combining a plurality of methods.

  FIG. 4 is an explanatory diagram relating to a process for specifying a handwritten entry area. FIG. 4 conceptually illustrates a state in which an answer (object) written by hand is extracted from a sheet image of a test sheet on which an answer has been written, and a handwritten writing area surrounding the extracted answer is specified.

  In the example of the image shown in the upper part of FIG. 4, the answer entered by the respondent is out of the range where the answer should be originally entered, and is partially overlapped with the question sentence (content). As described above, if the content area is masked as it is in such a state, a part of the answer becomes invisible. Therefore, the PC 1 extracts the handwritten answer from the paper image and specifies a handwritten entry area to be excluded from the masking target so as to surround the extracted answer.

  For example, the PC 1 compares a paper image in which a response has been entered and a paper image in which a response has not been entered, and extracts a handwritten response by extracting the difference between the two. For example, the PC 1 compares the pixel value in the paper image for which the answer has been entered with the pixel value of the same pixel as the pixel value of the paper image for which the answer has not been entered. . PC1 compares the pixel values of each pixel and extracts the difference between them.

  The middle part of FIG. 4 illustrates an image example showing the extracted answer. As shown in FIG. 4, the PC 1 extracts an object corresponding to a handwritten answer. The PC 1 sets (specifies), for example, a rectangular frame area as a handwritten entry area so as to surround the object.

  FIG. 5 is an explanatory diagram relating to the mask processing. FIG. 5 illustrates a state in which masking is performed on the sheet image of the test sheet in which the answer has been entered.

  The PC 1 performs image processing, that is, mask processing, on the paper image in which the answer has been entered, so that the area excluding the handwritten entry area from the content area specified above cannot be visually recognized. The process is conceptually illustrated on the right side of FIG. On the right side of FIG. 5, a black rectangular frame indicates a content area to be masked, and a white rectangular frame indicates a handwritten entry area. The PC 1 compares the content area shown in the upper right part of FIG. 5 with the handwritten entry area shown in the middle part of FIG. 5 and specifies the area shown in the lower right part of FIG. 5 excluding the handwritten entry area from the content area. . As a result, as shown by the dotted rectangular frame in the lower right part of FIG. 5, the overlapping part of the content area and the handwritten entry area, that is, the part where the question sentence and the answer overlap are excluded from the masking target.

  As shown on the left side of FIG. 5, the PC 1 performs masking to make the above-identified area invisible and generates a mask image for storage. The PC 1 transmits the generated mask image to the server 2 and stores it on the database.

FIG. 6 is a flowchart illustrating an example of a processing procedure executed by the PC 1. The processing executed by the PC 1 will be described with reference to FIG.
The control unit 11 of the PC 1 obtains, from the scanner 3, a sheet image obtained by scanning a test sheet having preprinted content and an object (answer) entered by handwriting (step S11). The content is a text, a picture, a photograph, or the like that may correspond to a copyrighted work, but the specific content is not particularly limited.

  The control unit 11 specifies the content from the paper image (Step S12). Specifically, the control unit 11 specifies the content from the paper image for which no answer has been entered. For example, the control unit 11 may extract a text printed on a test sheet by character recognition of a sheet image, and specify the extracted text as a content. Further, for example, the control unit 11 may specify the content in accordance with the distribution (pixel density) of the pixel values in the paper image. Further, for example, the control unit 11 may specify the content by using a learned classifier using a sheet image in which an object is not filled in as teacher data of the content.

  The control unit 11 extracts an object (answer) written on the test paper by hand from the paper image (step S13). Specifically, the control unit 11 compares the sheet image of the test sheet on which the answer has been entered with the sheet image of the test sheet on which the answer has not been entered, and extracts characters and the like written by hand. The control unit 11 specifies a handwritten entry area so as to surround the extracted object (step S14).

  The control unit 11 specifies a masking area on the sheet image, excluding the handwritten area specified in step S14, from the content area that is the peripheral area including the content specified in step S12, so that the specified masking area cannot be visually recognized. (Step S15). The control unit 11 transmits the generated mask image to the server 2 (Step S16), and ends a series of processing.

  In the above description, a test sheet has been described as an example of an entry sheet to be masked. However, a questionnaire sheet for answering a predetermined question (question) may be used.

  Further, in the above, the content area is made invisible by performing masking. However, the present embodiment is not limited to this. For example, the content area may be cut out (removed) from the paper image. You may. Even in this case, the portion corresponding to the content (work) can be appropriately processed, and the same effect as above can be obtained. As described above, the PC 1 only needs to be able to generate a paper image in which the content area is invisible, and the image processing method is not limited to masking.

  Further, in the above description, the content is specified from the sheet image in which the answer is not entered. However, the present embodiment is not limited to this, and the content may be identified from the sheet image in which the answer is entered. For example, the PC 1 may specify the content printed on the test paper by extracting characters, symbols, graphics, and the like that appear in each image from the paper images of a large number of respondents. Further, for example, the PC 1 may specify the content from the paper image with reference to a database storing image data predetermined as the content corresponding to the literary work. As described above, the PC 1 only needs to be able to specify the content from the sheet image obtained by scanning the test sheet, and the presence or absence of the entry of the answer is not particularly limited.

  In the above description, a handwritten answer (object) is extracted by taking a difference between an unfilled paper image and a filled paper image. However, the present embodiment is not limited to this. For example, the PC 1 may use a trained model (classifier) that has learned the characteristics of human handwritten characters to evaluate the likelihood of handwriting of characters included in the paper image and extract handwritten characters (objects). Good. As described above, the PC 1 only needs to be able to extract the object written by hand from the sheet image, and the method is not limited to the comparison between the unfilled sheet image and the filled sheet image.

  As described above, according to the first embodiment, masking can be performed so that objects such as characters written by handwriting are not covered, and document management can be appropriately performed.

  Further, according to the first embodiment, a handwritten entry area can be appropriately specified by comparing a filled-in sheet image with an unfilled sheet image.

  Further, according to the first embodiment, the content area can be appropriately specified by character recognition of the paper image.

  Further, according to the first embodiment, the content area can be appropriately specified by determining the distribution (pixel density) of the pixel values in the paper image.

  Also, according to the first embodiment, the content area can be specified with higher accuracy by using the classifier that has learned the characteristics of the content in the paper image.

(Embodiment 2)
In the present embodiment, an embodiment will be described in which a content area to be masked is specified according to the text content recognized from a test sheet. Note that the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.
FIG. 7 is an explanatory diagram relating to a process of specifying a content area according to the second embodiment. An outline of the present embodiment will be described with reference to FIG.

  The upper part of FIG. 7 shows an example of a sheet image (test sheet). In the present embodiment, the PC 1 performs character recognition on a paper image to extract a text printed on a test paper, and determines a content area to be masked according to the content of the extracted text.

  For example, the PC 1 specifies the question sentence from the text of the test sheet, and excludes the area where the question sentence is printed from the content area to be masked. The question sentence is a sentence instructing the respondent to answer, and it is unlikely that the question sentence itself contains a copyrighted work. Therefore, the PC 1 excludes the question sentence from the masking target, thereby preventing an unnecessary portion from being masked.

  Specifically, the PC 1 recognizes a specific keyword representing a question from the text of the test sheet, and specifies a sentence following the keyword as a question sentence. The keyword is, for example, a character string representing a question number. In FIG. 7, “fourth question” and “question 1” correspond to the question number. For example, the PC 1 refers to a table storing a template (rule) of a character string representing a question number, and specifies a keyword representing the question number from text on a test sheet. The PC 1 specifies the specified keyword (question number) and the text following the keyword as a question sentence. The PC 1 excludes the area where the question text is printed from the content area so that the question text is not masked.

  In the above description, the question sentence is specified based on the rule base. However, the PC 1 may directly specify (recognize) the question sentence from the text of the test sheet using a technique such as semantic analysis. For example, the PC 1 performs a syntax analysis on the text of the test paper, and specifies a syntax of a predetermined expression used in a question sentence such as a command sentence or a request sentence. For example, in the test sheet of FIG. 7, the PC 1 specifies a command sentence such as “answer the question” or “select the correct one”. The PC 1 specifies a series of sentences including the specified command sentence as a question sentence and excludes the sentence from the content area.

  As described above, the PC 1 specifies the question sentence from the text extracted by the character recognition, and excludes the area where the question sentence is printed from the content area. As a result, a portion having a low possibility of being a literary work is excluded from masking targets, and only a portion having a high possibility of being a literary work is masked.

  In the above, a technique is employed in which an area that is likely to be a copyrighted work is indirectly identified by identifying an area to be excluded from the masking target. On the other hand, the PC 1 may directly specify an area of the content that is highly likely to be a copyrighted work according to the content of the text on the test sheet.

  For example, the PC 1 specifies a sentence of a predetermined expression indicating the position where the content (work) is printed from the text, and specifies a content area to be masked according to the content of the sentence. The sentence is a sentence that suggests a location where the content is described, and is a sentence that can determine the question format, for example, whether or not it is a reading comprehension question. To explain using the example of FIG. 7, for example, the PC 1 determines (identifies) a sentence “read the next sentence A / B” that appears at the beginning of the question sentence following “fourth question”, and Judge that it is a reading problem. In this case, the PC 1 specifies a paragraph sentence next to the sentence as the content, and sets (specifies) a content area so as to cover the sentence. As described above, the PC 1 specifies the position where the content is described according to the content of the text extracted from the paper image. For example, the PC 1 may store a sentence of an expression suggesting the position of the content in a table, determine the presence or absence of the corresponding sentence while referring to the table, and specify the content area.

  Further, for example, the PC 1 may directly specify the content area by specifying a keyword of a sentence corresponding to the content (work) from the text extracted from the paper image. For example, the PC 1 stores in advance a keyword of a sentence corresponding to a copyrighted work in a database, searches (specifies) a sentence including the keyword from the text, and specifies only the sentence including the keyword as a content.

  As described above, the PC 1 identifies the area to be masked and the area not to be masked from the text extracted by the character recognition, and specifies the content area to be masked. In FIG. 7, the specified content area is indicated by hatching. The PC 1 performs a mask process on the content area as in the first embodiment, and generates a mask image. As described above, masking is performed only on a portion that is likely to include a copyrighted work, and masking of an unnecessary portion is prevented.

FIG. 8 is a flowchart illustrating an example of a processing procedure executed by the PC 1 according to the second embodiment.
After acquiring the paper image obtained by scanning the test paper (step S11), the control unit 11 of the PC 1 executes the following processing. The control unit 11 performs character recognition on the sheet image and extracts a text printed on the test sheet (step S201). The control unit 11 specifies the content according to the extracted text (Step S202). For example, the control unit 11 specifies a specific keyword representing a question from the text extracted in step S201, specifies a sentence including the keyword as a question sentence, and excludes the sentence from the content. For example, the control unit 11 performs a semantic analysis on the text extracted in step S201, specifies a syntax such as a command sentence or a request sentence, specifies a sentence including the syntax as a question sentence, and excludes the sentence from the content. Further, for example, the control unit 11 specifies a text of a predetermined expression indicating a position where the content is printed from the text extracted in step S201, and specifies a content to be masked according to the content of the text. Further, for example, the control unit 11 specifies a keyword of a text corresponding to the content from the text extracted in step S201, and specifies a text including the keyword as the content. The control unit 11 shifts the processing to step S13.

  As described above, according to the second embodiment, the question sentence is specified from the text printed on the test sheet and is excluded from the content area, so that the portion having a low possibility of including the content is excluded from the masking target. Can be.

  Further, according to the second embodiment, by estimating (identifying) the content area from the text printed on the test paper, it is possible to perform masking only on a portion having a high possibility of including the content.

(Embodiment 3)
FIG. 9 is a functional block diagram showing the operation of the PC 1 of the above-described embodiment. When the control unit 11 executes the program P, the PC 1 operates as follows.
The acquisition unit 91 acquires a sheet image obtained by scanning an entry sheet having preprinted content and a handwritten object entered by handwriting. The content specifying unit 92 specifies the content from the paper image. The extracting unit 93 extracts the handwritten object from the paper image. The generation unit 94 specifies a masking area excluding the handwritten object from a content area which is a peripheral area including the content, and generates the paper image subjected to image processing so that the masking area cannot be visually recognized.

  The third embodiment is as described above, and the other portions are the same as those in the first and second embodiments. Corresponding portions are denoted by the same reference numerals, and detailed description thereof will be omitted.

  The embodiment disclosed this time is an example in all respects and should be considered as not being restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 PC (information processing device)
Reference Signs List 11 control unit 12 main storage unit 13 communication unit 14 display unit 15 input unit 16 auxiliary storage unit P program 2 server 3 scanner

Claims (11)

An acquisition unit configured to acquire a paper image obtained by scanning an entry sheet having pre-printed content and a handwritten object filled in by hand,
A content identification unit that identifies the content from the paper image;
An extracting unit that extracts the handwritten object from the paper image;
A generation unit configured to specify a masking area excluding the handwritten object from a content area that is a peripheral area including the content, and generate the paper image that has been subjected to image processing so that the masking area cannot be visually recognized. Information processing device. An entry specifying unit that identifies a handwritten entry area surrounding the extracted handwritten object,
The information processing apparatus according to claim 1, wherein the generation unit specifies a masking area obtained by removing the handwritten entry area from the content area. The said extraction part extracts the said handwritten object based on the said paper image in which the said handwritten object was filled in, and the said paper image in which the said handwritten object was not filled in. The Claims 1 or 2 characterized by the above-mentioned. Information processing device. The information processing apparatus according to claim 1, wherein the content specifying unit specifies the content in accordance with a text extracted by character recognition on the paper image. The content specifying unit includes:
A specific keyword is specified from the extracted text,
The information processing device according to claim 4, wherein a sentence including the keyword is excluded from the content. The content specifying unit includes:
A specific keyword is specified from the extracted text,
The information processing apparatus according to claim 4, wherein only the text including the keyword is used as the content. The information processing device according to any one of claims 4 to 6, wherein the content specifying unit specifies a position where the content is printed according to the content of the extracted text. The information processing device according to any one of claims 1 to 7, wherein the content specifying unit specifies the content from a pixel density in the paper image or predetermined image data. The content specifying unit specifies the content from the paper image acquired by the acquisition unit using a plurality of paper images on which the handwritten object is not entered as learning data of the content using a learned classifier. The information processing apparatus according to claim 1, wherein the information processing apparatus includes: Obtain a paper image by scanning an entry paper having pre-printed content and a handwritten object filled in by hand,
From the paper image, identify the content,
Extracting the handwritten object from the paper image,
Specifying a masking area excluding the handwritten object from a content area that is a peripheral area including the content, and generating a paper image that has been subjected to image processing so that the masking area cannot be visually recognized. Information processing method. Obtain a paper image by scanning an entry paper having pre-printed content and a handwritten object filled in by hand,
From the paper image, identify the content,
Extracting the handwritten object from the paper image,
Specifying a masking area excluding the handwritten object from a content area that is a peripheral area including the content, and generating a paper image that has been subjected to image processing so that the masking area cannot be visually recognized. Program to do.

Images