JP2022070553A - Information processing device, control method and program - Google Patents

Abstract

To provide an information processing device, a control method and a program which easily can check or correct misrecognition in grading.SOLUTION: An information processing device comprises an acquisition unit, an identification unit, and a display unit. The acquisition unit acquires solution information showing a solution entered in an answer, and grading information showing grading marks representing marks for grading to the solution. The identification unit identifies classification to which each of the grading marks belongs. The display unit lists and displays a solution image showing the solution and the grading mark to the solution, and an identification result of the grading mark identified by the identification unit side by side as an identification result confirmation screen, based on the solution information and the grading information.SELECTED DRAWING: Figure 4

Description

The present invention relates to assistive techniques for scoring.

Traditionally, techniques have been known to assist students in scoring their test answers. For example, Patent Document 1 discloses a scoring system that acquires handwriting information entered by a student in an answer column of an answer sheet, displays it on the screen of a terminal used by a teacher, and accepts input of correct / incorrect judgment. Further, Patent Document 2 discloses a technique of comparing an answer written with an electronic pen with a correct answer and determining whether the answer is correct or incorrect.

Japanese Unexamined Patent Publication No. 2014-134854 Japanese Unexamined Patent Publication No. 2009-3227

Scoring support is provided to automatically calculate the score by scanning the answer sheet with a scoring mark (scoring mark) indicating the correctness, etc. written by the grader by hand and identifying the scoring mark. In this case, a classifier used for identifying the scoring mark can be obtained by performing machine learning with a large amount of learning data including a handwritten pattern of the scoring mark by a plurality of persons and the correct answer thereof. However, in the classifier learned using the learning data of the handwriting pattern by an unspecified number of people, erroneous recognition may occur due to the difference in handwriting habits, ink blurring, etc., so that it is final. Has the problem that it requires manual confirmation and requires a great deal of labor.

The present disclosure has been made to solve the above-mentioned problems, and mainly provides an information processing device, a control method, and a program capable of easily confirming and correcting misrecognition in scoring. It is an issue.

From one aspect of the present invention, the information processing apparatus includes an acquisition unit that acquires answer information representing an answer for each question written in an answer and scoring information representing a scoring mark that is a mark for scoring the answer. , An identification unit that identifies the classification to which each of the scoring marks belongs based on the degree of similarity to each classification, an answer image showing the answer and the scoring mark for the answer based on the answer information and the scoring information, and the above. It has a display unit for displaying the identification results of the scoring marks identified by the identification unit side by side as an identification result confirmation screen. According to this aspect, the grader can easily visually confirm whether or not the scoring mark is erroneously recognized.

In one aspect of the information processing apparatus, the display unit displays a list of the answer image corresponding to the question and the identification result side by side for each question. According to this aspect, the grader can easily confirm whether or not the scoring mark has a false recognition for a plurality of questions constituting the answer by looking at the list display.

In one aspect of the information processing apparatus, the acquisition unit acquires a plurality of answer information by a plurality of answerers and a plurality of the scoring information, and the display unit acquires the plurality of answerers. The answer image corresponding to a specific question and the identification result are displayed side by side in a list. According to this aspect, the information processing apparatus can display, for example, the answer images of all the class members corresponding to a specific question and the identification results side by side in a list. Therefore, the grader can easily confirm whether or not the scoring mark is erroneously recognized for a specific question by a plurality of answerers by looking at the list display.

In one aspect of the information processing apparatus, the display unit further displays a modification screen when the identification result modification request is acquired on the identification result confirmation screen, and the information processing apparatus displays the modification screen. Further has a correction unit for correcting the identification result based on the input information in. According to this aspect, when the grader confirms the erroneous recognition of the scoring mark from the list displayed on the identification result confirmation screen, the erroneous recognition can be easily corrected by the correction screen further displayed.

In one aspect of the information processing apparatus, the classification includes a correct answer mark indicating a correct answer, an incorrect answer mark indicating an incorrect answer, and an intermediate mark which is between the correct answer and the incorrect answer and is given a partial point. The correction request can be obtained for each answer, and the display unit displays a number for designating the partial point on the correction screen. According to this aspect, the grader can easily correct the erroneous recognition of the partial points by the numbers displayed on the correction screen.

In one aspect of the information processing apparatus, the display unit displays only numbers smaller than the points assigned to the question as numbers for designating the partial points. According to this aspect, the information processing apparatus can prevent a mistake such as the grader designating a partial point equal to or more than the score.

In one aspect of the information processing apparatus, the display unit displays a number smaller than the score assigned to the question as a number for designating the partial point in a selectable state, and selects a number equal to or higher than the score assigned to the question. Display in an impossible state. According to this aspect, the information processing apparatus cannot select a number equal to or higher than the score, so that it is possible to prevent a mistake such as designating a partial score equal to or higher than the score.

In one aspect of the information processing apparatus, the display unit also displays an answer image, which is an image of the answer with the scoring mark, on the identification result confirmation screen, and the identification unit displays the scoring mark. When the correction request is obtained for the answer for which the answer could not be identified, a predetermined range including the answer is highlighted in the answer image, and when a scoring mark outside the range is specified, the scoring mark and the corresponding scoring mark are specified. The identification result of the scoring mark by the identification unit is displayed as the identification result corresponding to the answer in association with the answer. According to this aspect, the information processing apparatus, for example, writes a scoring mark at a position away from the answer, and even if the scoring mark cannot be identified, the scoring mark can be easily specified by the grader. The scoring mark can be associated with the answer.

In another aspect of the information processing apparatus, the identification unit is learned by using a plurality of combinations of an image of a scoring mark handwritten by a plurality of persons and correct answer data relating to the classification of the scoring mark as learning data. Based on the classifier, the class is identified to which each of the scoring marks belongs, and the classifier identifies the image of the scoring mark corresponding to the correction request and the identification corrected by the correction unit as the correct answer data. The result and the learning data are learned. According to this aspect, the information processing apparatus can perform highly accurate identification by the classifier reflecting the correction contents.

Another aspect of the present invention is a control method executed by the information processing apparatus, which is answer information indicating an answer for each question entered in an answer and scoring information indicating a scoring mark which is a mark for scoring the answer. And, based on the degree of similarity with each classification, the classification to which each of the scoring marks belongs is identified, and based on the answer information and the scoring information, the answer and the answer image showing the scoring mark for the answer, and The identification results of the scoring marks are listed side by side as an identification result confirmation screen. By executing this control method, the information processing apparatus can display an identification result confirmation screen on which the grader can easily confirm whether or not the scoring mark is erroneously recognized.

From yet another viewpoint of the present invention, the degree of similarity between each classification and the acquisition unit that acquires the answer information representing the answer written in the answer and the scoring information representing the scoring mark that is the mark for scoring the answer. Based on the identification unit that identifies the classification to which each of the scoring marks belongs, the answer image showing the answer and the scoring mark for the answer based on the answer information and the scoring information, and the identification unit identified by the identification unit. The computer functions as a display unit that displays a list of the identification results of the scoring marks side by side as an identification result confirmation screen. By installing this program on a computer and executing it, the information processing apparatus according to the present invention can be configured.

According to the information processing apparatus according to the present invention, it is possible to easily confirm and correct the erroneous identification of the scoring mark.

The schematic configuration of the scoring system is shown. This is an example of a marked answer sheet. It is a block diagram which shows the functional structure of an information processing apparatus. It is a flowchart which shows the outline of the display process of the identification result confirmation screen. It is a figure which shows the result of giving a label to a part of the scoring marks written in the marked answer sheet shown in FIG. It is a figure which shows the scoring mark and the partial point attached to the scoring mark. This is a display example of the identification result confirmation screen. This is a display example of the correction screen in the correction of erroneous recognition. This is a configuration example of the scoring system according to the modified example.

Hereinafter, embodiments suitable for carrying out the present invention will be described with reference to the drawings.

[Scoring system configuration]
FIG. 1 is a schematic configuration diagram of a scoring system according to the present embodiment. The scoring system according to the present embodiment relates to an information processing device 1 used by a test grader such as a teacher, a scanner (optical device) 2, and correctness by the grader for each answer column of the answered answer sheet. It has a marked answer sheet 6 with a mark (also referred to as a “scoring mark”). In this embodiment, as an example, the scoring mark is given a correct answer mark indicating that the answer is correct, an incorrect answer mark indicating that the answer is incorrect, and a partial score between the correct answer and the incorrect answer. It shall be classified as an intermediate mark indicating that it can be used. Then, the scoring system accurately identifies the scoring mark written on the marked answer sheet 6 to automatically calculate the score of the answerer.

The information processing apparatus 1 is electrically connected to the scanner 2 by wire or wirelessly, and when the scanner 2 performs a reading process on the marked answer sheet 6, an image obtained by scanning the marked answer sheet 6 (“marked answer sheet 6”). Image Im ”) is received from the scanner 2. Then, the information processing apparatus 1 extracts the scoring mark from the marked answer image Im, and identifies the classification of the extracted scoring mark based on the degree of similarity with each classification of the scoring mark. Then, the information processing apparatus 1 displays a screen (also referred to as an "identification result confirmation screen") showing the identification result of the scoring mark.

FIG. 2 is an example of the marked answer sheet 6. As shown in FIG. 2, the marked answer sheet 6 is provided with a column for entering the student number and name of the student taking the test and an answer column for entering the answer to each question in addition to the question. .. Further, in each answer column of the marked answer sheet 6, the answer of the answerer to the corresponding question is written, and a scoring mark by the grader is attached.

Although FIG. 2 shows an example of an answer sheet for arithmetic, the marked answer sheet 6 according to the present embodiment may be an answer sheet for entering answers of any subject other than arithmetic. Further, the marked answer sheet 6 may be composed of a plurality of sheets.

[Information processing device configuration]
FIG. 3 is a block diagram showing a functional configuration of the information processing apparatus 1. The information processing apparatus 1 functionally includes a display unit 11, an input unit 12, a storage unit 13, an interface 14, and a control unit 15. Each of these elements is interconnected via a bus line 10.

The display unit 11 displays various information such as an identification result confirmation screen based on the control of the control unit 15. The display unit 11 may be, for example, a display or a projector, or may be an interface for connecting to these devices. The input unit 12 generates information to be input to the information processing device 1. The input unit 12 may be, for example, a voice input device such as a mouse, a button, a touch panel, a camera, or a microphone, or may be an interface for connecting to these devices. The interface 14 is an interface for connecting to the scanner 2 by wire or wirelessly, and supplies the marked answer image Im received from the scanner 2 to the control unit 15.

The storage unit 13 is composed of various memories such as a RAM, a ROM, and a readable / writable non-volatile memory. The storage unit 13 stores a program executed by the control unit 15. Further, the storage unit 13 functions as a working memory when the control unit 15 executes various processes.

Further, the storage unit 13 stores the classifier information 16 which is information about the scoring mark and the classifier for discriminating the numbers attached to the scoring marks. Here, the number attached to the scoring mark is, for example, a number representing a partial score (also referred to as a "partial score") given as a score at the time of the intermediate mark. The above-mentioned classifier is a learning model trained to output an identification result regarding classification of the scoring mark and identification of the partial point when an image showing a scoring mark or a partial point is input. This learning model may be any learning model used in machine learning such as a support vector machine or a neural network. The classifier information 16 is information obtained by learning the above-mentioned learning model. When the above-mentioned learning model is a neural network such as a convolutional neural network, the discriminator information 16 may include, for example, a layer structure, a neuron structure of each layer, a number of filters and a filter size in each layer, and a weight of each element of each filter. Contains various parameter information.

Here, the learning of the classifier will be supplementarily described. The classifier is learned using, for example, learning data that is a combination of an image of a handwritten scoring mark by a plurality of persons and correct answer data indicating the correct classification of the scoring mark. This training data contains a sufficient number of the above combinations for each classification that the classifier should identify. Further, when there are a plurality of scoring marks having the same classification, the discriminator is learned using the learning data including the combination of the scoring marks of each assumed mode and the correct answer data. For example, if there is a person who describes "x" and a person who describes "v" (check mark) as incorrect answer marks, learning data including a combination of the scoring marks of each aspect and the correct answer data is used. To learn the classifier. The learning of the classifier may be performed by the information processing device 1 or may be performed by a device other than the information processing device 1.

Further, the storage unit 13 may store information necessary for calculating the score of the answerer of the marked answer sheet 6 scanned as the marked answer image Im. For example, the storage unit 13 may store the score distribution information of each question to which the scoring mark is attached and the information indicating the arrangement in the answer sheet of each question for each test to be performed. Further, the storage unit 13 may store the calculation rule of the partial points in the intermediate mark as rule information. The rules for calculating partial points can be arbitrarily set for each question, such as "a predetermined ratio to the points allocated", "carrying up after the decimal point", and "uniformly 2 points".

Further, the storage unit 13 stores information necessary for extracting the scoring mark, such as the color information of the scoring mark written on the marked answer sheet 6.

The storage unit 13 may be an external storage device such as a hard disk connected to or built in the information processing device 1, or may be a storage medium such as a flash memory. In another example, the storage unit 13 may be a server device that performs data communication with the information processing device 1. In this case, the storage unit 13 may be composed of a plurality of server devices.

The control unit 15 is composed of a processor (computer) such as a CPU and a GPU, and controls each component of the information processing device 1. The control unit 15 performs a predetermined process based on the program stored in the storage unit 13. The control unit 15 is an example of an “acquisition unit”, an “identification unit”, a “display unit”, a “correction unit”, and a “computer” that executes a program.

In the present embodiment, when the marked answer image Im is received from the scanner 2, the control unit 15 identifies each classification of the scoring marks included in the marked answer image Im based on the classifier information 16. A label indicating the identified classification is attached to each scoring mark. Further, the control unit 15 identifies a partial point attached to the scoring mark to which the label of the intermediate mark is added, and associates it with the intermediate mark. Then, the control unit 15 generates and identifies the display information of the identification result confirmation screen in which the image for each question cut out from the marked answer image Im and the identification result of the scoring mark included in the image are arranged in a list. The result confirmation screen is displayed on the display unit 11.

[Display processing of identification result confirmation screen]
Next, the processing related to the display of the identification result confirmation screen will be described.

(1) Overview
FIG. 4 is a flowchart showing an outline of the display process of the identification result confirmation screen. The information processing apparatus 1 repeatedly executes the processing of the flowchart shown in FIG.

First, the information processing apparatus 1 determines whether or not the marked answer image Im is acquired from the scanner 2 (step S11). Then, when the information processing apparatus 1 determines that the marked answer image Im is acquired from the scanner 2 (step S11; Yes), the information processing apparatus 1 acquires it based on the information indicating the arrangement of each question stored in the storage unit 13. An image for each question (hereinafter, also referred to as a “question image”) is cut out from the marked answer image Im (step S12). The question image includes, for example, an answer column, an answer written by the answerer by hand, a scoring mark written by the grader by hand, and the like, and the question number is associated with the question image. In addition, the question number is expressed as "1-1" for (1) of large question 1 and "1-2" for (2) of large question 1.

On the other hand, when the information processing apparatus 1 has not acquired the marked answer image Im from the scanner 2 (step S11; Yes), the information processing apparatus 1 continues to determine whether or not the marked answer image Im has been acquired from the scanner 2. The information processing apparatus 1 is related to a request for displaying the identification result confirmation screen after acquiring the marked answer image Im from the scanner 2 instead of proceeding to step S12 immediately after acquiring the marked answer image Im from the scanner 2. When the user input is detected, the process may proceed to step S12. Further, the information processing apparatus 1 extracts a scoring mark from the acquired marked answer image Im (step S13). This extraction method will be described later.

Next, the information processing apparatus 1 identifies each classification of the scoring marks extracted in step S13, and assigns a label indicating the identified classification to each scoring mark (step S14). In this case, the information processing apparatus 1 acquires a label indicating the classification of the scoring mark by inputting an image of the scoring mark into the classifier configured based on the classifier information 16, and the acquired label is the target scoring mark. Correspond to. In this case, the label is information indicating, for example, a correct answer mark, an incorrect answer mark, or an intermediate mark to which a partial point is given.

Next, the information processing apparatus 1 determines whether or not there is an entry of a partial point attached to the scoring mark to which the label indicating the intermediate mark is attached (step S15). When the partial points are entered (step S15; Yes), the information processing apparatus 1 identifies the partial points and associates them with the target scoring mark as metadata (step S16). A specific example of the method of specifying the partial point will be described later. On the other hand, when the partial points are not entered (step S15; No), the information processing apparatus 1 proceeds to step S17.

Then, the information processing apparatus 1 displays an identification result confirmation screen including a list in which the question image cut out in step S12 and the identification result of the scoring mark are arranged for each question (step S17). Here, the identification result of the scoring mark is specifically a correct answer mark “◯”, an incorrect answer mark “×”, an intermediate mark “Δ”, a specified partial point, or the like. This display example will be described later with reference to FIG. 7.

(2) Extraction of scoring marks
Here, a process of extracting a scoring mark from the marked answer image Im, which is executed in step S13 of FIG. 4, will be described.

In the first example, the information processing apparatus 1 extracts a scoring mark based on the color information (pixel value) possessed by each pixel of the marked answer image Im. For example, in this case, the information processing apparatus 1 stores information on the color of the pen (for example, red) used by the grader to check the score mark in advance, and extracts pixels having color information similar to the stored color. Then, among the extracted pixels, adjacent pixels are recognized as pixels forming the same scoring mark. Then, the information processing apparatus 1 cuts out a rectangular area including the pixel group for each pixel group forming the same scoring mark, and generates a mask image of the scoring mark (an image displaying only the scoring mark) from the cut image. ..

In the second example, the information processing apparatus 1 stores in advance a scanned image of the answered answer sheet before the scoring mark is checked by the grader, and the scanned image and the marked answer obtained in step S11 are stored in advance. The difference value of the pixel value is calculated for each corresponding pixel of the image Im. Then, the information processing apparatus 1 recognizes the adjacent pixels among the pixels having the difference value as the pixels forming the same scoring mark. After that, the information processing apparatus 1 cuts out a rectangular area including the pixel group for each pixel group forming the same scoring mark, and generates a mask image of the scoring mark from the cut image.

(3) Labeling
Next, a specific example of label assignment executed in step S14 of FIG. 4 will be described.

FIG. 5 is a diagram showing the result of assigning a label to a part of the scoring marks written on the marked answer sheet 6 shown in FIG. In FIG. 5, among the scoring marks entered on the marked answer sheet 6, the scoring marks 21 to 25 to which the label indicating the correct answer mark “○” is attached are attached to the upper row, and the label indicating the intermediate mark “△” is attached. The scoring marks 31 and 32 are shown in the middle row, and the scoring mark 41 with a label indicating the incorrect answer mark "x" is shown in the lower row. Further, in FIG. 5, in order to make it easy to understand the correspondence with the marked answer sheet 6 shown in FIG. 2, a question number with the scoring mark is shown for each scoring mark.

When the information processing apparatus 1 acquires the marked answer image Im shown in FIG. 2 from the scanner 2, all the marked answer images Im including each scoring mark shown in FIG. 5 are based on step S13 of FIG. Extract the scoring mark. Then, the information processing apparatus 1 inputs the extracted image of each scoring mark (mask image) into the classifier configured based on the classifier information 16. In this case, the information processing apparatus 1 preferably normalizes the image of the scoring mark to be input to the classifier by enlarging / reducing the image so that the vertical and horizontal sizes become uniform, and the scoring after normalization. Enter the image of the mark into the classifier. Then, the information processing apparatus 1 assigns a label to the scoring mark input to the classifier based on the discrimination result output by the classifier. When the discriminator is a learning model based on a neural network, the output of the discriminator includes information on the degree of certainty indicating the certainty of the output discrimination result.

Further, the information processing apparatus 1 responds to the scoring mark and the question from the position of each scoring mark on the marked answer image Im based on the information indicating the position of the answer column for each question in the marked answer sheet 6. recognize. The question number of the question is associated with the scoring mark as metadata. The information indicating the position of the answer column for each question is stored in advance in, for example, the storage unit 13 or the like as information indicating the arrangement of each question.

(4) Identification of partial points
Next, specific specific examples of the partial points executed in steps S15 and S16 of FIG. 4 will be described.

FIG. 6 is a diagram showing a scoring mark 32 to which a label indicating an intermediate mark is attached and partial points 45 to 48 attached to the scoring mark 32. The method of writing partial points varies depending on the grader, but it is written in numbers around the triangle, which is an intermediate mark. Specifically, as shown in FIG. 6A, the partial point 45 is inside the scoring mark 32, the partial point 46 is on the outer upper right of the scoring mark 32, and the partial point 47 is on the outer upper left of the scoring mark 32. The partial point 48 is written in the lower outer part of the scoring mark 32. That is, the partial points are within a predetermined range from the scoring mark 32, and as shown in FIG. 6B in detail, from the center of gravity 51 of the triangle represented by the scoring mark 32 to the inside of the broken line frame 53 which is a circle having a predetermined radius 52. It is considered to be filled out. Here, the triangle is a triangle as a handwritten symbol, and is not a figure strictly surrounded by three straight lines. The center of gravity of the triangle is an approximate point where the line connecting each vertex and the midpoint of each opposite side intersects in the triangle as a handwritten symbol.

According to this, when the information processing apparatus 1 assigns a label indicating an intermediate mark, it identifies a target scoring mark, and from the marked answer image Im, within a range of a circle having a predetermined radius from the center of gravity of the scoring mark. Cut out a group of pixels. The information processing apparatus 1 generates a mask image of partial points from the clipped image. Further, the information processing apparatus 1 inputs the mask image of the generated partial point to the classifier configured based on the classifier information 16. Then, the information processing apparatus 1 identifies the number indicated by the partial point input to the classifier based on the discrimination result output by the classifier.

Since the partial points are not always entered when the intermediate mark is entered, if the mask image of the partial points cannot be generated or the number indicated by the partial points cannot be specified, the information processing apparatus 1 uses the partial points. It is judged that there is no entry.

(5) Display example
FIG. 7 is a display example of the identification result confirmation screen. In FIG. 7, the information processing apparatus 1 displays the marked answer image Im on the identification result confirmation screen, and displays the question image cut out from the marked answer image Im and the identification result for each extracted scoring mark for each question. It is listed side by side in.

Specifically, the information processing apparatus 1 displays a list of the question image and the identification result side by side for each question number associated with the question image and the identification result. As the identification result, the identification result for each extracted scoring mark is clearly indicated by "○", "Δ", and "×". If the scoring mark cannot be extracted or identified, the information processing apparatus 1 displays the identification result as "-". As described above, on the identification result confirmation screen, the question image cut out from the marked answer image Im and the identification result of the scoring mark included in the question image are listed side by side for each question number. As a result, the grader can easily visually confirm the erroneous recognition of the scoring mark.

Further, when the partial point is specified in the intermediate mark, the information processing apparatus 1 displays the partial point together with "Δ" as the identification result. As a result, the grader can confirm not only the scoring mark but also the identification result of the entered partial points. When the partial points are not specified in the intermediate mark, the information processing apparatus 1 calculates the partial points based on the rule information stored in the storage unit 13 for the points assigned to the questions with the intermediate mark. , Displayed together with the identification result "△". According to this, since the information processing apparatus 1 automatically calculates the partial points, the total score etc. can be accurately calculated even if the grader does not grasp the partial point calculation rule in the intermediate mark for each question. Is possible.

Further, the information processing apparatus 1 displays the score of the answerer (64 points in this case) in the total score column 61 on the identification result confirmation screen based on the score information of the target test and the identification result of the scoring mark. There is. The score of the answerer is not limited to the total score of the entire test, but the total score of each section may be calculated and displayed. For example, the information processing apparatus 1 displays the total score of (1) to (5) of the large question 1 in the section score column 62 and the total score of (1) and (2) of the large question 2 on the identification result confirmation screen. It is displayed in the section score column 63.

In this case, first, the information processing apparatus 1 acquires the score information of the target test from the storage unit 13 or the like based on the user input or the like that specifies the test corresponding to the marked answer image Im. The information processing apparatus 1 may identify the target test by using a known image recognition process (for example, template matching or the like) based on the arrangement of the answer column included in the marked answer image Im. Then, the information processing apparatus 1 calculates the score of the answerer based on the identification result of the scoring mark for each question and the score for each question. Here, the identification result of the scoring mark also includes the partial points attached to the intermediate mark. When the identification result of the scoring mark is changed, the information processing apparatus 1 recounts the points based on the changed identification result.

(6) Correction of misrecognition
FIG. 8 is a display example of the correction screen in the correction of erroneous recognition. When the misrecognition of the scoring mark is confirmed on the identification result confirmation screen, the grader can correct it. For example, on the identification result recognition screen shown in FIG. 7, if the identification result of question number 1-1 is the correct answer mark “○” but actually the intermediate mark “△”, the grader is the identification result item. Send a correction request by clicking or tapping 66. Upon receiving the correction request, the information processing apparatus 1 displays a pull-down menu 71 for correcting the identification result, as shown in FIG. 8A. The grader can easily correct the misrecognition of the identification result by designating the correct scoring mark.

When the grader selects the intermediate mark "Δ" in the pull-down menu 71, the information processing apparatus 1 further displays a pull-down menu 72 for designating a partial point. At this time, the pull-down menu 72 does not display or cannot select a number higher than the score of the target question. This makes it possible to prevent a mistake such as the grader selecting a number equal to or greater than the score when selecting a partial point attached to the intermediate mark. If you do not specify a partial point, for example, you may select the hyphen "-", or you may consider that the partial point is not specified by clicking or tapping any other place without selecting anything. ..

The method of correcting the misrecognition of the identification result is not limited to the method of displaying the pull-down menu as the correction screen as described above, and as shown in FIG. 8B, the software keyboard 73 is displayed as the correction screen. It may be that. In this case, when the grader clicks or taps the identification result item that is erroneously recognized, the information processing apparatus 1 displays the software keyboard 73. The grader can easily correct the misrecognition of the identification result by selecting the correct scoring mark. Further, the numbers on the software keyboard 73 select partial points, but the numbers above the points are inactive. Note that the information processing apparatus 1 may automatically designate an intermediate mark as the scoring mark when the grader selects a partial score.

When the erroneous recognition is corrected on the identification result confirmation screen, the information processing apparatus 1 can perform more accurate identification by reflecting the corrected content in the classifier. For example, the discriminator may learn by combining the image of the scoring mark corresponding to the correction request, that is, correcting the identification result, and the identification result corrected as correct answer data regarding the classification of the scoring mark. ..

(7) Association of scoring marks and questions outside the range
The information processing apparatus 1 recognizes the correspondence between the scoring mark and the question from the position of each scoring mark on the marked answer image Im based on the information indicating the position of the answer column for each question in the marked answer sheet 6. There is. Therefore, for example, as shown in FIG. 7, when the grader writes the scoring mark 26 at a position away from the answer column 67 of the question number 1-5, the information processing apparatus 1 performs the scoring corresponding to the question number 1-5. The scoring mark 26 cannot be extracted or identified as a mark. Therefore, on the identification result confirmation screen, "-" is displayed in the identification result item of question number 1-5. In such a case, the grader clicks or taps the identification result item 68. Then, the information processing apparatus 1 displays the broken line frame 69 in the answer column of question number 1-5 in the marked answer image Im displayed on the left side of the identification result confirmation screen. Then, when the grader designates the scoring mark 26 corresponding to the question number 1-5, the information processing apparatus 1 associates the question number 1-5 with the scoring mark 26. At this time, since the information processing apparatus 1 colors and emphasizes the broken line frame 69 and the scoring mark 26, the grader can easily recognize the associated question and the scoring mark. Then, the information processing apparatus 1 identifies the associated scoring mark 26, and reflects the identification result in the identification result item 68 of question number 1-5.

(8) Extraction of features for each grader
The information processing apparatus 1 acquires an N (that is, a feature vector) of N (N is an integer of 1 or more) dimension by applying an arbitrary feature amount extraction algorithm to the image of each scoring mark. This feature amount extraction algorithm may be, for example, any algorithm that extracts one or a plurality of feature amounts related to the shape. Further, the feature amount extraction algorithm may be a feature amount extractor obtained by learning a learning model of machine learning such as a neural network. In this case, the information processing apparatus 1 stores in advance the parameter information of the feature amount extractor obtained by learning in the storage unit 13. According to this, the information processing apparatus 1 can perform highly accurate identification based on the feature vector when identifying the scoring mark.

Further, the information processing apparatus 1 may extract the feature amount from the image of each scoring mark for each grader. In this case, it is assumed that the image of each grade mark is associated with the grader identification information for identifying the grader. For example, the information processing apparatus 1 also acquires the grader identification information when acquiring the marked answer image Im from the scanner 2 in step S11 or when acquiring the display request of the identification result confirmation screen, and metadata. It may be associated with the scoring mark. In addition, the grader identification information and the answer sheet to be graded are stored in advance in the storage unit 13, and the grader identification information is specified from the answer sheet on which the grade mark is entered, and the grader identification information is used as metadata in the score mark. It may be associated. According to this, the information processing apparatus 1 can extract the feature amount from the image of each grader for each grader based on the grader identification information. Further, the information processing apparatus 1 can create, for each grader, learning data which is a plurality of combinations of an image of a handwritten scoring mark and correct answer data regarding the classification of the scoring mark, that is, an identification result. By learning the feature amount extracted by the classifier and the created learning data, the information processing apparatus 1 can perform discrimination in consideration of the handwriting habits of each grader, and more accurate discrimination can be performed. It can be carried out.

Specifically, as described in the column of "(4) Identification of partial points", the method of writing the partial points attached to the intermediate mark varies depending on the grader, but if the same grader is used, the intermediate mark is used. In many cases, it is decided at which position of a certain triangle to write a partial point. For example, a grader who writes a partial point inside a triangle, which is an intermediate mark, has a low probability of writing a partial point outside the triangle. The information processing apparatus 1 acquires a feature vector for each grader by applying an arbitrary feature amount extraction algorithm to an image of a grade mark and a partial point associated therewith entered by the same grader. According to this, the information processing apparatus 1 performs identification based on the feature vector of each grader in consideration of the handwriting habit of each grader such as the position of the triangle which is the intermediate mark to write the partial point. This makes it possible to identify with high accuracy.

Further, in the above example, when specifying a partial point, a pixel group in a circle range of a predetermined radius is cut out from the center of gravity of the target scoring mark, and a mask image of the partial point is generated from the cut image. .. However, based on the feature vector for each grader, the information processing apparatus 1 can narrow the search and cut range of the pixel group such as "inside the target scoring mark", and can perform quicker identification. Become.

[Modification example]
Next, a modification suitable for the above-described embodiment will be described. The following modifications may be combined and applied to the above-described embodiment.

(Modification 1)
The information processing device 1 may function as a server device that transmits display data related to the identification result confirmation screen to the terminal.

FIG. 9 is a configuration example of a scoring system according to a modified example. In the example of FIG. 9, the information processing apparatus 1A is connected to the display terminal 7 used by the grader via the communication network 9. The display terminal 7 acquires the marked answer image Im from the scanner 2, and transmits the acquired marked answer image Im to the information processing device 1A.

When the information processing apparatus 1A receives the marked answer image Im from the display terminal 7, the information processing apparatus 1A executes the processing of the flowchart shown in FIG. In this case, the information processing apparatus 1A generates the display data "Dd" of the identification result confirmation screen in the display process of the identification result confirmation screen in step S19, and transmits the generated display data Dd to the display terminal 7. The identification result confirmation screen is displayed on the display terminal 7. As described above, in this modification, the information processing device 1A functions as a server device for transmitting the display data Dd of the identification result confirmation screen. Even in this case, the information processing apparatus 1A can preferably display the identification result confirmation screen on the display terminal 7 used by the grader, present the identification result of the scoring mark, and accept the correction.

(Modification 2)
In step S12 of FIG. 4, the information processing apparatus 1 asks questions from a plurality of marked answer sheet images Im obtained by scanning a plurality of marked answer sheets 6 (for example, for all the class members) with scoring marks by the same grader. The image may be cropped. Further, in step S13, the information processing apparatus 1 may extract scoring marks from the marked answer image Im. In this case, the information processing apparatus 1 performs processes in steps S12, S13, and subsequent steps on the question images cut out from the plurality of marked answer images Im and the extracted plurality of scoring marks. Even in this case, the information processing apparatus 1 displays the marked answer image Im on the identification result confirmation screen, the question image cut out from the marked answer image Im, and the identification result for each extracted scoring mark. Can be listed side by side for each question. Further, the information processing apparatus 1 does not display a list of question images and identification results for each question number for a plurality of questions constituting the marked answer image Im, but instead displays a question image for each answerer for one question number. It is possible to display a list of identification results. For example, the information processing apparatus 1 can display a list of question images and recognition results for all the class members for question number 1-1.

(Modification 3)
In the present embodiment, the answer sheet is a paper medium, the answerer writes the answer with a pencil or the like, and the grader writes the scoring mark with a red pen or the like. However, the present invention is not limited to this, and the answer may be displayed on a tablet or the like, the answerer may enter the answer with a stylus pen or the like, and the grader may also enter the scoring mark with the stylus pen or the like. .. Here, the tablet or the like is a plate-shaped computer terminal or hardware. The stylus pen or the like is a pen-type input device used in a tablet or the like.

In this case, it is not necessary for the scanner 2 to perform the reading process for the marked answer sheet 6 in step S11 of FIG. The information about the stroke of the scoring mark entered by the grader with a stylus pen or the like is acquired as the scoring stroke information. The stroke information includes the position information of the stroke on the entered tablet and the entry time. For example, the information processing apparatus 1 acquires answer stroke information and scoring stroke information from information about strokes written on a tablet or the like, based on the writing time.

Then, in step S12, the information processing apparatus 1 generates an image of the answer corresponding to each question based on the information indicating the position of the answer column for each question in the answer and the answer stroke information. Further, the information processing apparatus 1 generates an image of a scoring mark corresponding to each question based on the information indicating the position of the answer column for each question in the answer sheet and the scoring stroke information. Then, the information processing apparatus generates a question image by synthesizing the image of the answer corresponding to each question and the image of the scoring mark. Information indicating the position of the answer column is stored in advance in, for example, a storage unit 13 or the like. Further, in this case, unlike the case where the scanner 2 performs the reading process, the question image does not include the answer column.

Further, in step S14, the information processing apparatus 1 identifies each classification of the scoring mark based on the generated image of the scoring mark, and assigns a label indicating the identified classification to each scoring mark. In this case, the information processing apparatus 1 performs the processing after step S14 on the generated question image and the scoring mark.

1 ... Information processing device 2 ... Scanner 6 ... Marked answer sheet 11 ... Display unit 12 ... Input unit 13 ... Storage unit 14 ... Interface 15 ... Control unit

Claims (11)

An acquisition unit that acquires answer information that represents the answer for each question entered in the answer sheet and scoring information that represents the scoring mark that is a mark for scoring the answer.
An identification unit that identifies the classification to which each of the scoring marks belongs based on the degree of similarity with each classification.
Based on the answer information and the scoring information, the display unit that displays the answer image showing the answer and the scoring mark for the answer and the identification result of the scoring mark identified by the identification unit side by side as an identification result confirmation screen. When,
Information processing device with. The information processing apparatus according to claim 1, wherein the display unit displays a list of the answer image corresponding to the question and the identification result side by side for each question. The acquisition unit acquires a plurality of the answer information by a plurality of answerers and a plurality of the scoring information.
The information processing apparatus according to claim 1, wherein the display unit displays a list of the answer images corresponding to a specific question and the identification results of a plurality of answerers side by side. When the display unit acquires a correction request for the identification result on the identification result confirmation screen, the display unit further displays the correction screen.
The information processing device is
The information processing apparatus according to any one of claims 1 to 3, further comprising a correction unit that corrects the identification result based on the input information on the correction screen. The classification includes a correct answer mark indicating a correct answer, an incorrect answer mark indicating an incorrect answer, and an intermediate mark which is between the correct answer and the incorrect answer and is given a partial point.
The correction request can be obtained for each answer.
The information processing apparatus according to claim 4, wherein the display unit displays a number for designating the partial point on the correction screen. The information processing apparatus according to claim 5, wherein the display unit displays only numbers smaller than the points assigned to the question as numbers for designating the partial points. 5. The display unit displays a number smaller than the score assigned to the question as a number for designating the partial point in a selectable state, and displays a number equal to or higher than the score assigned to the question in a non-selectable state. The information processing device described in. The display unit is
On the identification result confirmation screen, an answer image, which is an image of the answer with the scoring mark, is also displayed.
When the correction request is obtained for the answer for which the scoring mark could not be identified by the identification unit, a predetermined range including the answer is highlighted in the answer image.
When a scoring mark outside the range is specified, the scoring mark is associated with the answer, and the identification result of the scoring mark by the identification unit is displayed as the identification result corresponding to the answer. The information processing apparatus according to any one of claims 4 to 7. The identification unit is
The classification to which each of the scoring marks belongs is identified based on the classifier learned by using a plurality of combinations of the image of the scoring mark handwritten by a plurality of people and the correct answer data related to the classification of the scoring mark as learning data. To do
The classifier 4 to 8 are characterized in that the image of the scoring mark corresponding to the correction request and the identification result corrected by the correction unit as the correct answer data are learned as the learning data. The information processing apparatus according to any one of the above. It is a control method executed by an information processing device.
Obtain the answer information indicating the answer for each question entered in the answer and the scoring information indicating the scoring mark which is a mark for scoring the answer.
Based on the degree of similarity with each classification, the classification to which each of the above scoring marks belongs is identified.
A control method in which an answer image showing the answer and a scoring mark for the answer and the identification result of the scoring mark are displayed side by side as an identification result confirmation screen based on the answer information and the scoring information. An acquisition unit that acquires answer information that represents the answer for each question entered in the answer sheet and scoring information that represents the scoring mark that is a mark for scoring the answer.
An identification unit that identifies the classification to which each of the scoring marks belongs based on the degree of similarity with each classification.
Based on the answer information and the scoring information, the display unit that displays the answer image showing the answer and the scoring mark for the answer and the identification result of the scoring mark identified by the identification unit side by side as an identification result confirmation screen. A program that makes a computer work as.

Images