JP2023070287A - Graphic correctness/incorrectness determination device, graphic correctness/incorrectness determination method, program, and recording medium - Google Patents

Abstract

To provide a graphic correctness/incorrectness determination device capable of improving the accuracy of figure recognition.SOLUTION: In a graphic correctness/incorrectness determination device 10 of the present invention, a graphic image acquisition unit 11 acquires a graphic image input for a question, a graphic image recognition unit 12 recognizes the obtained graphic image, and calculates the certainty of one or more recognized graphics and the recognition graphics to generate a certainty list, a correct answer information acquisition unit 13 acquires correct answer information of the question, a correctness/incorrectness determination unit 14 compares the recognition graphic with the correct answer information to generate match/mismatch information, and determines whether the graphic image is correct or incorrect based on the certainty list and the match/mismatch information to generate correctness/incorrectness information, and a correctness/incorrectness information output unit 15 outputs the correctness/incorrectness information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a graphic correct/incorrect judgment device, a graphic correct/incorrect judgment method, a program, and a recording medium.

Patent Literature 1 discloses a technique for recognizing handwritten characters using machine learning.

JP 2014-071813 A

However, there is a limit to improving the accuracy of character recognition due to technical progress in machine learning. In particular, there is a strong demand for improved accuracy in character recognition for quizzes, tests, questionnaires, and the like. This problem is not limited to letters, but is also a problem of graphic recognition in general including numbers, symbols, and the like.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a graphic correctness determination device, a graphic correctness determination method, a program, and a recording medium capable of improving the accuracy of graphic recognition.

In order to achieve the above object, the figure correct/incorrect judgment device of the present invention comprises:
including a graphic image acquisition unit, a graphic image recognition unit, a correct information acquisition unit, a correctness determination unit, and a correctness information output unit;
The graphic image acquisition unit acquires the graphic image input for the question,
The graphic image recognition unit recognizes the obtained graphic image, calculates the certainty of the recognized one or more recognized figures and the recognized figures, and generates a certainty list,
The correct information acquisition unit acquires correct information of the question,
The correct/wrong judging section compares the recognized figure with the correct information to generate match/mismatch information, and based on the certainty list and the match/mismatch information, judges whether the graphic image is correct/wrong and outputs the correct/wrong information. generate and
The correct/incorrect information output unit outputs the correct/incorrect information.
It is a device.

The figure correct/incorrect judgment method of the present invention includes:
including a graphic image acquisition process, a graphic recognition process, a correct information acquisition process, a correct/incorrect judgment process, and a correct/incorrect information output process;
The graphic image acquisition step acquires the graphic image input for the question,
The graphic image recognition step recognizes the obtained graphic image, calculates the certainty of the recognized one or more recognized figures and the recognized figure, and generates a certainty list,
The correct answer information acquisition step acquires correct answer information of the question,
In the correct/wrong decision step, the recognized figure is compared with the correct information to generate match/mismatch information, and based on the certainty list and the match/dismatch information, the correct/wrong decision of the graphic image is determined to generate the correct/wrong information. generate and
The correct/incorrect information output step outputs the correct/incorrect information.
The method.

According to the present invention, it is possible to improve the accuracy of figure recognition for a figure entered in an input format in which the character style or shape changes each time the character is written, such as by handwriting.

FIG. 1 is a block diagram showing an example of the configuration of the graphic correctness/incorrectness determination device of the first embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of the graphic correctness/incorrectness determination device of the first embodiment. FIG. 3 is a flow chart showing an example of processing in the graphic correct/wrong judging method of the first embodiment. FIG. 4(A) is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question, and FIG. 4(B) is generated from the graphic image shown in FIG. 4(A). FIG. 10 is a schematic diagram showing an example of a certainty list. FIG. 5(A) is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question, and FIG. 5(B) is generated from the graphic image shown in FIG. 5(A). FIG. 10 is a schematic diagram showing an example of a certainty list. FIG. 6(A) is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question, and FIG. 6(B) is generated from the graphic image shown in FIG. 6(A). FIG. 10 is a schematic diagram showing an example of a certainty list. FIG. 7(A) is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question, and FIG. 7(B) is generated from the graphic image shown in FIG. 7(A). FIG. 10 is a schematic diagram showing an example of a certainty list. FIG. 8(A) is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question, and FIG. 8(B) is generated from the graphic image shown in FIG. 8(A). FIG. 10 is a schematic diagram showing an example of a certainty list.

In the figure correct/incorrect judgment apparatus of the present invention, for example,
The correctness/incorrectness determination section may determine whether the graphic image is correct/incorrect based on the following criteria (1), (2), and (3), and generate correctness/incorrectness information.
(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition If the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .

For example, the figure correct/incorrect judgment device of the present invention is:
Furthermore, including a learning part,
The learning unit may calculate a correct answer rate of the output correct/wrong information, and adjust the correct threshold value and the incorrect answer threshold value in the correct/wrong judging unit based on the correct answer rate.

In the figure correct/incorrect judgment apparatus of the present invention, for example,
The graphic image recognition unit includes a trained model by machine learning,
The figure image may be recognized by the learned model, and the certainty factor of the recognized one or more recognized figures and the recognized figure may be calculated.

In the figure correct/incorrect judgment method of the present invention, for example,
The correctness/incorrectness determination step may be configured such that correctness/incorrectness information is generated by determining whether the graphic image is correct/incorrect based on the following criteria (1), (2), and (3).
(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition If the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .

For example, the figure correct/incorrect judgment method of the present invention is as follows.
Furthermore, including a learning process,
In the learning step, a correct answer rate of the output correct/wrong information may be calculated, and the correct threshold value and the incorrect answer threshold value in the correct/wrong determination step may be adjusted based on the correct answer rate.

In the figure correct/incorrect judgment method of the present invention, for example,
The graphic image recognition step includes a trained model by machine learning,
The figure image may be recognized by the learned model, and the certainty factor of the recognized one or more recognized figures and the recognized figure may be calculated.

The program of the present invention is a program for causing a computer to execute each step of the method of the present invention as a procedure.

A recording medium of the present invention is a computer-readable recording medium recording the program of the present invention.

The figures in the "graphic image" etc. of the present invention include letters, numerals, symbols, terms, words and the like.

For example, the present invention can be applied to subjects in which the answers (including answers or answers using options) to be asked of the respondent are known in advance, specifically questionnaires, tests, quizzes, learning drills, etc. . Furthermore, the present invention can be used as part of devices such as questionnaire devices, test devices, and quiz devices, methods corresponding to these devices, and programs.

Next, an embodiment of the present invention will be described with reference to the drawings. The invention is not limited to the following embodiments. In each figure below, the same reference numerals are given to the same parts. In addition, the description of each embodiment can be used with reference to each other's description unless otherwise specified, and the configurations of the respective embodiments can be combined unless otherwise specified.

[Embodiment 1]
FIG. 1 is a block diagram showing an example of the configuration of a graphic correct/incorrect judgment apparatus 10 of this embodiment. As shown in FIG. 1 , the device 10 includes a graphic image acquisition section 11 , a graphic recognition section 12 , a correct information acquisition section 13 , a correctness determination section 14 , and a correctness information output section 15 . Further, the device 10 may further include a learning unit 16 and the like as an arbitrary configuration. The units are interconnected, for example, by an internal bus.

The device 10 may be, for example, a single device including each of the above units, or may be a device in which each of the above units can be connected via a communication network. Further, the device 10 can be connected to an external device, which will be described later, via the communication network. The communication line network is not particularly limited, and a known network can be used, and may be wired or wireless, for example. The communication network includes, for example, the Internet line, WWW (World Wide Web), telephone line, LAN (Local Area Network), SAN (Storage Area Network), DTN (Delay Tolerant Networking), LPWA (Low Power Wide Area), L5G (local 5G), and the like. Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), local 5G, and LPWA. The wireless communication may be direct communication between devices (Ad Hoc communication), infrastructure communication, or indirect communication via an access point. The device 10 may be incorporated in a server as a system, for example. Further, the device 10 may be, for example, a personal computer (PC, for example, desktop type, notebook type), a smart phone, a tablet terminal, a wearable terminal, etc. in which the program of the present invention is installed. Furthermore, all or part of each unit of the device 10 may be realized on the cloud. Specifically, the device 10 is, for example, a form of cloud computing, edge computing, or the like in which at least one of the units is on a server (cloud) and the other units are on a terminal. may

FIG. 2 illustrates a block diagram of the hardware configuration of the device 10. As shown in FIG. The apparatus 10 may include, for example, a central processing unit (CPU, GPU, etc.) 101, a memory 102, a bus 103, a storage device 104, an input device 105, an output device 106, a communication device 107, and the like. Note that these are only examples, and the hardware configuration of the device 10 is not limited to this as long as the processing of each section can be executed. Also, the number of central processing units 101 and the like included in the apparatus 10 is not limited to the example shown in FIG. Each part of the hardware configuration of the apparatus 10 is interconnected via a bus 103 by each interface (I/F).

A central processing unit 101 is responsible for overall control of the apparatus 10 . In the apparatus 10, the central processing unit 101 executes, for example, the program of the present invention and other programs, and reads and writes various types of information. Then, the central processing unit 101 can execute the processing of each unit of the apparatus 10 .

Bus 103 can also be connected to external devices, for example. Examples of the external device include an external storage device (external database, etc.), an external input device, an external output device, and the like. For example, the device 10 can be connected to an external network (the above-mentioned communication line network) by means of a communication device 107 connected to the bus 103, and can also be connected to other devices via the external network.

The memory 102 is, for example, a main memory (main storage device). When the central processing unit 101 performs processing, for example, the memory 102 reads various operating programs such as the program of the present invention stored in the storage device 104 to be described later, and the central processing unit 101 reads from the memory 102 Get the data and run the program. The main memory is, for example, RAM (random access memory). Also, the memory 102 may be, for example, a ROM (read only memory).

The storage device 104 is also called a so-called auxiliary storage device, for example, in contrast to the main memory (main storage device). As described above, the storage device 104 stores operating programs including the program of the present invention. Storage device 104 may be, for example, a combination of a recording medium and a drive that reads from and writes to the recording medium. The recording medium is not particularly limited, and may be, for example, a built-in type or an external type, and includes HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, memory card, and the like. be done. The storage device 104 may be, for example, a hard disk drive (HDD) in which a recording medium and drive are integrated, and a solid state drive (SSD).

In the device 10, the memory 102 and the storage device 104 store log information, information acquired from an external database (not shown) or an external device, information generated by each process of the device 10, and It is also possible to store various information such as information used for execution. At least part of the information may be stored in an external server other than the memory 102 and the storage device 104, or may be distributed and stored in a plurality of terminals using blockchain technology or the like. .

The device 10 may, for example, further include an input device 105 and an output device 106 . The input device 105 is, for example, a device for inputting characters, numbers, positions of objects displayed on the screen, images, sounds, and the like. devices, microphones, sensors, and the like. Examples of the output device 106 include a display device (LED display, liquid crystal display), printer, speaker, and the like.

Next, an example of the graphic correct/wrong judging method of the present embodiment will be described with reference to the flow chart of FIG. The graphic correct/wrong judging method of the present embodiment is carried out as follows using, for example, the graphic correct/wrong judging apparatus 10 shown in FIG. The graphic correct/wrong judging method of this embodiment is not limited to the use of the graphic correct/wrong judging apparatus 10 of FIG.

In the following, the graphic image acquisition step can be executed by, for example, the graphic image acquisition unit 11, the graphic image recognition step can be executed, for example, by the graphic image recognition unit 12, and the correct information acquisition step can be executed by, for example, correct information For example, the correct/wrong determination step can be performed by the correct/wrong determination unit 14, the correct/wrong information output step can be performed by the correct/wrong information output unit 15, and the learning step can be performed by, for example, a learning unit 16.

First, the graphic image acquisition unit 11 acquires the graphic image input for the question (S11). The input is an "answer" or an "answer" to the question. The form of input (entry) for the question is, for example, an input form such as handwriting input or line-of-sight input, in which the character style or shape differs each time an entry is made. The questions may be printed on a printing medium such as recording paper, or may be displayed on a terminal having a touch display or the like. In the case of a question printed on a print medium, the graphic image acquisition section 11 may acquire, as the graphic image, an electronic version of a graphic written on the print medium. The digitization method is not particularly limited, and may be scanning or photographing with a camera. On the other hand, in the case of a question displayed on a terminal, the graphic image acquisition unit 11 may acquire, as the graphic image, a graphic input to the terminal by handwriting or the like. The terminal may be the device 10, for example.

Next, the graphic image recognition unit 12 recognizes the obtained graphic image, calculates the certainty factor of one or more recognized figures and the recognized figure, and generates a certainty factor list (S12). . An example of the certainty list will be described later. The graphic image recognition unit 12 may perform the recognition using, for example, machine learning including deep learning. Specifically, the graphic image recognition unit 12 may include, for example, a trained model by machine learning. Then, the graphic image recognition unit 12 may, for example, recognize the graphic image using the learned model, and calculate one or more recognized figures and the certainty of the recognized figure. The learned model is not particularly limited. The trained model is, for example, a model learned by associating attributes (e.g., age, gender, race, nationality, etc.) of a respondent (person who inputs a figure to the question) with handwriting. Alternatively, it may be a model learned by associating the respondent with the handwriting of each respondent. In this way, the accuracy of the recognition by the graphic image recognition section 12 is further improved by taking into consideration the attributes of the respondent and information specific to a specific individual in the recognition.

Next, the correct answer information acquisition unit 13 acquires the correct answer information of the question (S13). The correct answer information is information indicating the correct answer to the question. When the question is a question with options (for example, a questionnaire) and the options are "1", "2", "3" and "4", for example, the options "1", " At least one of "2", "3" and "4" is the correct answer. On the other hand, in the case of a question with no options, for example, all or part of what can be derived from the question can be the correct answer. More specifically, if the question asks about the sum of 2 and 2, the correct answer is "4". In this way, there may be a plurality of correct answers indicated by the correct answer information of the question. For example, the correct information acquisition unit 13 may acquire the correct information pre-stored in the memory 102 and the storage device 104, may acquire the correct information input from the outside, or may acquire the correct information input from the outside. The correct answer information may be acquired from the device.

Next, the correct/wrong judging section 14 compares the recognized figure with the correct information to generate match/mismatch information, and judges whether the graphic image is correct or wrong based on the certainty list and the match/mismatch information. Correctness information is generated (S14). For example, in the above-described case where the question is a question with options, when the figure image recognition unit 12 recognizes the figure image as "9", the option does not have "9", The correct/wrong judging section 14 generates match/mismatch information indicating "mismatch".

The generation of the correctness/incorrectness information by the correctness/incorrectness determination unit 14 will be described more specifically. The correctness/incorrectness determination unit 14 may determine whether the graphic image is correct/incorrect based on, for example, the following criteria (1), (2), and (3) to generate correctness/incorrectness information.

Judgment Criteria (1): Correct Judgment Conditions The graphic image is judged to be correct when either one of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

Judgment Criteria (2): Incorrect Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an incorrect answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

Judgment Criterion (3): Judgment Impossible Condition If the following condition (3a) is satisfied, the graphic image is judged to be undecidable.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .

Arbitrary degrees of certainty can be set for the correct answer threshold and the incorrect answer threshold. For example, the correct answer threshold may be set to a degree of certainty smaller than the incorrect answer threshold. Specifically, when the certainty is represented by a numerical value from 1 to 100, the correct threshold may be a numerical value less than 50, and the incorrect threshold may be a numerical value of 50 or more.

Then, the correct/incorrect information output unit 15 outputs the correct/incorrect information (S15). The said output is performed via the output device 106, for example.

If the device 10 further includes a learning unit 16, for example, after the step S15, the learning unit 16 calculates the accuracy rate of the output correct/incorrect information, and based on the correct accuracy rate, the correct/incorrect judgment unit 14 may be adjusted (S16). Note that the present invention is not limited to this, and the correct threshold value and the incorrect answer threshold value can be manually adjusted by the user, for example.

The step S16 is an optional step and may not be performed. In the graphic correct/wrong judging method of the present embodiment, for example, the process ends (END) after the step 15 or the step S16.

In conventional OCR (Optical character recognition) technology, there are many cases in which it is difficult to draw a line of judgment criteria even if deep learning is used. Specifically, for example, a character to be recognized that is recognized as a specific character (for example, "4") by a conventional OCR technology is different from the specific character (for example, "9") when viewed by a person. There are cases that look like Further, the judgment criteria differ depending on the operational status of the OCR technology, and different characters may be recognized depending on the operational status. On the other hand, according to this embodiment, for example, by using the certainty factor and the correct answer information, it is possible to improve the accuracy of figure recognition. Moreover, when the shape of a product is recognized as a figure (for example, a circle, a triangle, a square, etc.), the invention described in this embodiment can also be applied to quality control in a manufacturing line.

[Embodiment 2]
The present invention will be described more specifically by taking the case of recognizing handwritten digits as an example. In addition, these are examples, and the present invention is not limited to these.

FIG. 4A is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question. In FIG. 4(A), on the touch display of the device 10, which is a smartphone, a question asking the sum of 2 and 2 and an answer column 1 in which the answer to the question is input by handwriting are displayed. . In this example, it is assumed that "4" is set in advance in the device 10 as the correct answer information corresponding to the question. In this case, the graphic image acquisition section 11 acquires the graphic image entered in the answer column 1 . Next, the graphic image recognition unit 12 recognizes the graphic image and generates a certainty list. An example of the certainty list in this example is shown in FIG. 4(B). In this reliability list, the column of "recognized number (recognition figure)" indicates the recognition figure recognized by the graphic image recognition unit 12, and the column of "certainty (%)" indicates their reliability. In this example, the upper limit of the certainty factor is 100%, and the higher the certainty factor, the higher the probability that the recognized figure is a figure image. In addition, the aspect of the said certainty is not limited to this example. Hereinafter, in the present example, it is assumed that the correct threshold is preset at a certainty of 10%, and the incorrect answer threshold is preset at a certainty of 85%.

The processing by the correct/incorrect determination unit 14 will be described more specifically with reference to FIG. 4 as an example. In this example, as described above, "4" is the correct answer. Matching/mismatching information is generated indicating that the recognized figure matches the information and the other recognition figures do not match the correct information. Next, the correctness determination by the correctness determination unit 14 will be described. In this example, in the certainty list shown in FIG. 4B, the recognition figure "4" has the highest certainty and matches the correct information. Furthermore, the certainty (98%) of the recognition figure "4" is equal to or higher than the correct threshold (10%). Therefore, since the recognition figure "4" satisfies both the above-mentioned conditions (1a) and (1b) of the correctness judgment condition, which is the above-mentioned judgment criterion (1), the correctness/incorrectness judging section 14 The image is determined to be correct, and correct/incorrect information indicating the correct answer is generated.

FIG. 5A is a schematic diagram showing an example of a question and an answer (graphic image) input by handwriting to the question. FIG. 5(A) is the same as FIG. 4(A) except that the answer handwritten in the answer column 1 is different from the answer shown in FIG. 4(A). In this example, it is assumed that "4" is set in advance in the device 10 as the correct answer information corresponding to the question. In this case, in the same manner as described above, the graphic image acquisition section 11 acquires the graphic image entered in the answer column 1, and the graphic image recognition section 12 recognizes the graphic image to generate a certainty list. An example of the certainty list in this example is shown in FIG. 5(B). The certainty degree list shown in FIG. 5B has the same content as the certainty degree lists shown in FIG.

The processing by the correct/incorrect determination unit 14 will be described more specifically with reference to FIG. 5 as an example. In this example, as described above, "4" is the correct answer. Matching/mismatching information is generated indicating that the recognized figure matches the information and the other recognition figures do not match the correct information. Next, the correctness determination by the correctness determination unit 14 will be described. In this example, although the recognition figure "3" has the highest certainty in the certainty list shown in FIG. 5B, it does not match the correct information. Also, the certainty factor of the recognition figure "4" that matches the correct answer information is equal to or less than the correct answer threshold. Therefore, in this example, since neither of the conditions (1a) and (1b) of the correctness judgment condition, which is the judgment criterion (1), is satisfied, the correctness judgment unit 14 determines the graphic image input in the answer column 1 is not judged as correct. On the other hand, in this example, the certainty (2%) of the recognition figure "4" matching the correct information in the certainty list shown in FIG. The certainty of the recognition figure that does not match the information (for example, the recognition figure with the highest certainty "3": 98%) is equal to or higher than the wrong answer threshold (85%). Therefore, in this example, in order to satisfy the incorrect answer judgment criterion (2a), which is the judgment criterion (2), the correct/incorrect judging unit 14 judges that the graphic image input in the answer column 1 is an incorrect answer, and Generates true/false information indicating the answer.

FIG. 6A is a schematic diagram showing an example of a question and an answer (graphic image) input by handwriting to the question. FIG. 6(A) is the same as FIG. 4(A) except that the answer handwritten in the answer column 1 is different from the answer shown in FIG. 4(A). In this example, it is assumed that "4" is set in advance in the device 10 as the correct answer information corresponding to the question. In this case, in the same manner as described above, the graphic image acquisition section 11 acquires the graphic image entered in the answer column 1, and the graphic image recognition section 12 recognizes the graphic image to generate a certainty list. An example of the certainty list in this example is shown in FIG. 6(B). The certainty degree list shown in FIG. 6B has the same content as each certainty degree list shown in FIG.

The processing by the correctness determination unit 14 will be described more specifically with reference to FIG. 6 as an example. In this example, as described above, "4" is the correct answer. Matching/mismatching information is generated indicating that the recognized figure matches the information and the other recognition figures do not match the correct information. Next, the correctness determination by the correctness determination unit 14 will be described. In this example, in the certainty list shown in FIG. 1) does not satisfy the condition (1b) of the correctness determination condition. However, the recognition figure "4" has the highest certainty in the certainty list shown in FIG. 6(B) and matches the correct answer information. 1a) is satisfied. Therefore, the correct/wrong determination unit 14 determines that the graphic image input in the answer column 1 is correct, and generates correct/wrong information indicating the correct answer.

FIG. 7A is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question. FIG. 7(A) is the same as FIG. 4(A) except that the answer handwritten in the answer column 1 is different from the answer shown in FIG. 4(A). In this example, it is assumed that "4" is set in advance in the device 10 as the correct answer information corresponding to the question. In this case, in the same manner as described above, the graphic image acquisition section 11 acquires the graphic image entered in the answer column 1, and the graphic image recognition section 12 recognizes the graphic image to generate a certainty list. An example of the certainty list in this example is shown in FIG. 7(B). The certainty degree list shown in FIG. 7B has the same content as each certainty degree list shown in FIG. 4B except that the certainty degree for each recognition pattern is different.

Processing by the correct/incorrect determination unit 14 will be described more specifically with reference to FIG. 7 as an example. In this example, as described above, "4" is the correct answer. Matching/mismatching information is generated indicating that the recognized figure matches the information and the other recognition figures do not match the correct information. Next, the correctness determination by the correctness determination unit 14 will be described. In this example, in the certainty degree list shown in FIG. The above condition (1a) of the conditions is not satisfied. However, since the certainty (50%) of the recognition figure "4" that matches the correct information is equal to or higher than the correct answer threshold (10%), the condition (1b) of the correct answer judgment condition, which is the judgment criterion (1), is changed. Fulfill. Therefore, the correct/wrong determination unit 14 determines that the graphic image input in the answer column 1 is correct, and generates correct/wrong information indicating the correct answer.

FIG. 8A is a schematic diagram showing an example of a question and an answer (graphic image) handwritten input to the question. FIG. 8(A) is the same as FIG. 4(A) except that the answer handwritten in the answer column 1 is different from the answer shown in FIG. 4(A). In this example, it is assumed that "4" is set in advance in the device 10 as the correct answer information corresponding to the question. In this case, in the same manner as described above, the graphic image acquisition section 11 acquires the graphic image entered in the answer column 1, and the graphic image recognition section 12 recognizes the graphic image to generate a certainty list. An example of the certainty list in this example is shown in FIG. 8(B). The certainty degree list shown in FIG. 8B has the same content as each certainty degree list shown in FIG.

The processing by the correct/incorrect determination unit 14 will be described more specifically with reference to FIG. 8 as an example. In this example, as described above, "4" is the correct answer. Matching/mismatching information is generated indicating that the recognized figure matches the information and the other recognition figures do not match the correct information. Next, the correctness determination by the correctness determination unit 14 will be described. In this example, in the certainty list shown in FIG. The above condition (1a) of the conditions is not satisfied. In addition, since the certainty (1%) of the recognition figure "4" that matches the correct information is less than the correct answer threshold (10%), the condition (1b) of the correct answer judgment condition, which is the judgment criterion (1), is also not satisfied. Furthermore, the confidence (1%) of the recognition figure "4" that matches the correct information is less than the correct threshold (10%), and the confidence of the recognition figure that does not match the correct information (for example, the confidence is the highest) Since the high recognition figure "3": 9%) is less than the wrong answer threshold (85%), the wrong answer judgment criterion (2a), which is the judgment criterion (2), is not satisfied. On the other hand, the confidence (2%) of the recognition figure "4" that matches the correct information is less than the correct threshold (10%), and the confidence of the recognition figure that does not match the correct information (for example, the confidence is Since the highest recognition figure "3": 9%) is less than the wrong answer threshold value (85%), the condition (3a) of the undecidable condition, which is the criterion (3), is satisfied. Therefore, the correct/wrong determination unit 14 determines that the graphic image input in the answer column 1 cannot be determined. As in this example, when it is determined that the judgment is impossible, for example, correct/incorrect information indicating an incorrect answer may be generated, or correct/incorrect information indicating that the judgment is impossible may be generated. Further, when it is determined that the determination is impossible, the correctness/incorrectness determination unit 14 may acquire the correctness/incorrectness determination result of the graphic image visually observed by the user, and generate correctness/incorrectness information, for example.

[Embodiment 3]
The program of this embodiment is a program for causing a computer to execute each step of the method of the present invention as a procedure. In the present invention, "procedure" may be read as "processing". Moreover, the program of this embodiment may be recorded on a computer-readable recording medium, for example. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples thereof include read-only memory (ROM), hard disk (HD), optical disc, and the like.

Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes can be made to the configuration and details of the present invention within the scope of the present invention that can be understood by those skilled in the art.

<Appendix>
Some or all of the above-described embodiments can be described as the following notes, but are not limited to the following.
(Appendix 1)
including a graphic image acquisition unit, a graphic image recognition unit, a correct information acquisition unit, a correctness determination unit, and a correctness information output unit;
The graphic image acquisition unit acquires the graphic image input for the question,
The graphic image recognition unit recognizes the obtained graphic image, calculates the certainty of the recognized one or more recognized figures and the recognized figures, and generates a certainty list,
The correct information acquisition unit acquires correct information of the question,
The correct/wrong judging section compares the recognized figure with the correct information to generate match/mismatch information, and based on the certainty list and the match/mismatch information, judges whether the graphic image is correct/wrong and outputs the correct/wrong information. generate and
The correct/incorrect information output unit outputs the correct/incorrect information.
Graphic correctness judgment device.
(Appendix 2)
The correctness determination unit determines whether the graphic image is correct or not according to the following criteria (1), (2), and (3), and generates correctness information.
The figure correctness/incorrectness determination device according to Supplementary Note 1.

(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition If the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .

(Appendix 3)
Furthermore, including a learning part,
The learning unit calculates an accuracy rate of the output correct/incorrect information, and adjusts the correct threshold value and the incorrect answer threshold value in the correct/incorrect judgment unit based on the correct answer rate.
The figure correctness/incorrectness determination device according to appendix 2.
(Appendix 4)
The graphic image recognition unit includes a trained model by machine learning,
Recognize the graphic image by the trained model, and calculate one or more recognized figures and a certainty factor of the recognized figure;
3. A figure correct/incorrect judgment device according to any one of Appendices 1 to 3.
(Appendix 5)
including a graphic image acquisition process, a graphic recognition process, a correct information acquisition process, a correct/incorrect judgment process, and a correct/incorrect information output process;
The graphic image acquisition step acquires the graphic image input for the question,
The graphic image recognition step recognizes the obtained graphic image, calculates the certainty of the recognized one or more recognized figures and the recognized figure, and generates a certainty list,
The correct answer information acquisition step acquires correct answer information of the question,
In the correct/wrong decision step, the recognized figure is compared with the correct information to generate match/mismatch information, and based on the certainty list and the match/dismatch information, the correct/wrong decision of the graphic image is determined to generate the correct/wrong information. generate and
The correct/incorrect information output step outputs the correct/incorrect information.
Figure correct/wrong judgment method.
(Appendix 6)
In the correctness/incorrectness determination step, the correctness/incorrectness information is generated by determining whether the graphic image is correct/incorrect based on the following criteria (1), (2), and (3):
The graphic correct/wrong judgment method described in Supplementary Note 5.

(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition When the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .

(Appendix 7)
Furthermore, including a learning process,
The learning step calculates a correct answer rate of the output correct/wrong information, and adjusts the correct threshold value and the incorrect answer threshold value in the correct/wrong determination step based on the correct answer rate.
The figure correct/incorrect judgment method according to appendix 6.
(Appendix 8)
The graphic image recognition step includes a trained model by machine learning,
Recognize the graphic image by the trained model, and calculate one or more recognized figures and a certainty factor of the recognized figure;
8. A graphic correct/incorrect judgment method according to any one of Appendices 5 to 7.
(Appendix 9)
A program for causing a computer to execute a procedure including a graphic image acquisition procedure, a graphic recognition procedure, a correct information acquisition procedure, a correctness judgment procedure, and a correctness information output procedure;
The graphic image acquisition procedure acquires the graphic image input for the question,
The graphic image recognition procedure recognizes the obtained graphic image, calculates the certainty factor of one or more recognized figures and the recognized figure to generate a certainty factor list,
The correct information acquisition procedure acquires correct information of the question,
The correctness/incorrectness determination procedure compares the recognized figure with the correct information to generate match/mismatch information, and based on the certainty list and the match/dismatch information, determines whether the graphic image is correct or not, and outputs the correct/incorrect information. generate and
The correct/incorrect information output procedure outputs the correct/incorrect information.
(Appendix 10)
The correctness determination procedure determines whether the graphic image is correct or not according to the following criteria (1), (2), and (3), and generates correctness information.
The program according to Appendix 9.

(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition When the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .

(Appendix 11)
further comprising a learning procedure,
The learning procedure calculates the accuracy rate of the correct/incorrect information that is output, and adjusts the correct threshold value and the incorrect answer threshold value in the correct/incorrect judgment procedure based on the correct answer rate.
The program according to appendix 10.
(Appendix 12)
The graphic image recognition procedure includes a trained model by machine learning,
Recognize the graphic image by the trained model, and calculate one or more recognized figures and a certainty factor of the recognized figure;
12. A program according to any one of Appendixes 9-11.
(Appendix 13)
A computer-readable recording medium recording the program according to any one of Appendices 9 to 12.

According to the present invention, it is possible to improve the accuracy of figure recognition for a figure entered in an input format in which the character style or shape changes each time the character is written, such as by handwriting. For this reason, the present invention is particularly useful, for example, when recognizing a figure input to an answer for which the answer to be asked of the respondent is known in advance.

1 Answer column 10 Graphic correctness determination device 11 Graphic image acquisition unit 12 Graphic image recognition unit 13 Correct information acquisition unit 14 Correctness determination unit 15 Correction information output unit 16 Learning unit 101 Central processing unit 102 Memory 103 Bus 104 Storage device 105 Input device 106 Output device 107 Communication device

Claims (10)

including a graphic image acquisition unit, a graphic image recognition unit, a correct information acquisition unit, a correctness determination unit, and a correctness information output unit;
The graphic image acquisition unit acquires the graphic image input for the question,
The graphic image recognition unit recognizes the obtained graphic image, calculates the certainty of the recognized one or more recognized figures and the recognized figures, and generates a certainty list,
The correct information acquisition unit acquires correct information of the question,
The correct/wrong judging section compares the recognized figure with the correct information to generate match/mismatch information, and based on the certainty list and the match/mismatch information, judges whether the graphic image is correct/wrong and outputs the correct/wrong information. generate and
The correct/incorrect information output unit outputs the correct/incorrect information.
Graphic correctness judgment device. The correctness determination unit determines whether the graphic image is correct or not according to the following criteria (1), (2), and (3), and generates correctness information.
2. The figure correctness/incorrectness determination device according to claim 1.

(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition If the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .
Furthermore, including a learning part,
The learning unit calculates an accuracy rate of the output correct/incorrect information, and adjusts the correct threshold value and the incorrect answer threshold value in the correct/incorrect judgment unit based on the correct answer rate.
3. The figure correctness/incorrectness determination device according to claim 2. The graphic image recognition unit includes a trained model by machine learning,
Recognize the graphic image by the trained model, and calculate one or more recognized figures and a certainty factor of the recognized figure;
4. The graphic correct/incorrect judgment apparatus according to any one of claims 1 to 3. Including a graphic image acquisition process, a graphic recognition process, a correct information acquisition process, a correct/incorrect judgment process, and a correct/incorrect information output process,
The graphic image acquisition step acquires the graphic image input for the question,
The graphic image recognition step recognizes the obtained graphic image, calculates the certainty of the recognized one or more recognized figures and the recognized figure, and generates a certainty list,
The correct answer information acquisition step acquires correct answer information of the question,
In the correct/wrong decision step, the recognized figure is compared with the correct information to generate match/mismatch information, and based on the certainty list and the match/dismatch information, the correct/wrong decision of the graphic image is determined to generate the correct/wrong information. generate and
The correct/incorrect information output step outputs the correct/incorrect information.
Figure correct/wrong judgment method. In the correctness/incorrectness determination step, the correctness/incorrectness information is generated by determining whether the graphic image is correct/incorrect based on the following criteria (1), (2), and (3):
6. The graphic correct/wrong determination method according to claim 5.

(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition If the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .
Furthermore, including a learning process,
The learning step calculates a correct answer rate of the output correct/wrong information, and adjusts the correct threshold value and the incorrect answer threshold value in the correct/wrong determination step based on the correct answer rate.
7. The graphic correct/wrong judging method according to claim 6. The graphic image recognition step includes a trained model by machine learning,
Recognize the graphic image by the trained model, and calculate one or more recognized figures and a certainty factor of the recognized figure;
8. The graphic correct/wrong judgment method according to any one of claims 5 to 7. A program for causing a computer to execute a procedure including a graphic image acquisition procedure, a graphic recognition procedure, a correct information acquisition procedure, a correctness judgment procedure, and a correctness information output procedure;
The graphic image acquisition procedure acquires the graphic image input for the question,
The graphic image recognition procedure recognizes the obtained graphic image, calculates the certainty factor of one or more recognized figures and the recognized figure to generate a certainty factor list,
The correct information acquisition procedure acquires correct information of the question,
The correctness/incorrectness determination procedure compares the recognized figure with the correct information to generate match/mismatch information, and based on the certainty list and the match/dismatch information, determines whether the graphic image is correct or not, and outputs the correct/incorrect information. generate and
The correct/incorrect information output procedure outputs the correct/incorrect information. The correctness determination procedure determines whether the graphic image is correct or not according to the following criteria (1), (2), and (3), and generates correctness information.
10. A program according to claim 9.

(1) Correct Judgment Conditions The graphic image is judged to be correct if either of the following conditions (1a) and (1b) is satisfied.
(1a) The recognition figure with the highest degree of certainty in the degree of certainty list matches the correct information.
(1b) The degree of certainty of the recognition image that matches the correct information is equal to or greater than a correct answer threshold.

(2) Wrong Answer Judgment Criteria If the following condition (2a) is satisfied, the graphic image is judged as an erroneous answer.
(2a) In the certainty list, the certainty of the recognition figure that matches the correct information is less than a correct threshold, and the certainty of the recognition figure that does not match the correct information is equal to or greater than the wrong answer threshold.

(3) Judgment Impossibility Condition If the following condition (3a) is satisfied, it is judged that the graphic image cannot be judged.
(3a) In the certainty list, the certainty of the recognition image that matches the correct information is less than the correct threshold, and the certainty of the recognition figure that does not match the correct information is less than the wrong answer threshold. .

Images