JP2007286409A - Marking assist device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform semiautomatic marking of essay questions by using a computer instead of for a marker to mark them by reading them one by one by using a computer. <P>SOLUTION: In a marking assist device 1001, an answer data storage unit 1002 stores answer data 1006 of answers made by a plurality of learners and marking information 1007 given to the answer data. The marking information 1007 is not initially made to correspond to the answer data 1007, but the marking information 1007 is gradually made to correspond to the answer data 1007 as the marker marks the answers by using the present invention. A similarity decision unit 1003 decides similarities among answers. A marking plan generating unit 1004 gathers marking information on answer data similar to answer data being marked from the answer data storage unit 1002 by using the similarity decision unit 1003 to generate a marking plan. A marking information generation unit 1005 shows a result of the generated marking plan to the marker to let the marker confirm employment of the generated marking plan or make corrections, thus generating marking information for the answer data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a scoring support apparatus suitable for use when a computer and a human jointly score a collection of answer data of descriptive questions in education.

As a conventional technique related to a system that supports scoring of a descriptive problem using a computer system, for example, a technique described in Patent Document 1 is known. This prior art is a communication correction system that uses electronic forms to improve the work efficiency of the corrector, and overlays the electronically collected data of the student's answers with the correction format, review entry sheet, and scoring electronic form. Display and browse the answer sheet, correct the description problem, enter comments, etc. In the prior art, when scoring a descriptive question, the answers of a plurality of students are displayed side by side, corrected, and a comment model sentence is inserted according to the score.
JP 2003-66820 A

  The above-mentioned scoring support for the description problem by the prior art is only to display a plurality of answers side by side and to insert a comment model sentence according to the score. There is a problem in that it has been necessary to read the written answer and make a judgment one by one.

  The object of the present invention is to solve the above-mentioned problems of the prior art, and to allow the computer to score semi-automatically the scoring of the descriptive problem scored by the scorer while reading manually one by one. To provide a scoring support apparatus and support method.

  According to the present invention, the problem is a scoring support device that supports scoring of answer data for a question for which a description type answer is to be obtained. In the computer system, a similarity determination unit, a scoring plan generation unit, and scoring information creation And an answer data accumulating unit, the answer data accumulating unit accumulates a set of answer data for the question, and the scoring information after scoring is sequentially given to the answer data, and the similarity determination The section searches the answer data storage section for answer data with scoring information having answer data similar to the answer data of the problem being scored, and the similarity determination section searches for the scoring plan generation section. The similar answer data is collected to generate a scoring plan for the answer data of the problem being scored, and the scoring information generating unit generates the scoring plan generating unit when assigning scoring information to the answer data This is achieved by displaying the scoring plan and the answer data of the graded question to the grader, asking for the input of scoring information for the answer data, and attaching the entered scoring information to the answering data that is being graded .

  According to the present invention, it is possible to eliminate the need for the grader to score all the answers manually.

  Hereinafter, an embodiment of a scoring support apparatus will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a functional configuration of the scoring support apparatus. In FIG. 1, 1001 is a scoring support device, 1002 is an answer data storage unit, 1003 is a similarity determination unit, 1004 is a scoring plan generation unit, 1005 is a scoring information creation unit, 1006 is answer data, and 1007 is scoring information.

  The scoring support apparatus 1001 includes an answer data storage unit 1002, a similarity determination unit 1003, a scoring plan generation unit 1004, and a scoring information creation unit 1005. The answer data storage unit 1002 stores answer data 1006 which are answers of a plurality of learners and scoring information 1007 attached to the answer data. There are various methods for collecting the answer data. For example, the scoring support device 1001 is connected to a network, and a descriptive question is distributed to learners by another device connected to the network, and the answer is sent to the network. It is possible to use a method of collecting and storing the answer data in the answer data storage unit 1002 or a method of collecting the answer data manually and inputting it to the answer data storage unit 1002. Although the scoring information 1007 is not associated with the answer data 1006 at the beginning, the scoring information 1007 is gradually associated with the scoring person using the scoring support device according to the embodiment of the present invention.

  The data structure of the answer data 1006 and the scoring information 1007 in the answer data storage unit 1002 will be described later. First, each unit other than the answer data storage unit 1002 will be briefly described. The similarity determination unit 1003 determines the similarity between the answers. The scoring plan generation unit 1004 uses the similarity determination unit 1003 to collect scoring information of answer data similar to the answer data being scored from the answer data storage unit 1002 to generate a scoring plan. The scoring information creation unit 1005 displays the result of the generated scoring plan for the grader and creates scoring information for the answer data by confirming or correcting the adoption of the scoring plan.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer system that implements the scoring support apparatus 1001. In FIG. 2, 2001 is a computer system, 2002 is a CPU, 2003 is a storage device, 2004 is an auxiliary storage device, 2005 is an input device, and 2006 is an output device.

  A computer system 2001 that realizes the scoring support apparatus 1001 includes a CPU 2002 that performs various processing operations, a storage device 2003 using a semiconductor memory, an auxiliary storage device 2004 using a hard disk device, and the like, as is well known as a PC or the like. An input device 2005 such as a keyboard and a mouse and an output device 2005 such as a display, a printer, and a speaker are connected to each other. In the auxiliary storage device 2004, the similarity determination unit 1003, the scoring plan generation unit 1004, and the scoring information creation unit 1005 that are functional elements constituting the scoring support device 1001 shown in FIG. The storage unit 1002 is stored. The functions constituting the scoring support apparatus 1001 are realized by a program embodying each function being read into the storage device 2003 at the time of execution and executed by the CPU 2002, and the input device 2004 and the output device 2005 are used. Thus, the scoring support apparatus according to the embodiment of the present invention can be realized by interacting with the scorer. Moreover, the function which comprises the scoring assistance apparatus 1001 can also be implement | achieved directly using hardware.

  FIG. 3 is a block diagram showing a configuration of an education system in which the scoring support device 1001 is incorporated. 3, 3001 is a learner client, 3002 is a teacher client, and other reference numerals are the same as those in FIG.

  In the education system shown in FIG. 3, the scoring support apparatus 1001, the plurality of learner clients 3001, and the teacher client 3002 according to the embodiment of the present invention configured as described with reference to FIGS. 1 and 2 are connected to the network. Has been configured. Then, the teacher who is the grader distributes the descriptive question to the learner using the teacher client 3002, and the learner transmits the answer data 1006 from the learner client 3001 to the scoring support apparatus 1001. And stored in the answer data storage unit 1002. The scoring support device 1001 attaches scoring information to the answer data 1006 while interacting with the teacher.

  Next, description will be given of a description type question targeted by the scoring support apparatus 1001 according to the embodiment of the present invention, its answer data, scoring information, and an image of scoring returned to the learner.

FIG. 4 is a diagram for explaining an example of the description formula problem. The example of the description formula problem 4001 shown in FIG.
Question 1: Consists of a question part that raises the question, “List items that you think are effective to increase the processing speed of your PC.” Requests that the answer to question 1 be listed in a bulleted list.

  FIG. 5 is a diagram for explaining an example of answer data 1006 returned by the learner. As shown in FIG. 5, in the answer column 5002 of the answer data 1006, the learner writes the answer by text data in an itemized form.

  FIG. 6 is a diagram for explaining an example of the scoring information 1007 attached to the answer data 1006. As shown in FIG. 6, the scoring data 1007 includes table data 6002 having an answer data column 6003, an evaluation column 6004, and a comment column 6005. The answer data column 6003 is an item in which each line is an itemized answer data in the answer column 5002 answered by the learner shown in FIG. On the other hand, the column 6004 and the column 6005 are scoring information associated with each item of answer items. The evaluation column 6004 adds “correct answer” indicated by ○ and “incorrect answer” indicated by × to the answer item of each item, and the comment column 6005 indicates the answer of each item. Comment information is attached to the item. Further, the scoring data shown in FIG. 6 is provided with a data area 6006 for describing a comment for the entire scoring data. In the example shown in FIG. 6, comment data indicating an insufficient solution in the answer data is provided. is described.

  FIG. 7 is a diagram for explaining a display image of answer data with grading embedded in the answer data described in FIG. 6 and returned to the learner. As shown in FIG. 7, the display image 7001 of answer data is composed of a question part that lists a question and an answer column 7002 that shows a scoring result. The answer column 7002 is an individual item answer item for the question, a circle mark or a cross mark that indicates whether the answer is correct or incorrect, and learning that is indented for each answer. Comment to the person. In addition, items that the learner did not notice (other correct answer items) are pointed out after the answer items in those bullets.

  FIG. 10 is a flowchart for explaining the processing operation when the scoring information creation unit 1005 creates scoring information, which will be described next.

(1) First, while the unscored answer data 1006 exists in the answer data storage unit 1002, the scoring information creation unit 1005 repeats a series of processes from step 10002 and creates scoring information 1007 for each. It is stored in the answer data storage unit 1002 (step 10001).

(2) Then, the scoring information creation unit 1005 sets x as unscored answer data, requests the scoring plan generation unit 1003 for the answer data x, and generates a scoring plan for the unscored answer data x About the scoring plan,
CAND: Set of itemized scoring plans AUX: Set of candidates for additional items (steps 10002 and 10003).

(3) Next, the scoring information creation unit 1005 displays the data of CAND and AUX, presents it to the grader, requests its selection, correction, and additional input, and finally obtained from the grader The information is set as scoring information 1007 of unscored answer data x and stored in the answer data storage unit 1002 (steps 10004 and 10005).

  FIG. 9 is a diagram showing an example of a dialogue screen presented to the grader in the process of step 10004 in the flow of FIG. This screen example 9001 is an example in which CAND and AUX data is displayed to the grader.

  In FIG. 9, the data portion indicating CAND includes answer data 9002 indicating itemized items in bulleted items, similar scored answer 9007 which is an answer similar to the answer data 9002 collected from the scored score information 1007, and similar scoring. A correct answer for the completed answer 9007, an evaluation of incorrect answer 9003, a comment 9004 for the similar scored answer 9007, and an adoption 9005 indicating a check box as to whether or not to adopt scoring information as the similar scored answer 9007. In this screen example 9001, the grader checks the check box for “recruitment” 9005 to determine whether the scoring information can be adopted or not in the answer data 9002 that is the answer item for the itemized item displayed there. It can be attached to the similar scored answer 9007. In the rows of columns 9003 and 9004, there are completely blank rows where the grader can freely enter text. If the scorer wants to attach other scoring information instead of the scoring information displayed here, new scoring information can be created by inputting here. Further, in the screen example 9001 shown in FIG. 9, another answer 9006 is also displayed as data indicating AUX.

  FIG. 11 is a flowchart for explaining the processing operation when the scoring plan generating unit 1004 generates a scoring plan, which will be described next.

(1) First, the scoring plan generating unit 1004 sets the answer data to be processed as x and empties the data of the variable CAND (steps 11001 and 11002).

(2) Next, the scoring plan generation unit 1004 sets to repeat a series of processing from step 11004 on all itemized answer item items of the processing target answer data x (step 11003).

(3) Next, the scoring plan generation unit 1004 asks the similarity determination unit 1003 to search for items that are similar to the answer item item and have scoring information attached, and the searched item Similar to, and to which scoring information is attached is defined as SIMS (step 11004).

(4) The scoring plan generation unit 1004 next adds a pair of item and SIMS to the CAND. At this time, even if SIMS is empty, a pair of item and sky is created and added to CAND (step 11005).

(5) Next, the scoring plan generation unit 1004 empties the variable AUX, and repeats the subsequent processing for all item answer items item2 with correct scoring information 1007 in the answer data storage unit 1002. Is set (steps 11006 and 11007).

(6) Next, the scoring plan generation unit 1004 checks whether there are similar items in the itemized items of the processing target answer data x in all the itemized item item2 with the correct scoring information 1007. When this condition is satisfied, nothing is done. When the condition is not satisfied, item2 is a correct item not included in the currently scored answer data, and thus item2 is added to the AUX (steps 11008 and 11009).

(7) Finally, the scoring plan generation unit 1004 returns a set AUX of correct list item candidates that are not described in the CAND that is a set of scoring candidates for all item items in the answer data x and the answer data x. (Step 11010).

  FIG. 12 is a flowchart for explaining a process in which the similarity determination unit 1003 determines the similarity of the answer data of the scoring information that has already been scored with respect to the answer data. Next, this will be described.

(1) First, the similarity determination unit 1003 extracts a set of keywords registered in advance from the two sentences A and B as answer data. These sets are designated as KAs and KBs (step 12001).

(2) Next, the similarity determination unit 1003 calculates an average D of the distances between keywords included in the sets KAs and KBs. The average is, for example,
(Σdistance (a, b) a ∈KAs, b ∈KABs) / (| KAs | · | KBs |)
It can be calculated using the following formula. Here, distance (a, b) is the distance between the bullet item a and the bullet item b (step 12002).

(3) Finally, the similarity determination unit 1003 determines whether or not the average distance D is smaller than a preset threshold value. When this condition is satisfied, the similarity is determined to be similar. When not established, it is answered that they are not similar (steps 12003 to 12005).

  FIG. 8 is a diagram showing the relationship of similarity determination results between itemized answer items as links. FIG. 8 shows answer data 8001 to 8004 of a learner having individual item items surrounded by a dotted line. The individual item items in the answer data 8004 and the answer data 8001 to 8003 are shown in FIG. It shows the relationship with itemized items. In FIG. 8, the first item in the answer data 8004 is similar to the first item and the fourth item in the answer data 8002, and the first item in the answer data 8003. ing. The second item in the answer data 8004 is similar to the second item in the answer data 8001 and the third item in the answer data 8002. Thus, when the similarity of a sentence is investigated, generally a plurality of similarities are detected. Therefore, as shown in FIG. 9, when the scoring information creation unit 1005 inquires the scoring person about scoring information, a plurality of similar itemized items found are displayed.

  Although the above-described embodiment has been described as for a description type question that asks for an answer in an itemized form, if the answer type of the description type question is determined, it can be applied in the same manner as described above. The format is fixed, for example, not only in bulleted items, but also in hierarchical forms such as each bulleted item becoming a bulleted list, etc. There is a string of words to go.

  FIG. 13 is a flowchart for explaining another process in which the similarity determination unit 1003 determines the similarity of the answer data of the scoring information that has already been scored with respect to the answer data. In the similarity determination process described with reference to the flow shown in FIG. 12, the similarity determination unit 1003 has been described to be called for each answer data. However, the similarity determination unit 1003 is not called for each answer data, but is first set to 1 The similarity may be determined by calling it once and establishing a link between all similar item items. Next, the processing operation in this case will be described with reference to the flow shown in FIG.

(1) First, the similarity determination unit 1003 performs subsequent processing for all combinations of the item A of items A and the item B of items B for all sets A and B (A ≠ B) of answer data. It is set to repeat the processing (steps 13001 and 13002).

(2) After that, if the similarity determination unit 1003 can determine that the item A of item A and the item B of item B are similar by a method of calculating the average of keyword distances, a link is established between them. The process is repeated to link all similar itemized items (step 13003).

  Through the processing described above, the scoring plan generation unit 1004 can search for a desired similar item by checking only whether or not there is a link instead of calling the similarity determination unit 1003.

  FIG. 14 is a flowchart for explaining another processing operation when the scoring information creating unit 1005 creates scoring information, which will be described next. An example of the process described here is that when a similar item is first linked by the process described with reference to FIG. 13 and the grader determines the scoring information for the answer data. Is propagated along the link. The propagation is performed after the scoring information creation unit 1005 determines and writes one scoring information.

(1) The scoring information creation unit 1005 first sets the items to be repeated for each item item of the answer data A, and repeats the following items for the item item 2 linked to the item. It is set to repeat the process (steps 14001 and 14002).

(2) Next, the scoring information creation unit 1005 examines items2 that do not have scoring information, and when this condition is satisfied, performs a series of subsequent processes (step 14003).

(3) The scoring information creation unit 1005 adds item scoring information (correct answer, incorrect answer, and comment) to item 2 (step 14004).

(4) Next, the scoring information creation unit 1005 applies the processing here recursively to propagate the scoring information for the unscored item items linked to item 2 (step 14005).

  Through the above-described processing, the determined scoring information can be propagated to all the bulleted items linked by the link, and the scoring plan generator does not generate the scoring plan every time, but the scoring information is included in the answer data. It is possible to create scoring plan information of other unscored answer data that is linked at the time of being attached, and display the scoring plan data to the grader to promote scoring.

  The embodiment described above is a process for determining the similarity of the answer data of the already-scored scoring information with respect to the answer data of the flow described with reference to FIG. The information shown in FIG. 9 is returned to the scorer in the process of step 10004 of the scoring information creation unit described by the flow shown in FIG. When the CAND element is displayed on the screen, it is possible to display the CAND elements in descending order of similarity.

  Further, in the above-described embodiment, when the similarity is examined, even if they are similar as keywords, it is possible to prevent the meaning from being reversed due to the negative wording in one of them.

  FIG. 15 is a flowchart for explaining the processing operation of the similarity determination unit 1003 in consideration of the inclusion of a negative wording, which will be described next.

(1) First, the similarity determination unit 1003 checks whether or not the two sentences A and B are similar in the process 15001, using the method for obtaining the average distance between keywords described above. It is determined whether it is not (Step 15001, 15002).

(2) If it is determined in step 15002 that the conditions are not similar, if it is determined that they are not similar, it is determined that they are similar. It is determined whether or not only one of B includes negation (steps 15003 and 15004).

(3) If the condition is satisfied in the determination by the condition determination processing in step 15004, it is answered that they are not similar, and if not, it is answered that they are similar (steps 15005, 15006).

  In the embodiment described above, priority is given to the answer data, and it is possible to display them in order from the highest priority among items similar to the currently scored item. In this case, when the scoring information creation unit 1005 creates an interactive screen as shown in FIG. 9, the priority should be taken into consideration so that the higher the one, the higher the number displayed.

  According to the above-described embodiment, first, a scorer manually determines and attaches scoring information. However, when several scoring is finished, it is confirmed that similar answer data has already been scored. Since the scoring data is displayed as part of the answer data, it is not necessary for the scorer to manually score all of them, and after several scoring, the correct and incorrect items are already correct Since it is expected that the correct answer is judged and commented, they can be used.

  In addition, it is possible to automatically generate a draft of scoring information for the entire answer data as well as each item in the answer data. In addition, since all similarities can be determined first, it is not necessary for the scorer to manually score all, and it is not necessary to search for similar answer data at the time of scoring. It is possible to respond promptly.

  Further, it is possible to create scoring plan information of similar answer data immediately from the inputted scoring information.

  Also, it is possible to refer to the one with the highest degree of similarity, and avoid the risk of scoring wrongly when the meaning is completely reversed by negation even if the similarity degree calculation by keyword is high. Can do.

  Furthermore, since the priority of the answer data to be used can be specified, for example, the grader prepares the model solution and its scoring information in advance, and gives priority to the model solution among those similar to the answer data to be scored. It is possible to perform scoring according to the scorer's intention.

  The present invention can be used for education such as elementary and secondary education, higher education, cram school, correspondence course, and in-house education in a company.

The block diagram which shows the function structure of a scoring assistance apparatus. The block diagram which shows the example of the hardware constitutions of the computer system which implement | achieves a scoring assistance apparatus. The block diagram which shows the structure of the education system incorporating a scoring support apparatus. The figure explaining the example of a description type problem. The figure explaining the example of the answer data which a learner returns. The figure explaining the example of the scoring information attached | subjected to answer data. The figure explaining the display image of the answer data with a grading which embeds the scoring information demonstrated by FIG. 6 in answer data, and returns to a learner. The figure which shows the relationship of the result of the similarity determination between the answer items of a list item with a link. The figure which shows the example of the dialogue screen shown to a grader by the process of step 10004 in the flow of FIG. The flowchart explaining the processing operation at the time of a scoring information creation part creating scoring information. The flowchart explaining the processing operation at the time of a scoring plan production | generation part producing | generating a scoring plan. The flowchart explaining the process in which the similarity determination part determines the similarity of the answer data of the scoring information already scored with respect to the answer data. The flowchart explaining the other process which the similarity determination part determines the similarity of the answer data of the scoring information already scored with respect to the answer data. The flowchart explaining other processing operation at the time of a scoring information creation part creating scoring information. The flowchart explaining the processing operation of the similarity determination part in consideration of including the negative wording.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1001 Scoring support apparatus 1002 Answer data storage part 1003 Similarity determination part 1004 Scoring plan production | generation part 1005 Scoring information creation part 1006 Answer data 1007 Scoring information 2001 Computer system 2002 CPU
2003 Storage device 2004 Auxiliary storage device 2005 Input device 2006 Output device 3001 Client for learner 3002 Client for teacher

Claims (12)

In a scoring support device that supports scoring of answer data for questions that require a written answer,
In the computer system, a similarity determination unit, a scoring plan generation unit, a scoring information creation unit, and an answer data storage unit are constructed,
The answer data accumulating unit accumulates a set of answer data for the question, and the answer data is sequentially given scoring information after scoring,
The similarity determination unit searches the answer data storage unit for answer data with scoring information having answer data similar to the answer data of the problem being scored,
The scoring plan generation unit collects similar answer data searched by the similarity determination unit, generates a scoring plan for the answer data of the problem being scored,
When the scoring information creation unit gives scoring information to the answer data, the scoring information generated by the scoring plan generation unit and the answer data of the problem being scored are displayed to the scorer, and the scoring information for the answer data A scoring support device characterized in that the input scoring information is added to the scoring answer data.   The scoring support device according to claim 1, wherein the answer data includes itemized items listing sentence data.   3. The scoring support device according to claim 2, wherein the scoring information includes correct / incorrect answer determinations and comment data for itemized items that are answer data.   The similarity determination unit compares similarities between the itemized items that are the answer data of the scored problem and the itemized items of the other answer data in the answer data storage unit, and similar itemized items 3. The scoring support apparatus according to claim 2, wherein a scoring system with scoring information is searched.   2. The scoring support apparatus according to claim 1, wherein the input requested by the scoring information creating unit is to adopt the presented scoring plan or to modify the scoring plan.   The scoring information further includes comment data for the entire answer data, and the comment data for the entire answer data is determined to be correct by the scoring plan generation unit in the itemized item of the answer data in the answer data storage unit. A search is made for candidates that are not similar to any of the itemized items in the answer data for the graded question, and the search result is passed to the scoring information creation unit. Item 3. The scoring support device according to item 3.   The similarity determination unit first determines the similarity between all the answer data, establishes a link between similar answer data, and the scoring plan generation unit scores using the link The scoring support device according to claim 1, wherein answer data similar to the answer data of the question is searched.   The answer data has not only scoring data but also scoring plan data, and the scoring plan generation unit scores scoring plans for other unscored answer data that is linked when scoring information is added to the answer data. The scoring support device according to claim 7, wherein information is created.   The similarity determination unit searches a plurality of items with scoring information similar to itemized items that are answer data of the problem being scored, and the scoring information creation unit includes a plurality of scoring information creation units. 5. The scoring support device according to claim 4, wherein the scoring device is preferentially presented to the scorer from items with high similarity among similar itemized items.   When comparing the sentence data as the answer data, the similarity determination unit extracts keywords from the respective sentences, determines how similar they are, and determines that they are not similar if they are not similar 5. The scoring support apparatus according to claim 4, wherein, even if they are similar, if there is a keyword that indicates negation on one side, it is determined that they are not similar.   The scoring information creation unit gives priority to similar answer data and preferentially presents a scoring plan obtained from a higher priority among similar ones to the scorer. The scoring support device according to 1. In a scoring support method that supports scoring of answer data for questions that require a written answer,
In the computer system, a similarity determination unit, a scoring plan generation unit, a scoring information creation unit, and an answer data storage unit are constructed,
The answer data accumulating unit accumulates a set of answer data for the question, and the answer data is sequentially given scoring information after scoring,
The similarity determination unit searches the answer data storage unit for answer data with scoring information having answer data similar to the answer data of the problem being scored,
The scoring plan generation unit collects similar answer data searched by the similarity determination unit, generates a scoring plan for the answer data of the problem being scored,
When the scoring information creation unit gives scoring information to the answer data, the scoring information generated by the scoring plan generation unit and the answer data of the problem being scored are displayed to the scorer, and the scoring information for the answer data A scoring support method characterized in that the input scoring information is given to the answering data scoring.

Images