JP2017182101A - Comprehension tendency measuring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a comprehension tendency measuring system capable of more quickly, more accurately measuring many answers in a limited time.SOLUTION: A comprehension tendency measuring system includes: answer data storage means; question sentence data storage means for storing question sentence data of a question sentence group composed of a plurality of question sentences which demand selection from two alternatives such as affirmation or negation; marked answer transmission means for transmitting question sentence data and selected answer image data to a marker terminal; check result reception means for receiving check results which are either a check pattern composed of a combination of affirmation and negation relative to the question sentence group on the paper or a reserved check from the marker terminal; and check result storage means for storing the received check results. The marked answer transmission means further include marking reservation answer transmission means for transmitting question sentence data and answer image data corresponding to an answer whose check result is the reserved check, to a specific marker terminal.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an understanding tendency measurement system that measures an understanding tendency of a group of examinees by classifying answers using descriptive questions.

  In providing school education and various kinds of education, what kind of education learners have given to the contents of the education, such as examining the results of the education actually provided or trying to extract future issues It may be necessary to measure whether it shows a tendency to understand. In such measurement, “examination (test)” using a descriptive question is performed, and the collected tendency of answers is classified to measure an understanding tendency of a group of examinees.

  The above "examination (test)" is different in character from competitive examinations such as entrance examinations for selecting examinees, and the method of creating questions and the way of handling the answers are different. That is, for example, a problem is created so that multiple expected answers reflecting the understanding tendency are created for each learning item to be measured extracted from the educational content, and this problem is actually solved and examined. It is necessary to repeatedly correct both the expected answer and the problem so that the relation with the expected answer becomes closer. In addition, by taking an examination (test) using such a question, the answer obtained from the candidate is similar to and related to which expected answer, that is, by classifying each answer into the expected answer, The tendency of understanding for each learning item is measured.

  By the way, in the answer to the descriptive question, since a certain amount of free description is allowed for the examinee, variations of the answer become very large. While this makes it possible to measure the understanding tendency of each examinee's learning items from various viewpoints, it makes it difficult to measure objectively by eliminating the subjectivity of the scorer. In particular, if the number of examinees increases, the number of graders must be increased in order to complete the measurement within a set time, and as a result, the influence of the subjectivity of each grader increases, resulting in measurement results. Tends to be inaccurate.

  Therefore, Patent Document 1 does not aim to measure the understanding tendency of the examinees as described above. However, in the scoring of the answer to the descriptive question, the influence of subjectivity among a plurality of scorers is shown. Disclosed answer scoring support system. In such a system, one scorer re-evaluates the score result while referring to the score basis of another scorer, and corrects the score result to reduce the influence of subjectivity among a plurality of scorers. If the scoring results do not match among multiple scorers and there is a question about the objectivity of the scoring results, review the scoring results while referring to the scoring grounds of the other scoring personnel, and the registered scoring results It can be corrected.

  There has also been proposed a method for increasing the accuracy of measurement by reducing the influence of subjectivity in the work of the scorer itself.

  For example, in Patent Documents 2 and 3, the answer is based on a check pattern that is a combination of affirmative or negative checks for a question sentence group consisting of a plurality of question sentences that the grader selects with a choice of affirmative or negative. A method for classifying and performing the above-described measurements is disclosed. The grader can narrow the discretion range to only two choices for the question sentence, thereby reducing the influence of the subjectivity of each grader and increasing the accuracy of the measurement.

  Moreover, in patent document 4, like patent document 2 or 3, it is the combination which added further hold to affirmation or denial with respect to the question sentence group which consists of the question sentence made to select by two choices of affirmation or denial It discloses a method for classifying answers and measuring them based on check patterns. By giving the grader a hold decision, it is possible to reduce the ambiguity of the affirmative or negative decision of the grader and increase the accuracy of the measurement.

JP 2006-277086 A JP 2009-229606 A JP 2010-078637 A JP 2012-022185 A

  As in the method disclosed in Patent Document 4, by giving the grader a hold decision on the question sentence, it is possible to reduce the ambiguity of the affirmative or negative decision of the grader and increase the accuracy of the measurement. obtain. On the other hand, for the check pattern answer including the hold, the measurement has not been completed and it is necessary to measure again, and it is also required that a large number of answers can be measured more quickly and accurately within a limited time. It is done.

  The present invention has been made in view of such a situation, and the purpose of the present invention is to present the examinee with a descriptive question created in association with a plurality of predicted answers reflecting the understanding tendency of each measurement target item. In addition, a measurement system that measures the understanding tendency of test takers by classifying the answers using a test that requires the creation of an answer can measure a large number of answers quickly and more accurately within a limited time. It is to provide an understanding tendency measurement system.

  The understanding tendency measuring system according to the present invention uses a test that asks a test taker to create a written answer after presenting a descriptive question created in association with a plurality of predicted answers reflecting an understanding tendency for each measurement target item. A measurement system for measuring an understanding tendency of a group of examinees by categorizing the answer, and obtaining image data for each answer and attaching it as an answer image data with an answer identification code for identifying it Answer data accommodation means, question sentence data accommodation means for accommodating question sentence data of a question sentence group consisting of a plurality of question sentences to be selected in two choices of affirmation or denial with respect to the answer, and a request from the grader terminal In response, a grader sending means for sending the question sentence data and the selected answer image data to the grader terminal, and a grader for identifying the grader who operates the grader from the grader terminal A check result receiving means for receiving a check pattern or a check pending check result comprising a combination of affirmative or negative with respect to the question sentence group of the answer corresponding to the answer identification code together with another code, and the check result receiving A series of the grader identification code received by the means, the answer identification code, and a check result accommodating means for sequentially accommodating the check results, wherein the score answer transmitting means responds to a request from a specific grader terminal The grader terminal further has a pending grade answer sending means for sending the question sentence data and the answer image data corresponding to the answer identification code of the answer whose check result is put on check to the grader terminal. It is characterized by that.

  According to this invention, when the grader selects the hold, the check result storage means stores the check hold check result in place of the check pattern, and the corresponding answer is identified again from the score answer sending means. Can be easily retrieved for transmission to the grader terminal. At the same time, it is possible to provide an opportunity to correct the judgment of the question sentence that has been suspended and the judgment of the question sentence that can be subordinate to the judgment, thereby enabling more accurate measurement. That is, a large number of answers can be measured more quickly and accurately within a limited time.

  In the above-described invention, the scoring answer transmission means displays an image of the answer on the scorer terminal and displays the question sentence group, and affirmative or negative for each question sentence in the scorer terminal. It is also possible to input check pending for the question sentence group instead of this input. According to this invention, it is possible to reduce the judgment time of the grader for the hold input and the operation time of the terminal by making the check hold input to the question sentence group instead of an affirmative or negative input to the question sentence. Thus, a large number of answers can be measured more quickly and accurately within a limited time.

  In the above-described invention, the check pending input ratio for each grader is calculated from the grader identification code, and includes grade assigning means for grading the grader, based on the grader result of the grader, The said pending | scoring grade answer transmission means may specify the said grader terminal which should transmit data. According to this invention, the answer corresponding to the check pending check result can be transmitted again from the grade answer sending means to the graded specific grader based on the check pending input ratio, and a large number of answers can be sent within a limited time. Can be measured more accurately.

  In the above-described invention, the check pattern is classified into several types so as to reflect an understanding tendency for each measurement target item, and the type corresponding to the check pattern input to the scorer terminal is scored. It may be characterized by further including a type presenting means for presenting to a person. According to such an invention, it is possible to reduce a scorer's input mistakes and judgment mistakes, and to measure a large number of answers more accurately within a limited time.

  In the above-described invention, when the type presentation means presents the type corresponding to the predicted answer that is a correct answer to the grader, the type presentation unit presents the grader that the type is correct together with the type. This may be a feature. According to such an invention, it is possible to reduce a scorer's input mistakes and judgment mistakes, and to measure a large number of answers more accurately within a limited time.

It is a figure which shows the structure of the understanding tendency measurement system by this invention. It is a figure which shows the structure of the principal part of the understanding tendency measurement system by this invention. It is a figure which shows the example of a problem sentence. It is a figure which shows the example of a check box sheet. It is a flowchart of the waist | hip | lumbar part of the understanding tendency measurement system by this invention. It is a figure which shows the example of the image displayed on the monitor of a client computer. It is a figure which shows the example of the image displayed on the monitor of a client computer. It is a figure which shows the example of the image displayed on the monitor of a client computer. It is a figure which shows the example of the image displayed on the monitor of a client computer.

  First, the principle of the understanding tendency measurement system according to the present invention will be described.

  When you want to measure the understanding tendency for educational contents by test (test), create a question and multiple assumed answers to it, and examine whether the answer obtained by actually performing the test is closer to the expected answer Then, the number of classifications can be statistically processed to perform measurement.

  Specifically, questions are created corresponding to the measurement target items selected from the learning items that subdivide the educational content, and the question sentences are carefully prepared so that the description about the measurement target items can be included in the answer. The For example, in the case of a problem having a plurality of solutions until a correct answer is derived, the preconditions of the question sentence and the like are determined so that an answer can be obtained by being limited to only the solution related to the measurement target item. In other words, in consideration of the purpose of the problem, the question text is sometimes obtained from a plurality of mock examinees, and the sentence expression is corrected while checking whether these answers can be classified into assumed answers.

  By the way, even if it is the same answer, even if it is the same answer, it will be unpreferable that measurement accuracy will fall if the assumption answer classified is different. Therefore, as a method that can be classified while reducing the range of judgment of the worker, the answer is divided into multiple types by a check box consisting of a plurality of two-choice question sentences and a check pattern that is a combination of the two-choice results. A method of classifying was found. In addition, the question sentence is set so that the check pattern is different if it is an expected answer with a different understanding tendency.

  On the other hand, when many answers are classified in a short time, it is necessary to secure workers who perform more check operations, and the diversity of workers is further increased. Even within this, there is a limit to creating a check box question so that the check pattern does not depend on the operator. Therefore, if there is any doubt about the determination of the two-choice result of the check box, this check box can be put on hold, a check pattern including a hold answer is once obtained, and another worker checks the hold answer again. A way to do the work was found. This makes it possible to classify many answers in a short time without reducing measurement accuracy.

  Here, the check boxes for one answer have a lot of relations with each other, and the check operation after the check answer can be influenced by the check answer, and the reliability decreases. Therefore, the check process for the pending answer can be done not only for the pending check box but also for all the check boxes. Therefore, there is little need to obtain a check pattern including the pending answer, and the obtaining of the check pattern is suspended. It is more efficient to do. That is, it is efficient for the worker who gave the check box hold and for other workers who work again.

  Next, an understanding tendency measurement system according to the present invention will be described with reference to FIGS.

[System configuration]
As shown in FIG. 1, an understanding tendency measurement system 1 includes a server computer 22 used by an administrator including a problem creator and the like, and a plurality of computers connected to the computer via a communication line 20 such as an Internet line or a LAN. Client computer 21. The client computer 21 includes input / output devices such as a monitor, a keyboard, and a mouse.

  The server computer 22 includes a large-capacity storage device 23 such as a hard disk device, a CPU 12 as a control unit, a ROM 13 and a RAM 14, and a communication interface (not shown). The server computer 22 is connected as a user interface to a monitor 22a, a keyboard 22b, and an input / output device such as a scanner (not shown) that enables input of answer image data.

  The storage device 23 has a program storage area 30a and a database (DB) area 30b for storing various data.

  In the program storage area 30a, a question sentence data storage means 31, a response data storage means 32, a scoring answer transmission means 33, a check result storage means 34, and a type presentation means 35 as programs executed by the CPU 12 are stored. The grade assigning means 36 is stored. The scoring answer sending means 33 further includes a pending scoring answer sending means 33a as a subprogram.

  Referring also to FIG. 2, the database 30 b includes a question sentence database 24, a question sentence database 25, an answer database 26, a type information database 27, a grader information database 28, and a check result database 29.

  Although details will be described later, each means of the program storage area 30a will be outlined below.

  The question sentence data accommodating means 31 causes the administrator to input a question sentence including a question sentence group for creating a check box sheet from the keyboard 22b as a user interface as character data, and stores it in the question sentence database 25, and This is a program for correcting the question text stored in the question text database 25.

  The answer data storage means 32 is a program for storing image data of an answer in an answer database 26 from an image input device (not shown) such as a scanner connected to the server computer 22.

  The scoring answer transmission means 33 responds to the signal for requesting the first scoring work received from the client computer 21 with the answer data corresponding to the unscored answer to be scored by the scorer operating the client computer 21. A program for searching and selecting the answer data database 26 while referring to the check result database 29, attaching the answer code, and sending the answer data to the client computer 21 together with the question text stored in the question text data database 25 It is. The scoring answer sending means 33 takes over the processing to the type presenting means 35 when receiving a signal requesting the type presentation from the client computer 21 during the scoring work, and receives the check result as the result of the scoring work. It is also a program for sometimes taking over the processing to the check result accommodation means 34.

  The pending grade answer sending means 33a corresponds to an answer having a check pending check result to be given to a specific grader who operates the client computer 21 in response to a signal for requesting a grade work of the pending answer received from the client computer 21. The answer data to be searched is selected by searching the answer data database 26 with reference to the check result database 29, and sent to the client computer 21 together with the question text stored in the question text data database 25 with the answer code attached. It is a program to do.

  The check result accommodation means 34 is a program for receiving the check result, which is the result of each scoring work, from the client computer 21 together with the grader code and the answer code and storing them in the check result database 29.

  The type presentation unit 35 extracts the type information corresponding to the received check pattern from the type information database 27 and transmits it to the client computer 21 in response to the signal requesting the type presentation received from the client computer 21, and sends the type information to the client computer 21. Is a program for causing the monitor of the client computer 21 to display an image based on the above.

  The grade assigning means 36 refers to the check result database 29, calculates the ratio of transmitting the check result for which the check pending is input for each grader, determines the grade of the grader based on this, and determines the grade. This is a program for storing in the grader information database 28.

[Preparation]
Next, of the understanding tendency measurement procedure using the understanding tendency measurement system 1, a preparation procedure for performing answer classification processing will be described with reference to FIGS.

(Create question)
First, the administrator creates a problem sentence of a description type problem. The question sentence is created so as to obtain an answer including a description about the measurement target item selected from the learning items of the educational contents provided to the examinee to be measured.

  As shown in FIG. 3, the administrator, for example, “uses the three-square theorem formula” and “one-on-two” which are measurement target items selected from the learning items of the educational content provided in a predetermined grade. An attempt is made to create a question sentence q that requires an answer including a description relating to “solving the following equation”. The question sentence q is “If the lengths of two sides sandwiching a right angle of a right triangle are 1 cm and 2 cm, respectively, find an x with an equation where x is the length of the hypotenuse”. . It requires a description of the process of formulating a unitary quadratic equation using the formula of the three square theorem and solving it to derive the answer. The administrator inputs such a problem sentence from an input device such as a keyboard 22b connected to the server computer 22 or a scanner (not shown), and stores it in the problem sentence database 24 of the storage device 23.

(Creating a check box sheet question)
As shown in FIG. 4A, the first check box sheet 50 is an image that is transmitted to the client computer 21 and displayed on the display screen of the monitor as an input screen for input by the grader. Each of the question sentences CQ1 to CQ7 that can be answered by either “Y (positive)” or “N (negative)”, and select either “Y” or “N” A check column 52 (Y, N) for adding a check and a hold check column 53 for holding a check for a question sentence group by adding a check are included. In the first check box sheet 50, each of the graders can answer either “Y” or “N” as an answer to the question sentences CQ1 to CQ7 by checking the corresponding position in the check column 52 on the display screen. In addition to being able to select, it is also possible to select to put a check in the hold check column 53 in order to finish the scoring work without answering the remaining question sentences. The question sentences CQ1 to CQ7 are preferably contents that can be the same judgment for the same answer as long as it is a general scorer who can easily make an affirmative or negative decision. That is, the question sentences CQ1 to CQ7 are preferably questions that formally determine the description of the answer, not a question that is determined based on whether or not specific contents are described in the answer.

  As shown in FIG. 4B, the second check box sheet 51 is transmitted to the specific client computer 21 and displayed on the display screen of the monitor as an input screen for input by a specific grader. It is an image. The hold check item CQ0 is deleted from the first check box sheet 50. The question sentence group of the second check box sheet 51 is the same as that of the first check box sheet 50. In the second check box sheet 51 as well, by checking the corresponding position in the check field 52 on the display screen, each grader can answer each of the question sentences CQ1 to CQ7 from either “Y” or “N”. You can choose.

  The created question sentences CQ1 to CQ7 are input from the input device such as the keyboard 22b by the administrator along the question sentence data storage unit 31, and are stored in the question sentence data database 25 as question sentence data including character data. In addition, the question text based on the question text data is displayed on the monitor 22a along the question text data storage means 31, and an input by the administrator for correcting these is received from the keyboard 22b or the like, and the correction input is received. The question sentence data corresponding to the asked question sentence can be overwritten and saved in the question sentence data database 25. Thereby, the administrator can correct each question sentence.

(Memory of type check patterns)
The administrator prepares a plurality of predicted answers from the question sentence q so as to correspond to various understanding tendencies, assuming processing by the examinee in the above two measurement target items. The administrator applies the question sentence group of the second check box sheet 51 to the predicted answer to obtain a check pattern, and inputs it to the server computer 22 together with a type number indicating a type corresponding to the expected answer. . The server computer 22 attaches a type number to the input check pattern and stores it in the type information database 27. In the type information database 27, in addition to the check pattern and the type number, an expected answer corresponding to the type, an explanation about the understanding tendency, and the like may be input and stored.

  At this time, the obtained check patterns must be classified into corresponding types without overlapping at least for each predicted answer having a different understanding tendency. For this reason, the administrator changes the question content that is the basis of the question sentence as necessary, such as when check patterns corresponding to a plurality of types overlap. That is, there are cases where the administrator repeatedly creates a check box sheet question and stores a type check pattern. In addition, the administrator must also express the question text so that it is easy to make an affirmative or negative decision for each question sentence, that is, the two choices for each question sentence regarding a specific answer can be the same for any grader. Correct as appropriate.

(Memorization of answer data)
The administrator presents the question sentence q to a plurality of examinees, makes them answer, and creates an answer (implementation of a test). The administrator activates the answer data storage means 32, and takes in the answer sheet on which the collected answer is written as image data by an input device such as a scanner (not shown) and stores it as answer data in the answer database 26. At this time, an answer code for identifying each answer is attached to each answer data. For the answer code, for example, a combination of a code for identifying the examinee and a code for identifying the question sentence can be used so as to easily refer to the question sentence corresponding to the examinee who created the corresponding answer. Further, each answer data is provided with two kinds of flags, a scoring flag and a result received flag for searching answer data to be transmitted to the client computer 21. Both the initial value of the scoring flag and the result received flag are set to “0”.

[Scoring and classification of answers]
Next, of the understanding tendency measurement procedures using the understanding tendency measurement system 1, a procedure for scoring the answers and classifying the answers will be described with reference to FIGS.

(First scoring work)
A case will be described in which one of a plurality of scorers (hereinafter referred to as “scorer A”) scores one of a plurality of answers.

  First, the grader A accesses the server computer 22 by the client computer 21 operated by the grader A. Then, the server computer 22 displays the login image 40 on the monitor of the client computer 21 using a web browser or the like (see FIG. 6). After the grader code and password are entered, the grader A performs an input of pressing the login button 40a on the screen on which the login image 40 is displayed. That is, for example, the user operates the mouse to superimpose the pointer on the screen on the login button 40a and clicks the left button of the mouse. Upon receiving such input, the client computer 21 transmits the grader code and password to the server computer 22. The server computer 22 compares the grader code and password received from the client computer 21 with the grader information stored in the grader information database 28, confirms that the grader A has been registered, and then logs in. To give permission.

  As shown in FIG. 5, the grader A operates the client computer 21 and requests the server computer 22 to perform the scoring work (S11). Specifically, referring also to FIG. 7, the server computer 22 displays the menu image 41 on the monitor of the client computer 21 using a web browser or the like. The grader A selects “first grade” from the menu. That is, on the screen on which the menu image 41 is displayed, an input for pressing the “first scoring” button 41a is performed. In response to this, the client computer 21 transmits a signal requesting the first scoring operation to the server computer 22.

  The server computer 22 that has received the signal selects answer data to be returned from the answer data stored in the answer data database 26 by the scoring answer sending means 33 while referring to the check result database 29. In this selection, answer data corresponding to an unscored answer that is not graded at another client computer is selected. That is, the answer data in which the check result is not in the check pending state among the answer data in which both the scoring flag and the result received flag are “0” is searched for, and the “holding” will be described later). One is selected and “1” is set to the scoring flag (the scoring flag is set to ON) (S12). A plurality of answers can be selected such as allowing the grader to specify the number of answers to be selected, but one answer data is selected here.

  Further, the server computer 22 asks the client computer 21 to display the first check box sheet 50 on the monitor of the client computer 21 together with the answer data selected by the client computer 21 by the scoring answer sending means 33 and the answer code attached thereto. The sentence data and other standard data are returned (S13).

  Referring also to FIG. 8, the server computer 22 displays the scoring work image 42 on the monitor of the client computer 21 that has received the answer data etc. by the scoring answer sending means 33 (S14). Moreover, the client computer 21 is made to accept the input regarding the scoring work by the grader A (S15). That is, the grader A performs a scoring operation while operating a pointer on the scoring operation image 42 using an input device such as a keyboard or a mouse of the client computer 21.

  Specifically, the grader A is positive or negative with respect to the question sentences CQ1 to CQ7 of the first check box sheet 50 regarding the answer read from the answer image 43 of the scoring work image 42 displayed based on the answer data. The answer is determined by selecting two choices, selecting “Y” or “N” from the determination result, and performing an input for checking the check column 52. Alternatively, if there is a doubt in the determination of the two options, a check is put in the hold check column 53. By checking the pending check column 53, the grader A can finish the scoring work for the answer without answering the remaining other questions.

  If the hold check column 53 is not provided, the grader A is forced to make his own judgment based on his / her own subjectivity when he / she is at a loss in the two choices. An answer based on such a judgment may be different from the answer of other scorers, and as a result, the answer is low in reliability. In addition, in the case where a decision is lost, the reliability of the grader's answer to other question sentences having an associated relationship with the answer to the question sentence becomes low. By ending the scoring work without giving such an answer, for the grader A, the work amount can be assigned to the scoring work of another answer.

  During the scoring work by the grader A, the server computer 22 is in a standby state to receive signals from the client computer 21 in response to pressing of the type display button 44 or the scoring completion button 45 (S16; N, S21; N).

  When the type display button 44 is pressed (S16; Y), the server computer 22 confirms whether all the answers to the question sentences CQ1 to CQ7 are input by the scoring answer sending means 33, and then combines the answers. A signal for requesting the presentation of a type together with a grader code for a certain check pattern is transmitted to the server computer 22 and the processing is taken over by the type presentation means (S17). The server computer 22 having received this (S18) refers to the type information database 27 by the type presenting means 35 and transmits the type information of the type corresponding to the received check pattern to the client computer 21. The type information is, for example, a type number for specifying the type, an expected answer corresponding to the type, or an explanation about understanding tendency (S19). Further, the type information is displayed in the type information column 47 on the monitor of the client computer 21 that has received this, and the processing is passed to the scoring answer sending means 33 (S20).

  The grader A receives the type information during the scoring work, compares the answer with the type information, makes it easy to find the input mistake and judgment error, and corrects the input to the check column 52 as necessary. Thus, these mistakes can be reduced. The type information database 27 is attached with a code for determining the correct or incorrect answer of the predicted answer, and the code is included in the type information to display whether the corresponding expected answer is a correct answer or an incorrect answer. Also good. This can also reduce input errors and determination errors.

  When the scoring completion button 45 is pressed (S21; Y), the server computer 22 has entered all the answers to the question sentences CQ1 to CQ7 by the scoring answer transmission means 33, or enters the hold check column 53. After confirming whether the check is added, the check pattern or the check pending check result is transmitted to the server computer 22 together with the answer code and the grader code. Further, the process is taken over by the check result accommodating means 34 (S22). The server computer 22 receives the check result and the like by the check result accommodating means 34, adds the answer code and the grader code to the check result, and stores them in the check result database 29. Further, the server computer 22 sets the scoring flag of the corresponding answer data in the answer data database 26 to “0” (OFF) (S23). Note that it is preferable that the check pending check result also includes a part of the check pattern that is an already input answer. Thereby, in the pending | scoring work of the pending | holding answer mentioned later, the answer of the grader A with respect to a part of question sentence can be displayed previously.

  Furthermore, the server computer 22 classifies the answer based on the check result. When the check result is a check pattern, the answer is classified into each type according to the check pattern. That is, the corresponding type number is added to the corresponding check result in the check result database 29 and stored. Further, the result received flag of the corresponding answer data is set to “1” (ON). If the check result is check pending, no classification is performed (S24). Here, both the scoring flag and the result received flag for the answer data of the pending answer whose check has been suspended are both “0” and return to the initial value.

(Scoring work for pending answers)
The pending answer is scored by a specific grader who has a certain level of understanding about the intent of the question. That is, the grader B is a grader who does not hesitate to make an affirmative or negative decision on the question sentences CQ1 to CQ7 in the answer scoring operation. In particular, it is preferable that the intention of the question sentence is deeply understood, and it is possible to make a two-choice determination even for an answer with a specific content that is difficult to make the two-choice judgment. A case will be described in which one of the specific scorers (hereinafter referred to as a scorer B) scores one of a plurality of answers. As with the grader A in the first scoring operation, the grader B has accessed the server computer 22 in advance and transmitted the grader code and logged in.

  As shown in FIG. 5, the grader B operates the client computer 21 and requests the server computer 22 to perform the scoring work (S11). Specifically, referring also to FIG. 7, the server computer 22 displays the menu image 41 on the monitor of the client computer 21 using a web browser or the like. Grader 2 selects “pending grade” from the displayed menu. That is, an input operation is performed on the menu image 41 by operating a pointer with the mouse for the “pending scoring” button 41b. In response to this, the client computer 21 transmits a signal for requesting a scoring operation regarding the pending answer to the server computer 22. Note that the grader B can also select the “first-time scoring” and perform the above-mentioned first scoring work in the same manner as the grader A.

  The server computer 22 that has received the signal for requesting the scoring work related to the pending answer has the scorer information database 28 based on the grader code received from the client computer 21 by the pending score answer sending means 33a of the scoring answer sending means 33. Refer to the corresponding grade to confirm that it is certified to grade the pending answer. That is, it is confirmed that the grader B is a specific grader who has a certain level of understanding about the intention of the question sentence. After the confirmation, the server computer 22 selects answer data to be returned from the answer data stored in the answer data database 26 by referring to the check result database 29 by the pending score answer sending means 33a. In this selection, answer data that is not being scored at another client computer but is a pending answer is selected. That is, answer data whose check result is pending check is searched from answer data in which both the scoring flag and the result received flag are “0”, and one of them is selected as the scoring flag. “1” is set (grading flag is set to ON) (S12). The check result can be easily searched because it is only necessary to search for whether there is a check pending check. Although the number of answers to be selected can be made plural by allowing the grader to specify the answer, one answer data is selected here.

  Further, the pending scoring answer sending means 33a causes the server computer 22 to display the second check box sheet 51 on the monitor of the client computer 21 together with the answer data selected by the client computer 21 and the answer code attached thereto. The question sentence data and other fixed form data are returned, and the process is passed to the scoring answer sending means 33 (S13). Note that, as described above, when an already input answer (part of the check pattern) is included in the check pending check result, this is also returned.

  The server computer 22 displays an image obtained by deleting the pending check item CBp including the pending check column 53 from the scoring work image 42 (see FIG. 10) on the monitor of the client computer 21 that has received the answer data and the like by the scoring answer sending means 33. Let That is, the scoring work image including the second check box sheet 51 is displayed (S14). Since the subsequent processing (S15 to S22) up to the transmission of the check result regarding the scoring work for the pending answer is the same as the first scoring work, the description thereof is omitted. However, since the second check box sheet 51 is used in the scoring work regarding the pending answer, the pending check column 53 cannot be checked, and the check result always becomes a check pattern. Here, the grader B has a high degree of reliability with respect to the determination without questioning whether the question sentence CQ1 to CQ7 is positive or negative. In addition, when the answers to some of the questions that have already been answered in the first scoring operation are displayed in advance, the scorer B may use them as they are or modify them. In this way, the grader who has a certain level of understanding about the intent of the question sentence will be graded for the answer that has made the decision to choose between the two choices by the grader, so the reliability of the obtained check pattern is high. Become.

  The server computer 22 that has received the check result from the client computer 21 attaches the answer code and the grader code to the check result by the check result accommodation means 34 that has taken over the processing from the scoring answer sending means 33 and stores it in the check result database 29. Let By attaching a grader code to the check result, a scoring history of the first scoring work and the scoring work of the pending answer is left, and these can be searched later if necessary. Further, the server computer 22 sets the scoring flag of the corresponding answer data in the answer data database 26 to “0” (OFF) (S23).

  Furthermore, the server computer 22 classifies the answer based on the check result. All the check results are check patterns, and the answers are classified into each type according to the check patterns. That is, the corresponding type number is added to the corresponding check result in the check result database 29 and stored. Further, the received flag of the corresponding answer data is set to “1” (ON) (S24).

  The first scoring operation and the scoring operation for the pending answer are repeated by a plurality of scorers. The server computer 22 accepts the above-mentioned scoring work request until it is confirmed that the received flag for all the answers is “1”. At this time, for example, when there is no unscored answer when a request for the first scoring operation is received, if answer data to be transmitted cannot be selected, this is returned to the client computer 21 and displayed. In addition, you may classify | categorize an answer, after receiving the check pattern regarding all the answers.

  As described above, the server computer 22 can select the answer data that has become the pending answer by an easy search only for the presence or absence of the check pending check in the check result database 29 in order to use the pending answer scoring. In addition, for the answer whose check has been suspended, it is possible to give a score to a specific grader who has a deep understanding of the intentions of the question sentences CQ1 to CQ7 and does not hesitate to make a choice between two choices. As a result, it is possible to correct the judgment on the question sentence on which the pending answer is determined and the question sentence on which the judgment can be subordinated. Therefore, a large number of answers can be measured more quickly and accurately within a limited time.

  On the other hand, in the case where the client computer 21 is at a loss in the choice of the grader A, the client computer 21 scores the answer without answering the remaining question sentence groups related to the answer that caused the decision to be lost. End the work and prevent answers based on original judgment based on the subjectivity of the graders. Further, instead of answering the remaining question sentence groups, the grader A can perform the scoring work for other answers.

  As a result, the grader A performs a lot of scoring work for answers that are easy to make a two-choice decision, and the specific scorer B performs a lot of scoring work for answers that are relatively difficult to make a two-choice decision. , The tendency to understand many answers in a limited time can be measured more quickly and accurately.

  Note that the check pending check result may not be in the pending check column 53. For example, a “hold” button is separately provided in the scoring work image 42, and a check result of check hold is transmitted to the server computer 22 together with the answer code and the grader code by pressing the “hold” button.

  In addition, in the classification into types, “others” is generally provided as an unexpected type of check pattern that does not apply to any expected answer. If there are a large number of specific check patterns that are classified as “Others”, analyze the tendency of understanding the answers corresponding to the check patterns, and define them as new types as necessary. May be.

  In the above-described embodiment, for example, when “blank paper answer” is one of the types, the answer to the question sentence CQ1 “blank paper answer?” (See FIG. 4) is “Y”. It is not necessary to answer the question text, but it is classified into a “blank paper answer” type. In this way, in the case where the type is determined solely or by answering only some of the question sentences, prior to the above-mentioned first scoring work, only for some of the question sentences for all the answers. You can also answer and classify the relevant answers in advance. In this case, for the answers that are not classified here, the remaining question sentences may be answered in the first scoring work and the scoring work for the pending answer. In this way, by classifying a part of the answers with only answers to a small number of question sentences in advance, the total number of answers to the question sentences can be reduced, and the understanding tendency can be measured more quickly.

  In addition, as an example that can be distinguished from other answers by such an answer to a single question or only a part of the question, for example, “extruding” that causes a part of the answer data to be missing due to the protruding of the answer from the answer column, Special answers such as written answers in words and dialects. As for special answers other than blank papers, there is at least some description in the answer, so when measuring the understanding tendency, for example, it is based on the description rather than just categorizing it as “extruding” It is desirable to perform the original classification. Therefore, as in the case of the “blank paper answer”, it is preferable to make a procedure in which only a part of the question sentences are answered prior to the first scoring operation, but not separately classified but scored separately. About such an answer, it is good for an administrator and some graders equivalent to it to refer to an answer sheet as needed, and to grade while checking an overhang | projection part.

  In addition, there are few such special answers, and if the number of question sentences is increased only to classify them, the scoring work may be prolonged. In such a case, a check result indicating that the answer is a special answer may be input in the first scoring operation, instead of answering as a question sentence regarding all the answers. In this case, a check box indicating that the answer is a special answer is separately provided in the check box sheet, and a check is added to the check box so that the answer is marked as a pending answer with a reason. That is, also in this case, it is not scored by the grading work of the pending answer, but is scored by the manager or a part of the graders corresponding thereto. At this time, the scoring work image may be provided with a comment field describing such reasons.

  In the above-described embodiment, the description has been made with one question sentence. However, in practice, a plurality of question sentences in a plurality of subjects are tested, and an answer is created. Therefore, it is preferable that the grader can select a subject to be graded when making a request for a scoring operation, and can further select a question sentence. Therefore, when the grader selects “first grade” or “pending grade” from the menu (see FIG. 7), a question sentence to be graded may be designated.

  That is, when the menu image 41 is displayed on the monitor and the “first scoring” button 41a or the “pending scoring” button 41b is pressed, the server computer 22 is connected to the client computer as shown in FIG. The scoring target selection image 60 is displayed on the 21 monitors. The grader selects the question sentence of the subject to be graded in the question sentence selection column 61, designates it with the question number 62, and inputs it. In response to the pressing of the “next” button 63, the server computer 22 selects answer data for the designated question sentence, and causes the client computer 21 to select the answer data selected so that the selected answer data is scored. Corresponding answer codes, question sentence data, etc. are transmitted. As a result, the scoring work image 42 (see FIG. 8) is displayed on the monitor of the client computer 21, and the scoring work can be performed on the answer to the designated question sentence, as in the above-described embodiment.

  By the way, referring to FIG. 1, the program storage area 30 a includes grade assigning means 36 as a program executed by the CPU 12. That is, it is possible to calculate a ratio of inputting check suspension for each grader and transmitting it as a check result, and to determine the grade of the grader based on this. For example, when a signal requesting grade update is received together with the grader code from the client computer 21, the server computer 22 refers to the check result database 29 by the grade assigning means 36 and checks with the same grader code. The ratio of the number of check pending check results to the total number of results, that is, the check pending input ratio is calculated. The grade corresponding to the input ratio of the pending check result is returned to the client computer 21 by referring to a predetermined table in which the lower grade is associated with the higher ratio and the higher grade is associated with the lower ratio. And the corresponding grader information in the grader information database 28 is overwritten. It is preferable that the grade of each grader can be updated by an operation of the server computer 22 by the administrator. The grade of each grader may be calculated for each subject or for each subject unit. In addition, since the check result made into the above-mentioned pending answer with a reason is not lost in the decision of two choices, it is not included in the number of check results of check pending.

  Thereby, according to the grade according to the performance of each grader, the 1st scoring work and the scoring work of the pending answer can be shared, and a large number of answers can be measured more accurately within a limited time.

  The grade can be multi-stage. In this case, each grader can be assigned by graders belonging to each grade as appropriate, such as allowing graded work on pending answers only for those grades that have been suspended by graders belonging to a grade lower than their own grade. Work can be differentiated. Further, when the grader information is stored in the grader information database 28 and a new grader is registered, the answer and check box sheet used for the scoring work in the measurement of understanding tendency performed in the past, the virtual answer In addition, the scoring work can be performed by the check box sheet, and the grade can be determined based on the result.

  As described above, in order to measure the understanding tendency more accurately, it is important to reduce the original judgment based on the scorer's subjectivity. Therefore, it is also important to confirm the presence or absence of such judgment intervention in the check result by narrowing down to a set of check results that are highly likely to have intervened in the original judgment based on subjectivity. Therefore, answer data and check results that meet specific conditions may be searched and extracted from the answer data database 26 and the check result database 29, and the check results may be corrected.

  Further, for example, a grader of a grade higher than a predetermined level may be able to confirm whether or not the check pattern is correct with respect to a check result by another grader of a lower grade. That is, the server computer 22 prompts the grader to specify a problem number to be confirmed through the client computer 21, and obtains a check pattern by another grader below the grade of the grader. The check result is searched under the condition that the check result is found, the corresponding check result is extracted, and returned to the grader's client computer 21 together with the corresponding answer data. In this case, it is preferable that the search condition can be automatically set. For example, as shown in FIG. 7, when the confirmation work is selected by the “confirmation work” button 41c of the menu image 41, the scoring target selection image 60 shown in FIG. 9 is displayed. Next, a question number is obtained by prompting the designation by the grader, and other conditions necessary for the search are automatically set so that the search can be performed. The check result retrieved and extracted is confirmed by the grader, and the result of approval or denial is input. Such a result is transmitted to the server computer 22.

  Here, the rejected answer plan in which the rejected result is input is further confirmed by, for example, another grader of a higher grade, and the check result is corrected as necessary. For example, as shown in FIG. 7, a “denied scoring” button 41d of the menu image 41 can be used to select a scoring operation for the rejected answer plan, and in the same way, the target check result is automatically displayed according to the operation of the scoring person. Be able to search and extract.

  Further, for example, the answer to the question sentence that has already been checked in the first scoring work may be corrected by the scoring work of the pending answer. In that case, regarding the question text with the corrected answer, either the grader who performed the first scoring work or the scorer B who performed the pending work scoring work had a judgment error. Therefore, if there is a disagreement answer that has a disagreement in such an answer, the administrator and some of the graders that conform to it will check which is wrong, and if necessary, check results May be modified. Also in this case, as described above, the scoring operation of the mismatched answer can be selected by the “mismatched scoring” button 41e of the menu image 41, and the target check result can be automatically searched and extracted. Here, for a grader having a judgment mistake, information indicating that a judgment mistake has been made may be added to update the grader information. However, since “inconsistent scoring” is performed by the manager or a part of the scoring person corresponding thereto, it is set so that it cannot be selected by other scoring persons.

  As described above, when searching for a check result, it is preferable that a button that can automatically set a search condition can be similarly selected on the menu image 41 or the like so that a typical search can be easily performed.

  For example, when a judgment error is detected in the above-mentioned “mismatch scoring”, it is also desirable to search for other check results input by the grader who made the judgment mistake so that this can be confirmed. Therefore, an “other search” button 41f that can be selected by the administrator or some graders corresponding to the search can be added to the menu image 41 so that various searches including such cases can be freely performed. Good. By pressing the “other search” button 41f, the search condition is set in detail, and a transition is made to a search screen (not shown) in which check results can be searched.

  In addition, in such confirmation work or grade update by the grade assigning means 36, for a grader with many judgment mistakes or a grader with an extremely high check pending input ratio, for example, all the check results of the grader are reviewed. Or may take measures such as preventing further scoring. This also makes the measurement more accurate.

  Although exemplary embodiments according to the present invention have been described so far, the present invention is not necessarily limited thereto, and those skilled in the art will recognize various exemplary embodiments without departing from the scope of the appended claims. Examples and modifications can be found.

DESCRIPTION OF SYMBOLS 1 Comprehension tendency measurement system 21 Client computer 22 Server computer 23 Storage device 31 Question sentence data accommodation means 32 Answer data accommodation means 33 Grading answer transmission means 33a On-hold grade answer transmission means 34 Check result accommodation means

Claims (5)

Candidates by classifying the answers using a test that asks them to create an answer after presenting them with a descriptive question created in association with multiple expected answers reflecting the understanding tendency of each measurement item A measuring system for measuring the understanding tendency of a group,
Answer data storage means for storing answer image data after attaching an answer identification code for identifying each answer,
Regarding the answer, question sentence data storage means for storing question sentence data of a question sentence group consisting of a plurality of question sentences to be selected by two choices of positive or negative,
In response to a request from the grader terminal, a score answer sending means for sending the question sentence data and the selected answer image data to the grader terminal;
A check pattern or check pending comprising a combination of a scorer identification code for identifying a scorer who operates the scorer terminal from the scorer terminal, and a positive or negative combination of the answer identification code and the corresponding question sentence group of the answer A check result receiving means for receiving the check result of
A series of the grader identification code received by the check result receiving means, the answer identification code, and a check result accommodating means for sequentially accommodating the check results,
The scoring answer transmission means, in response to a request from a specific grader terminal, sends the question sentence data and the answer identification code of the answer whose check result is put on check to the questioner terminal. A comprehension tendency measuring system, further comprising a pending grade answer sending means for sending the corresponding answer image data.   The scoring answer transmission means displays the image of the answer on the scorer terminal and displays the question sentence group, and causes the scorer terminal to input positive or negative for each question sentence, 2. The understanding tendency measuring system according to claim 1, wherein a check pending input for the question sentence group is input instead of the input.   The check pending input ratio for each grader is calculated from the grader identification code, and includes grade assigning means for grading the grader, and the pending grade answer is transmitted based on the graded result of the grader 3. The understanding tendency measuring system according to claim 2, wherein the means specifies the grader terminal to which data is to be transmitted.   The check pattern is classified into several types so as to reflect the understanding tendency for each measurement target item, and the type corresponding to the check pattern input to the scorer terminal is presented to the scorer 3. The understanding tendency measurement system according to claim 2, further comprising a presentation means.   In the case of presenting the type corresponding to the predicted answer that is a correct answer to the grader, the type presenting means presents the grader that the answer is correct together with the type. The understanding tendency measurement system according to claim 4.