JP2000122517A - Electronic marking system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve an efficiency of marking by putting together answer images read from a lot of examination paper forms for each question, letting markers mark the answers to the questions in accordance with marker's capability, and simplifying the marking work. SOLUTION: A lot of answer images in one examination paper form are cut out by reading a lot of examination paper forms with an image scanner 12. The answer images are filed for each problem. This file is sent to a server 21. When a problem number is inputted at a client 22 for judgment, the answer image of the file of the problem is displayed on the CRT of the client 22. Looking at this display, a marker judges correct or false. The judgment result for each examinee is totaled in a server 21.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for handling images (images such as characters, symbols, figures, etc.) displayed or described on forms, and more particularly, to the solutions described in the answer column of a large number of answer sheets, especially An electronic scoring system suitable for evaluating the answer to a written question, an image reading system (more generally, a system for reading an image represented on a form), a scoring and tallying system, and a scoring system. The present invention relates to a grader management system for evaluating the grading quality of a computer, and a medium recording a program for controlling a computer included in these systems.

[0002]

BACKGROUND OF THE INVENTION There are many problems with scoring written answers. In particular, when a large number of answer sheets must be scored in a short period of time to obtain comprehensive evaluation results, as in the case of the kanji test, a large number of graders with high abilities must be secured.

[0003] One answer sheet describes answers to a plurality of questions. Naturally, the answers are different because the questions are different. Taking the kanji test as an example, there is a problem that a simple kanji with a small number of strokes is the correct answer, and a problem that a complex kanji with a large number of strokes is the correct answer. Therefore, the grader is required to be able to judge the answer to the highest level question. It is difficult to collect many high-level scorers, and it is necessary to pay for their ability.

[0004] When the number of graders increases, there is a high possibility that answers will be leaked to the outside, making it difficult to secure security.

[0005] Insufficient ability of the grader, misunderstanding of the grader,
The possibility of erroneous scoring due to poor physical condition or the like also increases as the number of graders increases. Efforts are also needed to ensure that the graders have a high level of scoring quality.

[0006]

DISCLOSURE OF THE INVENTION An object of the present invention is to simplify the task of marking (determining) answers and to increase the efficiency thereof.

Another object of the present invention is to provide a scoring system that does not necessarily require a large number of graders at the same level.

Another object of the present invention is to improve the security of the answer.

It is a further object of the present invention to improve the scoring quality as much as possible.

The answer sheet to which the electronic scoring system according to the present invention is suitably applied is provided with a plurality of answer columns for a plurality of questions and is provided with an identification code. The electronic scoring system reads the identification code of the plurality of answer sheets and the image of the answer column for each of these areas, and reads the image of the answer column obtained from the plurality of answer sheets by reading by the reading unit. A first storage means for storing, in association with each identification code, an identification code, in response to the designation of a specific question, reading an image of an answer column of the designated question from the first storage means; Means for displaying, means for inputting the evaluation of the image of the answer column displayed on the display means, and storing the evaluation inputted from the input means for each question in association with the identification code.
And a means for calculating an overall result in which the evaluations stored in the second storage means are combined for each identification code with respect to the questions included in one answer sheet.

This electronic scoring system can be realized in various forms. One is, in principle, a system composed of one computer system and one image scanner. In this case, the reading means corresponds to an image scanner (controlled by a computer as necessary). First storage means, display means, input means, second
Are implemented by a computer system (including a computer main body, an external memory, a display device, an input device, and the like). Two or more image scanners may be provided as necessary, or two or more computer systems may be provided.

Second, the above-mentioned computer system is constituted by a client / server system. The image scanner is controlled by a server computer. The first storage unit, the second storage unit, and the calculation unit can be realized by a server computer, and the display unit, the input unit, and the second storage unit can be embodied by a client computer. The second storage means may be interpreted as any one of the storage device of the server computer and the storage device of the client computer.

Third, the electronic scoring system comprises an image reading system and a scoring and tallying system. The image reading system is provided with a reading unit and a first storage unit. The first storage means, the display means, the input means, the second storage means and the calculation means are provided in the scoring and tallying system. The scoring and tallying system can be realized by a client / server system as in the embodiment described later. The first storage means may be interpreted as any one of the storage device of the image reading system and the storage device of the scoring system.

The association between the image in the answer column and the identification code of the answer sheet, or the association between the evaluation and the identification code can be performed in the reading order as in the embodiment described later, or can be associated with the image or the evaluation in the answer column. It can be realized by attaching an identification code or using a pointer or the like.

The evaluation can be expressed as correct, incorrect, etc., can be expressed in a plurality of steps (excellent, good, acceptable, unacceptable), or can be expressed as a score.

[0016] The comprehensive result obtained by summarizing the questions included in one answer sheet for each identification code can be expressed by the total score of all the questions, or can be obtained by discriminating at one or more levels. It could also be expressed as a result (eg, pass, fail, or excellent, good, good, bad).
If the problem includes not only the answer to the descriptive answer but also the answer to another form of answer (for example, a multiple-answer answer), the total result of the answer to the descriptive answer alone may be calculated. , A total result including both the answer in the descriptive formula and the answer in another format may be calculated.

According to the present invention, the images of the plurality of answer columns of each answer sheet are read and sorted for each question, and an answer image file for each question is created. When a specific question is designated, an answer image of the question is displayed on the display means. Therefore, the grader only needs to grade (determine) the answer to one of the questions displayed on the display means.

Since the grader only has to score the answer to one question, the work is simplified, and the efficiency of answer scoring is improved. In addition, erroneous determinations are reduced. Graders with a higher grader can be assigned grades for higher grades, and graders with a lower grader can score grades for the appropriate level. Since questions to be assigned can be assigned according to the grader's ability, it is not necessary to gather only highly qualified persons. Therefore, it is rational because it is easy to secure scorers and it is only necessary to pay a fee according to the ability.

The electronic scoring system can be implemented using one or more computer systems.
The computer system can be installed in one place (or an appropriate number), and the scorer performs the scoring on the spot, thereby increasing security. Since at least the written answer is stored in the computer system, there is no problem that the grader takes the answer sheet out.

The evaluation of the image in the answer column input by the grader is stored in association with the identification code. Finally,
The stored evaluations are combined for each identification code to calculate an overall result. In this way, an overall result for each answer sheet (answerer) is finally obtained. The overall result will be output by printing or the like as needed.

The present invention also provides a reading system for reading images from a plurality of answer columns of a plurality of answer sheets and filing them in association with identification codes for each question. This reading system does not require scoring. Therefore, the reading target of the reading system is not limited to the answer sheet. In this sense, the term report is used instead of the answer sheet. The form is a concept including various sheets including answer sheets, slips, sheets and the like.

According to the system for reading and filing a plurality of images of a form according to the present invention, a form provided with a form identification code is provided with a plurality of areas each having an area identification code. Reading means for separately reading the images of a plurality of areas from the printer, and means for controlling the file of the area image read by the reading means in association with the form identification code for each area identification code. Things.

This system reads each image of a plurality of areas on a form, and files the images (data) for each area identification code.

The present invention further provides a scoring and tallying system. This scoring and tallying system is a first storage means for storing images of answer columns obtained from a plurality of answer sheets for each question in association with an identification code, and in response to a specific question being designated, Means for reading and displaying the image of the answer column of the designated question from the first storage means, means for inputting the evaluation of the image of the answer field displayed on the display means, and the evaluation input from the input means. The second storage means for storing each question in association with the identification code, and the evaluation results stored in the second storage means are combined into a comprehensive result for each identification code for the questions included in one answer sheet. It has means for calculating.

In the above-described form reading file system, a file obtained when using an answer sheet as a reading target is stored in the first storage means. Assuming that there is a file for each question of the image of the answer column, the scoring and tallying system according to the present invention
The grader determines the answer to the question that is suitable for the grader, and the results of the determination are compiled for each solver to obtain the overall result of the solver.

In one embodiment, the scoring and tallying system further includes third storage means for storing the correct answer image. The input means is provided with means for setting whether to display a correct answer image. The display means reads out the correct answer image from the third storage means and displays it when the display of the correct answer image is set by the setting means.

The scorer can display the correct answer image on the display screen as needed, and judge the answer image while comparing it with the answer image, so that the possibility of erroneous judgment due to misunderstanding or the like can be reduced. .

In another embodiment, the scoring system is implemented by a client / server system. That is, this scoring and tallying system includes at least one server computer and at least one client computer connected to the server computer via a communication line. In the memory of the server computer, images of answer columns obtained from a plurality of answer sheets are stored for each question in association with identification codes of the answer sheets. The client computer receives the image of the answer column of the specified question from the server computer in response to the designation of the specific question, and displays the received image of the answer column on the display device by a predetermined number. Then, the evaluation of the image in the answer column input from the input device is sent to the server computer.
The server computer calculates an overall evaluation summarizing the evaluation of the image of the answer column sent from the client computer for each identification code for the questions included in one answer sheet.

By providing a plurality of client computers, a plurality of graders can simultaneously grade.

The present invention further provides a management system for objectively determining the grader's scoring quality. As a premise of the grader management system, a management answer sheet in which an answer whose evaluation has been determined in advance corresponding to the identification code is mixed in the answer sheets of a plurality of examinees. The management system stores the evaluation result of the plurality of answer sheets by the grader in association with the identification code of the grader and the identification code of the answer sheet. It is provided with means for determining whether the evaluation result of the answer sheet is correct, and means for counting the determination results of the determination means for each identification code of the grader.

The grader performs the evaluation without recognizing whether the examinee's answer sheet or the management answer sheet. The evaluation of the answers on the administrative answer sheet is predetermined. The degree to which the grader's evaluation matches (is similar to) the determined evaluation is tabulated. The quality of the grader's assessment will be measured objectively. By giving the measurement result to the grader to be used for the next evaluation, the quality of the evaluation (scoring) is improved as a result.

The present invention further provides a recording medium storing a program for operating a computer of the above-described scoring and tallying system, particularly a client computer. This recording medium, in response to the designation of a specific question, causes an image of the answer column of the designated question to be obtained, displays the acquired images of the answer column by a predetermined number, and displays the answer in the input answer column. Read the image rating,
A program for controlling the scoring computer to store the read evaluation in association with the identification code of the answer sheet is stored.

The present invention also provides a recording medium storing a program for operating a computer of the above-mentioned grader management apparatus. In this recording medium, a score sheet for a plurality of examinees, in which a management answer sheet in which an answer whose evaluation has been determined in advance corresponding to the identification code is entered is included in the answer sheets of a plurality of examinees. The evaluation result is read from the storage means storing the result of the evaluation by associating the evaluation result with the identification code of the grader and the identification code of the answer sheet, and it is determined whether or not the evaluation result read on the management answer sheet is correct. It stores a program for controlling the grader management computer so that the determination results are totaled for each grader identification code.

The recording medium is FD, CD-ROM, M
It includes not only portable recording media such as D and MOD, but also hard disks, magnetic tapes, semiconductor memories, and the like.

[0035]

FIG. 1 shows the overall configuration of an electronic scoring system.

The electronic scoring system includes an image (answer) reading system 10 and a scoring and tallying system 20.

The image reading system 10 comprises an image scanner 12 and a computer 11 for controlling the scanner, and a storage device 13 is connected to the computer 11.

This electronic scoring system is particularly suitable for a written answer. The answer sheet to be scored is provided with a plurality of answer columns, as will be clear from the description below, and the answerer (examinee) describes the answer in each answer column. It goes without saying that the answer sheet may have an answer column other than the description type, for example, a column for answering by marking.

Under the control of the computer 11, the image scanner 12 reads answers (images) described on answer sheets (including a grader's answer sheet for quality control, which will be described later), and reads the read image for each answer column. And output in a certain order. The control computer 11 organizes and stores the image data obtained from the image scanner 12 in the storage device 13. The storage device 13 is a hard disk device, F
It is realized by a D drive, an MO drive, a magnetic tape writing / reading device, and others.

The scoring and tallying system 20 comprises a scoring server 21 and a plurality of judgment client computers 22, which are connected by a LAN, another communication network or a transmission line. The storage device 23 is connected to the scoring server 21. The storage device 23 is also realized by a hard disk device, FD drive, MO drive, magnetic tape writing / reading device, and the like.

The image data (answer image) stored in the storage device 13 of the image reading system 10 is ISD
N, transmitted to the scoring server 21 of the scoring and tallying system via other networks, and stored in the storage device 23.
The image data stored in the storage device 13 of the image reading system 10 is temporarily recorded on a recording medium such as FD, MO, or magnetic tape, and this recording medium is transported by a person to a place where the scoring and counting system 21 is provided. You may. The image data in the recording medium is read by the scoring server 21 and stored in the storage device 23.

The grader uses the judgment client computer 22 to judge correctness of each answer. As will be apparent from the following description, on the display screen of the display device of the client computer 22 for determination, answers for one question (question) cut out from a large number of answer sheets are sequentially displayed in plurals. You. Therefore, the grader concentrates the answers on one question. The scoring server 21 stores the answer image in response to a request from the client computer 22 for judgment.
And sends it to the client computer 22. The scoring server 21 is a client computer 22
The results determined in (1) are tabulated for each answer sheet (examinee, examinee number) to calculate a scoring result (total score). The scoring server 21 further calculates data (correct judgment rate) indicating the scoring quality of the grader.

FIG. 2 shows an example of the answer sheet. This is a typical answer sheet for the kanji test.

In this test, it is assumed that there are m questions (problems) as a problem of a descriptive formula. In response to m questions,
The answer sheet has m answer columns. Suppose there are n answer sheets. The answer column (or answer) of the i-th (question i) of the answer sheet j is represented by (ji) (j = 1 to n, i = 1 to m).

The answer sheet has an entry column for data for identifying the answer sheet. In this box, the class, times,
The year and examination number are pre-filled (printed). There is also a space for test takers to write their names.

There are two types of answer sheets. One is written by the test taker (this is generally referred to as the answer sheet unless otherwise specified). The examinee's examination number is printed (filled in) on the answer sheet in advance, and the examinee's name is entered by the examinee. The second is a grader's answer sheet for quality control. The examination number on the quality control answer sheet is predetermined, and it is known in advance that the answer is correct or incorrect. For example, below 2
On a quality control answer sheet on which an examination number having all digits of 0 is printed (written), correct answers are described in all the answer columns. In addition, an incorrect answer is described in all the answer columns on the quality control answer sheet on which the examination number whose last two digits are all 9 is printed (written). It is not necessary to enter the name on the quality control answer sheet, but an appropriate name may of course be entered. The examinee number on the quality control answer sheet is excluded from the examinee number assigned to the examinee. It is possible to distinguish between the examinee's answer sheet and the quality control answer sheet by the examinee number.

FIG. 3 shows an example of a directory provided in the storage device 13 for storing image data read by the image scanner 12 of the image reading system 10.

The kanji test is conducted several times a year.
The year and times indicate the year in which the kanji test was conducted and the number of times in that year. In addition, there are grades 1 to 7 for kanji tests. This is the class. The type of test is specified by year, times and class. However, it goes without saying that the data specifying the type of test is not limited to the year, times and grades.

A large number of sheets (for example, 1
There are 10,000 to tens of thousands of answer sheets (including quality control answer sheets). These answer sheets are divided into lots.
One lot is, for example, 1000 sheets. In each lot, quality control answer sheets are randomly mixed (for example, 1 to 1).
About several percent). The examination numbers of 1000 answer sheets included in one lot need not be consecutive numbers at all.

A holder is created for each lot. The holder includes files for each question (files for answer images) (files for question 1, files for question 2, ..., files for question i, ..., files for question m) and files for additional information.

FIG. 4 shows an example of the contents of each file included in the holder of the first lot (lot number 1) (lot 1) (the contents of the holders of other lots are the same). .

The file of the question 1 (the file of the image of the answer to the question 1) is all the answer sheets included in the lot 1 (the answer sheet for the quality control is mixed as described above).
The image of the answer to the question 1 (answers (1-1) to (1000-1)) read and cut out by the image scanner 12 is stored. Similarly, the file of the question i is read from all the answer sheets included in the lot 1 and the answer (answer (1-i) to
(1000-i)). The additional information file can be obtained by entering the image scanner from the answer sheet identification data entry fields of all answer sheets included in lot 1.
The examination number recognized by the computer 11 based on the image data obtained by being read by the computer 12, the name data cut out from the image data, and other data (these are referred to as additional information) are stored.

In these files, the positions where the answer image and the additional information are respectively arranged are predetermined (for example, the order of scanning in the image scanner 12). In FIG. 4, sequence numbers are shown at these positions for convenience. The answer image and the additional information having the same sequence number in all the files in the holder of the lot 1 are obtained from the same answer sheet. In all the processes, the order of the answer image and the additional information in each file is maintained.

The image data output from the image scanner 12 is data compressed (for example, MMR
Compression format). This image data is, for example, TIFF
Stored in a file in format.

An example of the file for question i is shown in FIG. The attribute of the file is image data,
Includes data representing the number of stored images and the like.
Next is a pointer to the first image data storage location. The position indicated by the pointer has an attribute of the image data, and subsequently, a pointer to the next image data storage position and the image data are stored. Since the attribute of the image data includes the size of the attribute data, it is recognized from this size where the image data starts.

FIG. 6 shows an operation procedure of the image reading system 10 (the image scanner 12 and the control computer 11).

The answer sheet for one lot including the answer sheet for quality control is put on the stacker of the image scanner 12 by the operator (step 101). The image scanner 12 starts operating.

In the control computer 11, the operator inputs the lot number, the year, the number of times and the class (step 102). It goes without saying that the year, number and grade of the answer sheets included in one lot are all the same.

The image scanner 12 scans the answer sheet one by one and reads the image (step 103).

The data indicating the positions of the identification data entry field and the answer field on the answer sheet are stored in the control computer 11.
To the image scanner 12 in advance. The image scanner 12 cuts out the images included in the identification data entry field and the answer field on the image read from the answer sheet based on the position data. After the extracted image data is compressed (the data need not necessarily be compressed), the image data in the identification data entry field, and then the image data in the answer field of question 1 are fixed. Data, the image data in the answer column for question 2,..., The image data in the answer column for question m), and are given to the control computer 11 (step 104).

The control computer 11 expands the image data in the identification data entry column and recognizes the mark data (examination number, year, number, class). Check that the recognized mark data matches the year, times, and grades entered earlier, and add the recognized examination number and name image data to the last part of the additional information file for the corresponding lot number entered earlier. Add to Further, the image data (compressed image data) of each answer column is added to the last part of the file of the corresponding problem of the corresponding lot number (step 105).

The processing of steps 103 to 105 is repeatedly executed for all the answer sheets for one lot set in the stacker (step 106).

When the above-described processing is completed for one lot of answer sheets, holders for one lot (from the file of question 1 to the file of question m and the additional information file) are completed as shown in FIG. Step 106).

The number of records (the number of answer images or the number of additional information) included in the file created in this way is checked (step 107). If the number of answer sheets read by the image scanner 12 and the number of records respectively stored in the created file all match, the image reading processing for one lot is completed.

The image reading process shown in FIG. 6 is executed for all the answer sheets of the year and the time when the image reading process is executed for each grade. When the number of answer sheets belonging to one class is equal to or more than one lot, the answer sheets are executed for each lot.

As described above, the data representing the answer images of all answer sheets are classified into classes and lots, and classified into questions in each lot, and stored in the storage device 13 of the image reading system 10. (See FIG. 3 described above.
See the directory shown below).

The answer image data stored in the storage device 13 of the image reading system 10 is transferred to the storage device 23 of the scoring server 21 of the scoring and tallying system 20 through the network or the recording medium as described above. It is.

FIG. 7 shows an example of a directory provided in the storage device 23 of the scoring system 20. The directory created in the storage device 13 of the image reading system 10 is copied as it is, and, for each lot, a file for judging each question (a file for storing the result of a grader judging the correctness of the answer to each question). Information is additionally described.

In the holder of each lot provided in the storage device 23 of the scoring and tallying system 20, in addition to the file (of the answer image data) and the additional information file for each question (these files are stored in the image reading system). The files in the storage device 10 are copied as they are), and a file for determining each problem is included. The determination file for each question stores the determination result for the answer to each question for each answer.

FIG. 8 shows the contents of the holder of the lot 1 provided in the storage device 23. The file of each question and the file of additional information are copies of the storage device 13 as they are. From the file of judgment of problem 1 to problem m
M files are newly provided up to the determination file (only the determination file for the problem 1 and the file for the determination of the problem i are shown).

Also in these judgment files, for each question, data representing the judgment result of the answer sheet of one lot of answer sheets is stored in a certain order (as described above, the order of reading the answer sheets by the image scanner 12). ) Is stored. The data representing the determination result is accompanied by a grader ID for identifying the grader who made the determination. One record is composed of the data representing the determination result and the grader ID.

The judgment results include a correct answer, an incorrect answer, no entry, and suspension. With respect to the suspension, another grader re-determines or is determined based on the answer sheet, so that it is finally determined to be a correct answer or an incorrect answer.

FIG. 9 shows the judgment processing using the judgment client computer 22 in the scoring and tallying system 20.
This will be described with reference to FIG.

The grader inputs his / her grader ID to the determination client computer 22 together with the start of the determination. The client computer 22 notifies the scoring server 21 of the start of the judgment together with the grader ID (step 111). Subsequently, the grader enters in the client computer 22 the grade of the question to be graded and the number of the question. These input data are also sent from the client computer 22 to the scoring server 21 (step 112).

The scoring server 21 reads the answer image data of the designated question number of the designated class from the storage device 23 and transmits it to the client computer 22 (step 113). A predetermined amount (for example, one lot) of answer image data is downloaded to the client computer 22 and stored in its memory.

In the client computer 22,
Based on the answer image data sent from the scoring server 21, an appropriate number (eg, 12) of answer images are displayed on the display screen of the display device (step 114).
).

FIG. 11 shows an example of the answer image display screen on the client computer 22. In this example, 11 answer images are displayed in two rows. The character in the lower rightmost column is a correct answer image. This correct answer image is displayed only when the grader presses a specific key described later.

The number at the upper left of this screen is the answer ID, which represents the year, times, class, question number, order of answer images, and the like. The date is displayed at the top right of the screen.

The cursor is initially in the upper leftmost column. The grader looks at the answer image in this column, makes a decision, and inputs the result using the numeric keypad (step 115). As described above, the judgment includes a correct answer, an incorrect answer, a no answer, and a hold (those for which the grader cannot judge correctness). The grader presses 1 on the numeric keypad when the answer is correct, 2 when the answer is incorrect, 3 when the answer is no answer, and 4 when the answer is pending. The number representing the judgment result input by the grader is displayed in a box below the answer image. In addition, a symbol representing the determination result is displayed at the upper left position of the answer image. Circles indicate correct answers, crosses indicate incorrect answers, triangles indicate no answers, and squares indicate pending.

Each time the grader presses one of the numbers 1 to 4 on the numeric keypad, the cursor moves to the next answer image column (step 116).

When the judgment on all the answer images displayed on one screen is completed, the grader generally proceeds to
Click Y in Y / R at the bottom right of the screen. Then, the character of Y is displayed in the square section at the lower right of the screen,
The next 12 (or 11) answer images are displayed on the screen.

When the grader wants to re-score the answer image on the displayed screen, the Y / R
Click on R of. The cursor returns to the first answer image field, and you can start scoring again.

When the grader presses a specific key (for example, PF12 key) on the keyboard, if the correct answer image has already been displayed as shown in FIG. 11, the correct answer image disappears from the next screen. When 12 answer images are displayed on the screen and the above-mentioned specific key is pressed when the correct answer image is not displayed, the correct answer image is displayed from the next screen as shown in FIG. 11 (step 117). , 118). Data representing the correct answer image is sent from the server 21 to the client computer 22 together with the answer image data.
Needless to say, it is stored in the memory. Preferably, the background color of the correct answer image column is different from the background color of the answer image column.

Steps 114 to 118 are repeated until all the answer images (for example, one lot) downloaded from the server 21 have been graded. When the judgment on all the downloaded answer images is completed, the judgment result is transmitted to the scoring server 21 (steps 119 and 120). In the scoring server 21, the judgment result is stored in the storage device 23 together with the grader ID as a judgment file of a problem corresponding to each lot as shown in FIGS.

If all the judgments for the specific question number have not been completed yet (step 121), the server 21
Then, the next answer image (of the lot) is downloaded to the client computer 22 (step 113), and the same processing (scorer determination operation) is repeated.

When the determination for all the answer images stored in the storage device 23 for the specific question number is completed (step 121), the grader inputs the end of the work (step 122). The completion of the determination of the answer image for one question number is notified from the client computer 23 to the scoring server 21.

In the scoring server 21, the final determination of the answer image determined to be suspended and the input operation thereof are performed. The specific grader is the client computer 22
The answer image determined to be on hold is stored on the server 21 using
May be downloaded to the client computer 22, the client computer 22 makes a final determination, and sends the result to the server 21.

The judgment may be made by using a score instead of a right or wrong. In this case, the grader inputs the score using the numeric keys on the keyboard. The input score is displayed in a section below the answer image on the display screen.

In any case, when the judgment on all the questions of all the lots of a specific class is completed, the scoring and counting for each examinee is performed by the scoring server 21. As shown in FIG. 7, the determination result is stored in a determination file for each problem. The examination number is stored in the additional information file. The examination number in the additional information file and the judgment result of the answer image in the judgment file of the answerer of the examination number are associated with each other in the arrangement order. Therefore, the judgment results of each judgment file are rearranged for each examination number.

FIG. 12 shows the result of this rearrangement. The number of correct answers in the judgment result is counted for each examination number, and the total score is calculated based on the count value. The total score is finally printed out along with the examination number (and name). In the answer sheet, there are other types of answer columns other than the descriptive answer column (for example, a mark-type one that marks the correct answer in a multiple-choice question), and the answers in these answer columns are also judged. When the total score is calculated, these determination results are generally considered (they need not be considered).

As described above, the image reading system 10
The answer sheet read in step 2 also includes a quality control answer sheet. Therefore, the answer image in which the grader judges correctness using the client computer 22 in the scoring and tallying system 20 includes the answer described in the quality control solution. In addition, the last two digits of the examination number of the quality control answer describing only the correct answer are 00, and the last two digits of the examination number of the quality management answer describing only the wrong answer are 9
9 By utilizing these facts, the scoring server 21 manages the scoring quality of the scoring person.

FIG. 13 shows the quality control totaling process executed by the scoring server 21. This process is performed at any time by the administrator.

First, the range (gradation period, grade, problem number, lot number, etc.) in which quality control is to be totaled is input to the scoring server 21 (step 131).

In response, the grader server 21 reads the test number, the judgment result data, and the grader ID in the input range from the storage device 23 and transfers them to its internal memory (step 132). The following processing is performed on the data stored in the internal memory.

Examination number whose last two digits are 00 and 9
It is checked whether it is 9 or not (steps 133 and 134). If the last two digits of the test number are 00, the answer is on the quality control answer sheet in which the correct answer is described, so it is checked whether or not the grader has determined that the answer is correct (step 135). If it is determined that the answer is correct, 1 is incremented as the number of correctly determined scorer IDs that have scored (step 137). If it is determined that the answer is wrong, 1 is counted up as the number of erroneously determined for the grader ID (step 138). Similarly, if the judgment result corresponding to the examination number whose last two digits are 99 is judged to be an incorrect answer (step 136).
), 1 for the number judged correctly according to the grader ID
In addition (step 137), if it is determined that the answer is correct, 1 is added to the incorrectly determined number corresponding to the grader ID (step 138).

The above processing is repeated for all the data read into the internal memory (step 139). When the processing is completed for all the data, the grader quality table is printed out (step 140).

FIG. 14 shows an example of the grader quality table printed out. The name of the grader, the number of checked data (the number of management data), the number of correctly judged (the number of correct judgments), and the correct judgment rate are displayed.

The processing shown in the flowcharts of FIGS. 9 and 10 is performed by the client computer 22 in accordance with the determination processing program. This program is CD-R
What is recorded on the OM, FD, or the like is installed in the client computer 22 or downloaded from the scoring server 21 and stored in the memory (including the hard disk) of the client computer 22. Similarly, the processing shown in FIG. 13 is executed in the scoring server 21 according to the quality management totaling processing program. This program, which has been stored in a recording medium such as a CD-ROM or FD, is installed in the scoring server 21 and stored in its memory (including a hard disk). The determination processing program and the quality management totalization processing program may be recorded on a single recording medium such as a CD-ROM or FD, or may be recorded on separate recording media.

As described above, since the grader continues to judge the answer to one question, the operation is simplified, and the efficiency of answer scoring is improved. Since the correct answer image can also be displayed on the screen, it is possible to reduce erroneous judgment.

[0100] Since the determination of the answer to the question corresponding to the grader's level can be assigned to the grader, graders of various levels can be employed. That is, it is not necessary to prepare only the highest grader.

Since the client computer can be arranged at one place or a small number of places and the grader can perform the judgment work only at that place, sufficient security can be ensured.

Using the quality control answer, the judgment quality (scoring ability) of the grader can be objectively evaluated. By feeding back the grader's quality to the grader, the grader's grader quality is improved.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an entire electronic scoring system.

FIG. 2 shows an example of an answer sheet.

FIG. 3 shows a directory in a memory of the image reading system.

FIG. 4 shows an example of the contents of a holder for an answer image for one lot.

FIG. 5 shows an example of a file structure (TIFF file) for each problem.

FIG. 6 is a flowchart showing the operation of the image reading system.

FIG. 7 shows a directory of a memory of the scoring system.

FIG. 8 shows an example of the contents of a holder for a solution image and a determination result for one lot.

FIG. 9 is a flowchart illustrating a procedure of a determination process in the scoring and tallying system.

FIG. 10 is a flowchart illustrating a procedure of a determination process in the scoring and tallying system.

FIG. 11 shows an example of a display screen.

FIG. 12 shows an example of a scoring result file.

FIG. 13 is a flowchart showing a procedure of a quality management totaling process.

FIG. 14 shows an example of a grader quality table.

[Explanation of symbols]

 10 Image reading system 11 Control computer 12 Image scanner 13 Storage device 20 Scoring system 21 Scoring server 22 Client computer for judgment 23 Storage device

Claims (8)

[Claims] An answer sheet provided with an identification code is provided with a plurality of answer columns for a plurality of questions. The identification code of such a plurality of answer sheets and the image of the answer column are stored in those areas. Reading means for reading each answer sheet, first storage means for storing images of answer columns obtained from a plurality of answer sheets by the reading means in association with identification codes for each question, and a specific question is designated. Means for reading and displaying the image of the answer column of the designated question from the first storage means in response to the request, means for inputting the evaluation of the image of the answer field displayed on the display means, and input means The second storage means for storing the evaluation input from the user in association with the identification code for each question, and the evaluation stored in the second storage means for the questions contained in one answer sheet. An electronic scoring system comprising: means for calculating a total result summarized for each identification code. 2. A form provided with a form identification code is provided with a plurality of areas each having an area identification code.
Reading means for separately reading the images of a plurality of areas from such a plurality of forms, and controlling the image of the areas read by the reading means to be filed in association with the form identification codes for each area identification code. Means to do,
A system for reading and filing multiple images of a form provided with. 3. A first storage means for storing images of answer columns obtained from a plurality of answer sheets in association with identification codes for each question, and in response to designation of a specific question, designation of the answer. Means for reading and displaying the image of the answer column of the question from the first storage means, means for inputting the evaluation of the image of the answer field displayed on the display means, and identifying the evaluation input from the input means. A second storage means for storing each question in association with a code, and an evaluation result stored in the second storage means is calculated for each identification code for a question included in one answer sheet to calculate an overall result. Scoring system with means to perform 4. The apparatus according to claim 1, further comprising third storage means for storing a correct answer image, wherein said input means includes means for setting whether or not to display the correct answer image, and said display means displays said correct answer image by said setting means. 4. The scoring and counting system according to claim 3, wherein when is set, the correct answer image is read from the third storage means and displayed. 5. At least one server computer and at least one client computer connected to the server computer via a communication line, wherein the memory of the server computer includes:
The image of the answer column obtained from the plurality of answer sheets is stored for each question in association with the identification code of the answer sheet, and the client computer responds to the designation of the specific question, The image of the answer column of the designated question is received from the server computer, the received image of the answer column is displayed on the display device by a predetermined number, and the evaluation of the image of the answer column input from the input device is evaluated by the server computer. The server computer calculates an overall evaluation summarizing the evaluation of the image of the answer column sent from the client computer for each identification code for the questions included in one answer sheet. Is a scoring system. 6. A score sheet for a plurality of test-takers, wherein a management answer sheet in which an answer whose evaluation has been determined in advance corresponding to the identification code is entered in a plurality of test-taker answer sheets. Storage means for storing the result of evaluation by the grader in association with the identification code of the grader and the identification code of the answer sheet, reading the evaluation result from the storage means, and determining whether the evaluation result for the management answer sheet is correct And a means for summing up the results of the judgment by the judgment means for each identification code of the grader. 7. In response to the designation of a specific question, an image of the answer column of the designated question is obtained, the acquired images of the answer column are displayed by a predetermined number, and the input answer column is displayed. A recording medium storing a program for controlling a scoring computer to read an evaluation of an image and store the read evaluation in association with an identification code of an answer sheet. 8. A plurality of answer sheets for management, in which an answer sheet for management in which an answer whose evaluation has been determined in advance is entered corresponding to the identification code is mixed in the answer sheets of a plurality of examinees. The evaluation result is read from the storage means storing the result of the evaluation by associating the evaluation result with the identification code of the grader and the identification code of the answer sheet, and it is determined whether or not the evaluation result read on the management answer sheet is correct.
A recording medium storing a program for controlling a grader management computer so as to total the determination results for each grader identification code.