WO2007018344A1

WO2007018344A1 - System and method for managing questions of test to prevent cheating using random arrangement of answers

Info

Publication number: WO2007018344A1
Application number: PCT/KR2006/000530
Authority: WO
Inventors: Young-San Yoo; Hyun-Sik Choi
Original assignee: Uwayedu Inc.
Priority date: 2005-08-10
Filing date: 2006-02-15
Publication date: 2007-02-15

Abstract

A system for managing questions of a test arranges answers of each question in random to prevent cheating. A main test paper setting unit sets questions of a main test paper and answers dependent on each question. A random test paper generating unit endows a unique management number of a random test paper newly generated by arranging answers based on the main test paper, and then changes an arrangement order of answer numbers to generate random answer numbers. A correct answer mapping unit changes a correct answer number of the main test paper into that of the random test paper since the answer numbers of the main test paper are changed into the random answer numbers. A correct answer distribution ratio verifying unit accumulates the changed correct answer numbers to verify whether the correct answer numbers are regularly distributed, and then amends the arrangement of the correct answer numbers.

Description

SYSTEM AND METHOD FOR MANAGING QUESTIONS OF TEST TO PREVENT CHEATING USING RANDOM ARRANGEMENT OF ANSWERS

Technical Field

[1] The present invention relates to system and method for managing questions of a test to prevent cheating using random arrangement of answers, and more particularly to system and method for managing questions of a test in which answers of each question of the test are arranged in random when questions are set and printed so as to prevent cheating of examinees, and answers of an individual test paper set as mentioned above is constructed into a database. Background Art

[2] A test is a system executed for verifying a learning ability or a necessary qualification of an examinee, which is mainly directed to evaluate an accurate ability of the examinee. Thus, it is most important to evaluate ability of the examinee impartially.

[3] However, there are always troubles every year in discriminating fairness of a test since cheatings occur in various qualification or entrance examinations, held by institution agencies or state organs.

[4] In particular, though there have been unfair examinees in the national qualification exams or the national scholastic achievement examination for the college entrance, which requires fairness above all, there are systematic limits in exposing a cheating, so many tests are undergone without finding out such a cheating. In addition, a cheating is sometimes found out after a test is undergone completely, so an agency holding the test and all of examinees are unexpectedly damaged, which may be spotlighted as a serious social problem. As mentioned above, a cheating in a test destroys public trust and reliability of the test, thereby giving unrecoverable losses in aspect of costs and time. Disclosure of Invention

Technical Problem

[5] The present invention is designed in consideration of the above problems, and therefore it is an object of the invention to provide a technique of managing questions of a test, which has systematic tools for preventing any cheating in the test managing system itself in a way that answers of individual questions are arranged in random so that individual examinees take unique test papers. Technical Solution

[6] In order to accomplish the above object, the present invention provides a system for managing questions of a test to prevent cheating by arranging answers of each question in random, which includes a main test paper setting unit for setting questions (or, examination questions) of a main test paper and at least one answer (or, selective answer) dependent on each of the questions; a random test paper generating unit for endowing a unique management number of a random test paper newly generated by arranging answers based on the main test paper, and then changing an arrangement order of answer numbers of the main test paper to generate random answer numbers; a correct answer mapping unit for changing a correct answer number of the main test paper into a correct answer number of the random test paper with respect to the movement of a correct answer number caused by changing the answer numbers of the main test paper into the random answer numbers; and a correct answer distribution ratio verifying unit for accumulating the correct answer numbers changed by the correct answer mapping unit to verify whether the correct answer numbers are regularly distributed, and then amending the arrangement of the correct answer numbers.

[7] In another aspect of the present invention, there is also provided a method for managing questions of a test using a question managing system that arranges answers of each question in random to prevent cheating, which includes (1) setting and storing questions (or, examination questions) of a main test paper and at least one answer (or, selective answer) dependent on each of the questions; (2) endowing a unique management number of a random test paper newly generated by arranging answers based on the main test paper, and then storing random answer numbers generated by changing an arrangement order of answer numbers of the main test paper; (3) changing a correct answer number of the main test paper into a correct answer number of the random test paper with respect to the movement of a correct answer number caused by changing the answer numbers of the main test paper into the random answer numbers, and then storing the correct answer number of the random test paper; and (4) accumulating the correct answer numbers per each number to verify whether the correct answer numbers are regularly distributed, and then amending and storing the arrangement of the correct answer numbers. Brief Description of the Drawings

[8] These and other features, aspects, and advantages of preferred embodiments of the present invention will be more fully described in the following detailed description, taken accompanying drawings. In the drawings:

[9] FIG. 1 shows an inner configuration of a question managing system of a test according to an embodiment of the present invention;

[10] FIG. 2 shows a network configuration of the question managing system of a test under to a client-server environment according to an embodiment of the present invention; [11] FIG. 3 shows an example of random arrangement of answers, made by the question managing system of a test according to an embodiment of the present invention; [12] FIG. 4 shows an inner configuration of a random test paper printing unit according to an embodiment of the present invention; [13] FIG. 5 is a flowchart illustrating a question managing method of a test according to an embodiment of the present invention; [14] FIG. 6 is a detailed flowchart illustrating the process of generating a random test paper according to an embodiment of the present invention; [15] FIG. 7 shows an example of a random test paper generating program according to an embodiment of the present invention; [16] FIG. 8 is a detailed flowchart illustrating a random test paper printing process according to an embodiment of the present invention; and [17] FIG. 9 shows an example of a main test paper and a random test paper according to an embodiment of the present invention.

Best Mode for Carrying Out the Invention [18] Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. Terms frequently used in the specification are defined in the following table 1. [19] Table 1

[20] FIG. 1 shows an inner configuration of a question managing system 1 of a test according to an embodiment of the present invention.

[21] The question managing system 1 of this embodiment includes a main test paper setting unit 11, a random test paper generating unit 12, a correct answer mapping unit 13, and a correct answer distribution ratio verifying unit 14. In addition, there are provided a main test paper DB 111, a random test paper DB 121, and a correct answer DB 131, as a DBMS structure.

[22] The main test paper setting unit 11 receives information related questions (or, examination questions) composing a test paper and information related to at least one answers (or, selective answers) dependent on each question from an examiner, and stores the information in the main test paper DB 111. A data structure of the main test paper DB 111 is defined in the following table 2, but not limitedly.

[23] Table 2

Main test Test subject code + Question code + Question data + Numbers of paper DB answers + (Answer number + Answer data) + Correct answer number +

[24] The random test paper generating unit 12 changes arrangement of answers of each question in the same question sequence based on the main test paper set by an examiner and then stores the arrangement order information in the random test paper DB 121. The random test paper DB 121 stores only the arrangement order information of answer numbers, and contents of the answers refers to the answer data of the table 2. A data structure of the random test paper DB 121 is defined in the following table 3, but not limitedly. FIG. 3 shows its example.

[25] Table 3

Random test Test subject code + Management number + Question number + paper DB Number of answers + (Question number) + ...

[26] For example, it is assumed that an examiner sets a subjective question having five selective answers. Each question has five answers, so the answers may be differently arranged in 120 cases (24 cases in case of having four selective answers). That is to say, when answers for one question are arranged in random, 120 questions different from each other may be provided. Here, assuming that the examiner sets ten questions, random test papers may be provided as much as 120 squaring 10 from one main test paper having ten questions. Thus, there is very low possibility that different examinees receive the same random test paper though there are so many examinees, and thus a cheating is substantially impossible.

[27] The correct answer mapping unit 13 obtain a correct answer number location in the random test paper by tracking a movement of a correct answer number of the main test paper when a correct answer number is moved since answer numbers of the main test paper are changed into answer numbers of the random test paper, and then stores the correct answer number in the correct answer DB 131. Since every examinee takes an examination using a random test paper distinguished with different management number, the random test paper should be individually graded using the management number of the random test paper stored in the correct answer DB 131. A data structure of the correct answer DB 131 storing correct answers of the random test paper is defined in the following table 4, but not limitedly. [28] Table 4

Correct answer Test subject code + Management number + Question number + DB Correct answer number + ...

[29] The correct answer distribution ratio verifying unit 14 accumulates the number of answers for each answer number with respect to an individual random test paper so as to verify whether an accumulated distribution of each answer number is regular. For example, it is assumed that a random test paper has hundred questions in total and five answers are given to each question. Then, a maximum answer distribution ratio of each answer should be 20%. Here, it is assumed that an answer distribution ratio is 18 to 22% in consideration of an examiner error. Thus, if the number of questions whose answer number is 1 is 25 (corresponding to 25%), a correct answer number is gathered to a specific answer number, so it is required to adjust the answer distribution ratio more regularly. In the present invention, a user may change a correct answer to a specific answer number and then arrange the other answer numbers in random, or correct answers may be automatically arranged in random so that a threshold correct answer distribution ratio of an examiner is satisfied.

[30] If the correct answer DB 131 is constructed as mentioned above, after the test ends, the test paper may be graded by checking the test subject code and the peculiar ID of the test paper, recorded on the examiner answer sheet.

[31] FIG. 2 shows a network configuration of the question managing system of a test under a client- server environment according to an embodiment of the present invention.

[32] First, the question managing system 1 of the present invention shown in FIG. 1 may provide a question managing service of a test in both of a local environment and a client-server environment. Here, only the system utilizing the client-server environment will be explained for convenience.

[33] The question managing system 1 joins an examiner accessing thereto through a wire/wireless network as a member and provides the question managing service of the present invention with/without charge, for example, the question managing system 1 allows a logging-in from an examiner terminal 2 and provides the service of the present invention by constructing a homepage interface means using a web server so that the examiner may access the system 1 and directly set/print/manage a main test paper and a random test paper. The homepage interface means of an external web server may be not directly constructed inside the system 1, but linked using data transmission.

[34] The main test paper question setting service of the present invention has three modes using the main test paper setting unit 11 shown in FIG. 1 : namely an automatic question setting mode, a selective question setting mode, and a manual question setting mode. I) In the automatic question setting unit, an examiner designates the degree of difficulty and then automatically extracts and sets questions from a question bank DB 200 based on the degree of difficulty. II) In the selective question setting mode, an examiner selectively extracts and sets predetermined questions from the question bank DB 200. Ill) In the manual question setting mode, an examiner directly inputs contents of a question and answers and then determines arrangement of the answers.

[35] In addition, when setting main test paper questions, some questions should not be arranged in random by their nature. For example, if selective answers are already arranged in a certain order like the "unchangeable question" in the table 1, if the arrangement of answers is changed, the consistency of the answers is destroyed. If the consistency is destroyed, an examinee should make an additional struggle for finding a correct answer in the arrangement though he/she already knows the correct answer. It also includes a case that selective answers should be arranged in a certain order with consistency due to their relations or a case that answers are arranged in a dictionary order for facilitating better reading of the examinee. In such a case, an examiner selects corresponding questions and previously stores information related to the unchangeable questions in the main test paper DB 111. This question will be printed so that selective answers are arranged in the same order for all examinees, thereby having a correct answer in the same location.

[36] FIG. 3 shows an example of the random answer arrangement of the question managing system 1 of a test according to an embodiment of the present invention.

[37] In case of a main test paper, all answer numbers are arranged in order of 1 to 5 (in case five selective answers are given). For example, a question 1 is set to have a correct answer number "2" in the main test paper. Here, if the answers of the question 1 are arranged in random into the order of 4, 1, 3, 5, 2, a correct answer number is changed into 5 since "2" is located in the fifth order.

[38] FIG. 4 shows an inner configuration of the random test paper printing unit 15 according to an embodiment of the present invention. [39] The random test paper printing unit 15 of this embodiment includes a main test paper reader 151, a random test paper reader 152, an output file recorder 153, and an output file printer 154.

[40] The main test paper reader 151 reads all questions and all answers of each question from the main test paper DB 111 and then stores them in an internal memory.

[41] The random test paper reader 142 reads answer numbers of individual questions according to management codes from the random test paper DB 121 and then stores them in an internal memory.

[42] The output file recorder 153 firstly records data of an individual question read by the main test paper reader 151 to an output file, and then records the answer data of the main test paper DB 111 in the order of the random answer numbers read by the random test paper reader 152 to the output file, thereby completing one question. Then, the output file recorder 153 reads data of a next question and repeats the above process.

[43] Assuming that each question is recorded as a XML file, 600 questions for the national scholastic achievement examination is composed 600 questions for a main test paper, and assuming that there are 500,000 examinees, about 300,000,000 (=500,000 x 600) questions are generated. Also in this case, the probability that any two or more random test papers are identical to each other among 500 million random test papers is substantially zero.

[44] The output file printer 154 classifies individual random test papers depending on output management numbers, and then outputs the individual random test papers to a printing means 300 for printing.

[45] In this connection, an examinee should mark a management number, printed in an upper portion of a test paper, on an answer sheet (e.g., an OMR card) when taking an examination so that the test paper may be graded correctly.

[46] FIG. 5 is a flowchart illustrating a question managing method of a test according to an embodiment of the present invention.

[47] In a main test paper setting and storing step (SlO), the system 10 receives questions

(or, examination questions) of a main test paper and at least one answer (or, selective answer) depending on each question, set by an examiner, and then stores them in the main test paper DB 111. The system 1 may be constructed locally to the examiner terminal 2, or may be constructed under a client-server environment having a homepage interface means through a wire/wireless network. Both cases allow the examiner to directly set/print/manage a main test paper and random test papers.

[48] In addition, in the question setting process of the main test paper in the step SlO, the system 1 may receive the degree of difficulty designated by the examiner terminal 2 and then automatically extract questions from the question bank DB 200 based on the degree of difficulty in order to set questions (Sl 1), or may provide questions stored in the question bank DB 200 to the examiner terminal 2 and then receive predetermined selected questions in order to set questions (S 12), or may receive questions, contents of answers, and arrangement order directly from the examiner terminal 2 so as to set questions.

[49] In addition, the examiner may receive any question whose answer numbers should not be arranged in random (namely, "unchangeable question" in the table 1) among the questions of the main test paper and then store its information in the main test paper DB 111.

[50] Then, the system 1 arranges answers based on the main test paper and endows a unique management number, and the system 1 stores random answer numbers, generated by changing arrangement order of the answer numbers of the main test paper, to the random test paper DB 121 (S20). This step S20 will be explained later in more detail with reference to FIG. 6.

[51] If a random answer arrangement of the random test paper is completed by executing the step S20, the system 1 changes a correct answer number of the main test paper into a correct answer number of the random test paper with respect to the change of answer numbers caused by changing the answer numbers of the main test paper into answer numbers of the random test paper, and then stores the correct answer number of the random test paper (S30). This step S30 will be explained with reference to FIG. 3.

[52] The system 1 executes the step S30 to generate the random test paper and construct the correct answer DB 131, and then accumulates correct answer numbers of the random test paper to verify whether individual answer numbers are regularly distributed, and then, if correct answer numbers are too gathered to a specific answer number, the system 1 amends the arrangement of correct answer numbers and stores it (S40). In addition, such amendment is also made even in the case that correct answers are found in the same answer number successively.

[53] FIG. 6 is a detailed flowchart illustrating the process of generating a random test paper according to an embodiment of the present invention.

[54] In a random number and remainder generating step (S21), the order of all answers of each individual question of the main test paper is arranged in random. First, a certain random number is generated, and then the random number is divided by the number of all answers to obtain a remainder. For example, in case there are five answers, the remainder may have five cases, namely 0 to 4.

[55] Then, a number obtained by adding 1 to the remainder is determined as a answer number of the random test paper for a current answer number of the main test paper, thereby mixing the order of answers in random (S22).

[56] If the answer number determined by the remainder is already allocated to another answer, the above process is repeated from the step S21 until a random answer number not overlapped with a previous one occurs (S23).

[57] If the random answer number is not overlapped, the answer number of the random test paper, determined by the remainder, is stored in the random test paper DB 121 (S24).

[58] It is determined whether all answer numbers of an individual question of the main test paper are arranged in random, and then, if the random arrangement of all answer numbers is completed, the above process ends, and then answer numbers of a next question are arranged in random. If there is remained an answer number, the above process is executed again from the step S21.

[59] FIG. 7 shows an example of a random test paper generating program according to an embodiment of the present invention.

[60] In a main function, a random_generate function is executed to ten questions by setting the number of total questions to "count 10". In addition, 1 and 5 are transferred as parameter values to transmit five answer information, and 'arrangement information num' is transmitted to contain a result of the random-processed answer number.

[61] In an answer number random arrangement function, random numbers are generated for five answer numbers in total, and then its remainder is stored. Then, it is determined whether the obtained remainder is overlapped with previous answer numbers, and then, if overlapped, another remainder is obtained to determine orders of five answer numbers with different values.

[62] In a print output result, the result in which answer numbers are arranged in random in the order of 1 to 5 for ten questions of the main test paper is shown. For example, it would be noted that an answer number 4 of the main test paper is allocated to the number 1 in the random test paper for the question 1, and an answer 1 of the main test paper is allocated to the number 2 in the random test paper. If a correct answer in the main test paper is 2, a correct answer in the random test paper becomes 3 (see FIG. 9).

[63] FIG. 8 is a detailed flowchart illustrating the random test paper printing process according to an embodiment of the present invention.

[64] In a main test paper reading step (S51), the system 1 reads data related to entire questions and answers for each question from the main test paper DB 111 and stores them in an internal memory.

[65] Then, the system 1 reads an answer number of each question according to a management number from the random test paper DB 121 and then stores it in an internal memory (S52).

[66] The system 1 firstly records the data of each question, read in the step S51, to an output file, and then records the answer data of the main test paper DB 111 to the output file according to the order of the random answer numbers read in the step S52 (S53). [67] The system 1 generates the output file of the random test paper for each management number by executing the step S53, and then transmits the output file to the printing device to print it (S54).

[68] FIG. 9 shows an example of a main test paper and a random test paper according to an embodiment of the present invention.

[69] The random test paper of FIG. 9 shows a question 1 after reflecting answer numbers changed by the printed output result of FIG. 7.

[70] The system and method for managing questions of a test to prevent cheating using random arrangement of answers according to the present invention are configured as described above. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description. Industrial Applicability

[71] The system and method for managing questions of a test to prevent cheating using random arrangement of answers according to the present invention arranges answers dependent on each question of a test paper in random to provide every examinee with test papers different from each other, thereby capable of giving a test question managing service by which any cheating is systematically impossible.

Claims

[1] A system for managing questions of a test to prevent cheating by arranging answers of each question in random, the system comprising: a main test paper setting unit for setting questions (or, examination questions) of a main test paper and at least one answer (or, selective answer) dependent on each of the questions; a random test paper generating unit for endowing a unique management number of a random test paper newly generated by arranging answers based on the main test paper, and then changing an arrangement order of answer numbers of the main test paper to generate random answer numbers; a correct answer mapping unit for changing a correct answer number of the main test paper into a correct answer number of the random test paper with respect to the movement of a correct answer number caused by changing the answer numbers of the main test paper into the random answer numbers; and a correct answer distribution ratio verifying unit for accumulating the correct answer numbers changed by the correct answer mapping unit to verify whether the correct answer numbers are regularly distributed, and then amending the arrangement of the correct answer numbers.

[2] The system for managing questions of a test according to claim 1, wherein the question managing system includes a homepage interface means constructed by means of a web server so that an examiner directly sets/ prints/manages the main test paper and the random test paper at an examiner terminal accessing the question managing system through a wire/wireless network.

[3] The system for managing questions of a test according to claim 1 or 2, wherein the main test paper setting unit includes: an automatic question setting unit for automatically extracting questions from a question bank DB based on the degree of difficulty designated by an examiner so as to set questions; a selective question setting unit for allowing the examiner to selectively extract predetermined questions from the question bank DB so as to set questions; and a manual question setting unit for allowing the examiner to directly input questions and contents of answers and determining an arrangement order so as to the questions.

[4] The system for managing questions of a test according to claim 3, wherein the main test paper setting unit receives information related to a unchangeable question whose answer numbers should not arranged in random when the examiner sets questions in the main test paper, and then stores the information in a main test paper DB.

[5] The system for managing questions of a test according to claim 1, further comprising: a main test paper DB for storing data of entire questions of the main test paper and answers of each question; a random test paper DB for storing a management number designating a unique ID of the random test paper, and the answer numbers changed by arranging answer numbers of each question in random according to the order of question numbers of the main test paper; and a correct answer DB for storing the correct answer number of the random test paper mapped to the correct answer number of the main test paper.

[6] The system for managing questions of a test according to claim 1, wherein the random test paper generating unit generates a random number for an answer number of the main test paper, and then determining a remainder obtained after dividing the random number by the number of answers as an answer number of the random test paper.

[7] The system for managing questions of a test according to claim 1, further comprising a random test paper printing unit for printing the random test paper according to an instruction of a user, wherein the random test paper printing unit includes: a main test paper reader for reading data of entire questions and answers of each question from the main test paper DB and storing the data in an internal memory; a random test paper reader for reading answer numbers of each question dependent on the management number from the random test paper DB and then storing the answer numbers in an internal memory; an output file recorder for recording the data of individual questions read by the main test paper reader to an output file, and recording the answer data of the main test paper DB to the output file according to the order of random answer numbers read by the random test paper reader; and an output file printer for transmitting the output file for the individual management number generated by the output file recorder to a printing device for printing.

[8] A method for managing questions of a test using a question managing system that arranges answers of each question in random to prevent cheating, the method comprising:

(SlO) setting and storing questions (or, examination questions) of a main test paper and at least one answer (or, selective answer) dependent on each of the questions;

(S20) endowing a unique management number of a random test paper newly generated by arranging answers based on the main test paper, and then storing random answer numbers generated by changing an arrangement order of answer numbers of the main test paper;

(S30) changing a correct answer number of the main test paper into a correct answer number of the random test paper with respect to the movement of a correct answer number caused by changing the answer numbers of the main test paper into the random answer numbers, and then storing the correct answer number of the random test paper; and

(S40) accumulating the correct answer numbers per each number to verify whether the correct answer numbers are regularly distributed, and then amending and storing the arrangement of the correct answer numbers.

[9] The method for managing questions of a test according to claim 8, wherein, in the step SlO, a homepage interface means constructed to the question managing system so that an examiner directly sets/prints/manages the main test paper and the random test paper at an examiner terminal accessing the question managing system through a wire/wireless network.

[10] The method for managing questions of a test according to claim 8 or 9, wherein the step SlO includes: receiving the degree of difficulty designated by an examiner terminal, and then automatically extracting questions from a question bank DB based on the degree of difficulty so as to set the questions; providing questions stored in the question bank DB to the examiner terminal, and then allowing the examiner terminal to select questions so as to set the questions; and receiving questions, contents of answers and an arrangement order directly from the examiner terminal so as to set the questions.

[11] The method for managing questions of a test according to claim 8 or 9, wherein, in the step SlO, information related to a unchangeable question whose answer numbers should not arranged in random among the questions of the main test paper is received from an examiner terminal and then stored in a main test paper DB.

[12] The method for managing questions of a test according to claim 8 or 9, wherein, in the step SlO, the data of entire questions of the main test paper and answers of each question is stored in a main test paper DB.

[13] The method for managing questions of a test according to claim 8, wherein, in the step S20, the management number indicating a unique ID of the random test paper and the answers numbers by arranging answer numbers of each question in random according to a question number order of the main test paper are stored in a main test paper DB.

[14] The method for managing questions of a test according to claim 8 or 13, wherein the step S20 includes:

(521) generating a random number for an answer number of the main test paper, and then obtaining a remainder get after dividing the random number by the number of entire answers of an individual question;

(522) determining the remainder as an answer number of the random test paper in which individual answer numbers are arranged in random;

(523) checking whether the remainder is overlapped with a previously endowed answer number of the random test paper, and then executing the above process from the step S21 in case the remainder is overlapped;

(524) determining and storing the remainder as an answer number of the random test paper in case the remainder is not overlapped; and

(525) determining whether all of answers of the main test paper are completely arranged in random, and then ending the process in case all answers are completely arranged in random, but executing the above process from the step (S21) in case there is remained any answer number.

[15] The method for managing questions of a test according to claim 8, wherein, in the step S30, a correct answer number of the random test paper mapped to a correct answer number of the main test paper is stored in a correct answer DB.

[16] The method for managing questions of a test according to claim 8, further comprising:

(550) printing the random test paper according to an instruction of a user.

[17] The method for managing questions of a test according to claim 16, wherein the step (S50) includes:

(551) reading data of entire questions and answers of each question from a main test paper DB and storing the data in an internal memory;

(552) reading answer numbers of each question dependent on the management number from a random test paper DB and then storing the answer numbers in an internal memory;

(553) recording the data of individual questions read in the step S51 to an output file, and recording the answer data of the main test paper DB to the output file according to the order of random answer numbers read in the step S52; and

(554) transmitting the output file for the individual management number generated in the step S53 to a printing device for printing.