KR100355665B1 - On-line qualifying examination service system using the item response theory and method thereof - Google Patents

Abstract

The present invention relates to a system and method for online qualification and certification testing services, and in particular, to calculate the item parameters of accurate and unchanging criteria using item response theory (IRT), and systematically according to the area (category) and format of the item The present invention relates to a qualification and certification test service system and method for providing and evaluating items for individual examinees through computerized adaptive examination (CAT).

In the existing test was a method of calculating the score according to the number of questions with the same questions, but in the present invention, all examinees are tested with different questions. As each test item progresses, the question gradually converges to the examinee's ability, and finally the test ends when the estimated error falls within the allowable range.

When the system according to the present invention receives the administrator confirmation information and the information related to the test execution from the central administrator, an area (category) and a parameter are designated to select the test items of the corresponding test among the registered test questions, and the authentication encryption process A central server for securely compressing the test items through a transport file to prepare a transmission file; An item database for storing items registered and managed by the central server; A central server that securely compresses the registered test questions and prepares them as a transmission file when receiving administrator confirmation information and information related to a test execution from a local administrator; An item database for storing items registered and managed by the central server; A plurality of local databases, each of which is managed by central servers through a communication network, when receiving local administrator confirmation information and information related to test execution from a local administrator; And receiving a plurality of examinee terminals receiving the examinee identification information from the examinee, accessing a local server of the local server through a communication network, receiving test items stored in a corresponding local database, and proceeding for the examinee to perform a qualification test. do.

Description

ON-LINE QUALIFYING EXAMINATION SERVICE SYSTEM USING THE ITEM RESPONSE THEORY AND METHOD THEREOF}

The present invention relates to a qualification test service using a communication network, and more particularly, to an online qualification and certification test service system and method using an Item Response Theory (IRT).

The current national qualification test method is an extremely inefficient paper-based test method that does not deviate significantly from the conventional test form. Regardless of exams in all fields, when the exams are conducted, a large number of examinees across the country have to travel to gather to a certain place. It causes huge financial and time waste due to the examinee's movement, and mobilizes a lot of manpower in management. On the other hand, security issues are not efficient. In spite of the fact that the test members were detained / detained several months before the test, the test questions were leaked and the adequacy of the correctness of the test questions became a social problem.

The most fundamental cause of the security problem in the conventional written test is that all examinees conduct the same question. On the contrary, if the test is conducted with different items, the outflow of the item is not only difficult, but also the meaning of the outflow is lost. The adequacy of the question is because the question depends only on the personal judgment of the expert. It reflects the need for a means to manage questions scientifically and systematically.

In general, agency trials have consistently failed to establish an integrated test implementation environment, with only the administrative convenience of each ministry, despite the inherent inefficiencies in the procedures involved in testing. There are also new types of certification exams. Currently, developed countries, for example, are shifting to conducting trials to the private sector to solve administrative difficulties (eg, Sullivan Computer, USA). In the domestic realities, various private certifications are already appearing one after another, and the state is facing limitations in managing them.

Item Response Theory (IRT) means that there is an item characteristic function for each test item, and this characteristic is invariant by the examinee group. In the conventional test evaluation method, when a group with high ability and an inferior group see the same test, even the same test is easily evaluated by the superior test and difficult by the inferior group. If the evaluation is managed on such a changing basis, it is impossible to estimate the examinee's objective ability. Therefore, by using the item response theory, the questions such as difficulty, discrimination, and conjecture are automatically analyzed by a computer system to create a test question, and by using the analyzed item parameter values. Evaluate abilities objectively and specifically.

It is possible to secure the objectivity of the examinee's ability evaluation by the item parameter invariance and the ability parameter invariance by the item response theory. First, the concept of item parameter invariance is that the item itself is immutable. Therefore, if the parameters of the item characteristics are applied as a criterion for judging ability, it is possible to independently judge the ability of the examinee group. Also, the concept of capability parameter invariance is that the examinee's ability is not changed by the content of the test. However, in the conventional written test, the accuracy of the examinee's correct answer depends on the difficulty of the test questions in a manner that depends only on the number of correct answers, resulting in a change in the examinee's ability.

The present invention has been made in order to solve the problems of the conventional paper-based test method as described above, according to the item response theory, by using the communication network authentication and authentication of the test questions specified by the parameter, by performing an online test, accuracy To increase the reliability of the evaluation through standardization of tests. On the other hand, it aims to provide an online qualification and certification test service system and method for shortening all the time until the test execution time and notification of passing result and maximizing the efficiency of test management.

In addition, the computer-based test (CBT) similar to the paper-based test, as well as the computer-adaptive test (CAT), which is similar to the conventional paper-based test through the test questions composed of the designated questions, the individual's ability It aims to provide a standardized qualification and certification testing service system and method for providing accurate questions to increase the precision, efficiency and accuracy of evaluation.

1 is a block diagram showing an online qualification and certification test service system using an item response theory according to the present invention;

Figure 2 is a flow chart for explaining the online qualification and certification test service method using the item response theory according to the present invention,

3A is a flowchart for specifying step 200 (test service method at the central server side, S200) of FIG. 2 performed at a central server;

3B is a flowchart for specifying a registration step (item database construction, S210) of the parameter specification item of FIG. 3A,

3C is a flowchart for specifying an item analysis step S213 using the item response theory of FIG. 3B;

3D shows a graph of the item characteristic function,

4A is a flowchart for explaining an online qualification and certification test execution method using an item response theory performed at the local server side specifying step 400 of FIG. 2,

FIG. 4B is a flowchart for specifying a random parameter (particularly, difficulty) designation item assignment step S430 of FIG. 4A;

Figure 4c is a schematic diagram showing the test development algorithm according to the difficulty of the parameter,

4D is a flowchart for specifying a test termination step S440 of FIG. 4A;

5A and 5B are flowcharts showing a method of estimating the examinee's ability examinee's ability to be used in the statistical and analytical processing step (S270) of FIG. 3A or the initial item selection (S431a) of FIG. ,

6A to 6J illustrate screen examples for item management and screen examples of actual test items.

Hereinafter, described in detail by the accompanying drawings of the present invention.

On-line qualification and certification test system using the item response theory according to the present invention for achieving the above problem is shown in FIG. The system comprises a central server 100, a plurality of local servers 120 connected to the central server 100 through a communication network 150, and a plurality of examinee terminals 140 connected to each local server through a communication network 150. Include.

In the online qualification and certification test system using the item response theory according to the embodiment of the present invention, when the administrator receives the administrator identification information and the information related to the test execution from the central administrator, a parameter consisting of category, difficulty, discrimination degree, and conjecture is designated and registered. A central server 100 that selects test items of the test from among test items, securely compresses the test items through an authentication encryption process, and prepares them as a transmission file; An item database 114 in which items are registered and managed by the central server 100; When receiving the administrator confirmation information and the information related to the test execution from the local administrator, access to the central server 100 via the communication network 150 to request the questions of the test, and through the authentication approval process to transmit the prepared transmission file Receiving a plurality of local servers 120; A plurality of local databases 136, each managed by the plurality of local servers 120, for storing security decompressed items applied by the central server 100; And receiving the examinee identification information from the examinees, accessing the local server of the local server 120 through the communication network, and receiving the questions stored in the corresponding local database 136, and different tests according to the individual capabilities of the examinees. It characterized in that it comprises a plurality of examinee terminal 140 to proceed.

Here, the central server 100 receives and confirms the questionnaire confirmation information, registers the test items designated by the category and the parameter from the questionnaire, and assigns a code code to each test item to store in the fraud question database 114. Register 102; The item correction / deletion unit requests a different approval number each time when accessing a registered test item, and checks the corresponding approval number inputted from the item manager and grants the right to correct or delete the test item stored in the item database 114. 104; Upon receiving at least ID, password, test execution date, and time from the central manager, the predetermined multiple items of the corresponding test items to be executed in the parameter specification items stored in the item database 114 are selected, and the test is performed through an authentication encryption process. A security encryption unit 106 for securely compressing items to prepare a transmission file; And an item transmitter 108 which transmits the prepared transmission file to the local server after checking and confirming confirmation information from the connected local server 120. The various DBs may be managed by having a separate DB management server.

In addition, the item transmission unit 108 receives and confirms confirmation information from the connected local server 120, and then further inquires the confirmation code code of the test given by the central server 100, and confirms that each transmission starts. The security code may be strengthened by requiring the local server 120 to continue the authentication approval process by using an encryption code assigned to each test item.

In addition, the central server 100 includes a test result receiver 110 for receiving the test results of the examinee through the local server 120 at the end of the test; A test result database 116 for storing the received test results; And it is preferable to further include a statistics and analysis unit 112 to notify the test management institution by the statistical and analysis processing the test results.

Meanwhile, when each of the plurality of local servers 120 receives at least an ID, a password, a test date, and a time from a local administrator, the plurality of local servers 120 access the central server 100 through the communication network 150 and request the test items of the corresponding test. Question request unit 122; Receives the transmission file prepared through the authentication approval process, and after completion of the transmission of the security decompressed test questions applied in the central server 100 stored in the local database 136 Receiving unit 124; After receiving and confirming the examinee identification information from the connected examinee terminal 140, the test enforcement unit 126 randomly allocating the items stored in the local database 136 so that different parameter designating questions are provided according to individual examinee-response results. ); And a test termination unit 128 that determines whether the test continues through the connected examinee terminal according to whether the test maximum allowable time, maximum number of questions, or ability estimation error tolerance is exceeded.

In addition, each of the plurality of local servers, at least during the test is carried out in the examinee terminal 140, at least a password code of each question, a password code of the test, the central administrator of the test, local administrator, examinee identification information and scoring results A test result recorder 130 for recording any test information including; And a test result transmission unit 132 that informs the examinee of the grading result through the examinee terminal 140 and retransmits the contents recorded in the test result recording unit 130 to the central server 100. It is preferable to further include.

In addition, for security purposes, each of the plurality of local servers 120 further includes a deletion unit 134 for deleting contents stored in at least the corresponding local database 136 after the operation of the test result transmission unit 132 is completed. desirable.

2 is a flowchart for explaining the online qualification and certification test service method using the item response theory according to the present invention. 2, the operation of the central server 100 and the local server 120 will be described. In the following, S means a step.

First, when (a) the central server 100 receives the management confirmation information and the information related to the test execution from the central manager, a parameter consisting of a category, difficulty, discrimination degree, and conjecture is designated and the test of the corresponding test among the registered test items. The items are selected, securely compressed through an authentication encryption process, and prepared as a transmission file (S200). Specifically, the central server 100 generates one of the security code table determined by the system when receiving the administrator confirmation information and the information related to the test execution from the central administrator. Next, the central server 100 selects test items of the corresponding test among pre-registered category and parameter specification test items. Here, the selected test items are items that are appropriately assigned to the category, difficulty, discrimination degree, and conjecture of the items defined in the test regulations, and each selected item is given a separate authentication code code. Next, the central server 100 securely compresses the selected test items and prepares them as transmission files to be transmitted to each local server 120.

After step 200, (b) for the item request from the local server 120 connected through the communication network 150, the transmission file prepared through the authentication approval process is transmitted (S300). In detail, when the local server 120 receives input of its confirmation information and information related to the test execution from a local administrator, that is, a test supervisor, the local server 120 connects to the central server 100 through the communication network 150 and displays the test items of the corresponding test. Require. In this regard, the central server 100 checks whether the information input by the local administrator is correct. After the verification process is completed, the security code automatically assigned by the central server 100 is asked, and if the local server 120 responds correctly, transmission starts. The local server 120 receives the prepared transmission file from the central server 100 through an authentication approval process. At this time, even during transmission, the central server 100 systematically checks whether or not the system is authenticated using an authentication passcode assigned to each test item, and the local server 120 receives the test item for each test item to continuously receive the test items. desirable. When the reception is completed, the security compression algorithm applied in the central server 100 is applied to the transmission file and decompressed.

After the step 300, (c) the secure compression decompressed parameter specification items received from the local server 120 is stored in the local database 136 to complete preparation for the test (S310).

After step 310, (d) after confirming the examinee from the examinee terminal 140 connected through the communication network, the test is provided by providing a test item, and the test is scored during the test (S400). Specifically, when the plurality of terminals 140 receive the examinee identification information from the examinees at the test date and time, the plurality of terminals 140 accesses the local server 120 having jurisdiction among the local servers 120 through the communication network 150 to access the local database. Receive the test questions prepared in (136). Here, examinees are first identified as supervisors. The plurality of terminals 140 proceed with the examinees to perform the qualification test with the support from the local server 120.

During the test, the local server 120 aggregates all the test results, and records all information such as the authentication password code of the item, the central administrator, the local administrator, the examinee's confirmation information, and whether or not the answer is correct.

After step 400, the system is deactivated after the end of the test, and the test result score is counted (S500). Since the qualification and certification test service of the present invention employs a CBT or CAT method, it is possible to notify the examinee on the examinee terminal 140 as soon as the test result score is counted.

Finally, the test result is notified to the examinee and transmitted to the central server 100 after secure compression (S510). All test results of the examinee are transmitted to the central server 100, and the results are reported to the test administration through various statistical analysis and all test procedures are terminated.

3A is a flowchart for explaining a qualification and certification test support method performed at the central server 100 side. The central server, or host computer, plays the most important role in registering and managing test questions, so security issues should be given top priority. For this purpose, it is also necessary to consider the need to keep physically external and thoroughly offline, but more importantly, the structure of the item database 114 should be designed with security in mind. Hereinafter, the operation of the components in the central server 100 will be described in detail with reference to FIG. 3A.

First, (a1) the item registration unit 102 receives and confirms the questionnaire confirmation information, and registers the test items by specifying the item category, the item type, the parameter, etc. from the questionnaire, and assigns and stores a code code for each test item. (S210). At this time, the item registration unit 102 assigns a code code automatically generated by the system for each test item and stores it in the item database 114. Once a test item has been registered, it must be approved by a procedural practitioner. The registered items are tested on actual sample or population candidates, and the result parameters are applied to the item response theory, that is, difficulty, discrimination degree, and guess degree, are stored in the database.

Managing difficulty, discrimination, and speculation as the parameter of the question will be described in detail with reference to FIG. 3B.

First, a new question in which a questionnaire and a category according to a test subject are designated by a questionnaire is registered in the item register 102 (S211). The test questions composed of these new items are conducted through a sample test or a population test or CAT or CBT test (S212).

Next, IRT item analysis is performed (S213), which will be described in detail with reference to FIG. 3C.

First, assuming that the response result will be a normal distribution, the pre-distribution selects 10 or 20 pre-distributions at equal intervals of the capability values -4 to +4 to calculate the expected values by the pre-distribution (S213a), This is called a quadrature point. Next, the likelihood function (LF) at each quadrature is calculated by calculating a probability value that matches the examinee's correct answer at that point (S213b). Next, the weight at the quadrature point selected from the standard normal distribution is calculated, which is calculated by substituting the probability of the post-probability density by reflecting the weight of the newly calculated distribution (S213c). Then, the candidate's response to each question and the expected frequency of each correct examinee are calculated (S213d). The above steps 213a to 213d are called E steps.

The following is the process of solving the Likelihood Equation in M steps. First, the likelihood function is differentiated into the item parameters (S213f), and the calculations are repeated through the Taylor Series and the Newton Raphson Procedure (S213g).

If the variation difference (delta, Δ) becomes small enough to be negligible by repeatedly estimating the E-step and the M-step, that is, within the allowable range (for example, within 0.0001 to 0.005), the calculation is stopped to calculate the final value, Otherwise, E and M steps are repeated again.

The functions for obtaining the item characteristic function through the IRT item analysis include the regular ozive model function and the logistic model function. Among them, an equation of a logistic model function that is easy to operate and obtains the same result is as follows. An example of a graph of this function is shown in FIG. 3D.

P (θ) = c + (1-c) (1 / (1 + e ^{-a (θ-b)} ))

Where a is the degree of discrimination, b is the degree of difficulty, c is the degree of conjecture, and θ represents the examinee's ability value.

In FIG. 3D, the X axis represents the difficulty of the item, the Y axis represents the correct hit ratio, the Y intercept of the curve represents the guess degree, and the slope of the graph represents the discrimination degree.

The discrimination degree (a) is an index indicating a degree of distinguishing the examinee's ability to answer the question, and a high discrimination question is a good question for distinguishing the examinee who is excellent and the examinee who does not. According to the item response theory, it is managed in the range of -10 to +10, and the items with negative discrimination should be modified or used.

The difficulty (b) is an index indicating the degree of difficulty of the test, it is numerically managed in the range from -4 to +4.

And the degree of guessing (c) is a probability of correct answer that the question itself has, for example, in the case of the selective five-choice multiple choice, the probability that the item is selected and corrected is higher when 1/5, that is, 0.2, is a good question. Can't see The guess plot uses exponents between 0 and 1.

The above items with the parameters of difficulty, difficulty, and conjecture specified are entered into the item database 114, and each time the test is executed in the constructed item database, the test results are analyzed by the examinee and reflected again. As a result, the quality of the item database can be managed more accurately over time.

As a method of determining the three parameters, for example, a marginal maximum likelihood estimation or Bayesian estimation method is applied.

In brief, the maximum likelihood estimation method is a method of selecting and presenting an item that gives the maximum information from the ability analyzed in the examinee's response, while the information function of each item is calculated. After estimating this, it is a method to increase the precision of capability estimation by selecting an item that can minimize variance after the fact at the capability level.

Referring back to FIG. 3B, after the step 213, if the item is not suitable, through the item correction (S215), the process returns to step 212, and if the item is suitable, it is stored in the item database (S216) and goes through the determination step (S214). . A questionnaire for each category (area) is randomly generated from the question pool stored in the question database 114 (S217), and transmitted to the local server 120 (S218) to perform a CAT test (S219).

6A to 6B illustrate screen examples for item management. These screens may correspond to some of the web pages when the present invention is implemented on a web basis. FIG. 6A is related to the 211 new item registration step of FIG. 3B and includes items required for item registration, such as a subject, an item area, an item type, a questionnaire information, an input type, a problem, an answer, a time limit, a category, and the like. do. These items are entered by the questions. 6B relates to a field for inputting a category for a test question. For example, a candidate may enter a department name as a parent category and a detailed field as an additional category. FIG. 6C illustrates a screen related to item management and problem paper management capable of searching an item editor and a general item.

3A, after step 210, (a2) if necessary, the item correction / deletion unit 104 checks the ID, password, and approval number of the item manager, and then registers the registered test items stored in the item database 114. Allows modification / deletion and reports the details (S220).

The item correction / deletion unit 104 requests a different approval number each time a registered test item is accessed. The item manager can access the system to modify or delete it, but it is possible only if the test authority is given a predetermined number, for example, a code set consisting of 100 (10-digit) codes. Therefore, even if the ID or password of the item manager is exposed, the test item can be prevented from leaking. The item manager must enter the approval number that the system requires differently each time, even at the designated location (computer), so that the item can be entered into the edit or delete mode of the test item. Meanwhile, the item registration unit 102 or the item correction / deletion unit 104 automatically collects the result when the registration of the item and the correction / deletion work if necessary is completed, and the test authority and the questionnaire are provided to the questionnaire if necessary. Give details in detail. FIG. 6D illustrates a problem paper management screen related to production, correction, and deletion of a problem paper, and a registration, correction, and deletion screen related to events such as a test schedule and a place.

After step 220, next (a3) when receiving at least an ID, password, test date and time from the central administrator, the predetermined multiple items of the test question number of the test to be carried out in the stored parameter designation items are selected and authenticated. After the encryption process, the test items are securely compressed and prepared as a transmission file (S230). After verifying the input, the security encryption unit 106 selects the test questions of the test stored in the question database 114, and securely compresses the test questions through an authentication encryption process and transmits them to the transmission file. That is, prepare to send test items to each local server.

At this time, the security encryption unit 106, in the process of selecting the items of the test, selects a predetermined number of items, for example, 100 times the number of test items to be carried out substantially individually. This is to allow the local server 120 to automatically select the test questions at random to answer different test questions to the examinees.

After the step 230, (a4) receives the confirmation information from the connected local server and confirms, and transmits the prepared transfer file to the local server (S240).

Next, (a5) after performing the test for each local server 120, the test result receiving unit 110 through the local server receives the entire examinee and individual test results (S250). 6E shows a sample test screen for a conventional CBT, and FIG. 6F shows an item flag management screen in which a category and a parameter for a CAT of the present invention are designated.

Then, (a6) stores the received test result in the test result database 116 (S260). Next (a7) the test results are reported and statistically analyzed and analyzed to the relevant test management institution (S270).

4A is a flowchart for explaining a method for conducting an online qualification and certification test using an item response theory performed on a local server side. The local system including the local server 120 is in charge of all the management in which the actual test is performed, and is composed of a local area network system (LAN) having a plurality of examinee terminals, for example, 30 to 100 clients for one local server. In order for the local server, or local computer, to transfer the file prepared for testing, the test supervisor must request the transmission of the file in a normal manner and with qualification, at least for a period of time, eg, 5 to 3 hours before the test time. Hereinafter, the operation of the components in the local server 120 will be described in detail with reference to FIG. 4A.

First, (c1) when the item request unit 122 receives at least an ID, password, test execution date and time from the local administrator, accesses the central server 100 through the communication network 150 to specify the parameter specification item of the test. Request them (S410). At this time, the local server 120 is confirmed by further inquiring the encryption code of the test given by the central server side. This is to prevent unauthorized access to the central server 100 from the other server side.

Next, (c2) the item receiving unit 124 receives the transmission file prepared through the authentication approval process, and by storing the security decompressed test questions applied by the central server 100 in the local database 136 after completion of the transmission Complete the test preparation (S420). In order to maintain transmission efficiency and security, the special secure compression algorithm applied by the central server 100 undergoes a decompression process after the transmission file is received by the local server 120.

Then, (c3) after receiving and confirming confirmation information such as the examinee's examinee's examinee's number and resident's code from the connected examinee's terminal 140, the test items stored in the local database 136 by the examiner 126 for each examinee. Randomly allocated so that different parameter specification items are provided according to the response result (S430). In this process, the test may be scored during the test run.

Referring to Figure 4b it will be described in detail how to assign the test questions as follows. The following description with respect to FIG. 4B shows the progress of the initial item presentation and testing process based on difficulty for the convenience of explanation among parameters consisting of discrimination, speculation and difficulty.

First, it is determined whether to proceed to step 431b or step 432a according to whether there is information on the examinee's ability (S431a). If there is the examinee's ability information, select an item of difficulty level suitable for the examinee's ability (S432a), if not, then select an item having a difficulty that satisfies the acceptance criteria if the type of test is a qualification test (S432b), If it is not the qualification test, the item having a medium difficulty is selected (S432c) (S431b).

In this way, the initial item is selected and the initial item is presented through the examinee terminal 140 (S433). Depending on whether the examinee's response is correct, the following question is presented based on the pyramid difficulty algorithm illustrated in FIG. 4C (S434). For example, if the first question is asked in D and the examinee corrects the answer, the question will be one of E, F, and G (Difficulty of B + delta) (S435b). However, if the answer is incorrect, one of the questions A, B, C (B-delta difficulty) proceeds to (S435a). The choice of questions may become more or less difficult depending on the examinee's abilities.

Next, the items of the same area are continuously presented depending on whether the minimum number of items per category (area) is satisfied (S437a), or the items of other areas (S437b) are processed (S436).

Since different questions are randomly assigned to all examinees systematically, the order of the items is different as well as the composition of the different items. Therefore, even if the test is carried out several times on the same test date, it is possible to solve the problems caused by the leakage of questions. On the other hand, even if different items are randomly assigned, the level of difficulty is equally assigned to all examinees, so that the system checks and tests them, so that equity does not matter.

In selecting items according to the characteristics of these parameters, items with too low discrimination or high guess should be excluded. If a question database with such a constant parameter is properly constructed, even if each examinee is given a different question, the item can be configured to reflect the difficulty evenly, thus minimizing complaints even if the questions are different. In addition, by managing the evaluation criteria objectively, it is possible to prevent excessive fluctuations in the passing rate every time, and to enable more rational and standardized qualification and certification test operation.

Referring back to FIG. 4A, after step 430, (c4) the test termination unit 128 continues the test according to whether the maximum test time allowance, the maximum number of questions, or the capacity estimation error tolerance range are exceeded through the connected examinee terminal. It is determined whether or not (S440). Information on the time remaining during the test is displayed on the screen of the examinee's terminal, and when a predetermined test time has elapsed, the system is automatically deactivated so that the examinee can no longer correct the problem.

4D shows a flow chart of this test termination. Prior to the description of the capability estimation error in FIG. 4D, the capability value refers to a capability (score) calculated by the IRT method, and has a value between -4.0 and +4.0, and recognizes the zero point as the intermediate score. For example, the number of questions in the Microbiology and Clinical Chemistry subjects of the National Examination for Clinical Pathologist Qualification Examinations should be 40, with 40% (16 questions) per subject and 60% (24 questions) of all subjects. This translates into an IRT score of about -0.5 and a fallover score of 40% and +0.5 of 60%.

On the other hand, the ability estimation error is a number indicating how accurately the examinee's ability value is estimated to the actual ability. For example, if the ability value is 0.5 and the error is 0.1, it means that the examinee's actual ability is in the range of 0.5 ± 0.1. The smaller the error, the closer the examinee's ability is to the actual ability. However, an error of 0.05 or higher indicates that the examinee's response was inconsistent. In other words, the result of the evaluation is not reliable.

After the step 440 in FIG. 4A, (c4) while the test is performed in the examinee terminal 140, the test recorder 130 of the local server 120 may include at least the encryption code of each question, the encryption code of the corresponding test, and the corresponding test. It is preferable to record all test information including the central manager, local manager, examinee confirmation information, and scoring results of the test (S450).

In step 450, the next step (c5) informs the examinee of the grading result through the corresponding terminal 140, and the test result transmission unit 132 retransmits the recorded contents to the central server 100 after secure compression. It is for (S460).

Finally, for security (c6) after the step 460, the deletion unit 134 deletes the data including at least the test items stored in the local database (S470).

The above has described the qualification and certification test service system and method.

As described above, the central server 100, the local server 120, and the examinee terminal 140 exchange data through the communication network 150. No matter how good your security logic is, if you get hacked into the channel while data is being transferred, you can't expect complete security. Therefore, the communication means should be based on a regular telephone line or a leased line. Transmitting and receiving through the modem is generally slow, but due to the strength of hacking is being used as a means of transfer in the bank network.

Therefore, if the speed of the modem is best managed and the efficiency of data transmission and reception is used, the item transmission time will not be a problem even if many items are transmitted. In addition, since the compression algorithm is applied, the actual communication data amount is much reduced, so that fast communication is possible.

In addition to the security of the communication, the security of the question should be considered. No matter how secure the questions are, if all the examinees take the same test questions as they do now, the examinees will quickly create the original questions with just a few. However, in this system, the test is conducted in a different question and order for each examinee, each examinee can only remember their own questions, it is difficult to combine all the original questions.

On the other hand, securing enough test items enables screening tests of different items each time and makes it difficult for memorizing the questions to examinees, thus preventing the leakage of questions. In addition, having different difficulty levels allows even levels of difficulty to be reflected evenly, even if the test is repeated many times or if different test items are presented to all examinees.

Spatially and centrally, where the database is stored, access should be strictly restricted. For this purpose, it is best to have a central server in the test management organization, but in case of administrative problems, it is best to leave it to the private company and take full responsibility for security, and select a company that has appropriate control means based on access. It is desirable to.

FIG. 5A is a flowchart of a process of performing statistical and analysis processing of test results for step 270 of FIG. 3A or step 431A of FIG. 4B. First, the examinee's response pattern is analyzed (S271), the weight (Weight) by the pre-distribution described above with reference to FIG. 3C is calculated (S272), and then the ability values (theta, θ) are calculated (S273). Then, the capacity estimation error is calculated (S274), and if this error is not within the allowable range (for example, within 0.01 to 0.05), the next item is repeated (S276), and if it is within the allowable range (S275), the process ends. . 6G to 6J show examples of statistics and analysis implemented on a web basis. FIG. 6G is a screen for setting an option of an IRT analysis, FIG. 6H is a graph of a test (test) information function for a whole test or a subject, FIG. 6I shows an analysis result of an item parameter, and FIG. 6J shows a question about one item. The item characteristic function (IRF) is shown.

These statistics and analyzes have progress points in addition to the capability values and capability estimation errors described above.

In the classical test, the true score was taken as the mean score of the observation scores obtained from the tests. However, the true score according to the item response theory can be calculated by adding the item characteristic function for each item and adding up the probability of fitting the item. The problem with the classical test is that theoretically infinite trials are required to calculate the correct true scores. For difficult questions, the probability of fitting is very low, and for easy questions, the probability of fitting is very high. Therefore, the incorrect answer does not directly affect the calculation of the true score, but the noon of the response is already reflected in the ability estimation, so it is reflected in the true score calculation. In the Computer-Adaptive Test (CAT), questions are tailored to individual questions, so comparing the scores with other test takers is meaningless.

FIG. 5B is a graph of ability change for each examinee's item, and indicates that ability estimation is correctly performed when the graph converges about one point or is distributed around one point.

The qualification and certification test service systems and methods according to the present invention described so far target most large scale qualification tests. For example, qualifications and selection tests conducted in various countries, such as the driver's license test and the college entrance exam, can be considered.

On the other hand, the problem of establishing a qualification test center nationwide is also a difficult problem for the national budget, and even if it is established, management problems are also difficult. Therefore, the management of the system should be handed over to the private sector, and the authorities of the information will have to move towards the minimum costs necessary to manage the company. In contrast, the present invention provides an optimal system for managing in a private company.

As described above, if the computerized qualification test center according to the present invention is established as a city hall unit in each ward office and a small city in a large city, for example, it can be used as a convenient location close to the examinees and can easily perform various national qualification tests. In addition, it can be utilized for a variety of purposes, such as cultivating the people's computer skills. In addition, the cost and test time due to the examinee's movement can be significantly reduced, and the leakage of test questions can be suppressed.

In addition, the system of the present invention solves the basic security problem of the test site, and makes separate management for security unnecessary. In addition, there is an advantage accompanying verification of the adequacy of the test item itself.

As a result, the present invention can provide a qualification test service system and method which can not only reduce the time required for producing the test and reporting the grading result by using the CAT, but also can completely prevent fraudulent acts. have.

Claims (14)

When the administrator receives the administrator identification information and the information related to the test execution from the central manager, the parameters consisting of category, difficulty, discrimination degree, and conjecture are designated, and the test items of the corresponding test are selected from the registered test items, and the test is performed through the authentication encryption process. A central server for securely compressing items to prepare a transmission file; An item database for storing parameter specification items registered and managed by the central server; Upon receiving the administrator confirmation information and the information related to the test execution from the local administrator, a plurality of locals are connected to the central server through a communication network, requesting parameter specification items of the corresponding test, and receiving the prepared transfer file through an authentication approval process. Servers; A plurality of local databases managed by the plurality of local servers, respectively, for storing the security decompressed parameter specification items applied by the central server; And Upon receiving the examinee identification information from the examinees, the user accesses a local one of the local servers through a communication network, receives the parameter specification items stored in the corresponding local database, and performs a plurality of tests according to the individual capabilities of the examinees. On-line qualification and certification test service system using the item response theory, characterized in that the examinee terminal. The method of claim 1, wherein the central server is An item registration unit configured to receive test questions received from the questionnaire, and to receive test items designated by the category and parameter from the questionnaire, and to assign a code code to each test item and to store them in a fraudulent item database; An item correction / deletion unit for requesting a different approval number each time a user accesses a registered test item, and confirming a corresponding approval number inputted from the item manager and granting the right to modify or delete a test item stored in the item database; Upon receiving input of at least ID, password, test execution date and time from the central manager, the predetermined multiple items of the corresponding test items to be executed in the parameter specification items stored in the item database are selected, and the test items are secured through the authentication encryption process. A security encryption unit that compresses and prepares a transmission file; An item transmitter for transmitting the prepared transmission file to the local server after receiving and confirming confirmation information from a connected local server; A test result receiver configured to receive the examinee's entire and individual test results through a local server at the end of the test; A test result database for storing the received test results; And On-line qualification and certification testing service system using the item response theory, characterized in that it comprises a statistics and analysis unit for reporting the test results to the test management institution by statistical and analysis processing. The method of claim 2, wherein the item transmission unit After receiving and confirming the confirmation information from the connected local server, the inquiry is made by further inquiring about the cipher code of the test given by the central server. When the transmission starts, the local server approves authentication using the cipher code assigned to each test question. On-line qualification and certification test service system using the item response theory, characterized in that the process is required to continue the process. delete The method of claim 1, wherein each of the plurality of local servers An item request unit requesting test items of a corresponding test by accessing the central server through a communication network when at least an ID, password, test execution date, and time are input from a local administrator; An item receiving unit configured to receive the prepared transmission file through an authentication approval process and to store, in a local database, the test items decompressed security applied by the central server after completion of the transmission; A test execution unit for randomly allocating the items stored in the local database so that different parameter designation items are provided according to the examinee's individual response results after receiving and confirming the examinee identification information from the connected examinee terminal; And On-line qualification and certification using the Item Response Theory, which includes a test end to determine whether the test is continuously conducted according to whether the test maximum allowable time, maximum number of questions, or ability estimation error tolerance is exceeded through the connected examinee terminal. Trial service system. The method of claim 5, wherein each of the plurality of local servers A test result recorder that records at least one test information, including at least a password code for each question, a password code for the test, a central administrator, a local administrator, examinee identification information and scoring results, during the test at the examinee's terminal. ; And Online qualification using the item response theory, characterized in that further comprising a test result transmission unit for informing the examinee through the examinee terminal through the corresponding examinee terminal, securely compressing the contents recorded in the test result record and then re-transmitting to the central server And certification testing service system. The method of claim 6, wherein each of the plurality of local servers On-line qualification and certification test service system using the item response theory, characterized in that it further comprises a deletion unit for deleting at least the contents stored in the local database after completion of the operation of the test result transmission unit. A qualification and certification test service method using a communication network in a qualification test service system having a central server, a plurality of local servers connected to a central server via a communication network, and a plurality of terminals connected to each local server via a communication network, (a) When the central server receives the management confirmation information and the information related to the test execution from the central administrator, the test items of the corresponding test are selected from among the registered test questions with the parameters consisting of category, difficulty, discrimination degree, and conjecture. Preparing a transmission file by securely compressing the data through an authentication encryption process; (b) transmitting a prepared transmission file through an authentication approval process for an item request from a local server connected through a communication network; (c) storing the secure decompressed parameter specification items received from the local server in a local database and completing preparations for a trial; And (d) an online qualification and certification test service method using the item response theory, characterized in that it comprises the step of providing the test questions after the examinee from the examinee terminal connected through the communication network to perform the test. The method of claim 8, wherein step (a) comprises: (a1) receiving and confirming the questionnaire confirmation information, and registering test items designated by the category and parameter from the questionnaire, and assigning and storing a code code for each test question; (a2) if necessary, confirming the corresponding approval number inputted from the item manager by accessing the registered test item every time, requesting another approval number, and granting the right to modify or delete the stored test item. ; (a3) Upon receiving input of at least ID, password, test date, and time from the central administrator, the predetermined multiple items of the test question number of the test to be executed in the stored parameter designation items are selected, and the test items are passed through the authentication encryption process. Secure compression to prepare a transmission file; And (a4) After receiving and confirming confirmation information from a connected local server, the online qualification and certification test service method using the item response theory comprising the step of transmitting the prepared transfer file to the local server. The method of claim 9, wherein step (a4) After receiving and confirming the confirmation information from the connected local server, the inquiry is made by further inquiring about the cipher code of the test given by the central server side. On-line qualification and certification test service method using the item response theory, characterized in that to continue the process. The method of claim 9, wherein the steps (a1) to (a4) (a5) after conducting the test, receiving the examinee's entire and individual test results through a local server; (a6) storing the received test results in a test result database; And (a7) Online qualification and certification test service method using the item response theory, characterized in that further comprising the step of reporting the statistical and analytical processing of the test results to the test administration. The method of claim 8, wherein step (c) comprises: (c1) receiving at least an ID, a password, a test date, and a time from a local administrator, accessing the central server through a communication network, and requesting parameter designation questions of the test; (c2) receiving a transmission file prepared through a certification approval process, and storing the extracted security test items, which are applied by the central server, in a local database after the transmission is completed; (c3) after receiving and confirming the examinee identification information from the connected terminal, randomly allocating the stored items so that different parameter designating questions are provided according to individual test result of the examinee; And (c4) Online qualification using the item response theory, comprising determining whether to continue the test through the connected examinee terminal according to whether the test maximum allowable time, maximum number of questions, or ability estimation error tolerance is exceeded. And certification test service methods. The method of claim 12, wherein the steps (c1) to (c4) (c4) while the test is performed on the terminal, recording all test information including at least the passcode of each question, the passcode of the test, the central administrator of the test, the local administrator, the examinee identification information and the scoring results; ; And (c5) an online qualification and certification test service method using an item response theory, further comprising the step of informing the examinee of the grading result through the corresponding terminal, and re-transmitting the recorded contents to the central server after secure compression. . The method of claim 13, wherein each step is (c6) after the step (c5), further comprising the step of deleting the data including the test questions stored in the local database, the online qualification and certification test service method using the item response theory.