JP2022045158A - Test result analysis system and method - Google Patents

Abstract

To provide a technique enabling analysis of a test result before a pass/fail is announced.SOLUTION: A test result analysis system includes: an answer data collection part collecting answer data representing answers to respective questions submitted by a plurality of examinees who took a test containing a plurality of questions; a pass/fail estimation part estimating respective passes/fails of the plurality of examinees based on the answer data of the plurality of examinees; and an answer analysis part analyzing answers to the questions of estimated failures estimated to have failed the test based on answer data of estimated passers estimated to have passed it.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a technique for analyzing a candidate's test results before the pass / fail of the test is announced.

In qualification exams, etc., there is a high demand to know your own score, possibility of passing, and what you are missing if you cannot pass the exam before the result is announced. However, usually, the correct answer and passing line of this test are not announced. Therefore, educational institutions that provide learning courses and contents for the exam publish their own model answers and expected passing lines on their website immediately after the exam. Candidates can predict their score and chances of passing by browsing the website.

Patent Document 1 discloses a system for predicting a ranking in a test and a possibility of passing the test. In the system of Patent Document 1, various problems of the mock test are stored in the database, and the server extracts the corresponding problem from the database in response to the request of the user to select the problem and displays the corresponding problem on the display of the corresponding user terminal. It is displayed so that the user can perform a mock test. In addition, the user can obtain information on the pseudo-ranking and pass / fail prediction in the mock test from the system.

Japanese Unexamined Patent Publication No. 2002-72857

The system of Patent Document 1 predicts the possibility of passing this test from the result of the mock test. The mock exam measures the level of the learner's academic ability at the time of conducting the mock exam by using the exam questions and answers prepared by the educational institution itself, and the result of this exam is based on the actual answer in the main exam. Is not an analysis. Therefore, after taking the mock exam, the examinee can obtain information that can be used as a reference for learning up to the main exam, but does not satisfy the demand for promptly obtaining information on the results of the main exam.

One object of the present disclosure is to provide a technique that allows the test results to be analyzed even before the pass / fail announcement.

A test result analysis system according to one aspect of the present disclosure includes an answer data collection unit that collects answer data indicating answers to each question submitted by a plurality of examinees who have taken a test including a plurality of questions, and the above-mentioned. A pass / fail estimation unit that estimates the pass / fail of each of the plurality of examinees based on the answer data of a plurality of examinees, and an estimation failure estimated to be rejected based on the answer data of the estimated passers estimated to pass. It has an answer analysis unit that analyzes the answers to the above-mentioned problems of successful applicants.

According to one aspect of the present disclosure, a plurality of examinees are divided into estimated passers and estimated failers, and the answers to each question of the estimated failers are analyzed based on the answer data of the estimated passers. , It will be possible to analyze the test results even before the pass / fail is announced.

It is a block diagram which shows the test result analysis system by this embodiment. It is a block diagram which shows the computer system by this embodiment. It is a flowchart which shows the test result analysis process performed by a test result analysis system. It is a flowchart which shows the problem classification process. A list showing examples of problem types. It is a figure which shows an example of a correctness problem. It is a figure which shows an example of an alternative problem. It is a figure which shows an example of the combination problem. It is a figure which shows an example of a choice-type problem. It is a figure which shows an example of a computational problem. It is a figure which shows an example of the number problem. It is a figure which shows an example of a descriptive question. A list showing examples of problem levels. It is a figure which shows an example of a problem information. It is a flowchart which shows an example of the prerequisite knowledge determination process. It is a figure which shows an example of the field information. It is a flowchart which shows an example of answer format specification processing. It is a flowchart which shows an example of a problem level evaluation process. It is a flowchart of evaluation processing by answer format. It is a figure which shows an example of the problem level judgment information. It is a figure which shows an example of the judgment logic information of the alternative problem comprehensive evaluation process. It is a flowchart which shows an example of the alternative problem comprehensive evaluation process. It is a flowchart which shows an example of an answer analysis process. It is a figure which shows an example of the judgment criterion information. It is a figure which shows an example of the judgment result by subject. It is a table showing the analysis result of the problem-specific analysis. It is another table showing the analysis result of the problem-specific analysis. Yet another table showing the analysis results of the problem-specific analysis. Yet another table showing the analysis results of the problem-specific analysis.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a test result analysis system according to the present embodiment. The test result analysis system 10 is a system that enables an examinee's test result to be analyzed before the pass / fail is announced in a pass / fail test.

The test referred to here is, for example, a test conducted on a regular basis. Examples of exams that are held regularly, such as once a year, include qualification exams such as national exams and entrance exams for various schools. In addition, there are many tests that are conducted regularly, such as once a year to several times a year. In many cases, failed candidates will continue to study and take the next exam again. Therefore, analyzing the test results is useful for learning for the next test, especially for those who fail.

The exam contains multiple questions. Candidates pass if the total score obtained by answering the question correctly exceeds the pass line, and fail if the total score falls below the pass line. For example, the residential land and building trader qualification test is a test format in which 50 questions are answered in 2 hours.

It is empirically known that some of the questions included in the exam need to be answered correctly in order to pass the exam, and some questions have little effect on pass / fail even if they cannot be answered correctly. It is said that you can pass the exam if you can surely answer the questions of the level that other examinees answer correctly, and it is said that the shortcut to passing is to acquire the ability to surely answer the questions of that level.

Referring to FIG. 1, the test result analysis system 10 includes a pass line acquisition unit 11, an answer data collection unit 12, a pass / fail estimation unit 13, a question classification unit 14, and an answer analysis unit 15.

The pass line acquisition unit 11 acquires the minimum number of questions (correct answers) to be answered correctly in order to pass the examination as a pass line. If the number of questions included in the exam is 50 and if 35 or more questions are answered correctly, the pass line is 35. For example, the pass line acquisition unit 11 is given a predetermined pass rate (for example, an average value) based on the pass rate of each year, and determines the pass line based on the number of examinees for whom answer data was obtained and the pass rate. You may. Assuming that answer data is obtained from 100 examinees and the pass rate is 20%, the pass line acquisition unit 11 may determine the pass line so that the top 20 pass.

The answer data collection unit 12 collects answer data of a plurality of examinees. The answer data is data showing the answer of each question submitted by the examinee in the examination. For example, the answer data collection unit 12 provides a web page including a form for the examinee to input an answer so that the examinee can access it by an information processing terminal (not shown) such as a personal computer, a tablet terminal, or a smartphone. Get the answer data entered from the web page. It is not always necessary to obtain answer data from all the examinees who took the test. However, it is more likely that better analysis results will be obtained if the answer data is obtained from as many examinees as possible.

The pass / fail estimation unit 13 estimates the pass / fail of each of the plurality of examinees based on the collected answer data of the plurality of examinees. By comparing the number of correct answers of the examinee with the pass line, it is possible to decide whether to presume to pass or fail. Here, the examinee presumed to pass is referred to as an estimated passer, and the examinee presumed to fail is referred to as an estimated failer.

The question classification unit 14 classifies the questions included in the test into a plurality of question levels based on the correct answer rate in the answer data of the estimated passers. The question classification unit 14 calculates the correct answer rate of each question in the answer data of the estimated passers, and compares the correct answer rate with the threshold value to determine which question level each question corresponds to. The details of the problem level will be described later.

The question classification unit 14 further classifies the questions included in the test into a plurality of question types based on the answer format. The question classification unit 14 specifies the answer format of each question based on the question sentence of each question included in the question data, and determines the question type of each question according to the answer format. The types of answer formats and the method of specifying the answer format for each question will be described later.

The answer analysis unit 15 analyzes the answers to each question of the estimated passers based on the answer data of the estimated passers. According to this aspect, a plurality of examinees are divided into estimated passers and estimated failers, and the answers to each question of the estimated passers are analyzed based on the answer data of the estimated passers, so that the pass / fail is announced. The results of the test can be analyzed before it is done.

For example, if the examination includes questions of a plurality of subjects, the answer analysis unit 15 calculates the deviation value of the estimated failing person based on the answer data of the estimated passing person for each subject. The average score of the estimated passers is the deviation value 50.

Further, the answer analysis unit 15 calculates the estimated passer correct answer rate, which is the ratio of the estimated passers who correctly answered the question to the whole estimated passers, for each question in the answer data of the estimated passers, and estimates the question. Based on the passer correct answer rate, the answer to the question of the estimated unsuccessful person to be analyzed is analyzed.

According to this aspect, the answers to each question of the estimated passers are analyzed based on the correct answer rate of the estimated passers, so the percentage of the estimated passers who answer correctly before the pass / fail is announced. It is possible to analyze in consideration of. In estimating the pass / fail of an examinee, it is not always necessary to estimate the pass / fail accurately. Even if the pass / fail cannot be determined accurately, the correct answer rate of the top performers shows the same tendency as the correct answer rate of the passers of this test, so it is possible to provide useful information by analysis considering the tendency of the passers. ..

For example, the answer analysis unit 15 analyzes the answers to the questions of the estimated failers based on the question level of each question classified by the question classification unit 14 and the estimated passer correct answer rate. The analysis referred to here includes calculating the number of correct answers and / or the correct answer rate of the estimated unsuccessful persons for each question level. The questions that many presumed passers answer correctly are likely to be questions that cannot be mistaken for passing. For example, if the correct answer rate of the estimated passers is 80% or more, there is a high possibility that the question should be answered correctly in order to pass. If the estimated failer to be analyzed does not answer such a problem correctly, the estimated failer's academic ability may be insufficient, or the problem may be a problem that the estimated failer is not good at. .. The answer analysis unit 15 presents the analysis result to the estimated unsuccessful person. The answer analysis unit 15 displays, for example, the correct answer rate for each question level for the estimated unsuccessful persons to be analyzed on the web page in a table format or a graph format based on the analysis result.

According to this aspect, the answer to each question of the estimated failing person is analyzed based on the degree to which the question is required to be answered correctly in order to pass the examination and the correct answer rate of the estimated passing person of the question. An effective analysis that considers how much you are required to answer the question correctly in order to pass the exam and what percentage of the estimated passers are correct before the pass / fail is announced. It will be possible.

Further, for example, the answer analysis unit 15 further analyzes the answers to the questions of the estimated failers based on the question type of each question classified by the question classification unit 14 and the estimated passer correct answer rate. The analysis here includes calculating the number of correct answers and / or the percentage of correct answers for the estimated failers by question type. The answer analysis unit 15 presents the analysis result to the estimated unsuccessful person. The answer analysis unit 15 displays, for example, the correct answer rate for each question type for the estimated unsuccessful persons to be analyzed on the web page in a table format or a graph format based on the analysis result.

According to this aspect, the answer to each question of the estimated failing person is analyzed based on the format of the answer to the question and the correct answer rate of the estimated passing person of the question. It is possible to make an effective analysis considering the format of the answer to the question and the percentage of the estimated passers who answered the question correctly.

Further, when the target test includes questions of a plurality of subjects, the answer analysis unit 15 calculates the deviation value of the estimated failing person based on the answer data of the estimated passing person for the whole examination and each subject in the examination. You may.

According to this aspect, in a test that includes a plurality of subjects and the pass / fail is determined by the total score of the subjects, the deviation value for each subject of the estimated failing person is calculated based on the answer data of the estimated passing person. It is possible to objectively analyze the results of each subject.

The test result analysis system 10 of this embodiment can be realized by one or more computers.

FIG. 2 is a block diagram showing a computer system according to the present embodiment. The computer system 20 is a computer that realizes the test result analysis system 10 shown in FIG. The computer system 20 has a processor 21, a main memory 22, a storage device 23, a communication device 24, an input device 25, and a display device 26 as hardware, and these are connected to the bus 27.

The storage device 23 is a device that stores data so that it can be written and read. The pass rate data, the answer data, the correct answer data, the question data, and the analysis result shown in FIG. 1 are stored in the storage device 23.

The processor 21 is a processor that reads the data stored in the storage device 23 into the main memory 22 and executes the processing of the software program by using the main memory 22. The processor 21 realizes the pass line acquisition unit 11, the answer data collection unit 12, the pass / fail estimation unit 13, the question classification unit 14, and the answer analysis unit 15 shown in FIG.

The communication device 24 is a device that transmits / receives data to / from an external device such as an information processing terminal via a communication line (not shown) such as a communication network. For example, the answer data is input via the communication device 24 and stored in the storage device 23.

The input device 25 is a device that receives information from an operator's operation input such as a keyboard and a mouse, and the input information is used for software processing by the processor 21. When the operator performs an operation instructing the desired process to the input device 25, the processor 21 executes the process instructed by the operation.

The display device 26 is a device that displays image or text information on a display screen as a result of software processing by the processor 21.

FIG. 3 is a flowchart showing a test result analysis process executed by the test result analysis system.

In step 101, the answer data collecting unit 12 acquires answer data for each question entered in the answer sheet in the actual test from a plurality of examinees. The answer data collection unit 12 displays an input screen of the self-scoring site by a web page. When a plurality of examinees access the self-scoring site and input the answers entered in the answer sheet in the actual examination, the answer data collection unit 12 records the input answers as answer data.

In step 102, the pass / fail estimation unit 13 predicts the pass / fail of each of the plurality of examinees based on the collected answer data of the plurality of examinees. For example, the pass / fail estimation unit 13 compares the correct answer data indicating the correct answer of each question given in advance with the answer data obtained from the examinee, and the examinee correctly answers which question and which question is incorrect. And calculate the number of correct answers. Then, the pass / fail estimation unit 13 compares the number of correctly answered questions with the passing line given in advance, and if the number of correctly answered questions is equal to or greater than the passing line, the examinee estimates that the examinee has passed, and the number of correctly answered questions passes. If it is below the line, the candidate is presumed to have failed.

In step 103, the question classification unit 14 classifies the questions included in the test. Here, the process of classifying the questions included in the test is referred to as a question classification process.

FIG. 4 is a flowchart showing the problem classification process. The question classification process is a process of classifying exam questions according to two criteria, question type and question level.

FIG. 5 is a list showing examples of problem types.

As a problem type, type (1) is a correct / incorrect question, and type (2) is a knowledge question of an alternative question.

Type (3) is a degree of alternative question, type (4) is a combination question, type (5) is a judicial question of alternative question, and type (6) is a multiple choice question. The type (7) is a hardness case problem of the alternative problem, the type (8) is a calculation problem, the type (9) is a number problem, and the type (10) is a descriptive problem.

The test targeted in this embodiment includes correct / incorrect questions, alternative questions, combination questions, multiple choice questions, computational problems, number questions, and descriptive questions as typical answer formats. Further, the alternative question includes a case law question, a case law question, and a knowledge question. Further, since the difficulty level of the case problem of the alternative problem differs depending on the number of subjects appearing in the option, it is divided into a low-level case problem and a high-level case problem having different difficulty levels.

The question type (answer format) itself needs to be conscious in the measures for passing the exam, and at the same time, it can be a factor that determines the question level because the ease of deriving the correct answer changes depending on the answer format.

The format of the question sentence of the question is also determined to some extent by the format of the answer to the question. Therefore, the question classification unit 14 analyzes the question text of the target question in the question data and determines whether or not the predetermined conditions for each answer format are met, thereby determining which answer format the target question belongs to. Can be determined.

The following is an example of each answer type question.

FIG. 6 is a diagram showing an example of a correct / incorrect problem.

The right / wrong question is an alternative question that asks you to answer whether a certain matter is correct or equivalent, or incorrect or equivalent. FIG. 6 shows, as an example, a correct / incorrect question that asks whether a given sentence is correct or appropriate, and whether the sentence is incorrect or inappropriate. The correct / incorrect question is that the answer is an alternative, and the question sentence contains the phrase "correct" or a predetermined phrase equivalent to it, and / or a predetermined phrase "wrong" or equivalent. There is a feature. "Appropriate" in the correctness problem shown in FIG. 6 is an example of a phrase equivalent to "correct". In addition, "inappropriate" is an example of a phrase equivalent to "wrong".

FIG. 7 is a diagram showing an example of an alternative problem.

The alternative question is a question in which a predetermined number of sentences (limb sentences) are selected as options, and one correct answer (correct answer limb) is selected from the options. The number of choices (number of limbs) includes, for example, limbs, five limbs, six limbs, etc., depending on the test. FIG. 7 shows, as an example, an alternative question in which one of the five alternative sentences (limb sentences) is answered. The correct one is the correct limb. As another example, there is also an alternative question in which one limb sentence that is incorrect is answered from among a plurality of limb sentences. In that case, the wrong one is the correct limb. The alternative question is characterized in that the question sentence includes a plurality of limb sentences, and the limb sentence itself is an answer option.

In addition, alternative question options such as legal exams include knowledge question options, case question options, and case law choices. Case question options are options that ask about specific cases. The case question option is characterized in that the question sentence contains a word (case subject term) indicating a predetermined subject appearing in the case question. Typical case-based terms that appear in legal examinations include A, B, Hei, Ding, Company A, Company B, Company C, and Company D. Case law options are options that ask about understanding of a particular case. Case law option is characterized in that the question sentence contains a phrase or a combination of phrases peculiar to a specific case. The knowledge problem option is an option that asks for knowledge about a specific matter. Problems that do not correspond to case law choices or case law choices can be presumed to be knowledge question choices.

FIG. 8 is a diagram showing an example of a combination problem.

The combination problem is a problem in which a plurality of combinations for a plurality of limb sentences are selected as options, and a combination of limb sentences corresponding to the conditions given in the problem sentence is selected from the options. The combination referred to here includes the case of 0 and the case of 1. In FIG. 8, as an example, there is a combination problem in which four combinations 1, 2, 3, and 4 are selected for the three items a, b, and c, and the wrong combination is selected from the options. It is shown. The combination problem is characterized in that the question sentence has a plurality of limb sentences and each of the options is a combination of 0 or more limb sentences.

FIG. 9 is a diagram showing an example of a multiple choice problem.

The multiple-choice question is a question in which there are multiple blanks in the question sentence, and a plurality of words and phrases including words and phrases that apply to those blanks are given as options separately from the question sentence, and the words and phrases that apply to each blank are selected from the options. .. As an example, FIG. 9 shows a problem in which a question sentence has four blanks of a, a, c, and d, and a word or phrase applicable to these blanks is selected from 20 options of 1 to 20. The multiple-choice question is characterized by having a question sentence containing multiple blanks and a part in which candidates for words and phrases that apply to the blanks are listed as options.

FIG. 10 is a diagram showing an example of a computational problem.

A computational problem is a problem that derives a correct answer by performing numerical calculations according to the conditions of the problem statement. In FIG. 10, as an example, four amounts of 1 to 4 are given as options, and a problem is shown in which the amount requested in the problem statement is calculated and the option matching the calculated amount is selected. ing. Computational problems are characterized in that each of the answer options contains a numerical value, and the question sentence also contains a numerical value used for the calculation.

FIG. 11 is a diagram showing an example of the number problem.

The number problem has multiple limb sentences in the question sentence, has the number of multiple limb sentences as an option, and from the options, the number of limb sentences corresponding to the conditions given in the problem sentence is selected. It's a matter of choice. The number here includes 0. In FIG. 11, as an example, the question sentence has three limb sentences a, b, and c, and four limbs 1 (1), 2 (2), 3 (3), and 4 (none). It shows a number problem that has choices and lets you choose from the choices how many of the three limb sentences are incorrect. The number problem is characterized in that each of the plurality of options is a number included in the range from 0 to the number of limbs. The symbol here may be anything as long as it identifies each limb sentence, and may be, for example, a number, an alphabet, a katakana, a hiragana, a kanji, a figure, or the like.

FIG. 12 is a diagram showing an example of a descriptive question.

A descriptive question is a question in which a predetermined item is described based on a question sentence. FIG. 12 shows, as an example, a descriptive question in which a matter required by a question sentence is described in a sentence having a predetermined number of characters. Descriptive questions are characterized by containing certain words and phrases that are required to be written in sentences. "Write" in the question sentence shown in FIG. 12 is an example of a predetermined phrase that is required to be described in the sentence. Other examples include "describe," "discuss," and "explain."

FIG. 13 is a list showing examples of problem levels. There are four question levels, levels A, B, C, and D, from the one with the highest degree of being required to answer the target question correctly in order to pass the exam. Level D is a problem that can be passed without a correct answer. Level C is a problem in which the probability of a correct answer is assumed to be close to 1/2. Level B is a difficult question that should be answered correctly in order to pass. Level A is an easy question that should be answered correctly in order to pass. The problem classification unit 14 determines to which level the target problem corresponds.

In this way, a question that is expected to pass without a correct answer, a question that is expected to have a correct answer probability of around 1/2, a difficult question that is expected to be answered correctly to pass, and a pass. The response required in studying for passing differs depending on the four problem levels, which are the easy questions that should be answered correctly.

In exams such as qualification exams, questions such as super-difficult questions and strange questions that almost all examinees, including successful applicants, cannot solve may be asked. Treating such questions as any other question and studying may not be effective as a study for passing. Level D is a low-priority question in studying exam preparation that allows you to pass without having to answer correctly. The other levels A to C are questions that require study to be answered correctly in order to pass. In studying about problems that require high study, the three levels are useful information for studying according to the academic ability of the examinee and guidance for the examination.

The three-level division, excluding level D, which allows students to pass even if they cannot answer correctly and has a low priority for studying, is an example of the number of effective steps for advancing studying step by step. The three levels A to C are useful information for advancing the study step by step.

Returning to FIG. 4, first, in step 201, the problem classification unit 14 executes the prerequisite knowledge determination process. The prerequisite knowledge discrimination process is a process of discriminating the field of prerequisite knowledge required to solve the target problem. By this processing, it is specified which field the target problem is premised on, that is, which field the problem asks whether or not there is knowledge. When the examination includes multiple subjects, the subject of the target question can be determined by determining the field of the target question. The details of the prerequisite knowledge determination process will be described later with reference to FIG.

Next, in step 202, the problem classification unit 14 executes the problem sentence length calculation process. The problem sentence length calculation process is a process for determining the problem sentence length of the target problem. The question sentence length is the length of the problem sentence. The unit for expressing the problem sentence length is not particularly limited, but may be expressed by, for example, the number of characters, the number of words, the number of lines, and the like. When expressed by the number of characters, the problem classification unit 14 may count the number of characters included in the problem sentence.

FIG. 14 is a diagram showing an example of problem information. In the question information 30, the corresponding field 32, the question sentence length 33, the answer format (question type) 34, and the question level 35 are registered in association with the question number 31 of the question. In this step 202, the question sentence length 33 is registered in the question information 30 in association with the question number 31 of the target question.

Next, in step 203, the question classification unit 14 executes the answer format specifying process. The answer format specifying process is a process of specifying the answer format, that is, the question type of the target question. The details of the answer format specifying process will be described later with reference to FIG. By this process, the answer format of the target question is specified.

Next, in step 204, the problem classification unit 14 executes the problem level evaluation process. The problem level evaluation process is a process for evaluating the problem level of the target problem. For example, the problem level of each problem may be determined. The expression of the problem level is not particularly limited, and can be expressed by, for example, numerical scores, rankings, and the like. In this embodiment, the problem level is represented by a ranking of levels A to D. The difficulty increases from level A to D, but the impact on passing the exam decreases. The question level can be determined based on the field of prerequisite knowledge of the target question, the question sentence length, and the answer format. The details of the problem level evaluation process will be described later with reference to FIG.

FIG. 15 is a flowchart showing an example of the prerequisite knowledge determination process.

The problem classification unit 14 extracts the field-related terms included in the target problem by acquiring the target problem text from the problem data and analyzing it.

Field-related terms are terms that appear in problems in each field. The problem classification unit 14 manages field information in which field-related terms in each field are predetermined. FIG. 16 is a diagram showing an example of field information. The field information 40 is a table in which the number of terms 42, the field-related terms 43, and the knowledge difficulty level 44 are registered in association with the field 41. The term number 42 indicates the number of terms related to the corresponding field. The field-related term 43 indicates a term related to the corresponding field. The knowledge difficulty level 44 indicates the difficulty level of acquiring knowledge in the corresponding field. Here, as an example, it is assumed that the knowledge difficulty level is shown in 10 levels. For example, four field-related terms 43 such as "abb", "acc", "add", and "aee" are set in the field "1". The knowledge difficulty level of the field "1" is "4". Similarly, in the field "2", seven field-related terms 43 such as "bcc", "bdd", "bee", "bff", "bgg", "bhh", and "bii" are set. The knowledge difficulty level of the field "2" is "7".

Specifically, the problem classification unit 14 collates the words appearing in the target problem with the field-related terms associated with each field in the field information 40, and selects the field-related terms of each field appearing in the target problem. Extract.

Next, the problem classification unit 14 calculates a field score as an index as to how likely it is that the problem is a problem in each field based on the collation result in step 301 (step 302). As an example, the field score of field X is calculated by the formula (1).

Field score of field _X SX = (ST / N _X ) x N _m ... (1)

Here, ST is a constant value, and a value common to all fields is used. Nx is the total number of field-related terms set in the field X, that is, the number of terms 42. Nm is the number of field-related terms in field X included in the subject problem.

The value obtained by dividing the constant value ST by the number of terms Nx of the field X is evenly distributed to each field-related term B33 as the score of each field-related term B33 so that the total score is constant among the fields. The value obtained by integrating the scores of the field-related terms B33 included in the target problem is the field score for the field X of the target problem. For example, when the target problem includes two field-related terms B33, "abb" and "acc", which are related to the field A, the field score S ₁ for the field "1" of the target problem is S ₁ = ( ST / 4) × 2 = ST / 2.

Next, the problem classification unit 14 identifies the field (corresponding field) to which the target problem corresponds (step 303) based on the field score of each field calculated in step 302. For example, a field whose field score exceeds a predetermined threshold value may be a field to which the target problem is concerned. There may be a plurality of applicable fields of the subject problem.

Next, the problem classification unit 14 registers the corresponding field of the target problem specified in step 303 in the problem information 30. In this step 303, the corresponding field 32 is registered in the problem information 30 exemplified in FIG. 14 in association with the problem number 31 of the target problem.

FIG. 17 is a flowchart showing an example of the answer format specifying process.

First, in step 401, the problem classification unit 14 determines whether or not the target problem is a correct / incorrect problem. The correct / incorrect question illustrated in FIG. 6 has an alternative answer, and the question sentence includes the phrase "correct" or a predetermined phrase equivalent thereto, and / or a predetermined phrase "wrong" or equivalent. It has the characteristic that it contains. The question classification unit 14 analyzes the question text of the target question and determines whether or not the question text of the target question contains the characteristics of the correct / incorrect question. If the question text contains the characteristics of the correct / incorrect question, it can be determined that the target question is a correct / incorrect question. If the target question is a correct / incorrect question, the question classification unit 14 proceeds to step 409, and registers the “correct / incorrect question” in the question information 30 in the answer format 34 corresponding to the question number 31 of the target question.

If the target problem is not a correct / incorrect problem in step 401, in step 402, the problem classification unit 14 determines whether or not the target problem is an alternative problem. The alternative question shown in FIG. 7 is characterized in that the question sentence includes a plurality of limb sentences, and the limb sentence itself is an answer option. The question classification unit 14 determines whether or not the question text of the target question contains the characteristics of the alternative question. If the question text contains the characteristics of the alternative question, it can be determined that the target question is an alternative question. If the target problem is an alternative problem, the problem classification unit 14 proceeds to step 403.

In step 403, the question classification unit 14 determines whether each option of the target question, which is an alternative question, is a case law option, a case question option, or a knowledge question option. The decision problem option is characterized in that the question sentence contains a phrase or a combination of phrases peculiar to a specific case. The case problem is characterized in that the case-based term is included in the problem sentence. The problem classification unit 14 determines that the target problem is a case problem if the problem text contains the characteristics of the case law, and if the problem text contains the characteristics of the case problem, the target problem is the case problem. If the question text does not include the characteristics of the case law problem or the characteristics of the case law problem, it is judged that the target problem is a knowledge problem. After that, the question classification unit 14 proceeds to step 409, and registers the "alternative question" in the question information 30 in the answer format 34 corresponding to the question number 31 of the target question. Further, the question classification unit 14 records whether each option of the alternative question is a case law option, a case question option, or a knowledge question option.

If the target problem is not an alternative problem in step 402, the problem classification unit 14 determines in step 404 whether or not the target problem is a combination problem. The combination problem illustrated in FIG. 8 is characterized in that the problem sentence has a plurality of limb sentences and each of the options is a combination of 0 or more limb sentences. The problem classification unit 14 determines whether or not the problem sentence text of the target problem contains the characteristics of the combination problem. If the question text contains the characteristics of the combination problem, it can be determined that the target problem is a combination problem. If the target question is a combination question, the question classification unit 14 proceeds to step 409, and registers the "combination question" in the question information 30 in the answer format 34 corresponding to the question number 31 of the target question.

If the target problem is not a combination problem in step 404, the problem classification unit 14 determines in step 405 whether or not the target problem is a selective problem. The multiple-choice question illustrated in FIG. 9 is characterized by having a question sentence including a plurality of blanks and a part in which candidates for words and phrases applicable to the blanks are listed as options. The question classification unit 14 determines whether or not the question text of the target question contains the characteristics of the multiple-choice question. If the question text contains the characteristics of the multiple-choice question, it can be determined that the target question is a multiple-choice question. If the target question is a multiple-choice question, the question classification unit 14 proceeds to step 409, and registers the "selective question" in the question information 30 in the answer format 34 corresponding to the question number 31 of the target question.

If the target problem is not a selective problem in step 405, the problem classification unit 14 determines in step 406 whether or not the target problem is a computational problem. The computational problem illustrated in FIG. 10 is characterized in that each of the answer options includes a numerical value, and the question sentence also contains a numerical value used for the calculation. The problem classification unit 14 determines whether or not the problem sentence text of the target problem contains the characteristics of the computational problem. If the problem text contains the characteristics of a computational problem, it can be determined that the target problem is a computational problem. If the target problem is a computational problem, the question classification unit 14 proceeds to step 409, and registers the "computational problem" in the question information 30 in the answer format 34 corresponding to the question number 31 of the target question.

If the target problem is not a computational problem in step 406, the problem classification unit 14 determines in step 407 whether or not the target problem is a number problem. The number problem illustrated in FIG. 11 is characterized in that each of the plurality of options is a number included in the range from 0 to the number of limbs. The problem classification unit 14 determines whether or not the problem sentence text of the target problem includes the feature of the number problem. If the question text contains the characteristics of the number problem, it can be determined that the target problem is the number problem. If the target question is a number question, the question classification unit 14 proceeds to step 409, and registers the "number question" in the question information 30 in the answer format 34 corresponding to the question number 31 of the target question.

If the target problem is not a number problem in step 407, the problem classification unit 14 determines in step 408 whether or not the target problem is a descriptive problem. The description problem illustrated in FIG. 12 is characterized in that it contains predetermined words and phrases that are required to be described in sentences. The question classification unit 14 determines whether or not the question text of the target question contains the characteristics of the descriptive question. If the question text contains the characteristics of the descriptive question, it can be determined that the target question is a descriptive question. If the target question is a descriptive question, the question classification unit 14 proceeds to step 409, and registers the "descriptive question" in the question information 30 in the answer format 34 corresponding to the question number 31 of the target question.

If the target question is not a descriptive question in step 408, the question classification unit 14 outputs an error indicating that the answer format of the target question does not correspond to any of them.

FIG. 18 is a flowchart showing an example of the problem level evaluation process.

First, in step 501, the question classification unit 14 evaluates the question frequency of the target question. If the period during which no question is asked from the field corresponding to the prerequisite knowledge of the target problem exceeds the predetermined period threshold value, the target problem is determined to be level D. In general, many test takers often exclude areas that have not been asked for a long time from their study targets or lower their priority. Therefore, it is assumed that many examinees cannot answer questions in fields that have not been asked for a long time because they do not know the prerequisite knowledge. Problems from such fields can be presumed to have little impact on pass / fail.

Next, in step 502, the problem classification unit 14 evaluates the knowledge difficulty level of the corresponding field of the target problem. The relevant field of the target problem can be known by referring to the problem information 30 exemplified in FIG. The target field is not limited to one, and may be multiple. The knowledge difficulty level of the relevant field can be obtained by referring to the field information 40 exemplified in FIG. As shown in FIG. 16, for each of the fields 41, a knowledge difficulty level 44 indicating the difficulty level of acquiring the knowledge of the field 41 is preset. As an example, the instructor or the like may set the knowledge difficulty level of each field in advance. Alternatively, the knowledge difficulty level may be automatically set based on the amount of the field in the teaching material. For example, a field with many pages of text may have a high degree of knowledge difficulty. Alternatively, a field with a long e-learning time may have a high degree of knowledge difficulty.

If the period during which the question is not given from the relevant field of the target problem is equal to or less than the period threshold value, the problem classification unit 14 sets the problem level of the target problem according to the knowledge difficulty level of the relevant field. In this way, the difficulty level of acquiring knowledge in the relevant field of the target problem is reflected in the problem level, so it is possible to provide information that reflects the knowledge difficulty level that is useful for studying to acquire knowledge in the field to be acquired. can.

Specifically, the problem classification unit 14 classifies the problem level of the target problem into any of levels A, B, and C by comparing the maximum value of the relevant field of the target problem with a predetermined knowledge difficulty threshold. .. As mentioned above, level C is a problem in which the probability of correct answer is assumed to be close to 1/2, level B is a difficult problem in which correct answer is assumed to pass, and level A passes. Is an easy question that should be answered correctly. The knowledge difficulty threshold is set in advance so that the target problem is appropriately classified into those levels A, B, and C. Specifically, the higher the knowledge difficulty level, the lower the problem level. This is because it is assumed that the higher the knowledge difficulty level, the higher the possibility that many examinees have not acquired knowledge in the field.

The problem level set here is not finalized. As will be described later, it is adjusted based on the question sentence length and the answer format.

Subsequently, in step 503, the problem classification unit 14 evaluates the problem sentence length of the target problem. Specifically, when the problem level set for the target problem is level A, the problem classification unit 14 determines that the problem of the target problem is a problem if the problem sentence length of the target problem exceeds a predetermined problem sentence length threshold. Adjust the level to level B. Even if the question is asked from a field with low knowledge difficulty, if the question sentence becomes long, the number of places to which the prerequisite knowledge in the question sentence is associated increases, and it becomes difficult to grasp the intention of the questioner. On the other hand, problems in fields with high knowledge difficulty of premise knowledge have different characteristics, such as a large amount of knowledge that should be known as a premise and complicated mutual relationships. For problems in fields with high knowledge difficulty of prerequisite knowledge, if the problem level that matches the difficulty of knowledge is set in advance, the complexity of the relationship between knowledge and the problem has already been taken into consideration, so further problem sentences are required. Even if it is long, it does not necessarily mean that the difficulty level will rise more rapidly and the problem level will rise. Based on these properties, in this embodiment, only for the target problem of level A, if the problem sentence length exceeds a predetermined problem sentence length threshold value, the problem level is changed to level B, and the problem level is changed. We are trying to improve the validity.

Subsequently, in step 504, the question classification unit 14 executes an evaluation process for each answer format. The answer format-specific evaluation process is a process of evaluating the question level of the target question according to the answer format of the target question. There are various types of answers to the questions. The difficulty of deriving the correct answer differs depending on the answer format. Depending on the answer format, there is a problem that it is difficult to derive the correct answer unless all related knowledge is accurately understood and memorized. On the other hand, if some related knowledge is understood and memorized, there is a problem that the correct answer can be derived by the elimination method or the like. Such a difference in answer format can be reflected in the question level.

FIG. 19 is a flowchart of the evaluation process for each answer format.

In step 601 the problem classification unit 14 determines whether or not the target problem is an alternative problem. Whether or not the target question is an alternative question can be determined from the processing result of the answer format specifying process.

If the target question is an alternative question, the question classification unit 14 evaluates each option included in the alternative question in step 602. At that time, the problem classification unit 14 determines the problem level of the option based on the option type of each option. The option type of each option can be known from the processing result of the answer format processing. The problem classification unit 14 refers to the problem level determination information set in advance, and identifies the problem level corresponding to the option type of each option.

FIG. 20 is a diagram showing an example of problem level determination information. The question classification unit 14 is a table in which the question level 52 is registered in association with the answer format of the target question or the option type 51 of the options. If the target question is an alternative question, the question level is set for each option type, and if it is not an alternative question, the question level is set for each answer format.

In the example of FIG. 20, the question level of the knowledge question option of the alternative question is set to be A or higher. The question level of the low-level case question choices of the alternative question is set to be B or higher. Case law of alternative questions The question level of the question choices is set to B or higher. Advanced case of alternative questions The question level of the choices is set to C or higher.

If the question level set for the option is lower than the question level "○" set for "○ or higher" in the question level determination information 50, the question level for the option is changed to "○". .. For example, if the option is a low-level case question option for an alternative question and the question level is set to A, the question level is changed to B.

The case problem option is not a simple problem that can be solved only by knowing the prerequisite knowledge, because it is necessary to apply the prerequisite knowledge to the problem, unlike the knowledge problem option that asks whether or not the prerequisite knowledge is known. Therefore, the case question option has a lower problem level than the knowledge question option to ensure its validity. In addition, among the case problem options, the advanced case problem options with many appearing subjects are more likely to be applied than the low-level case problem options with few appearing subjects, so lower the problem level and improve the validity. I have secured it. Based on these properties, in this embodiment, the low-level case problem option is set to level B and the high-level case problem option is set to level C to ensure the validity of the problem level.

Subsequently, the problem classification unit 14 executes the alternative problem comprehensive evaluation process in step 603. The alternative question comprehensive evaluation process is a process of setting the problem level of the target problem, which is an alternative problem, based on the problem level of each option included in the target problem. The details of the alternative problem comprehensive evaluation process will be described later with reference to FIGS. 21 and 22.

If the target problem is not an alternative problem in step 601 the problem classification unit 14 evaluates the problem level of the target problem in step 604. At that time, the question classification unit 14 refers to the preset question level determination information 50 to specify the question level corresponding to the answer format of the target question.

In the example of FIG. 20, the problem level of the correctness problem is set to be A or higher. The problem level of the combination problem is set to be B or higher. The question level of the multiple choice question is set to B or higher. The problem level of the computational problem is set to be C or higher. The problem level of the number problem is set to be C or higher. The question level of the descriptive question is set to C or higher.

If the question level set for the target question is lower than the question level "○" set in the question level judgment information 50 as "○ or higher", the question level of the target question is changed to "○". .. For example, if the target problem is a combination problem and the problem level is set to A, the problem level is changed to B.

That is, the problem classification unit 14 changes the target problem to level B if the problem level set by the prerequisite knowledge of the target problem and the problem sentence length is level A and the target problem is a combination problem or a multiple-choice problem. .. If the problem level set by the prerequisite knowledge of the target problem and the problem sentence length is level A or level B, and the target problem is either a computational problem, a number problem, or a descriptive problem, the target problem is set to level C. change. It is possible to provide a reasonable question level adjusted by the answer format.

FIG. 21 is a diagram showing an example of determination logic information of the alternative problem comprehensive evaluation process. FIG. 22 is a flowchart showing an example of the alternative problem comprehensive evaluation process.

The alternative question comprehensive evaluation is a process of setting the question level of the alternative question by the comprehensive evaluation based on the question level of each option. Comprehensive evaluation is performed based on the determination logic information shown in FIG.

In the alternative question, if it is not possible to make a correct judgment for all the options, the correct answer will be derived by narrowing down the options by elimination. Therefore, the ease of deriving the correct answer to the alternative question changes depending on the ease of narrowing down the options by the elimination method. In this embodiment, the degree of narrowing down is associated with the comprehensive evaluation.

As shown in FIG. 21, the determination logic information 60 includes a condition for determining whether or not the option of the target problem is applicable (condition content 62) and a problem level of the target problem corresponding to the condition (comprehensive evaluation B63). Is associated with the above, and the condition number 61 is assigned and set. In this example, the determination is made in ascending order of the number of the condition number 61.

Under the condition of condition number = 1, the problem level as a comprehensive evaluation of the target problem corresponding to the condition content "level A and the option of the correct limb is included" is A. Under the condition of condition number = 2, the problem level as a comprehensive evaluation of the target problem corresponding to the condition content "the number of choices that are level A and incorrect limb is included is one less than the total number of choices" is A. Is. Under the condition of condition number = 3, the problem level as a comprehensive evaluation of the target problem corresponding to the condition content "the narrowing condition 2 is not applicable and the option of level B and the correct limb is included" is B. The condition number = 4 corresponds to the condition content "the number of choices that do not correspond to the narrowing conditions 1 and 3 and are level A or B and are incorrect answers is included by one less than the total number of choices". The problem level as a comprehensive evaluation of the target problem to be performed is B. Under the condition of condition number = 5, the problem level as a comprehensive evaluation of the target problem corresponding to the condition content "the narrowing conditions 2 and 4 do not correspond to the level C and the option that is the correct limb is included" is C. be. Under the condition of condition number = 6, the condition content "Neither of the narrowing conditions 1 to 5 is applicable, and the number of choices that are level A or B and that are incorrect limbs is included by 2 less than the total number of choices. The problem level as a comprehensive evaluation of the target problem corresponding to "is" is C. Under the condition of condition number = 7, the problem level as a comprehensive evaluation of the target problem corresponding to the condition content "does not correspond to any of the narrowing conditions 1 to 6" is D.

The alternative problem comprehensive evaluation process of FIG. 22 is realized by a series of flowcharts.

First, in step 701, the problem classification unit 14 determines whether or not the target problem corresponds to the narrowing condition 1. If the target problem corresponds to the narrowing condition 1, the problem classification unit 14 sets the problem level of the target problem to A in step 703.

If the target problem does not correspond to the narrowing condition 1, the problem classification unit 14 then determines in step 702 whether or not the target problem corresponds to the narrowing condition 2. If the target problem corresponds to the narrowing condition 2, the problem classification unit 14 sets the problem level of the target problem to A in step 703.

If the target problem does not correspond to the narrowing condition 2, the problem classification unit 14 then determines in step 704 whether or not the target problem corresponds to the narrowing condition 3. If the target problem corresponds to the narrowing condition 3, the problem classification unit 14 sets the problem level of the target problem to B in step 706.

If the target problem does not correspond to the narrowing condition 3, the problem classification unit 14 then determines in step 705 whether or not the target problem corresponds to the narrowing condition 4. If the target problem corresponds to the narrowing condition 4, the problem classification unit 14 sets the problem level of the target problem to B in step 706.

If the target problem does not correspond to the narrowing condition 4, the problem classification unit 14 then determines in step 707 whether or not the target problem corresponds to the narrowing condition 5. If the target problem corresponds to the narrowing condition 5, the problem classification unit 14 sets the problem level of the target problem to C in step 709.

If the target problem does not correspond to the narrowing condition 5, the problem classification unit 14 then determines in step 708 whether or not the target problem corresponds to the narrowing condition 6. If the target problem corresponds to the narrowing condition 6, the problem classification unit 14 sets the problem level of the target problem to C in step 709.

If the target problem does not correspond to the narrowing condition 6, the problem classification unit 14 sets the problem level of the target problem to D in step 710.

The processing of FIG. 22 can also be organized as follows.

The problem classification unit 14 sets the target problem to level A if the target problem meets the narrowing condition 1 that the alternative question includes the option of level A and the correct answer. If the target problem corresponds to the narrowing condition 2 that the number of options that are level A and incorrect answer is included in the alternative question by one less than the number of options, the problem classification unit 14 sets the target problem to the level. Set to A. If the target problem does not correspond to the narrowing condition 2 and the target problem corresponds to the narrowing condition 3 that the option that is level B and the correct answer is included in the alternative question, the problem classification unit 14 sets the target problem to level B. Set. In the problem classification unit 14, the target problem does not correspond to the narrowing condition 1 or the narrowing condition 3, and the option of level A and incorrect answer or level B and incorrect answer is 1 from the total number of options for the alternative question. If the narrowing condition 4 that only a small number of pieces are included is satisfied, the target problem is set to level B. If the target problem does not correspond to the narrowing condition 2 or the narrowing condition 4, the problem classification unit 14 corresponds to the narrowing condition 5 that the option of level C and the correct answer is included in the alternative question. Set the target question to level C. In the problem classification unit 14, the target problem does not correspond to any of the narrowing conditions 1 to 5, and the option of level A and incorrect answer or level B and incorrect answer is 2 from the total number of options for the alternative question. If the narrowing condition 6 that only a small number of pieces are included is satisfied, the target problem is set to level C. If the target problem does not meet any of the narrowing conditions 1 to 6, the problem classification unit 14 sets the target problem to level D. About the alternative question The question level of the whole alternative question and each option can be provided as useful information for studying for passing.

Returning to FIG. 3, in step 104, the answer analysis unit 15 executes the answer analysis process.

FIG. 23 is a flowchart showing an example of the answer analysis process.

First, in step 801 the answer analysis unit 15 analyzes the test results by subject.

The answer analysis unit 15 calculates the deviation value of the estimated failing person based on the answer data of the estimated passing person for each subject included in the examination. The average score of the estimated passers is the deviation value 50. Then, the answer analysis unit 15 determines the performance of each subject by referring to the predetermined determination standard information based on the deviation value of each subject.

FIG. 24 is a diagram showing an example of determination criterion information.

In the determination reference information 70, the determination 72 and the countermeasure 73 are set in association with the deviation value range 71. Judgment 72 is information indicating the degree of test performance of the subject in a plurality of stages. There are from A judgment to D judgment, A judgment is the judgment with the best condition, and D judgment is the judgment with the worst condition. The measure 73 is information indicating measures for future learning for each determination. Judgments and measures for each subject will be provided to the examinees (estimated failers).

When the deviation value range 71 is 55 or more, the determination 72 is A, and the examinee may maintain the current state of the measure 73 without any special measures for the subject. When the deviation value range 71 is 50 or more and less than 55, the judgment 72 is B, and the examinee needs to take measures for the subject, and it is desirable to make an effort to make the subject more good. When the deviation value range 71 is 45 or more and less than 50, the judgment 72 is C, and the examinee needs to take measures for the subject, and the examinee is required to make an effort to aim at the average score or more. If the deviation value range 71 is less than 45, the determination 72 is D, and the measure 73 requires the examinee to take considerable measures for the subject.

Candidates can find out which subjects are good or bad among the multiple subjects included in the exam. Generally, data such as the score distribution of the estimated passers in the test is not published, but according to the present embodiment, the estimated failers obtain objective information comparing the score distribution of the estimated passers with their own score distribution. be able to.

FIG. 25 is a diagram showing an example of the determination result for each subject. FIG. 25 shows the judgment results for each subject of Mr. A in the home-building test. The subject-specific determination result 80 shows the score 82, the average score of the estimated passers 83, the score 84 of Mr. A, and the subject-specific determination 85 for each subject 81.

In the home-building test, the subject 81 is the Home-building Business Law, the score is 20 points, the average score of the estimated passers is 18.5 points, the score of Mr. A is 20 points, and the judgment by subject is 85. Is A. Regarding the legal restrictions on subject 81, the score is 8 points, the average score 83 of the estimated passers is 5.5 points, the score of Mr. A is 2 points, and the judgment 85 by subject is D. ing. For subjects 81 and other laws and regulations, the score is 8 points, the average score 83 of the estimated passers is 5.5 points, Mr. A's score is 3 points, and the subject-specific judgment 85 is C. There is. Regarding the rights of subject 81, the score is 14 points, the average score of the estimated passers is 9.5 points, the score of Mr. A is 10 points, and the judgment by subject 85 is B. ..

In the case of a qualification test, etc., when the test taker's performance is said to be unsuccessful immediately after the test, the person himself / herself makes a subjective judgment in most cases. On the other hand, according to the present embodiment, the examinee can objectively judge the success or failure of the main test. Further, in the present embodiment, the deviation value of the estimated failing person is calculated based on the average score of the estimated passing person for each subject, and the judgment is made. It can be known by comparison with the person.

Returning to FIG. 23, in step 802, the answer analysis unit 15 analyzes the test results by question.

The answer analysis unit 15 calculates the estimated passer correct answer rate for each question from the answer data of the estimated passers, and analyzes the answer to the question of the estimated failer based on the estimated passer correct answer rate of each question. Process the analysis result and output it in table format.

FIG. 26 is a table showing the analysis results of the problem-specific analysis.

Referring to FIG. 26, the result of analyzing Mr. A's answer to each question is shown. In the table 90 shown in FIG. 26, for each of the 50 questions included in the exam, the subject 92, the field 93, the correctness 94, the estimated passer correct answer rate 95, the question level 96, and the question are associated with the question number 91. Type 97 is recorded.

The answer analysis unit 15 records in the correct / incorrect 94 of the table 90 whether the answer is correct or incorrect determined by the pass / fail estimation unit 13. In FIG. 26, the correct answer is indicated by ◯ (circle mark) and the incorrect answer is indicated by × (cross mark). The answer analysis unit 15 records the subject, field, question level, and question type of the question specified by the question classification unit 14 in the subject 92, field 93, question level 96, and question type 97 of the table 90, respectively. Further, the answer analysis unit 15 records the calculated estimated passer correct answer rate in the estimated passer correct answer rate 95 of the table 90. For example, in the question of question number 1, the subject is related to rights, the field is a statement of intention, and Mr. A's correctness is correct.

FIG. 27 is another table showing the analysis results of the problem-specific analysis. In the table A0 shown in FIG. 27, the number of questions A2, the number of correct answers A3 of Mr. A, and the correct answer rate A4 of Mr. A are recorded in association with the range A1 of the estimated correct answer rate.

Estimated passer correct answer rate range A1 is 0% or more and less than 10%, 10% or more and less than 20%, 20% or more and less than 30%, 30% or more and less than 40%, 40% or more and less than 50%, 50% or more and 60%. The range of less than, 60% or more and less than 70%, 70% or more and less than 80%, 80% or more and less than 90%, 90% or more and 100% or less is set.

The answer analysis unit 15 identifies the questions whose estimated passer correct answer rate is included in each of the above ranges among the questions included in the examination, and the number of the questions (referred to as N1) and A among them. The number of questions answered correctly by Mr. A (assumed to be N2) and N1 / N2 are calculated and recorded in the number of questions A2, the number of correct questions A3 by Mr. A, and the correct answer rate A4 by Mr. A, respectively.

In order to pass the exam, it is important to ensure that successful applicants answer the correct questions. Looking at the table A0 in FIG. 27, the estimated correct answer rate is higher in the upper range. Therefore, in order to pass, it is required to answer the questions included in the upper range with a high correct answer rate, but in the example of FIG. 27, it can be seen that Mr. A has not been able to do so.

In addition, the answer analysis unit 15 calculates the estimated passer correct answer rate for each question from the answer data of the estimated passer, and based on the estimated passer correct answer rate and the question level of each question, the estimated failer to the question. Analyze the answer, process the analysis result and output it in table format.

FIG. 28 is yet another table showing the analysis results of the problem-specific analysis. In the table B0 shown in FIG. 28, the number of questions B2, the number of correct questions B3 of Mr. A, the correct answer rate B4 of Mr. A, and the correct answer rate B5 of the estimated passers are recorded in association with the question level B1.

Among the questions included in the test, the answer analysis unit 15 includes the number of questions whose question level corresponds to each level (referred to as N3), the number of questions answered correctly by Mr. A (referred to as N4), and N4 / N3. , The correct answer rate (referred to as P1) of the estimated passers is calculated, and recorded in the number of questions B2, the number of correct questions B3 of Mr. A, the correct answer rate B4 of Mr. A, and the correct answer rate B5 of the estimated passers, respectively. P1 is calculated by the following equation (1).

P1 = (Integrated value of the number of estimated passers who correctly answered each question whose question level corresponds to a predetermined level) / (N3 x (estimated number of passers)) ... (1)

Looking at the table B0 in FIG. 28, at levels A, B, and C, the correct answer rate B4 of Mr. A is lower than the correct answer rate B5 of the estimated passers. On the other hand, at level D, the correct answer rate B4 of Mr. A exceeds the correct answer rate B5 of the estimated passers. The level D problem is a problem that is expected to pass without a correct answer, and it can be said that it is a difficult problem to answer correctly with ability. It is highly possible that the correct answer rate B4 of Mr. 9A at level D is higher than the correct answer rate B5 of the estimated passers because the correct answer is a fluke. Mr. A is required to be able to obtain a correct answer rate B4 higher than the correct answer rate B5 of the estimated passers at levels A, B, and C.

The answer analysis unit 15 calculates the estimated passer correct answer rate for each question from the answer data of the estimated passers, and based on the estimated passer correct answer rate and the question type of each question, the answer to the question of the estimated failer is given. Analyze, process the analysis result and output it in table format.

FIG. 29 is yet another table showing the analysis results of the problem-specific analysis. In the table C0 of FIG. 29, the number of questions C2, the number of correct questions C3 of Mr. A, the correct answer rate C4 of Mr. A, and the correct answer rate C5 of the estimated passers are recorded in association with the question type C1.

Among the questions included in the test, the answer analysis unit 15 includes the number of questions for which the question type corresponds to each type (N5), the number of questions answered correctly by Mr. A (N6), and N6 / N5. , The correct answer rate (referred to as P2) of the estimated passers is calculated, and recorded in the number of questions C2, the number of correct questions C3 of Mr. A, the correct answer rate C4 of Mr. A, and the correct answer rate C5 of the estimated passers, respectively. P2 is calculated by the following equation (2).

P2 = (Integrated value of the number of estimated passers who correctly answered each question whose question type corresponds to the predetermined type) / (N5 x (estimated number of passers)) ... (2)

In order to pass the exam, it is important not to create question types that you are not good at, and it is necessary to obtain the same correct answer rate as successful applicants for all question types. Looking at the table C0 in FIG. 29, the correct answer rate C4 of Mr. A is lower than the correct answer rate C5 of the estimated passers in the case problem and the number problem of the alternative question. Mr. A will make an effort to increase the correct answer rate in the case question and the number of alternative questions with the goal of obtaining a correct answer rate that exceeds the correct answer rate C5 of the estimated passers in all question types. good.

The present embodiment described above includes the following items. However, the matters included in this embodiment are not limited to the following.

The test result analysis system has an answer data collection unit 12 that collects answer data indicating the answers of each question submitted by a plurality of examinees who have taken an examination including a plurality of questions, and answer data of a plurality of examinees. Based on the pass / fail estimation unit 13 that estimates the pass / fail of each of the plurality of examinees, and the answer data of the estimated passers that are presumed to pass, the estimated failers that are presumed to have failed to solve the problem. It has an answer analysis unit 15 for analyzing an answer. According to this aspect, a plurality of examinees are divided into estimated passers and estimated failers, and the answers to each question of the estimated passers are analyzed based on the answer data of the estimated passers, so that the pass / fail is announced. The results of the test can be analyzed before it is done.

Further, the answer analysis unit 15 calculates the estimated passer correct answer rate, which is the ratio of the estimated passers who correctly answered the question to the whole estimated passers, for each question in the answer data of the estimated passers, and estimates the question. Based on the passer correct answer rate, the answers to the above-mentioned questions of the estimated failers are analyzed. According to this aspect, the answers to each question of the estimated passers are analyzed based on the correct answer rate of the estimated passers, so the percentage of the estimated passers who answer correctly before the pass / fail is announced. It is possible to analyze in consideration of.

Further, the answer analysis unit 15 divides the estimated passer correct answer rate into a plurality of ranges, and for the questions belonging to each range, the number of questions correctly answered by the estimated passer to be analyzed with respect to the number of questions belonging to the range. / Or calculate the ratio of the number. According to this aspect, it is possible to analyze the test result by comparing the estimated correct answer rate of the passers with the number of correct answers and / or the correct answer rate of the estimated unsuccessful persons to be analyzed.

Further, the test result analysis system further has a question classification unit 14 that classifies a plurality of questions into a plurality of question levels indicating the degree to which the question is required to be answered correctly in order to pass the test, and the answer analysis unit. Fifteen analyzes the answer to the question of the estimated failers based on the question level of the question and the estimated passer correct answer rate. According to this aspect, the answer to each question of the estimated failing person is analyzed based on the degree to which the question is required to be answered correctly in order to pass the examination and the correct answer rate of the estimated passing person of the question. An effective analysis that considers how much you are required to answer the question correctly in order to pass the exam and what percentage of the estimated passers are correct before the pass / fail is announced. It will be possible.

Further, the answer analysis unit 15 divides the number of questions correctly answered by the presumed unsuccessful person among the questions corresponding to the question level by the number of questions corresponding to the question level for each of the plurality of question levels. Calculate the correct answer rate of the estimated failing person, calculate the total number of estimated passing people who answered correctly for each question corresponding to the problem level, and calculate the number of questions corresponding to the problem level and the estimated number of passing people. The correct answer rate of the estimated passer is calculated by dividing by the value obtained by multiplication, and the correct answer rate of the estimated passer and the correct answer rate of the estimated passer can be compared.

Further, the test result analysis system further has a question classification unit 14 that classifies a plurality of questions into a plurality of question types according to the format of the answer to the test, and the answer analysis unit 15 further has a question type of the question and an estimation pass. Based on the correct answer rate, the answers to the questions of the estimated failers are analyzed. According to this aspect, the answer to each question of the estimated failing person is analyzed based on the format of the answer to the question and the correct answer rate of the estimated passing person of the question. It is possible to make an effective analysis considering the format of the answer to the question and the percentage of the estimated passers who answered the question correctly.

In addition, the answer analysis unit 15 divides the number of questions correctly answered by the estimated failing person among the questions corresponding to the question type by the number of questions corresponding to the question level for each of the plurality of question types. Calculate the correct answer rate of the estimated failing person, calculate the total number of estimated passing people who answered correctly for each question corresponding to the question type, and calculate the number of questions corresponding to the problem type and the estimated number of passing people. The correct answer rate of the estimated passer is calculated by dividing by the value obtained by multiplication, and the correct answer rate of the estimated passer and the correct answer rate of the estimated passer can be compared.

In addition, the test result analysis system has a plurality of question levels indicating the degree to which the question is required to be answered correctly in order to pass the test, and a plurality of question types according to the answer format for the test. The answer analysis unit 15 further has a question classification unit 14 for classifying into ,, and analyzes the answers to the questions of the estimated failers based on the question level, the question type, and the estimated passer correct answer rate. According to this aspect, the presumed passer is concerned with the degree to which the question is required to be answered correctly in order to pass the exam and the format of the answer to the question before the pass / fail is announced. It is possible to make an effective analysis considering the percentage of correct answers to the problem.

Further, the problem classification unit 14 determines the prerequisite knowledge required to solve the problem, calculates the problem sentence length which is the length of the sentence of the problem, specifies the answer format for the problem, and determines the prerequisite knowledge and the problem sentence length. The question level is determined based on the answer format and the question type is determined based on the answer format, and the question level and the question type are used for the analysis of the answers of the presumed rejecters.

Further, the examination includes questions of a plurality of subjects, and the answer analysis unit 15 calculates the deviation value of the estimated failing person based on the answer data of the estimated passing person for each of the subjects. According to this aspect, in a test that includes a plurality of subjects and the pass / fail is determined by the total score of the subjects, the deviation value for each subject of the estimated failing person is calculated based on the answer data of the estimated passing person. It is possible to objectively analyze the results of each subject.

The embodiments of the present invention described above are examples for the purpose of explaining the present invention, and the scope of the present invention is not limited to those embodiments. One of ordinary skill in the art can practice the invention in various other embodiments without departing from the scope of the invention.

10 ... Test result analysis system, 11 ... Pass line acquisition unit, 12 ... Answer data collection unit, 13 ... Pass / fail estimation unit, 14 ... Problem classification unit, 15 ... Answer analysis unit, 20 ... Computer system, 21 ... Processor, 22 ... Main memory, 23 ... Storage device, 24 ... Communication device, 25 ... Input device, 26 ... Display device

Claims (11)

An answer data collection unit that collects answer data showing the answers to each question submitted by multiple test takers who have taken an exam containing multiple questions,
A pass / fail estimation unit that estimates the pass / fail of each of the plurality of examinees based on the answer data of the plurality of examinees.
Based on the answer data of the presumed passers who are presumed to pass, the answer analysis unit that analyzes the answers to the above-mentioned problems of the presumed failers who are presumed to fail,
Test result analysis system with. The answer analysis unit calculates the estimated passer correct answer rate, which is the ratio of the estimated passers who correctly answered the question to the total of the estimated passers, for each question in the answer data of the estimated passers, and estimates the question. Based on the passer correct answer rate, the answer to the question of the estimated failer is analyzed.
The test result analysis system according to claim 1. The answer analysis unit divides the estimated passer correct answer rate into a plurality of ranges, and for the questions belonging to each range, the number of questions correctly answered by the estimated passer to be analyzed with respect to the number of questions belonging to the range and / Or calculate the ratio of the number,
The test result analysis system according to claim 2. It further has a question classification unit that classifies the plurality of questions into a plurality of question levels indicating the degree to which the questions are required to be answered correctly in order to pass the examination.
The answer analysis unit analyzes the answers to the questions of the estimated failers based on the question level of the question and the estimated passer correct answer rate.
The test result analysis system according to claim 2. For each of the plurality of question levels, the answer analysis unit divides the number of questions correctly answered by the estimated unsuccessful person among the questions corresponding to the question level by the number of questions corresponding to the question level. Calculate the correct answer rate of the estimated passers, multiply the total number of estimated passers who answered each question corresponding to the question level by the number of questions corresponding to the question level and the number of estimated passers. The correct answer rate of the estimated passer is calculated by dividing by the value obtained in the above process, and the correct answer rate of the estimated passer can be compared with the correct answer rate of the estimated passer.
The test result analysis system according to claim 4. It further has a question classification unit that classifies the plurality of questions into a plurality of question types according to the format of the answer to the test.
The answer analysis unit analyzes the answers to the questions of the estimated failers based on the question type of the question and the estimated passer correct answer rate.
The test result analysis system according to claim 2. For each of the plurality of question types, the answer analysis unit divides the number of questions correctly answered by the estimated unsuccessful person among the questions corresponding to the question type by the number of questions corresponding to the question level. Calculate the correct answer rate of the estimated failers, multiply the total number of estimated passers who answered each question corresponding to the question type correctly, and multiply the number of questions corresponding to the question type by the number of estimated passers. The correct answer rate of the estimated passer is calculated by dividing by the value obtained in the above process, and the correct answer rate of the estimated passer can be compared with the correct answer rate of the estimated passer.
The test result analysis system according to claim 6. Question classification that classifies the plurality of questions into a plurality of question levels indicating the degree to which the question is required to be answered correctly in order to pass the exam, and a plurality of question types according to the answer format for the exam. Has more parts,
The answer analysis unit analyzes the answers to the questions of the estimated failers based on the question level, question type, and estimated passer correct answer rate of the question.
The test result analysis system according to claim 2. The problem classification unit
Determine the prerequisite knowledge required to solve the above problem,
Calculate the problem sentence length, which is the length of the problem sentence,
Identify the answer format for the question and
The question level is determined based on the prerequisite knowledge, the question sentence length, and the answer format.
Determine the question type based on the answer format and
The question level and the question type are used in the analysis of the answer of the presumed failer.
The test result analysis system according to claim 8. The exam includes multiple subject questions
The answer analysis unit calculates the deviation value of the estimated failing person based on the answer data of the estimated passing person for each of the subjects.
The test result analysis system according to claim 1. We collected answer data showing the answers to each question submitted by multiple test takers who took the test including multiple questions.
Based on the answer data of the plurality of examinees, the pass / fail of each of the plurality of examinees is estimated.
A test result analysis method performed by a computer to analyze the answers to the above-mentioned problems of the estimated failers who are estimated to fail based on the answer data of the estimated passers who are estimated to pass.

Images