JP4659433B2 - Problem automatic creation device and problem automatic creation program - Google Patents

Description

  The present invention relates to a technique for automatically creating a multiple-choice question, and in particular, an automatic question creation device for automatically creating a multiple-choice question used in, for example, so-called e-learning or mobile learning. And an automatic problem creation program.

  With changes in the economy and society, employment forms are beginning to change. The idea of improving one's own career by actively changing jobs is also spreading. For this reason, an increasing number of people are trying to acquire qualifications useful for work. Many people think that it is more advantageous to have some qualifications not only for changing jobs but also for the first employment, so preparatory schools, cram schools, correspondence courses, etc. for obtaining various qualifications are popular.

  One of the most popular qualification tests is to measure English communication skills. As a condition for studying abroad, it is often required to obtain a specified score or more in a specific examination.

  By the way, the problem in such tests is how to make the scoring accurate. In particular, when there are a large number of examinees, the time taken to obtain results increases because of the effort for scoring, the accuracy of scoring decreases due to an increase in the number of graders, and the cost of conducting the test. There is a problem that becomes high.

  Therefore, as a form of various test questions, a multiple-choice question in which a correct option is selected from a plurality of options provided for one question is frequently used. In the case of a multiple choice question, there are advantages that scoring is easy and can be automated, and results can be obtained in a short time. As a result, there is no need to increase the number of graders, and even if the number of examinees increases, an increase in cost can be suppressed.

  Also, in a society where network technology and communication technology have been developed as in modern Japan and communication tools such as mobile phones have become popular, learning services for qualification tests that can be used anytime and anywhere in a short time using such communication tools More users are seeking The multiple-choice question is also effective for such services.

  In such a learning form using a personal computer or a mobile device, it is necessary to efficiently create a problem on the practice question providing side. In order to create a multiple-choice question, it is necessary to prepare not only a question and a correct answer to the question but also an option corresponding to an incorrect answer. However, in order to create an option that will give an incorrect answer to a question, it is necessary to first create an incorrect answer candidate and confirm that the candidate cannot be the correct answer to the question. . In addition, in order to increase the difficulty level as a problem, it is also necessary to create an option that is misleading even if it is an incorrect answer.

  However, in order to perform such work, for example, in the case of a language test, it was thought that it was necessary to accurately understand the meaning of sentences as candidates for options. Therefore, in the past, humans with sufficient ability to create test questions have created such questions manually.

  However, in the future, it is possible that a larger number of people will learn using communication tools, and in order to respond to such demand, there is a technology to create appropriate multiple-choice questions efficiently and automatically. It has been demanded.

  Accordingly, an object of the present invention is to provide an automatic problem creation apparatus for a multiple-choice question that can automatically create an appropriate multi-choice question and a computer program therefor.

  Another object of the present invention is to provide a multiple-choice question automatic creation apparatus and a computer program therefor capable of automatically and efficiently creating an appropriate multiple-choice question. is there.

  An automatic problem creation device according to an aspect of the present invention selects an arbitrary condition-response pair as a correct answer pair from a corpus that includes a plurality of condition-response pairs each composed of a condition and a response to the condition. Correct answer selection means, a correct answer pair selected by the correct answer selection means, a set of conditional response pairs (hereinafter referred to as a first set) having a response equivalent to the response of the correct answer pair, and for correct answer Creating an incorrect answer candidate set for obtaining a set of incorrect answer option candidates by removing from the corpus a set of condition-response pairs (hereinafter referred to as a second set) having a condition equivalent to the pair condition. A correct answer is obtained by combining a correct number of incorrect responses selected from arbitrary condition-response pairs included in the set of means and incorrect answer option candidates obtained with the responses of the correct pairs. And a problem creating means for creating a multiple-choice question to select a response corresponding to the A conditions.

  The first set is a set having responses included in the expression set equivalent to the response of the correct pair, and these responses can be correct with respect to the condition of the correct pair. If the response included in this set is combined with the response of the correct answer pair, a plurality of correct answers exist, which is inappropriate as a problem. The second set is a set having a condition included in an expression set equivalent to the condition of the correct answer pair, and their responses can be correct with respect to the condition of the correct answer pair. If these responses are combined with the response of the correct answer pair, there are a plurality of correct answers, which is also inappropriate as a problem. For this reason, the correct answer pair and the first and second sets are deleted from the corpus to obtain an incorrect answer option set. In this set of incorrect answer choices, the existence of a response in which multiple correct answers may appear for the condition of the correct answer pair is excluded, and there is only one correct combination of condition and response, A proper multiple-choice question is created. Problem creation work is done automatically without human intervention, which is efficient and leads to cost reduction. In addition, since the number of options can be set freely, the degree of freedom of combination of options can be greatly improved, and by preparing conditional response pairs for natural dialogue and other various condition response pairs in the corpus, It is also possible to automatically create more advanced problems.

  Preferably, the automatic question creation apparatus further includes a set of conditional response pairs (hereinafter referred to as the first set of conditional response pairs) having a response similar to the response of the correct answer pair from the set of incorrect answer option candidates created by the incorrect answer candidate set creating means. Means for giving the question creation means a set of incorrect answer choices from which 3) is deleted.

  Since the response included in the third set is dissimilar to the response of the correct pair, the candidate can avoid this option without hesitation even if it is included in the option. By excluding these responses from the set of incorrect answer options, a response that is completely different from the correct answer is not set as an option. As a result, it is possible to create a question suitable for test practice, which has an option that may be confusing to the examinee.

  As specific examples of conditions and responses in a condition-response pair, if the condition is a question and the response is an answer to the question, the condition is an image and the response is an explanatory text about the image, or the condition is a sentence and a response May be a summary of a sentence.

  Further, the automatic question creation apparatus further includes means for repeatedly operating the correct answer selection means, the incorrect answer candidate set creation means, and the problem creation means until a predetermined number of multiple-choice questions are created by the question creation means. May be included.

  The automatic problem creation apparatus automatically creates a predetermined number of problems and automatically stops when the creation is completed. Since you can create as many questions as you need, you can create problems efficiently.

  A computer program according to another aspect of the present invention, when executed by a computer, causes the computer to operate as one of the above-described problem automatic creation devices.

  The embodiment of the present invention described below is realized by a computer and software operating on the computer. Of course, some or all of the functions described below can be realized by hardware instead of software.

  FIG. 1 shows an external view of a computer system 20 used in the present embodiment, and FIG. 2 shows a block diagram of the computer system 20. The computer system 20 shown here is merely an example, and various other configurations are possible.

  Referring to FIG. 1, the computer system 20 includes a computer 40, a monitor 42, a keyboard 46, and a mouse 48, all connected to the computer 40. The computer 40 further includes a CD-ROM (Compact Disc Read-Only Memory) drive 50 and an FD (Flexible Disk) drive 52.

  Referring to FIG. 2, computer system 20 further includes a printer 44 connected to computer 40, which is not shown in FIG. The computer 40 further includes a bus 66 connected to the CD-ROM drive 50 and the FD drive 52, a central processing unit (CPU) 56 connected to the bus 66, a boot-up program for the computer 40, and the like. ROM (Read-Only Memory) 58 storing RAM, RAM (Random Access Memory) 60 providing a work area used by the CPU 56 and a storage area for programs executed by the CPU 56, and a hard disk 54 storing a corpus to be described later. .

  The software for realizing the system of the embodiment described below is recorded and distributed on a recording medium such as a CD-ROM 62, and is read into the computer 40 via a reader such as the CD-ROM drive 50. Stored in the hard disk 54. When the CPU 56 executes this program, the program is read from the hard disk 54 and stored in the RAM 60, and an instruction is read from an address designated by a program counter (not shown) and executed. The CPU 56 reads data to be processed from the hard disk 54 and stores the processing result in the hard disk 54 as well.

  Since the operation itself of the computer system 20 is well known, details thereof will not be repeated here.

  The distribution form of software is not limited to the form fixed to the recording medium as described above. For example, data may be distributed in the form of receiving data from other computers connected through a network. Further, there may be a distribution form in which a part of software is stored in the hard disk 54 in advance, and the remaining part of the software is taken into the hard disk 54 via the network and integrated at the time of execution.

  In general, modern programs utilize the general purpose functions provided by a computer operating system (OS) and achieve the desired objectives described above by executing them in an organized manner according to the desired objectives. . Therefore, among the functions of this embodiment described below, a program (group) that does not include general-purpose functions provided by the OS or a third party, and specifies only a combination of execution orders of these general-purpose functions. Even so, as long as the program (group) has a control structure that achieves a desired object as a whole by using them, it is obvious that they are included in the technical scope of the present invention.

[Constitution]
FIG. 3 functionally shows the configuration of the automatic problem creating apparatus 82 on the assumption that the automatic problem creating apparatus 82 is realized by the program of the present embodiment. Referring to FIG. 3, this automatic question creating apparatus 82 stores text data of English questions (conditions) and voice data thereof and text data of answers (responses) to the question text stored in the hard disk 54 (or This is for creating a multiple-choice question regarding English listening ability using a corpus 80 including a large number of audio data pairs. That is, in this embodiment, text data or voice data of a question sentence is used as a condition, and a response to the condition is an answer sentence. The corpus 80 is a collection of a large number of such condition response pairs.

In the following description, as shown in FIG. 4, an English listening problem in which the condition is the question Q _K and the response is the answers A _K and A _i to the question will be described as an example. In the example shown in FIG. 4, the answer _AK is a correct answer. In order to simplify the explanation, the number of choices is two in the following example. However, in the actual problem, it is considered that the number of choices is in the range of 3 or more and 5 or less.

  The automatic question creation apparatus 82 includes a correct answer selection unit 90 for selecting a question sentence and a correct answer pair from the corpus 80, and a correct answer selected by the correct answer selection part 90 from the pairs included in the corpus 80. By removing from the corpus 80 a set of conditional response pairs having a response equivalent to the response of the correct pair, a correct response pair, and a set of conditional response pairs having a condition of the expression equivalent to the correct pair condition. A set computing unit 92 for obtaining a set of correct option candidates and creating a set of answer sentences that are candidates for incorrect answers, a correct pair selected by the correct answer selecting unit 90, and a set computing unit 92 A question creation unit 94 for creating a multiple-choice question from the set of answer sentences that are selected as incorrect answers.

A configuration example of the corpus 80 stored in the hard disk 54 is shown in FIG. Referring to FIG. 5, the corpus 80 is a set of problems C = {(Q _i → A _i ) | i∈N} in which a large number of pairs including a question Q _i and a correct response A _i to the question Q _i are collected ( N is a set of natural numbers). The question Q _i and the response A _i are prepared in advance so that the response A _i is a correct answer to the question Q _i .

  FIG. 6 shows a flowchart of a computer program for realizing each function of the automatic problem creation apparatus 82. Hereinafter, the function of each unit in FIG. 3 will be described with reference to each unit in FIG. 3 and FIG. 6.

Answer selection unit 90 has from corpus 80, any question answering pair (Q _{K, A K)} select as the correct use pairs of Q _K problem condition, a function of the choice of the correct answer A _K (Step 110 in FIG. 6).

  The set calculation unit 92 obtains a set of incorrect answer options (steps 112 to 124 in FIG. 6). Specifically, the set calculation unit 92 obtains a set of incorrect answer choices as follows.

First, a set C ′ = C− (Q _K , A _K ) is obtained by subtracting the correct pair (Q _K , A _K ) from the set C constituting the corpus 80 (step 112).

Next, the set operation section 92, of a set C ', extracts what response is a Genkae response A _K of the correct (ie) as a first set (step 114). Hereinafter, a set of sentences that is a paraphrase of a certain sentence X is hereinafter referred to as PARAPHRASE (X).

For example, if the response A _i = “Yes, I want to know his name” shown in FIG. 4 and FIG. 5 is compared with the response A _j = “Yes, I want to know his name” to be compared, the response A _i = “Yes, I ′d like to know his name” _j ∈ PARAPHRASE (A _i ). There are various of whether a method of determining a statement is in the word instead of the response A _K of the correct answer. For example, it is possible to determine by using a dictionary in which different expressions are collected with the same meaning and collating with this dictionary. In addition, “paraphrase (paraphrase)” means the same meaning in another way, that is, both are equivalent. On the other hand, “similarity” to be described later means that words or syntax appearing in two sentences are similar, and is determined by a predetermined criterion for determining whether or not they are similar. Therefore, “paraphrase” and “similarity” are different concepts.

Response that is the word instead of the response A _K correct answers are unsuitable as distractor. Accordingly, in step 116, the set calculation unit 92 deletes the first set described above from the corpus C ′ and newly sets it as C ′.

Further, in step 118, the set operation unit 92 creates a second set of pairs Q _j -A _j that satisfies the question Q _j εPARAPARASE (Q _K ) in the corpus C ′. This is because the response to the question that is the paraphrase of the question of the correct answer pair represents the same meaning as the correct answer sentence or is a correct response to the question of the correct answer pair. Such a response is inappropriate as a wrong choice. Therefore, in step 120, the second set is deleted from the set C ′ and newly set as C ′.

In the above processing, the set C ', the response is left only pair to be different meaning is the response A _K of the correct answer. Therefore, a question of an arbitrary pair in the set C ′ can be extracted and set as an incorrect answer option. However, as a multiple choice question, it is desirable to provide incorrect answer options that are as confusing as possible. Therefore, in the present embodiment, the following processing is further performed.

That is, the set operation unit 92, in step 122, of the set C ', to create a third set of comprising only pairs response does not similar to the response A _K of the correct answer. Since each response in the third set is dissimilar to the response of the correct pair, if it is adopted as another option together with the response of the correct pair, it is clearly different from the response for the correct answer. There are options to understand. Then, since the examinee is not confused and lacks suitability as a test practice question, such a third set is excluded from the set of incorrect answer choices. That is, in step 124, the third set is deleted from the set C ′ to form a new set C ′. This process is the response of the pair remain in the set C ', but both are similar to the response A _K of the correct answer, the statement not its word recombination. By using these as options, the multiple-choice question becomes more difficult.

  In step 126, a necessary number of arbitrary responses are extracted from the set C 'created in this way, and are set as incorrect answer options.

The problem preparation unit 94, thus the response A _i for incorrect extracted, the correct answer for the pair (Q _{K, A K)} in combination with the response A _K of options in response there are two or more A multiple-choice question is created and output to the hard disk 54 as one question (step 128).

  Further, in step 130, it is determined whether or not a problem creation end condition is satisfied. If the end condition is satisfied, the process ends. If the end condition is not satisfied, the process returns to step 110 to start creating the next question. As the termination condition, for example, whether or not a predetermined number of problems have been created can be employed.

  In the multiple choice problem thus created, there is only one correct combination of condition and response. In addition, the incorrect answer options are confused with the correct answer options, and an appropriate question is created as a multiple-choice question.

  The criteria for extracting the incorrect answer options described above are summarized in a table form in FIG.

  Referring to FIG. 7, row 204 indicates that the correct answer pair is (Q3, A3).

  The row 200 indicates that the pair (Q1, A1) having the correct response A3 and the dissimilar response A1 is inappropriate as a pair for creating an incorrect answer option.

  The row 202 indicates that the question response pair (Q2, A2) whose response belongs to PARAPHRASE (A3) is inappropriate as a pair for creating an incorrect answer option.

  The row 206 indicates that the question answer pair (Q4, A4) whose question belongs to PARAPHRASE (Q3) is inappropriate as a pair for creating an incorrect answer option.

  The row 208 shows a question answer pair (Q5, A5) selected from a set of incorrect answer choices excluding the question answer pairs (Q1, A1), (Q2, A2), (Q3, A3), (Q4, A4). Indicates that it is appropriate as a pair for creating an incorrect answer option.

[Operation]
Next, the operation of the automatic problem creation apparatus 82 shown in FIG. 3 will be described with reference to the flowchart of FIG. 6 as appropriate.

When the creation of a question is instructed, the correct answer selection unit 90 selects an arbitrary question answering pair (Q _K , A _K ) from the corpus 80 as a correct answer pair, and the question Q _K is a problem condition and a response. _AK is set as a correct answer option (step 110 in FIG. 6).

The set calculation unit 92 obtains a set C ′ = C− (Q _K , A _K ) obtained by subtracting the correct pair (Q _K , A _K ) from the corpus 80 (step 112), and then obtains the first set (step 112). Step 114). Then, the first set is deleted from the set C ′ to form a new set C ′ (step 116). The set operation unit 92 further creates a second set (step 118). This second set is deleted from the new C ′ to form a new set C ′ (step 120). The set operation unit 92 further creates a third set (step 122). Then, the third set is further deleted from the set C ′ to obtain a final incorrect answer option set C ′ (step 124).

By such processing, the set C 'wrong answers for choice, the correct answer pair (Q _{K, A K)} for the condition Q _K of the response that can be the same correct answers and response A _K, response A _K and non Similar responses will not be included.

Next, the problem creating section 94 converts one or a plurality of incorrect answer responses A _i selected from arbitrary condition-response pairs included in the incorrect answer option set C ′ into correct answer pairs (Q _K , combined with response _{a K} of a _K) to create a problem (step 126).

  The problems created in this way are appropriately shaped and then stored in the hard disk 54 as a multiple-choice question collection.

  When there is a request from the user to provide a multiple-choice question, the multiple-choice question is selected from this multiple-choice question collection according to some method, and the selected multiple-choice question is selected. The voice data of the question sentence is output to the requested personal computer or mobile device, and the text of the option is also transmitted and displayed on the screen, and the user selects a response from the text. When the user's response is received, it is determined whether or not the response matches the correct answer option, the correct / error determination result is transmitted, and the following problem is executed.

  Such problem creation processing and output are repeated each time a request is made from the examinee, and many problems are provided to the examinee.

  According to this embodiment, since problem creation is performed automatically without manual operation, the efficiency is high and the cost is reduced. In addition, since the number of options can be freely selected, the degree of freedom in combining options can be significantly improved. Further, by preparing a natural interaction condition response pair or various other condition response pairs in the corpus 80, it is possible to automatically create more advanced problems.

In this embodiment, the correct answer for the pair (Q _{K, A K)} (the set of the third) set of question and answer pairs has a response A _K dissimilar responses be deleted from the set of incorrect for choice Yes. Therefore, it is possible to add an option that makes the examinee confused, and to create a problem suitable for practice.

[Modification]
The above-described automatic problem creating apparatus 82 creates a multiple-choice question related to English listening. However, the present invention is not limited to the listening problem and is not limited to the English problem. It can be applied to the problem of presenting text, and can be applied to any language as long as it is a speech language.

In the above embodiment, the case where the condition is a question and the response is an answer to the question has been described as an example. However, the present invention is not limited to such an embodiment. For example, as shown in FIG. 8, there may be a problem in which a plurality of images P _K such as pictures and photographs and explanatory texts S _K and Sj are presented and an appropriate image description is selected. In this case, the image _{P i} and an explanation _{S i} and corpus collected pairs _{{(P i, S} i)}, such as a picture or photograph in advance by creating an image _{P i} on condition, description _{S i} It is possible to cause problem creation to be performed automatically by substituting each for the response and performing the above-described processing. In some cases, it may be difficult to determine whether two images are equivalent based on only the images. In such a case, it is effective to add some typical explanation to the images. This explanation is different from the explanatory text as a response.

Further, as shown in FIG. 9, the sentence C _K with a summary S _K, a plurality presenting Sj, may be in the form of the problem of selecting an abstract that matches the sentence C _K. In this case, a corpus {(C _i , S _i )} that collects pairs of the sentence C _i and the summary S _i is created, and the sentence S _i is replaced with the response with the sentence C _i as a condition. By performing the above process, problem creation can be automatically performed.

  In addition, it may be a problem of a format in which a plurality of translations are presented and a translation that matches the original text is selected, or a problem in which a plurality of word definitions are presented and a definition that matches the word is selected. Also good. Not a question consisting of one question and multiple responses, but a multiple-choice question in which multiple questions and one response are given and the most appropriate question is selected for the response. Good. In this case, in the above-described embodiment, the question creation can be automatically performed by replacing the question with the response and the response with the question and performing the above-described processing.

  In the above-described embodiment, the problem is output as voice data. However, the problem may be output as text. Also, the problematic voice may be generated from the text by voice synthesis without holding the problematic voice data in the corpus. Furthermore, an embodiment is also conceivable in which the voices of the choices are generated in order to let the user hear and select an answer from them.

  In the above-described embodiment, the multi-choice question is assumed to be related to languages such as English. However, the present invention is not limited to such an embodiment. As long as the corpus can collect a pair consisting of some condition that can be used as a question and the correct response to it, the appropriate multiple-choice question is automatically shortened regardless of the target. Many can be created efficiently in time.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

1 is an external view of a computer system that executes an automatic problem creation program according to an embodiment of the present invention. FIG. 2 is a block diagram of the computer system of FIG. 1. It is a block diagram which shows the structure of the automatic problem preparation apparatus 82 which concerns on one embodiment of this invention. It is a figure which shows an example of a multiple choice type | formula problem. It is a figure which shows an example of corpus. It is a flowchart which shows the content of the question creation process by the question automatic creation apparatus 82 of FIG. It is the table | surface which summarized the content of the question preparation process. It is a figure which shows the other example of a multiple choice type | formula problem. It is a figure which shows the further another example of a multiple choice type | formula problem.

Explanation of symbols

40 Computer 80 Corpus 82 Problem Automatic Creation Device 90 Correct Answer Selection Unit 92 Set Operation Unit 94 Question Creation Unit

Claims (7)

A storage device for storing a corpus including a plurality of condition-response pairs each comprising a condition and a response to the condition ;
From the corpus stored in the storage device, and the correct answer selection means you select any condition responsive pair as the correct for a pair,
A correct answer pair selected by the correct answer selecting means, a set of condition response pairs having a response equivalent to the response of the correct answer pair, and a conditional response having a condition equivalent to the condition of the correct answer pair by excluding a set of pairs from the corpus, the incorrect candidate set creating unit that obtains a set of incorrect for alternative candidates,
By combining a predetermined number of incorrect answer responses selected from arbitrary condition response pairs included in the determined incorrect answer option candidate set with the correct pair responses, the conditions of the correct answer pairs and a problem creating means to create a multiple-choice question to select a response corresponding to the automatic creation device problems. Further, a set of incorrect answer options obtained by deleting a set of conditional response pairs having a response dissimilar to the response of the correct answer pair from the set of incorrect answer option candidates created by the incorrect answer candidate set creating means. including hand stage Ru applied to the problem creating means, the device for automatically creating issues of claim 1. The automatic problem creation apparatus according to claim 1, wherein the condition in the condition-response pair is a question, and the response is an answer to the question. The automatic problem creating apparatus according to claim 1, wherein the condition in the condition-response pair is an image, and the response is an explanatory text about the image. The automatic problem creation apparatus according to claim 1, wherein the condition in the condition-response pair is a sentence, and the response is a summary of the sentence. Until a predetermined number of multiple choice problems are created by the problem creating means further comprises the correct answer selection means, the incorrect candidate set creating unit, and the hand stages Ru was repeatedly operate the problem creating means, claim The problem automatic creation apparatus according to any one of claims 1 to 5. A computer connected to a storage device that stores a corpus that includes a plurality of condition-response pairs each comprising a condition and a response to the condition,
Correct answer selection means for selecting an arbitrary condition-response pair as a correct answer pair from the corpus stored in the storage device;
A correct answer pair selected by the correct answer selecting means, a set of condition response pairs having a response equivalent to the response of the correct answer pair, and a conditional response having a condition equivalent to the condition of the correct answer pair An incorrect answer candidate set creating means for obtaining a set of option candidates for incorrect solutions by removing a set of pairs from the corpus;
By combining a predetermined number of incorrect answer responses selected from arbitrary condition-response pairs included in the determined incorrect answer option candidate set with the correct answer pairs, the conditions of the correct answer pairs A computer program for automatically creating a problem that functions as a problem creating means for creating a multiple-choice question for selecting a response corresponding to a question.

Images