JP5182954B2 - Cavity problem generator and program thereof - Google Patents

Description

  The present invention relates to individual learning using a computer, and more particularly to a question generation technique for generating a burial problem using a computer.

  Individual learning is known as a form in which a learner advances learning individually. Until now, problems used in individual learning have generally been created by specialists such as teachers. In recent years, along with the spread of e-learning, individual learning using a computer is expanding through a network. For this reason, individual learning requires a large amount of problems tailored to individual learners.

  For this reason, a technique for generating a burial problem in which a part of English text or the like is blank has been proposed (see Non-Patent Documents 1 and 2, for example). In the techniques described in Non-Patent Documents 1 and 2, verbs and adjectives appearing in English are blank. As a result, the techniques described in Non-Patent Documents 1 and 2 enable generation of a burial problem asking for a specific part of speech.

  In addition, in individual learning, it is necessary not only to determine a blank, but also to generate an erroneous option that becomes an incorrect option. For example, in the techniques described in Non-Patent Documents 1 and 2, synonyms and antonyms are generated as erroneous choices using a thesaurus. As a result, the techniques described in Non-Patent Documents 1 and 2 can also generate a burial problem that measures the vocabulary ability of the learner.

Eiichiro Sumida, Fumiaki Shibuya, Seiichi Yamamoto, “Automatic generation method of empty space filling problem for measuring English proficiency”, IEICE Technical Report, Vol.104, No.503 (2004), pp.17-22. Yi-Chien Lin, Li-Chun Sung, and Meng Chang Chen, “An Automatic Multiple-Choice Question Generation Scheme for English Adjective Understanding”, ICCE 2007 Workshop Proc. Of Modeling, Management and Generation of Problems / Questions in eLearning (2007) , pp.137-142.

  The conventional techniques including Non-Patent Documents 1 and 2 have the following problems. For example, in the techniques described in Non-Patent Documents 1 and 2, it is necessary for the learner himself to select one sentence in English from the plurality of sentences. However, it is difficult for the learner himself to select an English sentence suitable for the burial problem from an infinite number of sentences.

  In addition, it is important to select an appropriate English sentence for the burial problem because it affects the difficulty of the problem. For example, sentences that are grammatically correct and composed of commonly used words and parts of speech are suitable as question sentences. On the other hand, sentences containing many proper nouns, conversation sentences, and sentences that are not grammatically correct are inappropriate as question sentences. However, a burying problem generation technique that can determine whether a sentence is suitable for a burying problem has not been proposed.

  The blank in the burial problem indicates the grammar and vocabulary that the learner learns, and the word and the part of speech that are replaced by the blank are factors that greatly affect the difficulty of the problem. Here, as in the prior art, in a method of leaving only a specific part of speech blank or a method of determining a word close to the frequency of words specified by the learner in a question sentence as a blank, it is replaced with a blank in the question sentence. Words were often limited. However, since the structure of the question sentence influences the determination of the blank, there is a strong demand for a flexible burial problem generation technique that is not limited to a vocabulary level such as a specific part of speech or word frequency.

  In addition, wrong choices greatly affect the difficulty of the problem. For example, if the wrong choice is completely different from the part of speech or meaning of the correct choice, the learner can easily make the correct answer. On the other hand, when the wrong choice is a word having the same part of speech or similar meaning as the correct answer choice, the difficulty level of the problem becomes high. However, as in the prior art, a method for generating a wrong choice group from synonyms and antonyms of a correct choice word, or a method using a word with a close vocabulary level according to the correct choice word and its frequency as the wrong choice group Limited to the creation of burial questions that ask vocabulary. For this reason, there is a strong demand for a burial problem generation technique for generating a burial problem that can learn grammar.

  Accordingly, an object of the present invention is to solve the above-described problems and provide a burial problem generation technique that can generate a burial problem suitable for individual learning.

  When an expert creates a burial problem, based on experience, he selects a problem sentence according to the learning purpose, and determines blanks and wrong choices. That is, it can be considered that the expert's knowledge of problem creation can be acquired from the already created burial problem. Therefore, the inventors of the present application have found that the empirical knowledge of experts can be reflected in the burial problem by machine learning statistical information on the words and grammar of the existing burial problem, and have completed the present invention.

  Specifically, the burial problem generator according to the first invention of the present application replaces one or more words included in candidate burial problem sentences input from the outside with blanks, and replaces the blanks with the blanks. In the burial problem generating apparatus for generating a burial problem composed of a correct answer option that is the one or more words and an incorrect option that is different from the correct answer option, the burial problem sentence to which a morpheme is added. A plurality of candidates are input, and the priority is calculated for each burial problem sentence candidate based on the priority information learned in advance by priority learning, and the burial problem sentence candidates are selected in descending order of the priority. In the burial problem sentence candidate selected by the problem sentence selection means, based on blank statistical information previously learned by a problem sentence selection means and sequence labeling that treats the words included in the burial problem sentence candidate as a sequence. A blank determining means for determining the blank and a generation rule for generating a wrong choice candidate that is a candidate for the wrong choice are preset, and the blank corresponding to the blank determined by the blank determination means based on the generation rule. An erroneous option candidate generating means for generating an incorrect option candidate, and for each of the erroneous option candidates, an Internet search was performed using a search sentence in which the incorrect option candidate was inserted into the blank of the burial problem sentence, and the Internet search failed. And selecting an erroneous option candidate in the search sentence as the erroneous option, and an erroneous option selecting means for outputting the burial problem.

  First, the burial problem generating apparatus receives a plurality of sentences (for example, English sentences such as books, news, columns, papers, documents on the Internet, etc.) prepared individually by the learner as burial problem sentence candidates. That is, the learner prepares sentences corresponding to each preference and interest in advance as burial problem sentence candidates. At this time, for example, a morpheme (Part-of-speech tag) is added to each word included in each burial problem sentence candidate.

  Next, the burial problem generation device selects a sentence (clogging problem sentence candidate) suitable for the burial problem from the burial problem sentence candidates. When individual learning is performed using the input burial problem sentence candidates, it is preferable to generate burial problems in order from more appropriate burial problem sentence candidates. Therefore, the burial problem generating apparatus selects candidate burial problem sentences in descending order of priority using the priority information learned by the priority learning. Accordingly, the burial problem generating apparatus can reduce the selection of an inappropriate sentence (for example, a sentence or conversation sentence having many proper nouns) as the burial problem sentence as the burial problem sentence. Note that priority learning is a method of relatively classifying samples by calculating the priority in the input sample from the similarity between the samples. As this priority learning, for example, Ranking Voted Perceptron method (hereinafter abbreviated as “RVP method”) and RocSVM (Rocchio Support Vector Machine) method can be cited.

  Then, the burial problem generating device determines a blank in the burial problem sentence. Here, the sentence is composed of a string of words and morphemes. That is, the determination of a blank can be paraphrased as a problem of labeling a column of words or morphemes as “blank”. Therefore, the burial problem generating apparatus treats the determination of blanks in the burial problem sentence as a series labeling problem, and labels words included in the burial problem sentence to mean blanks. As a result, the burial problem generator can reduce the bias of words that are replaced with blanks in the burial problem sentence. At this time, the burial problem generating device stores blank statistical information obtained by machine learning using, as learning data, sequence information of words and morphemes included in the existing burial problem. Examples of the machine learning include a conditional random field (CRF), a hidden Markov model (HMM), and a maximum entropy Markov model (MEMM).

  Further, the hole filling problem generating device generates an erroneous option. At this time, the burial problem sentence filled with blanks with correct choices must be semantically and grammatically correct sentences, while the burial problem sentence filled with blanks with wrong choices is semantic and grammatical. It must not be a correct sentence. In other words, it is necessary for the burial problem generating apparatus to delete erroneous option candidates that are correct sentences and to select erroneous option candidates that are semantically and grammatically incorrect as incorrect options. Therefore, the burial problem generation device generates a search sentence in which the blank of the burial problem sentence is filled with the generated erroneous option candidates, and performs an Internet search using this as a search condition. Then, the burial problem generating apparatus selects, as an incorrect option, an incorrect option candidate in a search sentence in which Internet search has failed. In this way, the burial problem generator generates a burial problem suitable for individual learning.

  Further, the burial problem generating apparatus according to the second invention of the present application provides a correspondence between the generation rule and the morpheme of the one or more words that have been replaced with the blank in the already created burial problem (existing burial problem). Storage means for storing a statistical pattern indicating a relationship is further provided, wherein the erroneous option candidate generating means refers to the statistical pattern and uses the one or more word morphemes replaced with the blank in the burial problem sentence. The erroneous option candidate is generated using the corresponding generation rule.

  Here, the generation rule of the erroneous option in the existing hole filling problem is analyzed, and the statistical pattern is stored. Then, the burial problem generating device generates erroneous option candidates based on the statistical pattern. As a result, the burial problem generator generates burial problems that can learn not only vocabulary but also grammar.

  The burial problem generator according to the third invention of the present application is configured such that a plurality of the burial problem sentence candidates are input from the outside, the morphological analysis is performed on the input burial problem sentence candidates, and the morpheme is added to the hole. It further comprises morpheme adding means for outputting buried question sentence candidates to the question sentence selecting means. This eliminates the need for the learner himself to add a morpheme to the candidate burial problem sentence, which is convenient for the learner.

  A burial problem generating program according to the fourth invention of the present application causes a computer to operate as each means of any of the burying problem generating apparatuses from the first invention of the present application to the third invention of the present application. A computer in which this program is installed can realize each function based on this program.

  ADVANTAGE OF THE INVENTION According to this invention, the burial problem generation technique which can reflect the empirical knowledge of an expert and can generate the burial problem suitable for individual learning can be provided.

It is a figure explaining the outline of an individual learning system in the present invention. It is a block diagram which shows the structure of the individual learning system in this invention. It is a figure which shows an example of the problem input screen in this invention. It is a figure which shows an example of the learning screen in this invention. It is a figure explaining the function of the burial problem generating apparatus concerning a 1st embodiment of the present invention. It is a block diagram which shows the structure of the hole filling problem generation apparatus which concerns on 1st Embodiment of this invention. It is a figure explaining the algorithm of the RVP method which the burial problem generating apparatus of FIG. 6 uses. It is a block diagram of a computer in which a hole filling problem generating program according to the present invention is installed. It is a flowchart which shows operation | movement of the hole filling problem production | generation apparatus of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numerals and description thereof is omitted.

[Outline of individual learning system]
Hereinafter, an outline of the individual learning system 1 using the hole filling problem generating device 4 according to the first embodiment of the present invention will be described with reference to FIGS. 1 and 2. The individual learning system 1 generates, for example, an English burying problem and supports the learner's individual learning. Since the hole filling problem generating device 4 is built in the individual learning device 3, it is not shown in FIG.

  As described above, in the prior art, the learner himself needs to select one sentence in question from a plurality of sentences. However, it is difficult for the learner himself to select an English sentence suitable for the burial problem from a myriad of sentences. As shown in FIG. 1, the learner prepares a plurality of sentences (English sentences in FIG. 1) according to each preference and interest, and inputs these to the individual learning device 3. Then, the individual learning device 3 selects an appropriate burial problem sentence from the sentences input by the learner, generates an incorrect option, and outputs it as a burial problem (shown as a problem in FIG. 1). . Then, the learner performs individual learning by answering the burial problem output by the individual learning device 3.

  For this reason, as shown in FIG. 2, the individual learning system 1 includes a learner terminal device 2 and an individual learning device 3. Here, the learner terminal device 2 is connected to the individual learning device 3 via a network (not shown) such as the Internet. In FIG. 2, only one learner terminal device 2 is shown for explanation, but the individual learning system 1 may include a plurality of learner terminal devices 2.

  The learner terminal device 2 is a terminal device used by the learner for individual learning. For example, the learner terminal device 2 is a general computer that includes input means (not shown) such as a mouse and keyboard, output means (not shown) such as a display, and has an internet browser (not shown) installed. .

  The individual learning device 3 is a device that generates a burial problem used by the learner for individual learning, and includes a problem input screen presentation unit 31, a burial problem sentence candidate acquisition unit 32, a burial problem generation device 4, and learning. Screen display means 33.

  The question input screen presenting means 31 presents to the learner terminal device 2 a question input screen Sc1 (see FIG. 3) in which the learner inputs a sentence (filled question sentence candidate). For example, the question input screen presenting means 31 generates HTML (HyperText Markup Language) data to be the question input screen Sc1 and outputs it to the learner terminal device 2. As shown in FIG. 3, the question input screen Sc1 includes a text box Tx in which a learner can input a plurality of sentences (filled question sentence candidates) and a sentence (filled question sentence candidates) input in the text box Tx. Is transmitted to the individual learning device 3. In FIG. 3, “This is the building where we had our first office....” In the text box Tx is an example of a sentence (candidate question sentence candidate) input by the learner.

  Here, when the HTML data is output from the question input screen presenting means 31, the learner terminal device 2 displays the question input screen Sc1 on the Internet browser. Then, for example, the learner copies and pastes sentences such as books, news, columns, papers, and documents on the Internet to the text box Tx of the question input screen Sc1. Thereafter, when the learner clicks the send button Bt1 on the question input screen Sc1, the sentence (filled question sentence candidate) input in the text box Tx is transmitted to the individual learning device 3.

  The burial problem sentence candidate acquisition unit 32 acquires a sentence (a burial problem sentence candidate) transmitted from the learner terminal device 2 and outputs the sentence to the burial problem generation apparatus 4. Here, for example, the burial problem sentence candidate acquisition means 32 converts a plurality of sentences including “This is the building where we had our first office.” Input in the text box Tx of the problem input screen of FIG. Acquired as a buried question sentence candidate.

  The burial problem generating apparatus 4 includes a burial problem sentence in which one or more words included in the burial problem sentence candidate from the burial problem sentence candidate obtaining unit 32 are replaced with blanks, and one or more words replaced with blanks. A burial problem composed of a certain correct answer option and an incorrect option that is different from the correct answer option is generated. Details of the hole filling problem generating device 4 will be described later.

  The learning screen presenting means 33 generates a learning screen Sc2 (see FIG. 4) based on the hole filling problem generated by the hole filling problem generating device 4, and presents the learning screen Sc2 to the learner terminal device 2. For example, the learning screen presentation unit 33 generates HTML data to be the learning screen Sc2 and outputs the HTML data to the learner terminal device 2. As shown in FIG. 4, the learning screen Sc2 includes, for example, a burial problem sentence “This is the building () we had our first office.”, A correct answer option “where”, “what”, “howch”. ”And“ when ”. The learning screen Sc2 includes, for example, a radio button Rb indicating an option selected by the learner and an answer button Bt2 for determining whether the option selected by the radio button Rb is correct. Furthermore, when the answer button Bt2 is clicked on the learning screen Sc2, it is determined whether or not the choice selected by the learner is the correct answer, and a correct answer dialog box or an incorrect answer dialog box is output according to the choice. A script (not shown) is included. In this example, the blank in the burial problem sentence is a part written in parentheses.

  Here, when the HTML data is output from the learning screen presenting means 33, the learner terminal device 2 displays the learning screen Sc2 on the Internet browser. Then, the learner corresponds to the choice “what”, “who”, “where” or “when” appropriate for filling in the burial problem sentence “This is the building () we had our first office.”. Select the radio button Rb and click the answer button Bt2. Here, when the learner selects the correct answer option “where”, the learner terminal device 2 displays a correct answer dialog box including a correct message. On the other hand, when the learner selects the wrong choice of “what”, “who”, or “when”, the learner terminal device 2 displays an error answer dialog box including an error message. As described above, in the individual learning system 1, a learner can perform individual learning.

[Configuration of hole filling problem generator]
Hereinafter, the details of the hole filling problem generating apparatus 4 will be described. As shown in FIG. 5, the burial problem generating device 4 is largely divided into 1) selecting an appropriate problem sentence from sentences input by the learner, 2) determining a blank in the selected burial problem sentence candidate, and 3 ) Has the function of generating wrong choices. In order to realize these functions, as shown in FIG. 6, the burial problem generating apparatus 4 includes a storage unit 41, a priority learning unit 42, a blank statistical information learning unit 43, and a statistical pattern learning unit 44. A morpheme adding unit 51, a question sentence selecting unit 52, a blank determining unit 53, and an erroneous option generating unit 54.

  In FIG. 6, the priority learning means 42, the blank statistical information learning means 43, and the statistical pattern learning means 44 generate various information necessary for generating the burial problem and store them in the storage means 41. To do. In addition, the morpheme adding unit 51, the question sentence selecting unit 52, the blank space determining unit 53, and the erroneous option generating unit 54 generate a burial problem using various information stored in the storage unit 41.

  The storage means 41 includes existing burial problem sentence information 411, labeled existing burial problem sentence information 412, priority information 413, blank space statistical information 414, a synonym dictionary 415, a word dictionary 416, and a use word list. A memory or a hard disk for storing 417 and a statistical pattern 418.

  The existing burying problem sentence information 411 is information including a plurality of existing burying problem problems prepared by a specialist as a learning problem and morphemes for each word included in the problem sentence. In the first embodiment, for example, the TOEIC (registered trademark) filling problem is used as the existing filling problem. The existing burial problem sentence information 411 is stored in the storage unit 41 in advance by an operator, for example.

  The existing burial problem sentence information 412 with label is information in which a unique expression label is added to each word of the burial problem sentence included in the existing burial problem sentence information 411. This existing hole-filled problem sentence information 412 with label is stored in the storage means 41 in advance by an operator, for example. Details of the label will be described later.

  The priority information 413 is learning data generated by priority learning means 42 described later by priority learning. The priority information 413 is referred to by the question sentence selection unit 52 described later.

  The blank statistic information 414 is learning data generated by the blank statistic information learning means 43 to be described later by series labeling. The blank statistic information 414 is referred to by the blank decision means 53 described later.

  The synonym dictionary 415 is a dictionary in which a synonym having a meaning similar to a word included in the burial problem sentence and an antonym having a meaning opposite to the word are registered. The synonym dictionary 415 is referred to by the erroneous option generation means 54 described later. The synonym dictionary 415 is stored in the storage unit 41 in advance by the operator.

  The word dictionary 416 is a dictionary in which words included in the burial problem sentence are registered. This word dictionary 416 is referred to by the erroneous option generation means 54 described later. The word dictionary 416 is stored in the storage means 41 in advance by the operator.

  The utilization word list 417 is a list in which utilization words of words included in the burial problem sentence are registered. The use word list 417 is referred to by the erroneous option generation means 54 described later. The utilization word list 417 is stored in the storage unit 41 in advance by the operator.

  The statistical pattern 418 is information indicating a correspondence relationship between the generation rules and the morphemes of one or more words that have been replaced with blanks in the existing burying problem. Here, the generation rules indicate various rules for generating erroneous choices. The statistical pattern 418 is referred to by an erroneous option generation unit 54 described later.

  The priority learning unit 42 refers to the labeled existing burial problem sentence information 412 in the storage unit 41, performs priority learning, generates priority information 413, and stores it in the storage unit 41. Hereinafter, an example of priority learning by the RVP method will be described with reference to FIG. 7 (see FIG. 6 as appropriate). Details of the RVP method are described in, for example, the document “Michael Collins, and Nigel Duffy,“ New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron ”, Proc. Of 40th Annual Meeting of the Association for Computational Linguistics (2007), pp.263-270.

In FIG. 7, x _i0 is a sentence in which the question sentence of the existing burying problem is filled with the correct answer options, which is a positive example. In FIG. 7, x _i1 ,..., X _ij and y are arbitrary sentences, which are candidates. Further, in FIG. 7, h (x _i0 ) · h (y) indicates the similarity of words and grammar between the sentence x _i0 and the sentence y. In FIG. 7, h (x _ij ) · h (y) indicates the similarity of words and grammar between the sentence x _ij and the sentence y. In FIG. 7, G (y) indicates the priority between the sentence x _i0 and the sentence y, α _ij is a parameter indicating the weight, and argmax is a function that returns a value that maximizes G (x _ij ). Indicates.

Here, the priority learning means 42 calculates h (x _ij ) · h (y) by the following equations (1) and (2). Incidentally, as shown in equation _{(2), k (x ij} , y) is the sentence _{x ij} and unigram indicating the number of matched words and sentences y _(x ij, y) and the sentence _{x ij} and text y This is a total value with posunigram (x _ij , y) indicating the number of coincidence of morphemes. That is, when the similarity between the sentence x _i0 and the sentence y is high, the priority G (y) has a higher value. At this time, the priority learning means 42 sets the value to “1” when the similarity of the candidate sentence exceeds the similarity of the positive sentence so that the similarity of the candidate sentence does not become a high value. learn. Note that the priority learning means 42 may use the number of coincidence of word and part-of-speech sequences and the number of coincidence of syntax trees in addition to the equations (1) and (2).

  Note that the priority learning means 42 can also perform priority learning using the RocSVM method in addition to the RVP method. Details of the RocSVM method are described in, for example, the document “Xaioli Li, Bing Liu,“ Learning to Classify Texts Using Positive and Unlabeled Data ””.

  The blank statistic information learning means 43 refers to the labeled existing burying question sentence information 412 in the storage means 41, generates blank statistic information 414 using series labeling, and stores it in the storage means 41. For example, the blank statistic information learning means 43 treats blanks in the burial problem sentence as unique expressions and expresses them using the IOB2 format. In this IOB2 format, three types of tags are defined: an “I” tag, an “O” tag, and a “B” tag. Specifically, the blank statistic information learning unit 43 adds a “B” tag when the word included in the series (filled question sentence) is located at the head of the blank. The blank statistical information learning means 43 adds an “I” tag to the words positioned after the second word when the blank spans a plurality of words. The blank statistical information learning unit 43 adds an “O” tag to words other than blanks, that is, words that are not unique expressions.

  In the following, the IOB2 format will be described with an example of the burial problem sentence “His doctor urged him to () doing hard exercise.” And the correct answer option “give up” in this blank. In this example, as shown in Table 1, a “B” tag is added to the first word “give” in the blank, and an “I” tag is added to the next word “up”, and the word “give” is positioned in the blank. An “O” tag is added to the missing word.

  Here, the blank statistic information learning means 43 uses a conditional random field, for example, as series labeling. In this case, the blank statistic information learning unit 43 learns one or more word morphemes corresponding to the blank included in the labeled existing burial problem sentence information 412 using a conditional random field, and generates blank statistic information 414. The details of the conditional random field are described in, for example, the literature “John Lafferty, Andrew McCallum, and Fernando Pereira,“ Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data ”, Proc. Of ICML 2001, (2001), pp. .282-289. ".

  The blank statistical information learning means 43 may use the invention described in Japanese Patent Application Laid-Open No. 2008-225907 as sequence labeling. In this case, the blank statistic information learning unit 43 uses unlabeled data (that is, the existing burial problem sentence information 411), learns the learned identification model parameter vector set and the model integration parameter set, and generates the generated model. A parameter vector set is determined. In this case, the blank statistic information learning unit 43 learns the identification model parameter vector set and the generation model parameter vector set using the labeled data (that is, the labeled existing burying problem sentence information 412). Then, a parameter set for model integration is determined. Thereby, the hole filling problem generating device 4 can improve the prediction performance of the sequence labeling at a low cost. The blank statistical information learning means 43 may use a hidden Markov model or a maximum entropy Markov model as the sequence labeling.

  The statistical pattern learning unit 44 refers to the labeled existing burial problem sentence information 412 in the storage unit 41 and obtains the correspondence between the morphemes of one or more words replaced with blanks in the existing burial problem sentence and the generation rules. The statistical pattern 418 is generated and stored in the storage unit 41. In the first embodiment, the statistical pattern learning unit 44 converts the morphemes of the words replaced with blanks in the existing burial problem into Class I, II-utilized words, II-derived words, II-prefix / suffixes. The statistical pattern is generated in correspondence with the generation rule of either the class or the II-related class. Details of the generation rule and the wrong choice candidate will be described later.

  The morpheme adding means 51 receives a plurality of burial problem sentence candidates from the learner terminal device 2 (see FIG. 2), and converts the candidate burial problem sentence into the inputted burying problem sentence candidates by morphological analysis such as a hidden Markov model and a conditional random field. Add morphemes to the included words and add parts of speech. Then, the morpheme adding means 51 outputs these burying question sentence candidates to the question sentence selecting means 52.

The question sentence selection means 52 refers to the priority information 413 of the storage means 41, calculates the priority for each burying question sentence candidate input from the morpheme addition means 51, and fills the burial problem sentences in descending order of priority. Select a candidate. Specifically, the question sentence selection unit 52 uses the parameter α _ij of the priority information 413 for each of the input burial problem sentence candidates z _k (k = 0,..., M) to give priority. The degree G (z _k ) is calculated. Here, it can be said that the higher the priority G (z _k ), the more appropriate the problem sentence as a burial problem. The sentence selection unit 52 issues, each Anauma clue candidate z _k, calculated performs priority G (z _k) ranked in descending order, Anauma problems priority G (z _k) in descending order A sentence candidate z _k is selected. Here, when the number (number of questions) for generating the burial problem is set in advance, the question sentence selection unit 52 may select the burial problem sentence candidate z _k by this number of problems. Further, the question sentence selection unit 52 outputs the selected burial problem sentence candidate z _k to the blank space determination unit 53. As a method for calculating the priority G (z _k ), the calculation formula a for the priority G (y) in FIG. 7 can be used.

  The blank determining unit 53 refers to the blank statistical information 414 in the storage unit 41 and determines a blank in the burial problem sentence candidate selected by the question sentence selecting unit 52. Here, the blank statistic information 414 indicates the probability that the morpheme of the word will be each of the “I” tag, the “O” tag, and the “B” tag, that is, the probability that the word morpheme will be blank. . Here, an example of the burial problem sentence “This is the building () we had our first office.” Is shown in Table 2 below. In Table 2, the tag type item indicates the tag type determined by the blank space determination unit 53 and its peripheral probability. Also, in Table 2, the B tag, I tag, and O tag items indicate the peripheral probabilities of being the “I” tag, the “O” tag, and the “B” tag, respectively, and can be obtained from the blank statistical information 414. .

  The blank determination unit 53 determines the word having the highest peripheral probability among the “I” tag, the “O” tag, and the “B” tag among the words included in the burial problem sentence candidate as the tag of the morpheme. Here, as shown in Table 2, the blank determining means 53 determines the word “This” or the word “is” as an “O” tag, and determines the word “where” as a “B” tag, that is, a blank. . Here, when the blank determination unit 53 determines that all the words included in the burial problem sentence candidate are “O” tags, the word having the highest peripheral probability of being a “B” tag is set as a blank. good. Thereafter, the blank determination unit 53 outputs the burial problem sentence in which the word determined to be the “B” tag is replaced with a blank, and the word as the correct choice to the wrong choice generation unit 54.

  As shown in FIG. 6, the erroneous option generation unit 54 includes an erroneous option candidate generation unit 54a and an erroneous option selection unit 54b.

  The erroneous option candidate generation unit 54a refers to the statistical pattern 418 of the storage unit 41, and uses the morpheme of the word (that is, the correct option word) that is replaced with a blank in the burial problem sentence input from the blank determination unit 53. Determine the corresponding production rule. Then, the erroneous option candidate generation unit 54a generates erroneous option candidates using the generation rule. Here, the generation rules are classified into, for example, class I, class II-used words, class II-derived words, class II-prefix / suffix, or class II-related words. Hereinafter, specific examples of generation rules and erroneous option candidates will be described.

<Specific examples of generation rules and wrong choice candidates>
First, based on the morphemes of the words replaced with blanks, the generation rules are broadly classified into class I and class II. Class I is limited in the types of words that appear as erroneous choices. Specifically, in class I, the morpheme of the word replaced with a blank is either “preposition”, “subordinate conjunction”, “question”, “equal conjunction”, or “auxiliary verb”. For example, when the morpheme of the word replaced with a blank is “question word”, the wrong option is a word such as “which”, “what”, “who”, “when”, “where”.

  For example, in the existing burying problem, Table 3 shows the ratio of morpheme types in the burying problem in which “question word” is replaced with a blank. Further, for example, in the existing burial problem, the frequency of the word “question” in the question where “question” is blank is shown in Table 4, and the frequency of words other than “question” is shown in Table 5. Shown in As shown in Tables 3 to 5, in Class I, the wrong choice candidate generation unit 54a determines the wrong choice based on the ratio of the types of morphemes that appear as wrong choices and the frequency of words replaced with blanks. Can generate candidates. That is, in class I, the wrong choice candidate generation unit 54a sets the same part of speech as the word replaced with a blank and a high frequency of occurrence as the wrong choice candidate.

  Class II has a specific pattern between options. Specifically, in class II, the morpheme of the word replaced with a blank is either “verb”, “noun”, “adjective”, or “adverb”. Here, this class II is further subdivided into class II-utilization, class II-derivative, class II-prefix / suffix and class II-related.

  The II-used word class is composed of the used words of words whose erroneous choices become correct answer choices. For example, the II-used word class is a case where the correct answer option is “ask” and the wrong option is a used word such as “asked”, “asking”, “asks”, and the like. In this case, the wrong choice candidate generation unit 54a refers to the utilization word list 417 in the storage unit 41 and generates a wrong choice candidate.

  The II-derived word class is composed of a derivation word of a word in which an erroneous option is a correct answer option. For example, the II-derivative term is a case where the correct choice is “work” and the wrong choice is a derivative word such as “worker”, “works”, and “working”. In this case, the wrong choice candidate generation unit 54a refers to the word dictionary 416 of the storage unit 41, and acquires a word including a character string of 75% from the front of the word that is the correct answer option. Then, the wrong choice candidate generation unit 54a sets the words acquired from the word dictionary 416 in the order of the Levenshtein distance from the word that is the correct choice as the wrong choice candidate. Note that the Levenshtein distance is a numerical value indicating how much a word that is an incorrect option differs from a word that is a correct option, and can be obtained by, for example, dynamic programming (DP matching method).

  The II-prefix / suffix class is such that a part of a word (prefix or suffix) that becomes an incorrect option is common to a word that becomes a correct option. Specifically, the II-prefix / suffix class is when the correct answer option is “circulation” and the incorrect option is “circuitcumulation”, “circucus”, and “circucumstance”. In this case, the incorrect option candidate generation unit 54a refers to the word dictionary 416 of the storage unit 41, acquires a word that matches the first syllable with the word that is the correct option in the case of a prefix, and correct in the case of a suffix. Get a word whose final syllable matches the word to be selected. Then, the wrong choice candidate generation unit 54a sets the words acquired from the word dictionary 416 in the order of the Levenshtein distance from the word that is the correct choice as the wrong choice candidate.

  In II-derivatives, words that are erroneous choices often have a semantic relationship with words that are correct choices. On the other hand, in the II-prefix / suffix, a word that is an incorrect option often has no semantic relationship with a word that is a correct answer option.

  II-related words are those in which a word that is an incorrect option is a synonym or antonym of the word that is the correct option. For example, the II-related term is a case where when the correct answer option is “car”, the wrong option is a synonym such as “automobile”, “vehicle”, and “buggy”. In this case, the wrong option candidate generation unit 54a refers to the synonym dictionary 415 of the storage unit 41, and acquires the synonym and / or antonym corresponding to the word that is the correct answer option as the wrong option candidate.

  As an existing burial problem, statistics of generation rules are shown in Table 6 below for 77 questions whose word morpheme is “verb” in TOEIC (registered trademark). From Table 6, when the morpheme of the word that is the correct answer option is “verb”, it is a II-utilized word class with a probability of 62%.

  The erroneous option selection unit 54b performs an Internet search for each erroneous option candidate generated by the erroneous option candidate generation unit 54a using a search sentence in which the erroneous option candidate is inserted in the blank of the burial problem sentence from the blank determination unit 53. There are both semantically and grammatically correct and incorrect sentences on the Internet. Therefore, it is considered that the sentence (search sentence) that failed in the Internet search is a sentence that no one writes and is not a semantically or grammatically correct sentence. Therefore, the wrong choice selection means 54b sets only the wrong choice candidates in the search sentence that failed in the Internet search as the wrong choice.

  Here, in the case of any one of class I, II-derivatives, II-prefix / suffix, or II-related words, the erroneous option selection unit 54b selects the erroneous option by the above-described Internet search. It was decided. Also, in the case of II-used words, there is a possibility that a sentence in which a word that is a candidate for an incorrect option is inserted into a blank of a burial problem sentence is grammatically correct in consideration of a grammatical relationship between a correct answer option and an incorrect option candidate. There is. Therefore, if the morpheme of the word that becomes the correct option matches the morpheme of the word that becomes the error option candidate, the error option selection unit 54b does not select the error option candidate. Thereafter, the wrong option selection means 54b outputs a combination of the burial problem sentence, the correct answer option, and the selected wrong option as a burial problem.

  As described above, the burying problem generation device 4 has been described as an independent device. However, the present invention can also be realized by a burying problem generation program for operating a general computer as each unit of the burying problem generation device 4. . Here, as shown in FIG. 8, the computer 9 includes calculation means 91, storage means 92 such as a hard disk, input means 93 such as a mouse and keyboard, and output means 94 such as a display. Then, the computer 9 in which the burial problem generating program is installed has its hardware resources as priority learning means 42, blank statistical information learning means 43, statistical pattern learning means 44, morpheme adding means 51, question sentence selecting means. 52, blank space determining means 53, erroneous option candidate generating means 54a, and erroneous option selecting means 54b are operated in cooperation. The burial problem generation program may be distributed via a communication line or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

[Operation of the hole filling problem generator]
Hereinafter, the operation of the burial problem generating apparatus 4 in FIG. 6 will be described with reference to FIG. 9 (see FIG. 6 as appropriate). In FIG. 9, the storage unit 41 stores the existing burial problem sentence information 411, the labeled existing burial problem sentence information 412, the priority information 413, the blank space statistical information 414, the synonym dictionary 415, and the word dictionary. A description will be given on the assumption that 416, a utilization word list 417, and a statistical pattern 418 are stored in advance.

  First, the burial problem generating device 4 receives a plurality of burial problem sentence candidates from the learner terminal device 2 (see FIG. 2) by the morpheme adding means 51, and performs morphological analysis such as a hidden Markov model and a conditional random field. Then, morphemes are added to the words included in the input burial problem sentence candidates to give part of speech. Then, the burial problem generating device 4 outputs these burial problem sentence candidates by the morpheme adding means 51 (step S1).

  Subsequent to the processing of step S1, the burial problem generating device 4 refers to the priority information 413 of the storage means 41 by the question sentence selection means 52, and sets the priority for each burying question sentence candidate output in step S1. While calculating, the burial problem candidate is selected in descending order of priority. Then, the burial problem generation device 4 outputs the selected burial problem sentence candidate by the question sentence selection means 52 (step S2).

  Subsequent to the processing in step S2, the burial problem generating device 4 refers to the blank statistic information 414 in the storage means 41 by the blank determining means 53, and determines a blank in the burial problem sentence candidate selected in step S2. Then, the burial problem generating apparatus 4 outputs, as the correct choice, the burial problem sentence including the blank and one or more words replaced with the blank by the blank determination unit 53 (step S3).

  Subsequent to the processing in step S3, the burial problem generating device 4 refers to the statistical pattern 418 in the storage unit 41 by the erroneous option candidate generation unit 54a, and the morphemes of one or more words replaced in the blank in step S3. The generation rule corresponding to is determined. Then, the burial problem generating device 4 generates an erroneous option candidate using the generation rule by the erroneous option candidate generating unit 54a, and outputs it (step S4).

  Subsequent to the processing in step S4, the burial problem generating device 4 uses the erroneous option selection unit 54b to enter the erroneous option in the blank of the burial problem sentence from the blank determination unit 53 for each erroneous option candidate generated in step S4. Perform an Internet search using the search text with the candidate inserted. Then, the burial problem generating device 4 uses the erroneous option selection unit 54b as an erroneous option only for the erroneous option candidate in the search sentence that failed in the Internet search. At this time, if the morpheme of the word that is the correct answer matches the morpheme of the word that is the incorrect option candidate by the incorrect option selection unit 54b, the burial problem generating device 4 does not select the incorrect option candidate. It may be good (step S5).

  Subsequent to the process of step S5, the burial problem generating device 4 outputs a combination of the burial problem sentence, the correct option, and the erroneous option selected in step S5 as a burial problem by the erroneous option selection unit 54b. (Step S6).

  As described above, according to the burial problem generation device 4 according to the first embodiment of the present invention, it is possible to reduce selection of an inappropriate sentence as a burial problem as a burial problem sentence. The bias of the blank word can be reduced, and only the wrong choice candidates that are semantically and grammatically incorrect can be selected as wrong choices. Therefore, according to the burial problem generating device 4 according to the first embodiment of the present invention, it is possible to generate a burial problem suitable for individual learning by reflecting empirical knowledge of experts.

  Moreover, according to the burial problem generating device 4 according to the first embodiment of the present invention, the morpheme adding means 51 is provided, so that it is not necessary for the learner to add morphemes to the burial problem sentence candidates. Convenience is good for. In the present invention, the morpheme adding means 51 may not be provided as long as candidate burial problem sentences to which morphemes are added are input.

  Moreover, although 1st Embodiment of this invention demonstrated by the example which produces | generates an English hole filling problem, the object of this invention is not limited to English. For example, the present invention can generate a burial problem in languages other than English, such as French, German, and Japanese.

Example 1
The priority learning by the RVP method by the burial problem generating device 4 in FIG. 6 was evaluated. In this Example 1, 1560 sentences of the burial problem in TOEIC (registered trademark) were taken as positive examples, and the same number of sentences extracted from newspaper articles were evaluated as 1060 burial problem sentence candidates by 10-fold cross validation. Specifically, the burial problem generating apparatus 4 was made to learn the priority and the burial problem sentence candidate as a training set by each of 1404 sentences (9/10). Then, the remaining 156 sentences (1/10) of the positive example and the burial problem sentence candidate were subjected to priority learning as a test set. Table 7 below shows the results of performing this operation in 10 ways.

  In Table 7, the positive examples included in the upper half, that is, the average of the ratios of candidates ranked in the 156th or higher rank are shown. English sentences composed of words and morphemes that frequently appear in positive examples were ranked higher, and English sentences containing grammatical errors, conversational sentences, and English sentences containing colloquial expressions were ranked lower. As described above, it has been found that the burying problem generating apparatus 4 can select an appropriate burying problem.

(Example 2)
Evaluation of blank determination by a conditional random field by the burial problem generator 4 of FIG. 6 was evaluated. In Example 2, using the existing filling problem, the blank determined by the conditional random field was compared with the original blank in the existing filling problem. Specifically, a 10-fold cross-validation was performed on the hole filling problem 1560 sentences in TOEIC (registered trademark) in the same manner as in Example 1 (that is, 1404 sentences (9/10) were trained as 156 sentences ( 1/10) test set). Then, the following two comparison methods were compared with the method (method of the present invention) of the burial problem generating device 4 in FIG.

<Comparison method 1: Method to leave the leftmost verb in English blank>
This comparison method 1 is a method for determining a verb appearing at the beginning of an English sentence as a blank. Here, it applied to the clogging problem 1560 sentences in TOEIC (registered trademark).

<Comparison method 2: Method of making the morpheme with the highest frequency of blanks blank>
This comparison method 2 is a method of measuring the number of morphemes in the training set and the order of appearance, and setting the morpheme having the highest frequency as a blank morpheme. Here, blanks were determined in English from the test set based on the morphemes to be left blank and the order of their appearance.

  Table 8 below shows the results of the comparison method 1, the comparison method 2, and the method of the present invention. In Table 8, the item of matching rate indicates the matching rate (that is, accuracy) between the original blank in the existing hole filling problem and the blank determined by each method.

  As shown in Table 8, the method of the present invention has a higher matching rate than the comparison method 1 and the comparison method 2, can determine words corresponding to various morphemes as blanks, and determines a plurality of words as blanks. it can. Therefore, it can be said that the method of the present invention is more appropriate than the comparative methods 1 and 2.

DESCRIPTION OF SYMBOLS 1 Individual learning system 2 Learner terminal device 3 Individual learning apparatus 31 Problem input screen presentation means 32 Cavity question sentence candidate acquisition means 33 Learning screen presentation means 4 Cavity problem generation apparatus 41 Storage means 42 Priority learning means 43 Blank statistical information Learning means 44 Statistical pattern learning means 51 Morphological addition means 52 Problem sentence selection means 53 Blank determination means 54 Error option generation means 54a Error option candidate generation means 54b Error option selection means 9 Computer 91 Calculation means 92 Storage means 93 Input means 94 Output means

Claims (4)

An underfill problem sentence in which one or more words included in a candidate for an underfill problem sentence input from the outside are replaced with blanks, a correct answer option that is the one or more words replaced with blanks, and an option different from the correct answer option In a burial problem generator for generating a burial problem composed of wrong choices,
A plurality of the burial problem sentence candidates to which morphemes are added are input, and a priority is calculated for each burial problem sentence candidate based on priority information learned in advance by priority learning, and the priority is high. A question sentence selecting means for selecting candidate burial problem sentences in order;
A blank determination for determining the blank in the burial problem sentence candidate selected by the question sentence selection means based on blank statistic information learned in advance by series labeling that treats the word included in the burial problem sentence candidate as a series. Means,
A generation rule for generating an incorrect option candidate that is a candidate for the incorrect option is preset, and an incorrect option candidate generation for generating the incorrect option candidate corresponding to the blank determined by the blank determination unit based on the generation rule Means,
For each of the erroneous option candidates, an Internet search is performed using a search sentence in which the incorrect option candidate is inserted into the blank of the burial problem sentence, and the erroneous option candidate in the search sentence that has failed in the Internet search is used as the erroneous option. And selecting an erroneous option selection means for outputting the hole filling problem,
A burial problem generating apparatus comprising: A storage means for storing a statistical pattern indicating a correspondence relationship between the generation rule and the morpheme of the one or more words that have been replaced with the blank in the burial problem that has already been created;
The erroneous option candidate generation unit generates the erroneous option candidate using the generation rule corresponding to the morpheme of the one or more words replaced with the blank in the burial problem sentence with reference to the statistical pattern. The hole filling problem generating apparatus according to claim 1, wherein:   A plurality of the burial problem sentence candidates are inputted from the outside, and the inputted burial problem sentence candidates are subjected to morphological analysis, and the morpheme addition is performed to output the burial problem sentence candidates to which the morphemes are added to the question sentence selecting means The hole filling problem generating apparatus according to claim 1, further comprising means.   A burial problem generation program for causing a computer to operate as each means of the burial problem generation apparatus according to any one of claims 1 to 3.

Images