JP2008129449A - Automatic question creating device, automatic question creating method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an automatic question creating device, an automatic question creating method and a computer program, capable of automatically creating a fill-in-the blank question etc. of complicated grammar, without time and effort. <P>SOLUTION: The automatic question creating device includes a learning section and a classification section. The learning section receives multiple fill-in-the blank questions as reasonable data, and creates non-reasonable data by changing a position from a sentence head in each reasonable data of a blank equivalent part. The reasonable data and the non-reasonable data are characterized by an identity list of a plurality of kinds of identities for characterizing the data, and a reference data is created for distinguishing between a reasonable data group and a non-reasonable data group, by statistic processing. The classification section performs morphological analysis of an input test data and characterizes a candidate data in which each morpheme is specified as the blank equivalent part by the identity list, and outputs the candidate data which is determined to belong to the reasonable data group, by the reference data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an automatic problem creation apparatus, an automatic problem creation method, and a computer program that automatically create a hole filling problem and the like.

In a test such as an entrance examination, a hole filling problem and an option problem (hereinafter referred to as “a hole filling problem etc.”) are given, but it is useful for a problem creator if a hole filling problem etc. can be automatically created. On the other hand, there is a technology that performs morphological analysis on the input learning target sentence, extracts the target word for each word against the dictionary, and replaces the extracted word part with a blank to create a hole filling problem. It has been proposed (Patent Document 1). In addition, an arbitrary pair is selected from a dictionary (corpus) containing question and answer pairs, and an answer with an expression equivalent to the answer, a question with an equivalent expression to the question, and an answer not similar to the answer are deleted from the corpus. Thus, a technique for obtaining an incorrect answer option candidate and creating a multiple-choice question has been proposed (Patent Document 2).
Japanese Patent Laid-Open No. 06-95583 JP 2006-126242 A

  However, the technique of Patent Document 1 is such that if a year name, a person's name and a place name are registered in the dictionary in advance, and a word existing in the dictionary exists in the problem target sentence, that portion is replaced with a blank. Therefore, although it is possible to create simple filling problems such as year names and person names in social studies, etc., there is a problem that advanced filling problems such as foreign language grammar cannot be created.

  In addition, according to the technique of Patent Document 2, since an incorrect answer similar to the correct answer can be created, a more advanced multiple-choice question can be created. However, a large number of question / answer pairs are manually created in advance. Therefore, there is a problem that the burden of labor is large.

Therefore, an object of the present invention is to provide an automatic problem creation apparatus, an automatic problem creation method, and a computer program that can automatically create a complicated grammar filling problem without trouble.

  (1) An automatic problem creation apparatus including a machine learning unit and a classification unit, wherein the machine learning unit receives a large number of filling data having blank portions as valid data suitable for a learning application, In the valid data, by changing the position of the blank part from the head of the sentence, the invalid data generation unit that generates invalid data inappropriate for learning use, and the feature list of a plurality of types of features for characterizing the data, The valid data group is characterized by statistically processing the valid data and the feature analysis unit that characterizes the invalid data, the valid data group that is a set of the valid data, and the invalid data group that is the set of the invalid data. And a reference data generation unit that generates reference data for distinguishing between the invalid data group and the classification unit, A test data reception unit that receives force, a morphological analysis of the test data, a candidate data feature analysis unit that characterizes candidate data that defines each morpheme as a blank portion by the feature list, and the valid data group by the reference data And an output data generation unit for generating output data for outputting the candidate data classified as (1).

  According to the invention of (1), when valid data is received by the valid data receiving unit, the automatic problem creating apparatus automatically distinguishes between the valid data group and the invalid data group by the reference data generating unit. Reference data is generated, and the candidate data feature analysis unit characterizes the candidate data generated based on the test data. Then, the output data generation unit automatically outputs the candidate data determined to belong to the valid data group.

  As described above, according to the invention of (1), it is possible to automatically create reference data and generate appropriate output data using the reference data. Can be created automatically.

In this specification, “appropriate” means that it is appropriate for a learning application. And “invalid” means inappropriate for learning use. The “correct answer” means a correct answer that can objectively compose a correct sentence. The “wrong answer” means an erroneous answer that constitutes an objectively erroneous sentence.
The distinction between “valid” and “invalid” and the distinction between “correct answer” and “wrong answer” are separately independent concepts.

  (2) The automatic problem creation apparatus according to (1), wherein the reference data generation unit is a support vector machine, and the reference data is an identification plane defined by a support vector.

  According to the configuration described in (2), in addition to the effect of (1), a complicated means such as filling a complicated grammar can be automatically and easily performed by a reliable means such as a support vector machine (SVM). Can be created. Here, the support vector machine (SVM) is an AT & T V. It is a learning machine proposed by Vapnik in the framework of statistical learning theory, and is a learning system that uses a hypothesis space of a linear function in a high-dimensional feature space.

  (3) The automatic problem creation device according to (2), wherein the output data generation unit sets the candidate data closest to the identification plane as the output data.

  According to the configuration described in (3), the valid data closest to the identification surface and closest to the invalid data is output, not the candidate data that is clearly deviated from the identification surface and is valid data. In addition to the effects (1) and (2), it is possible to create a good question for which it is difficult to determine whether the answer is correct.

  (4) The automatic question creation apparatus according to any one of (1) to (3), wherein the valid data is a set of a question sentence and an answer given in a test of a national language or a foreign language.

  According to the configuration described in (4), since it is based on the test of the given language or foreign language, in addition to the effects of (1) to (3), appropriate filling in in the problem of national language or foreign language. Can create problems etc.

  (5) The feature analysis unit further includes first incorrect answer data generated by applying an incorrect answer option other than the correct answer included in the valid data to a blank portion, and the incorrect answer option that is not included in the valid data. And the second erroneous answer data generated by applying a word having a predetermined relationship to a blank part by the feature list and the additional feature list for erroneous answers, and the first erroneous data By statistically processing a first erroneous answer data group that is a set of answer data and a second erroneous answer data group that is a set of the second incorrect answer data, the first incorrect answer data group and the second incorrect answer data are processed. An option reference data generation unit that generates option reference data for distinguishing the data group, and the classification unit further has a predetermined relationship with a word corresponding to a blank portion of the output data Words in blanks The candidate error data generated by fitting the candidate error data feature analysis unit characterized by the feature list and the additional feature list and the option reference data are determined to belong to the first error data group. The automatic problem creation device according to any one of (1) to (4), further comprising: an option generation unit that outputs a word corresponding to a blank portion of the candidate incorrect answer data.

  According to the configuration described in (5), the erroneous answer data feature analysis unit characterizes the first incorrect answer data and the second incorrect answer data with the feature list and the additional feature list for erroneous answers, and for the option. The reference data generation unit generates option reference data. Then, the candidate error data is characterized by the candidate error data feature analysis unit by the feature list and the additional feature list, and the candidate error data determined by the option generation unit as belonging to the first error data group Output the word corresponding to the blank part.

  As described above, the automatic question creation apparatus automatically generates the reference data for options, and automatically outputs a word corresponding to the blank portion of the candidate wrong answer data determined to belong to the first wrong answer data group. Therefore, in addition to the effects (1) to (4), a complicated grammar filling problem including options can be automatically created without trouble.

(6) The option reference data generation unit is a support vector machine,
The automatic problem creation device according to (5), wherein the option reference data is an identification surface defined by a support vector.

  According to the configuration described in (6), in addition to the effect of (5), options can be automatically created without trouble by a reliable means such as a support vector machine.

  (7) The automatic problem creation device according to (6), wherein the option generation unit is configured to output a word corresponding to a blank portion of the candidate incorrect answer data closest to the identification plane. .

  According to the configuration described in (7), the candidate data is not the candidate data that deviates greatly from the identification plane and belongs to the first erroneous answer data group, but is closest to the identification plane and is the most similar to the second erroneous answer data group. Since a word corresponding to the blank portion of the first first incorrect answer data is output, in addition to the effect of (6), it is possible to create a good question in which it is difficult to determine whether or not the answer is correct.

(8) A method for automatically creating a hole-filling problem, in which a valid data receiving step for receiving a large number of hole-filling problems having blank portions as valid data, and by changing the position of each blank portion from the beginning of each valid data, the invalid data A non-valid data generation step for generating data, a feature analysis step for characterizing the valid data and the non-valid data by a feature list of a plurality of types of features for characterizing the data, and valid data that is a set of the valid data A reference data generation step for generating reference data for distinguishing between the valid data group and the invalid data group by statistically processing the group and the invalid data group that is a set of the invalid data; and a test A test data reception step for receiving data input;
A candidate data feature analyzing step of characterizing candidate data defining each morpheme as a blank part by the feature list by performing morphological analysis on the test data, and the candidate data determined to belong to the valid data group by the reference data And an output data generation step for generating output data for output.

  According to the invention of (8), as in the invention of (1), it is possible to automatically create reference data and generate appropriate output data using the reference data. It is possible to automatically create a hole filling problem without trouble.

  (9) A computer program for causing a computer to function as a device for automatically creating a hole-filling problem, wherein a valid data receiving step for accepting a large number of hole-filling problems having a blank equivalent part as valid data, and each of the valid data of a blank equivalent part Characteristic analysis for characterizing the valid data and the invalid data by a non-valid data generation step for generating invalid data by changing the position from the beginning of the sentence and a feature list of a plurality of types of features for characterizing the data Statistically processing a valid data group that is a set of valid data and an invalid data group that is a set of invalid data, thereby distinguishing the valid data group from the invalid data group Step of generating reference data to generate reference data and input of test data A test data receiving step for receiving, a morphological analysis of the test data, a candidate data feature analyzing step for characterizing candidate data defining each morpheme as a blank equivalent part by the feature list, and the reference data to the valid data group An output data generation step for generating output data for outputting the candidate data determined to belong is executed.

According to the present invention, it is possible to provide an automatic problem creation apparatus, an automatic problem creation method, and a computer program that can automatically create a complicated grammar filling problem without trouble.

(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of an automatic problem creation apparatus 10 (hereinafter referred to as “apparatus 10”) according to the first embodiment of the present invention. The apparatus 10 includes an input unit 12 that receives input of various information, a dictionary database (DB) 14, a learning result database (DB) 16, an output unit 18, and a control unit 20. The dictionary DB 14 is an English dictionary, for example, and is configured so that the control unit 20 can determine a word having a predetermined relationship with a certain word. Here, “a word having a predetermined relationship with a certain word” is, for example, “at” or “in” with respect to “on” in English. These have the common property of a preposition indicating a place, and are “words having a predetermined relationship”. The device 10 is a computer such as a normal PC or server, but may be a portable information terminal such as a mobile phone. The English dictionary of this embodiment is an English-Japanese dictionary, but unlike this embodiment, an English-English dictionary or the like may be used.

  The control unit 20 is, for example, a CPU (Central Processing Unit) that is a central processing unit that controls the apparatus 10.

  The control unit 20 includes a machine learning unit 30 and a classification unit 50. The machine learning unit 30 automatically learns the data input by the input unit 12 and performs statistical processing. The machine learning unit 30 performs appropriate processing (hereinafter, referred to as “valid data”) and invalid data (hereinafter, “ This is a configuration for generating reference data indicating a criterion for distinction with “invalid data”. The input unit 12 and the machine learning unit 30 function as learning means. The machine learning unit 30 includes a support vector machine (SVM: Support Vector Machine) that is software. As described above, the support vector machine is a learning system that uses a hypothesis space of a linear function in a high-dimensional feature space. The learning result includes an identification surface and a support vector for identifying a valid data set and an invalid data set, and is stored in the learning result DB 16. The classification unit 50 has a configuration for creating a new hole filling problem using the input English text (test data) and the learning result of the learning result DB 16.

  FIG. 2 is a block diagram showing details of the machine learning unit 30. As illustrated in FIG. 2, the machine learning unit 30 includes an invalid data generation unit 32, a feature analysis unit 34, and a reference data generation unit 36. The invalid data generation unit 32 is a configuration for generating invalid data based on the training data. The feature analysis unit 34 is configured to analyze features of valid data and invalid data. The reference data generation unit 36 is configured to calculate an identification plane and a support vector as a reference for identifying valid data and invalid data. Details of the function of each unit will be described later.

  FIG. 3 is a block diagram showing details of the classification unit 50. As illustrated in FIG. 3, the classification unit 50 includes a candidate data feature analysis unit 52 and an output data generation unit 54. The candidate data feature analysis unit 52 is configured to create candidate data from the test data and analyze the feature of the candidate data. The output data generation unit 54 is configured to select (generate) data to be output from candidate data. Details of the function of each unit will be described later.

  FIG. 4 is a diagram illustrating an example of a feature list used by the machine learning unit 30. A feature characterizes input data. The feature value is a score given to the feature. The feature value includes a binary value of “0” or “1”, a binary value of “0” or “−1”, or an arbitrary numerical value.

  The feature list is composed of a plurality of types of features. For example, as shown in FIG. 4, the feature list is composed of six types of features A to F. The feature A is a word corresponding to the blank part of the hole-filling problem, and is defined as a feature value 1 if there is a word, and is defined as 0 if it does not exist. The “blank part” indicates a hole part to be filled in the hole filling problem. In other words, the “blank part” means a position where a word (hereinafter referred to as “problem word”) that should enter a hole (vacant space) of the hole filling problem should be included in a sentence composed of a plurality of words.

  The feature B is the part of speech of the word corresponding to the empty space. If the part of speech can be determined, the feature value is defined as 1, and if the part of speech cannot be determined, the feature value is defined as 0. The feature C is a prototype of a word corresponding to a void. If the prototype can be determined, the feature value is defined as 1. If the prototype cannot be determined, the feature value is defined as 0. The feature D is a match between the word corresponding to the empty space and the prototype, and if they match, the feature value is defined as 1, and if not, the feature value is defined as 0. The feature E is the length of the sentence, and the feature value is defined by the number of words constituting the sentence. The feature F is a blank position in the sentence, and the feature value is defined by the number of words from the beginning.

  FIG. 5 is a diagram illustrating an example of training data. Training data is data used as a material for the apparatus 10 to learn. In the present embodiment, the training data is a question sentence and a correct answer of a hole filling problem having a blank part.

  As illustrated in FIG. 5, the training data is, for example, an English sentence “Thank you very much (for) taking care of my child.”. In this English sentence, the word “for” in parentheses is a blank word (question word) and is a correct answer. In the actual test, blanks are left blank. Note that the above-mentioned English sentences are the question sentences given in the actual English test and their correct answers. Since this training data is given in an actual English test, it is appropriate for learning purposes. For this reason, in this embodiment, the question sentence and the correct answer shown in the training data are set as valid data. Unlike the present embodiment, the training data may be a question sentence and its correct answer given in a Japanese (Japanese language) test, or in a test in a foreign language other than English (French etc.). It may be a question sentence and its correct answer.

  The input unit 12 (see FIG. 1) is adapted to accept a large number of the training data described above. Further, the input unit 12 is configured to receive input of options other than the correct answer when the training data is originally a multiple choice question. Options other than the correct answer are also valid data because they are valid as problems. Thus, the input unit 12 functions as a valid data receiving unit.

  The machine learning unit 30 (see FIG. 1) generates invalid data by changing the position of the blank portion from the beginning of each sentence in the training data. Non-valid data is data that is not valid for learning purposes. The hole filling problem created based on the invalid data cannot sufficiently achieve the learning effect of students studying English, and becomes an inappropriate problem for learning use. This invalid data is generated based on each training data.

  FIG. 6 is a diagram illustrating an example of invalid data. As illustrated in FIG. 6, the invalid data generation unit 32 of the machine learning unit 30 generates invalid data by changing the position of the blank portion of each training data from the beginning of the sentence. In FIG. 6, the part in parentheses is a blank part.

  FIG. 7 is a diagram illustrating an example of a case generated based on valid data and invalid data (hereinafter referred to as “valid data”). Examples are data indicating features such as valid data. Valid data is known to be valid and its label is “1”. It is known that invalid data is not valid and its label is “−1”.

  The feature analysis unit 34 (see FIG. 2) performs morphological analysis on valid data, etc., determines the features of the words that apply to the blank portion and the entire features such as valid data, and adds a label value (hereinafter referred to as “labeling”). "). For example, for valid data in which the word of the blank part is “for”, in the feature list of FIG. 4, the feature A is “for”, the feature value is “1”, and the feature B is “preposition”. The value is “1”, the feature C is “for”, the feature value is “1”, the feature D is “match”, and the feature value is “1”. The feature E is “sentence length”, the feature value is “10”, the feature F is “a position of a space in the sentence”, and the feature value is “5”. These features A to F and corresponding feature values are examples of valid data. Since it is valid data, the label is “1”.

  On the other hand, for example, for invalid data whose blank word is “taking”, the feature A is “taking”, the feature value is “1”, the feature B is “verb”, and the feature value is Is “1”, the feature C is “take”, the feature value is “1”, the feature D is “mismatch”, and the feature value is “0”. The feature E is “sentence length” and the feature value is “10”, the feature F is “position of a space in the sentence” and the feature value is “6”. Since the data is invalid, the label is “−1”.

  Similarly, feature determination and labeling are performed on all invalid data. Then, a case with a label about valid data or the like is stored in the learning result DB 16.

  FIG. 8 is a diagram illustrating an example of a learning result by the reference data generation unit 36 (see FIG. 2). The reference data is generated by the reference data generation unit 36. The reference data generation unit 36 is a support vector machine. The reference data generation unit 36 is a classifier that estimates a label for a case in which a valid or invalid label is unknown. Cases that are generated from valid data (hereinafter referred to as “valid cases”) and cases that are generated from invalid data (hereinafter referred to as “invalid cases”) after taking appropriate data as examples The identification surface is calculated such that the distance (margin) between the two identification surfaces is maximized. As described above, the reference data generation unit 36 learns after converting the valid data into the form of the case.

  The reference data generation unit 36 uses a valid case and an invalid case that are closest to the identification surface as support vectors, and uses them for classification of cases whose labels are unknown. In this way, the reference data generation unit 36 makes an identification for distinguishing between a valid data (case) group and an invalid data (case) group by performing statistical processing after taking a valid case and an invalid case as examples. An identification surface which is data is generated.

  As shown in FIG. 8, the reference data generation unit 36 includes a valid case group (a valid case group set) that is a set to which valid cases belong, and an invalid case group (invalid case set) that is a set to which invalid cases belong. An identification surface is generated for identifying. Then, the valid case and the invalid case that are closest to the identification surface are used as support vectors. In other words, the identification plane is defined by the support vector.

  FIG. 9 is a diagram illustrating an example of test data. As shown in FIG. 9A, in the present embodiment, the test data is an English sentence “Thank you very much for your help.” This test data is received by the input unit 12 (see FIG. 1). Thus, the input unit 12 also functions as a test data receiving unit.

  The candidate data feature analysis unit 52 (see FIG. 3) of the classification unit 50 is configured to perform morphological analysis on the test data and to characterize candidate data that defines each morpheme as a blank part using a feature list.

  FIG. 9B is a diagram illustrating an example of candidate data. As shown in FIG. 9A, the candidate data feature analysis unit 52 generates a plurality of candidate data having each morpheme as a blank portion from the test data.

  FIG. 9C is a diagram illustrating an example of a case generated based on each candidate data. As shown in FIG. 9C, a case is generated for each candidate data. Hereinafter, a case generated based on the candidate data is referred to as a “candidate case”. Since it is unknown whether the candidate case is valid, its label is unknown.

  FIG. 10 is a diagram illustrating an example of a method in which the output data generation unit 54 (see FIG. 3) labels candidate cases. As shown in FIG. 10, the output data generation unit 54 uses each identification case to identify each candidate case belonging to a valid case group (hereinafter referred to as a “valid candidate case”) and a candidate case belonging to an invalid case group. (Hereinafter referred to as “invalid candidate cases”). This means that candidate cases are labeled. The label of the valid candidate case is “1”, and the label of the non-valid candidate case is “−1”.

  The output data generation unit 54 is configured to output data for outputting candidate data that are valid candidate cases. Specifically, the output data generation unit 54 determines a valid candidate case closest to the identification surface, and uses candidate data corresponding to the valid candidate case as output data.

  The apparatus 10 is configured as described above. As described above, when the valid data is received by the input unit 12 (see FIG. 1), the apparatus 10 automatically distinguishes the valid data group from the invalid data group by the reference data generation unit 36 (see FIG. 2). An identification plane and a support vector are generated, and candidate data generated based on the test data is characterized by the candidate data feature analysis unit 52 (see FIG. 3). Then, the candidate data determined to belong to the valid data group is automatically output by the output data generation unit 54 (see FIG. 3).

  As described above, according to the apparatus 10, since the reference data can be automatically created and appropriate output data can be generated using the reference data, the complicated grammar filling problem can be automatically and easily performed. Can be created.

  The reference data generating unit 36 can automatically create a complicated grammar filling problem and the like without trouble by a reliable means called a support vector machine. The output data generation unit 54 uses the candidate data closest to the identification plane as output data. Therefore, the output data generation unit 54 is not the candidate data that is significantly deviated from the identification plane and is clearly valid data, but is closest to the identification plane. The valid data closest to the invalid data is output. For this reason, it is possible to create a good question that is difficult to determine whether or not it is a correct answer.

  Further, the training data (valid data) is a set of question sentences and answers given in an actual English test, and the apparatus 10 generates output data based on the English test actually given. Appropriate hole filling problems can be created.

  Hereinafter, an operation example of the apparatus 10 will be described mainly using FIGS. 11 and 12. 11 and 12 are schematic flowcharts showing an operation example of the apparatus 10.

  First, the apparatus 10 receives a large number of inputs by using a combination of a question sentence and an answer of a hole filling problem as training data (step S10 in FIG. 11). Subsequently, invalid data is generated (step S12). Then, the features of valid data (training data) and invalid data are determined, and a labeled case is generated (step S14). Since the training data (valid data) is known to be valid in advance, it is labeled as “1” indicating that it is valid, and the invalid data is not valid because it is known in advance. A label “−1” is attached.

  Subsequently, the apparatus 10 performs statistical processing on the labeled case and calculates an identification surface and a support vector (step S16). The above is the learning process by the apparatus 10. Next, the classification process by the apparatus 10 will be described with reference to FIG.

  First, the apparatus 10 accepts an English input as test data (step S30 in FIG. 12). Subsequently, morphological analysis is performed on the test data, and candidate data defining each morpheme as a blank equivalent part is generated (step S32). Subsequently, the candidate data is characterized by a feature list, and a candidate case is generated (step S34).

  Subsequently, the device 10 refers to the learning result and labels each candidate case (step S36). Subsequently, the apparatus 10 selects a candidate case closest to the identification surface among the valid candidate cases as an output case (step S38). Subsequently, the device 10 outputs candidate data corresponding to the output case (step S40).

  Through the above steps, a complicated grammar filling problem can be automatically created without trouble.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIGS. 13, 14, 15, 16 and 17. The apparatus 10A (see FIG. 13) of the second embodiment has a function for automatically generating appropriate options in addition to the function of the apparatus 10 of the first embodiment. As will be described below, the apparatus 10A can learn using the option when the option is also included in the training data (see FIG. 16). The apparatus 10A can generate a valid option when creating a problem using the test data.

  13, 14 and 15 are functional block diagrams of the apparatus 10A. FIG. 16 is a diagram illustrating an example of training data. FIG. 17 is a diagram illustrating an example of an additional feature list for incorrect answer data.

  As illustrated in FIG. 13, the control unit 20A of the apparatus 10A includes a machine learning unit 30A and a classification unit 50A. As illustrated in FIG. 14, the machine learning unit 30 </ b> A of the apparatus 10 </ b> A includes a first incorrect answer data generation unit 40. The first incorrect answer data generation unit 40 is configured to generate data (hereinafter referred to as “first incorrect answer data”) to which options other than correct answers of training data (hereinafter referred to as “valid options”) are applied. It is. The machine learning unit 30 </ b> A includes a second incorrect answer data generating unit 42. The second incorrect answer data generation unit 42 extracts words that are not included in the training data (valid data) but have a predetermined relationship with the valid option (hereinafter referred to as “invalid option”) from the dictionary DB 14, and training. Data (hereinafter referred to as “second erroneous answer data”) is generated by applying the blank portion of the data.

  As described above, the appropriate option is an option that is included in the training data and is not a correct answer. For example, in the training data of FIG. 16, “in” is a correct answer option. “Into” and “on” are options that are not correct answers. These are not correct answers, but are words that have a misleading meaning of “in” and are appropriate for students who study English to learn. For this reason, these non-correct answers are called valid options. On the other hand, a word that has a predetermined relationship with a word in the blank portion of the training data and is not included in the training data is called an invalid option. For example, “above” and “having” are invalid options. The second incorrect answer data generation unit 42 refers to the dictionary DB 14 to extract invalid options that are words having a predetermined relationship with “in” and apply them to blank portions of the training data. . The predetermined relationship is, for example, that the meaning is similar or the spelling (spelling) is approximate.

  As illustrated in FIG. 14, the machine learning unit 30 </ b> A includes an erroneous answer data feature analysis unit 44. The incorrect answer data feature analysis unit 44 uses a feature list (first error data in which valid options are applied to blank portions of training data and second incorrect answer data in which invalid options are applied to blank portions of training data. Characterized by an additional feature list (see FIG. 17) and a case are generated. Here, the feature list is also used because the blank portion corresponds to the same place in the same training data, so that the difference between the cases is extracted by analyzing the features in more detail. Since the case generated based on the first erroneous answer data is known in advance to be valid, the erroneous answer data feature analysis unit 44 attaches a label “1” indicating that it is valid. On the other hand, since it is known in advance that the case generated based on the invalid answer data is not valid, the label “-1” indicating that the answer data feature analysis unit 44 is not valid is attached. To do.

  As shown in FIG. 17, the additional feature list for incorrect answer data is composed of features G to J, for example. The feature G is one of the parts of speech with the correct answer, the feature H is “similarity of meaning with the correct answer”, the feature I is “word length”, and the feature J is “the number of meanings is a predetermined threshold value” whether or not α or more ”, and includes feature values corresponding to the features G to J.

  As illustrated in FIG. 14, the machine learning unit 30 </ b> A includes an option reference data generation unit 46. The option reference data generation unit 46 includes a first incorrect answer case group that is a set of cases generated from the first incorrect answer data and a second incorrect answer case that is a set of cases generated from the second incorrect answer data. This is a configuration for generating reference data (hereinafter referred to as “reference data for options”) for distinguishing between the first erroneous answer case group and the second erroneous answer case group by performing statistical processing on the group. . The option reference data generation unit 46 is a support vector machine, and the option reference data is an identification plane and a support vector that defines the identification plane.

  As illustrated in FIG. 15, the classification unit 50 </ b> A includes a candidate incorrect answer data feature analysis unit 60. The candidate error data feature analysis unit 60 adds a plurality of candidate error data generated by applying a word having a predetermined relationship with a blank word of the test data to the test data, and adds a feature list (see FIG. 4) and By characterizing with a feature list (see FIG. 17), a candidate wrong answer example is generated. Here, the phrase “having a predetermined relationship with the word in the blank portion of the test data” means that the meaning is similar, for example. Then, the candidate incorrect answer data feature analysis unit 60 labels candidate incorrect answer cases according to the identification surface.

  As illustrated in FIG. 15, the classification unit 50A includes an option generation unit 62. The option generation unit 62 distinguishes each candidate error answer case into a candidate error answer case belonging to the first error answer case group and a candidate error answer case belonging to the second error answer case group according to the identification surface. This means that the candidate wrong answer cases are labeled. The label of the candidate wrong answer case belonging to the first wrong answer case group is “1”, and the label of the candidate wrong answer case belonging to the second wrong answer case group is “−1”.

  The option generation unit 62 selects a predetermined number of candidate erroneous answer examples in order from the closest to the identification plane, and outputs a blank portion of data corresponding to the erroneous answer candidate examples as an erroneous answer option.

  As described above, the device 10A is not candidate data that deviates greatly from the identification plane and belongs to the first erroneous answer data group, but is closest to the identification plane and closest to the second incorrect answer data group. Since the first erroneous answer data is output, it is possible to create a good question for which it is difficult to determine whether the answer is correct.

[Hardware configuration of automatic problem creation apparatus 10]
FIG. 18 is a diagram illustrating a hardware configuration example of the devices 10 and 10A (hereinafter referred to as the information processing device 1000). The information processing apparatus 1000 includes a CPU (Central Processing Unit) 1010 (a plurality of CPUs such as the CPU 1012 may be added in a multiprocessor configuration), a bus line 1005, a communication I / F 1040, and a main memory 1050 that configure each control unit. , A BIOS (Basic Input Output System) 1060, a USB port 1090, an I / O controller 1070, an input means such as a keyboard and mouse 1100, and a display device 1022.

  Storage means such as a tape drive 1072, a hard disk 1074, an optical disk drive 1076, and a semiconductor memory 1078 can be connected to the I / O controller 1070.

  The BIOS 1060 stores a boot program executed by the CPU 1010 when the information processing apparatus 1000 is activated, a program depending on the hardware of the information processing apparatus 1000, and the like.

  The hard disk 1074 constituting the storage unit stores various programs for the information processing apparatus 1000 to function and programs for executing the functions of the present invention, and can configure various databases as necessary.

  As the optical disc drive 1076, for example, a DVD-ROM drive, a CD-ROM drive, a DVD-RAM drive, or a CD-RAM drive can be used. In this case, the optical disk 1077 corresponding to each drive is used. A program or data can be read from the optical disk 1077 by the optical disk drive 1076 and provided to the main memory 1050 or the hard disk 1074 via the I / O controller 1070. Similarly, the tape medium 1071 corresponding to the tape drive 1072 can be used mainly for backup.

  The program provided to the information processing apparatus 1000 is provided by being stored in a recording medium such as the hard disk 1074, the optical disk 1077, or a memory card. The program may be installed in the information processing apparatus 1000 and executed by being read from the recording medium via the I / O controller 1070 or downloaded via the communication I / F 1040.

  The aforementioned program may be stored in an internal or external storage medium. Here, in addition to the hard disk 1074, the optical disk 1077, or the memory card, a magneto-optical recording medium such as an MD, or a tape medium 1071 can be used as a storage medium constituting the storage unit. Further, a storage device such as a hard disk 1074 or an optical disk library provided in a server system connected to a dedicated communication line or the Internet may be used as a recording medium, and the program may be provided to the information processing apparatus 1000 via the communication line.

  Here, the display device 1022 displays a screen for accepting data input to the server administrator, or displays a screen of a calculation processing result by the information processing device 1000. The display device 1022 is a cathode ray tube display (CRT), liquid crystal display, or the like. A display device such as a display device (LCD) is included.

  Here, the input means accepts input by the server administrator, and may be constituted by a keyboard, a mouse 1100, and the like.

  The communication I / F 1040 is a network adapter that enables the information processing apparatus 1000 to be connected to a terminal via a dedicated network or a public network. The communication I / F 1040 may include a modem, a cable modem, and an Ethernet (registered trademark) adapter.

  In the above example, the information processing apparatus 1000 has been mainly described. However, the functions described above can also be realized by installing a program in a computer and causing the computer to operate as the information processing apparatus 1000. Therefore, the functions realized by the information processing apparatus 1000 described as an embodiment in the present invention are executed by executing the above-described method by the computer, or by introducing the above-described program into the computer and executing it. Is also feasible. The keyboard / mouse 1100 described above corresponds to the input unit 12 (see FIGS. 1 and 13), and the CPU 1010 and CPU 1012 correspond to the control units 20 and 20A (see FIGS. 1 and 13). The display device 1022 corresponds to the output unit 18 (see FIGS. 1 and 13). The hard disk 1074 corresponds to the dictionary DB 14 and the learning result DB 16.

  The apparatus 10, 10A (see FIGS. 1 and 13), which is an embodiment of the present invention, or the method used for controlling this apparatus can also be realized by a program on a computer. The storage medium storing the program can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of this storage medium include semiconductor or solid state storage devices, magnetic tape, removable computer readable media examples include semiconductor or solid state storage devices, magnetic tape, removable flexible disks, random Includes access memory (RAM), read only memory (ROM), rigid magnetic disk and optical disk. Examples of optical disks at the present time include compact disk read only memory (CD-ROM), compact disk read / write (CD-R / W) and DVD.

  As mentioned above, although embodiment of this invention was described, it only showed the specific example and does not specifically limit this invention. Further, the effects described in the embodiments of the present invention only list the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to the effects described in the embodiments of the present invention.

It is a block diagram which shows schematic structure of an automatic problem preparation apparatus. It is a block diagram which shows the detail of a machine learning part. It is a block diagram which shows the detail of a classification | category part. It is a figure which shows an example of a feature list. It is a figure which shows an example of training data. It is a figure which shows an example of invalid data. It is a figure which shows an example of a case. It is a figure which shows an example of a learning result. It is a figure which shows an example etc. of test data. It is a figure which shows an example of the method of labeling a candidate example. It is a schematic flowchart which shows the operation example of a problem automatic preparation apparatus. It is a schematic flowchart which shows the operation example of a problem automatic preparation apparatus. It is a block diagram which shows schematic structure of an automatic problem preparation apparatus. It is a block diagram which shows the detail of a machine learning part. It is a block diagram which shows the detail of a classification | category part. It is a figure which shows an example of training data. It is a figure which shows an example of an additional feature list. It is a figure which shows the hardware structural example of the server of a problem automatic preparation apparatus.

Explanation of symbols

10, 10A automatic problem creation device 12 input unit 14 dictionary DB
16 Learning result DB
18 Output unit 20, 20A Control unit 30, 30A Machine learning unit 32 Inappropriate data generation unit 34 Feature analysis unit 36 Reference data generation unit 40 First error data generation unit 42 Second error data property western part
44 erroneous answer data feature analysis unit 46 option reference data generation unit 50, 50A classification unit 52 candidate data feature analysis unit 54 output data generation unit 60 candidate error data feature analysis unit 62 option generation unit

Claims (9)

An automatic problem creation apparatus having a machine learning unit and a classification unit,
The machine learning unit
A valid data receiving unit that accepts a large number of blank filling problems with blank parts as valid data suitable for learning use;
In each of the valid data, an invalid data generator that generates invalid data inappropriate for learning use by changing the position of the blank part from the beginning of the sentence;
A feature analysis unit characterizing the valid data and the invalid data by a feature list of a plurality of types of features for characterizing the data;
Reference data for distinguishing between the valid data group and the invalid data group by statistically processing the valid data group that is the set of valid data and the invalid data group that is the set of invalid data A reference data generation unit to generate,
Have
The classification unit includes:
A test data reception unit for receiving input of test data;
A candidate data feature analysis unit that performs morphological analysis on the test data and characterizes candidate data that defines each morpheme as a blank portion by the feature list
An output data generation unit that serves as output data for outputting the candidate data classified into the valid data group according to the reference data;
An automatic problem creation apparatus characterized by comprising: The reference data generation unit is a support vector machine,
The automatic problem creation device according to claim 1, wherein the reference data is an identification surface defined by a support vector.   The automatic problem creation apparatus according to claim 2, wherein the output data generation unit sets the candidate data closest to the identification plane as the output data.   The automatic question creation apparatus according to any one of claims 1 to 3, wherein the valid data is a set of a question sentence and an answer given in a national language or foreign language test. The feature analysis unit further includes:
First error data generated by applying wrong answer options other than correct answers included in the valid data to blank parts, and blank parts that are not included in the valid data but have a predetermined relationship with the wrong answer options An erroneous answer data feature analysis unit that characterizes the second erroneous answer data generated by applying to the feature list and the additional feature list for erroneous answers;
By statistically processing a first erroneous answer data group that is a set of the first incorrect answer data and a second incorrect answer data group that is a set of the second incorrect answer data, the first incorrect answer data group and the An option reference data generation unit for generating option reference data for distinguishing the second erroneous answer data group;
Have
The classification unit further includes:
Candidate error data characteristic analysis unit characterized by the feature list and the additional feature list, the candidate answer data generated by applying a word having a predetermined relationship with the word corresponding to the blank portion of the output data to the blank portion; ,
An option generation unit that outputs a word corresponding to a blank portion of the candidate incorrect answer data determined to belong to the first incorrect answer data group based on the option reference data;
5. The automatic problem creating apparatus according to claim 1, further comprising: The option reference data generation unit is a support vector machine,
6. The automatic problem creation apparatus according to claim 5, wherein the option reference data is an identification surface defined by a support vector.   The automatic question creation apparatus according to claim 6, wherein the option generation unit is configured to output a word corresponding to a blank portion of the candidate incorrect answer data closest to the identification surface. A method for automatically creating a hole filling problem,
A valid data acceptance step for accepting a large number of hole filling problems having blank portions as valid data;
A non-valid data generation step of generating invalid data by changing the position of the blank part from the beginning of each valid data;
A feature analysis step for characterizing the valid data and the invalid data by a feature list of a plurality of types of features for characterizing the data;
Reference data for distinguishing between the valid data group and the invalid data group by statistically processing the valid data group that is the set of valid data and the invalid data group that is the set of invalid data A reference data generation step to generate;
A test data reception step for receiving input of test data;
A morphological analysis of the test data, a candidate data feature analysis step characterized by the feature list candidate data defining each morpheme as a blank part;
An output data generation step as output data for outputting the candidate data determined to belong to the valid data group according to the reference data;
A method for automatically creating a problem, characterized by comprising: A computer program for causing a computer to function as an automatic creation device for a hole filling problem,
A valid data receiving step for accepting a large number of hole filling problems having a blank equivalent part as valid data;
A non-valid data generation step of generating non-valid data by changing a position from the beginning of each valid data of the blank equivalent part;
A feature analysis step for characterizing the valid data and the invalid data by a feature list of a plurality of types of features for characterizing the data;
Reference data for distinguishing between the valid data group and the invalid data group by statistically processing the valid data group that is the set of valid data and the invalid data group that is the set of invalid data A reference data generation step to generate;
A test data reception step for receiving input of test data;
A morphological analysis of the test data, a candidate data feature analyzing step characterized by the feature list candidate data defining each morpheme as a blank equivalent part,
An output data generation step as output data for outputting the candidate data determined to belong to the valid data group according to the reference data;
A computer program for executing

Images