WO2022180989A1 - Dispositif de génération de modèle et procédé de génération de modèle - Google Patents

Dispositif de génération de modèle et procédé de génération de modèle Download PDF

Info

Publication number
WO2022180989A1
WO2022180989A1 PCT/JP2021/046088 JP2021046088W WO2022180989A1 WO 2022180989 A1 WO2022180989 A1 WO 2022180989A1 JP 2021046088 W JP2021046088 W JP 2021046088W WO 2022180989 A1 WO2022180989 A1 WO 2022180989A1
Authority
WO
WIPO (PCT)
Prior art keywords
question
sentence
model
generation
classification
Prior art date
Application number
PCT/JP2021/046088
Other languages
English (en)
Japanese (ja)
Inventor
熱気 澤山
Original Assignee
株式会社Ｎｔｔドコモ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｎｔｔドコモ filed Critical 株式会社Ｎｔｔドコモ
Priority to JP2023502088A priority Critical patent/JPWO2022180989A1/ja
Publication of WO2022180989A1 publication Critical patent/WO2022180989A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/42—Data-driven translation
    • G06F40/44—Statistical methods, e.g. probability models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/55—Rule-based translation
    • G06F40/56—Natural language generation

Definitions

  • One aspect of the present invention relates to a model generation device and a model generation method.
  • Patent Document 1 discloses a question sentence generation method for inputting the contents of a user's utterance (input sentence) and generating a question sentence according to the input sentence.
  • Patent Literature 1 discloses a mechanism for generating a question sentence for a user from a predetermined template based on a predicate argument structure extracted from an input sentence and an estimated dialogue act type.
  • one aspect of the present invention aims to provide a model generation device and a model generation method that can efficiently generate a model that generates a question sentence for an input sentence.
  • a model generation device includes a pair information acquisition unit that acquires pair information including a question sentence and an answer sentence to the question sentence;
  • a teacher data generation unit that generates teacher data in which the question sentences included in is associated with output data, and machine learning using the teacher data is executed to input an arbitrary first sentence and ask a question for the first sentence a model generation unit that generates a question generation model that outputs a second sentence that is a sentence;
  • a question generation model that inputs an arbitrary first sentence and generates a question sentence (second sentence) for the first sentence is obtained.
  • the question generation model it is natural to use a pair of an arbitrary sentence corresponding to the first sentence and a question sentence corresponding to the arbitrary sentence as training data. More specifically, since the first sentence input to the question generation model exists temporally before the second sentence output from the question generation model, in order to generate the question generation model, a teacher who associates an arbitrary sentence corresponding to the first sentence with the input data, and associates a question sentence generated temporally after the arbitrary sentence (that is, a question sentence for the arbitrary sentence) with the output data.
  • the teacher data in which the temporal context is reversed that is, the question sentence that occurred earlier in time corresponds to the output data, and the answer sentence that occurred later in time corresponds to the output data).
  • teacher data corresponding to the input data is used.
  • the question sentence (second sentence) corresponding to the input first sentence is output. It was discovered that a question generation model that Furthermore, pair information containing a question and an answer to the question is relatively easily available. Therefore, according to the model generation device, it is possible to efficiently generate a model for generating a question sentence for an input sentence.
  • a model generation device and a model generation method that can efficiently generate a model that generates a question sentence for an input sentence.
  • FIG. 4 is a diagram showing an example of a question sentence generated by a question generation model; 4 is a flow chart showing an example of a processing procedure of the model generating device;
  • FIG. 10 is a diagram illustrating a first example of processing by an analysis unit;
  • FIG. 10 is a diagram illustrating a second example of processing by the analysis unit;
  • It is a figure which shows an example of the process of a question sentence production
  • It is a figure which shows an example of the hardware constitutions of a model generation apparatus and a question generation apparatus.
  • FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to one embodiment.
  • the information processing system 1 includes a model generation device 10 , a question generation device 20 and one or more user terminals 30 .
  • the model generation device 10 is a device that generates a question generation model M.
  • the question generation model M is a trained model created by machine learning so that an arbitrary sentence is input and a question sentence for the sentence is generated.
  • the question generation device 20 is a device that generates a question sentence for a user's input sentence and presents the generated question sentence to the user. In this embodiment, the question generation device 20 generates a question sentence for the user's input sentence by using the question generation model M generated by the model generation device 10 .
  • the question generation device 20 is configured to communicate with the user terminal 30 .
  • the question generation device 20 is configured to receive from the user terminal 30 an input sentence input by a user operation via the user terminal 30 and to transmit the generated question sentence to the user terminal 30 .
  • a user terminal 30 is any terminal possessed and operated by a user. Specific examples of the user terminal 30 include smart phones, tablet terminals, desktop PCs, laptop PCs, wearable terminals, and the like.
  • Each of the model generation device 10 and the question generation device 20 may be configured by a single server device, or may be configured by a plurality of server devices. Also, the model generation device 10 and the question generation device 20 may be configured by the same server device.
  • the information processing system 1 is a system that supports a user's language learning (English learning in this embodiment).
  • the sentence input by the user is in English
  • the question sentence generated by the question generation device 20 is also in English.
  • the question generating device 20 acquires an arbitrary English sentence input by the user, presents the user with a question sentence corresponding to the content of the English sentence, and prompts the user to answer in English, thereby helping the user learn English.
  • generation and presentation of redundant (meaningless) question sentences should be eliminated as much as possible.
  • the question generating device 20 is configured to avoid generating and presenting such redundant question texts and to generate and present question texts with appropriate content according to input texts.
  • the model generation device 10 has a pair information acquisition unit 11 , a teacher data generation unit 12 , a model generation unit 13 and a model storage unit 14 .
  • the pair information acquisition unit 11 acquires pair information including a question and an answer to the question.
  • the pair information acquisition unit 11 may acquire such samples as pair information, for example.
  • the information source of the pair information is not limited to the above example.
  • the pair information acquisition unit 11 extracts question and answer pairs from any information source that can extract question and answer pairs, such as dialogue logs accumulated in any dialogue system that interacts with the user, answer results (Q&A) of web questionnaires, etc. , may acquire pair information. Examples of paired information include the question "What animal do you like?" and the answer to the question "I don't really like animals.” I don't like it.)" pair.
  • the teacher data generation unit 12 generates teacher data by associating the answer sentences included in the pair information acquired by the pair information acquisition unit 11 with the input data, and by associating the question sentences included in the pair information with the output data.
  • the teacher data generation unit 12 generates teacher data in which the temporal order of the question sentence and the answer sentence included in the pair information is reversed. More specifically, the teacher data generation unit 12 generates teacher data having a relationship of predicting a question sentence (output data) that occurred earlier in time from an answer sentence (input data) that occurred later in time. Generate.
  • the teacher data generation unit 12 associates the answer sentences included in the pair information with the input data, and further associates the classification information indicating the classification of the question sentences included in the pair information with the input data.
  • the classification information is a category related to question content (question target).
  • the classification information is information indicating the classification corresponding to English interrogative words.
  • the classification information is information indicating six classifications called 5W1H. That is, the taxonomy information includes "when", "where", “who”, “what”, “why", and “how”. is information indicating six classifications of
  • the training data generation unit 12 extracts interrogative words included in the question sentence, and uses the extracted interrogative words as classification information. For example, in the pair information example described above, the training data generation unit 12 extracts the interrogative word "what" included in the question sentence "What animal do you like?" and uses the interrogative word "what” as the classification information. In this case, the question word "what" and the answer sentence "I don't really like animals.” correspond to the input data, and the question sentence "What animal do you like?" corresponds to the output data. be done. For example, teacher data is expressed in the following data format.
  • the training data includes "a token specifying the division of interrogative words (5W1H), an answer sentence, a token ⁇ SEP> indicating a break between the answer sentence and the question sentence, a question sentence, and a token ⁇ EOS> indicating the end of the sentence.” It is composed by In the teacher data expressed in such a data format, the part before the token ⁇ SEP> corresponds to the input data, and the part after the token ⁇ SEP> corresponds to the output data.
  • the training data generation unit 12 Information indicating that there is no interrogative (other) (for example, information indicating null) may be set as the classification information.
  • the model generation unit 13 described later By performing learning by the model generation unit 13 described later using such teacher data (that is, teacher data including “other” classification information), the classification information input to the question generation model M becomes null. It is possible to generate a question generation model M capable of generating a question sentence (for example, a question sentence corresponding to a closed question) even in the case of (that is, when no interrogative is specified).
  • the model generation unit 13 executes machine learning using the training data generated by the training data generation unit 12 to input an arbitrary first sentence and generate a question sentence (second sentence) for the first sentence.
  • a question generation model M to be output is generated.
  • teacher data includes classification information as data corresponding to input data.
  • the question generation model M inputs an arbitrary first sentence and arbitrary classification information, and the classification indicated by the arbitrary classification information (in this embodiment, any interrogative of 5W1H or "other" It is configured to output a question sentence (second sentence) according to the classification shown).
  • Figure 2 corresponds to each classification information (interrogative) obtained from an input sentence "I studied English yesterday.” by the question generation model M created by the present inventor.
  • An example of a question sentence is shown.
  • a question generation model M capable of generating and outputting a question sentence (second sentence) asking a question about the item was obtained.
  • the model generation unit 13 may generate the question generation model M by performing additional learning on a language model prepared in advance.
  • the model generation unit 13 uses a trained large-scale language model such as BERT (Bidirectional Encoder Representations from Transformers), GPT-2 (Generative Pre-trained Transformer), etc.
  • the question generation model M may be generated by performing learning (eg, fine tuning, transfer learning, etc.). By generating the question generation model M based on the language model described above, it is possible to obtain the question generation model M capable of generating a more natural sentence (second sentence).
  • the question generation model M generated by the model generation unit 13 is stored in the model storage unit 14.
  • the question generation model M stored in the model storage unit 14 is used by the question generation device 20 (more specifically, the question sentence generation unit 23 described later).
  • the pair information acquisition unit 11 acquires pair information (pair information acquisition step, step S1).
  • the pair information acquisition unit 11 acquires a sufficient number of pair information for the question generation model M to learn.
  • the teacher data generator 12 generates teacher data based on the pair information acquired by the pair information acquirer 11 (teacher data generation step, step S2).
  • the pair information acquisition unit 11 generates teacher data in which the answer sentences included in the pair information correspond to the input data and the question sentences included in the pair information correspond to the output data.
  • the teacher data generation unit 12 generates teacher data by further associating classification information indicating the classification of question sentences included in the pair information (for example, interrogative words included in the question sentences) with the input data.
  • the model generator 13 generates a question generation model M by executing machine learning using the teacher data generated by the teacher data generator 12 (model generation step, step S3).
  • the generated question generation model M is stored in the model storage unit 14 .
  • the question generation device 20 includes a reception unit 21 , an analysis unit 22 , a question text generation unit 23 , a question text selection unit 24 , a question text correction unit 25 and a presentation unit 26 .
  • the reception unit 21 acquires (receives) the user's input sentence from the user terminal 30 .
  • the analysis unit 22 analyzes the user's input sentence acquired by the reception unit 21 to identify or presume that the input sentence includes one or more of a plurality of predetermined classifications related to the content of the sentence. Extract the first category.
  • the multiple classifications are the same as the classification information described above. That is, the multiple classifications are classifications corresponding to English interrogative words (eg, 5W1H).
  • a specific example of processing executed by the analysis unit 22 to extract the first classification will be described below.
  • FIG. 4 is a diagram showing a first example of processing by the analysis unit 22. As shown in FIG. FIG. 4 shows an example in which the input sentence is "I went to Kyoto yesterday." First, the analysis unit 22 extracts all named entities included in the input sentence. A known technique (eg, Bi-LSTM-CRF, etc.) can be used to extract the named entity.
  • a known technique eg, Bi-LSTM-CRF, etc.
  • a named entity has multiple predefined classes defined.
  • classes such as person's name, place name, organization name, time, date (including time), amount of money, terrain name, and facility name are defined.
  • Each class is associated in advance with the interrogative classification described above. For example, "person's name” is associated with the interrogative "who”. "Place name”, “geographic name”, and “facility name” are associated with the interrogative "where”. "Time” and "date” are associated with the interrogative "when". Such association can be arbitrarily set in advance by an operator or the like.
  • the named entity extracted from the input sentence by the analysis unit 22 belongs to one of the above classes. That is, the analysis unit 22 extracts the named entity included in the input sentence and acquires information about the class to which the named entity belongs.
  • the analysis unit 22 determines from the input sentence that the named entity "yesterday” belonging to the class "date” corresponding to the interrogative "when” and the class "place name” corresponding to the interrogative "where” Extract the named entity "Kyoto”. Therefore, in the example of FIG. 4, the analysis unit 22 extracts the interrogative words "when” and "where" as the first classification. In other words, the analysis unit 22 analyzes that the input sentence includes the content regarding "when” and "where".
  • the analysis unit 22 may extract the first classification by using a learned model (classification model).
  • FIG. 5 is a diagram showing a second example of processing by the analysis unit 22.
  • classification models include one classification model M1 that performs multi-class classification and a classification model M2 that performs binary classification prepared for each interrogative.
  • the classification model M1 that performs multi-class classification receives an input sentence and each classification (here, 6 classifications of 5W1H and a total of 7 classifications of "Others"). ) and the degree of fitness (probability that the input sentence contains the content related to each classification).
  • each classification here, 6 classifications of 5W1H and a total of 7 classifications of "Others"
  • the degree of fitness probability that the input sentence contains the content related to each classification.
  • the input sentence contains the named entity 'Nagano' corresponding to the class 'place name' corresponding to the interrogative 'where', and the named entity corresponding to the class 'date' corresponding to the interrogative 'when'.
  • the analysis unit 22 classifies (in the example of FIG. 5, the interrogative words "where", "when", and " who") may be extracted as the first classification.
  • a classification model M2 for binary classification is prepared for each interrogative classification.
  • the classification model M2 corresponding to the interrogative word "when" inputs an input sentence and calculates the probability that the input sentence includes "when” (that is, the input sentence includes the content corresponding to "when”). is configured to output the probability that
  • the classification model M2 corresponding to the interrogative word "how” inputs an input sentence and calculates the probability that the input sentence contains "how” (that is, the input sentence contains the content corresponding to "how”). is configured to output the probability that The same applies to the classification models M2 corresponding to other interrogative words.
  • the analysis unit 22 may refer to the probability values output by the classification model M2 corresponding to each interrogative, and extract the classification corresponding to the probability value equal to or greater than a predetermined threshold as the first classification.
  • the classification models M1 and M2 as described above can be configured by, for example, a one-layer or multi-layer neural network.
  • the classification models M1 and M2 for example, perform machine learning (for example, deep learning, etc.) using teacher data in which a certain sentence and interrogative information (i.e., correct label) included in the sentence are set. generated.
  • the question sentence generation unit 23 excludes one or more first categories extracted by the analysis unit 22 from among a plurality of predetermined categories (six categories of 5W1H in this embodiment). Obtain one or more secondary classifications. Then, based on the input sentence and the one or more second classifications, the question sentence generation unit 23 generates one or more question sentences corresponding to the input sentence and each second classification.
  • the analysis unit 22 extracts the classifications corresponding to the interrogatives "where", “when” and “who” as the first classification.
  • the question sentence generation unit 23 acquires three categories corresponding to the remaining interrogative words “what", “why", and “how” as the second categories.
  • the analysis unit 22 applies the Generate a corresponding question sentence.
  • the question generation unit 23 generates a question using the question generation model M generated by the model generation device 10 (the question generation model M stored in the model storage unit 14). More specifically, the question sentence generation unit 23 acquires a question sentence output from the question generation model M by inputting the pair of the input sentence and the second classification into the question generation model M.
  • the question generation unit 23 generates text data in the same format as the part corresponding to the input data in the teacher data (for example, " ⁇ what> I went to ski last winter to Nagano with Hanako.") to the question generation model M. to enter.
  • the question generation model M generates and outputs a question sentence corresponding to the designated interrogative ("what" in this example) and the input sentence.
  • the question sentence generation unit 23 generates "What did you do last winter?" as a question sentence corresponding to the pair "input sentence + what". Then, as a question sentence corresponding to the pair of "input sentence + why", "Why did you go skiing last winter?" is generated, and "input sentence + how" is generated. "How did you get there?" is generated as a question corresponding to the pair of .
  • the question text selection unit 24 selects a question text to be presented to the user from among the plurality of question texts based on the input text and the plurality of question texts. to select. For example, when presenting only one question sentence to the user, the question sentence selection unit 24 selects one question sentence to be presented to the user from among the plurality of question sentences generated by the question sentence generation unit 23. Decide on a sentence. When a predetermined number of questions (N (N is an integer equal to or greater than 2)) can be presented to the user, the question text selection unit 24 selects the questions generated by the question text generation unit 23. If the number of sentences is greater than N, N question sentences may be selected. The details of the processing of the question text selection unit 24 will be described later.
  • the question text correction unit 25 converts the question text generated by the question text generation unit 23 (for example, one question text selected by the question text selection unit 24) into a more appropriate question that takes into account the content of the input text. Correct the sentence. It should be noted that if there is no part to be corrected, the question text correction unit 25 does not need to correct the question text. Further, the timing at which the question sentence correction unit 25 corrects the question sentence may be before the processing of the question sentence selection unit 24 . In this case, the question text correction unit 25 may perform correction processing (including processing for determining whether correction is necessary) for each question text generated by the question text generation unit 23 .
  • the question sentence correction unit 25 compares the question sentence generated by the question sentence generation unit 23 and the input sentence, and determines that the first word contained in the question sentence and the input sentence having the same named entity classification are When the included second word is extracted, the first word in the question sentence is replaced with the second word. The details of the processing of the question sentence correction unit 25 will be described later.
  • the presentation unit 26 presents the question text generated by the question text generation unit 23 (the question text selected by the question text selection unit 24 when multiple question texts are generated) to the user.
  • the presentation unit 26 presents the question to the user by transmitting the question to the user terminal 30 and displaying the question on a display unit such as a display of the user terminal 30 .
  • the reception unit 21 acquires the user's input sentence from the user terminal 30 (step S11).
  • the analysis unit 22 analyzes the user's input sentence acquired by the reception unit 21 to extract the first classification that is specified or presumed to be included in the input sentence (step S12). For example, as shown in FIG. 4, the analysis unit 22 selects one of a plurality of predetermined classifications (in this embodiment, a classification corresponding to each interrogative word of 5W1H) by the technique of named entity extraction described above.
  • the classification (interrogative term) corresponding to the named entity class may be specified as the first classification.
  • the analysis unit 22 uses the classification models M1 and M2 shown in FIG. 5, for example, to estimate a classification (interrogative) for which a probability value equal to or greater than a predetermined threshold value is output as the first classification.
  • a classification interrogative
  • the question text generation unit 23 removes the one or more first classifications extracted by the analysis unit 22 from among the plurality of classifications (six classifications of 5W1H), and removes the remaining one or more second classifications. is obtained (step S13).
  • the question text generation unit 23 removes the one or more first classifications extracted by the analysis unit 22 from among the plurality of classifications (six classifications of 5W1H), and removes the remaining one or more second classifications. is obtained (step S13).
  • four categories corresponding to the interrogative words “who”, “what”, “why”, and “how” are acquired as the second category through this process.
  • the question text generation unit 23 generates a question text for each set of the input text and each second classification (step S14).
  • the question text generation unit 23 uses the question generation model M to generate a question text.
  • the question sentence generation unit 23 inputs pairs of the input sentence and the second classification to the question generation model M, and outputs the output result from the question generation model M to each pair. Generate as a corresponding question sentence.
  • step S15: YES when a plurality of question texts are generated by the question text generating unit 23 (that is, when there are a plurality of second classifications) (step S15: YES), the question text selection unit 24 selects the plurality of question texts. A question sentence to be presented to the user is selected from among them (step S16). When the number of question sentences generated by the question sentence generation unit 23 is one (step S15: NO), the process of the question sentence selection unit 24 is omitted.
  • the question sentence is corrected to a more appropriate question sentence that takes into account the content of the input sentence (step S18).
  • the predetermined correction condition is, for example, that the above-described first word and second word are extracted from the question sentence and the input sentence. That is, when the above-described first word and second word are extracted from the question sentence and the input sentence, the first word included in the question sentence is replaced with the second word included in the input sentence in step S18.
  • the presentation unit 26 presents the question text generated by the question text generation unit 23 (if multiple question texts are generated, the question text selected by the question text selection unit 24) to the user (step S19).
  • FIG. 8 is an example of a screen displaying the processing contents and processing results of the question generation device 20 as a CUI.
  • the input sentence "I went to ski last winter to Nagano with Hanako.” is acquired by the reception unit 21 (step S11).
  • the analysis unit 22 generates a named entity “winter” belonging to the class “date” corresponding to the interrogative “when”, a named entity “Nagano” belonging to the class “place name” corresponding to the interrogative “where”, A named entity “Hanako” belonging to the class "person's name” corresponding to the interrogative "who” is extracted (step S12). That is, “when”, “where", and “who” are extracted as the first classification.
  • the remaining interrogative words “what", "how”, and "why” are extracted as the second category by the analysis unit 22 (step S13).
  • the question text generation unit 23 generates a question text corresponding to each second classification (step S14).
  • “What did you do last winter?” corresponds to the interrogative "what", “How did you get there?" corresponds to the interrogative "how”, and "Why Three question sentences, "did you go skiing last winter?" are generated.
  • the question text selection unit 24 selects one question text to be presented to the user from among the three question texts (step S16). For example, the question sentence selection unit 24 calculates the degree of similarity between each question sentence generated by the question sentence generation unit 23 and the input sentence, and preferentially selects the question sentence with the lower similarity to the input sentence. It may be selected as a question sentence to be presented to
  • the question sentence selection unit 24 creates input sentences and 2-grams of each question as shown below.
  • 2-gram indicates the 2-gram of the sentence.
  • the question text selection unit 24 creates an intersection of the 2-grams of the input text and the 2-grams of each question text.
  • the intersection of the 2-gram of the input sentence and the 2-gram of the what question sentence is ⁇ 'last winter' ⁇ and the intersection of the 2-gram of the input sentence and the 2-gram of the how question sentence is The intersection is ⁇ 'last winter' ⁇ , and the intersection of the 2-gram of the input sentence and the 2-gram of the why question sentence is the empty set ⁇ .
  • the question text selection unit 24 may set a higher priority to a question text having a smaller number of elements in the product set (that is, a question text having a lower degree of similarity).
  • the priority of the why question (the number of elements in the intersection set is "0") is set higher than the priority of the what and how questions (the number of elements in the intersection set is "1").
  • the question sentence selection unit 24 selects the question sentence "Why did you go skiing last winter?" as the question sentence to be presented to the user.
  • step S17 NO
  • step S18 the presentation unit 26 presents the question "why" selected by the question selection unit 24 to the user (step S19).
  • FIG. 9 is an example of a screen displaying the processing contents and processing results of the question generation device 20 as a CUI.
  • the input sentence "I like to go to Kyoto.” is acquired by the receiving unit 21 (step S11).
  • the analysis unit 22 extracts the named entity "Kyoto" belonging to the class "place name” corresponding to the interrogative "where” (step S12). That is, "where" is extracted as the first classification.
  • four categories of "where", “when”, “how”, and “why” are predetermined as a plurality of categories (interrogative candidates). Therefore, the remaining interrogative words "when", "how", and "why” are extracted as the second category by the analysis unit 22 (step S13).
  • the question text generation unit 23 generates a question text corresponding to each second classification (step S14).
  • “When do you like to go to Kyoto?” corresponds to the interrogative "when", "How would you like to travel, by train or by bus?”
  • Three question sentences of "Why are you interested in Japan?” corresponding to the lyric "why" are generated.
  • the question text selection unit 24 selects one question text to be presented to the user from among the three question texts (step S16).
  • the question text selection unit 24 may select a question text based on 2-grams, as in the first embodiment described above.
  • the intersection of the 2-gram of the input sentence and the 2-gram of the when question sentence is ⁇ 'like to', 'to go', 'to Kyoto', 'go to' ⁇ The intersection of the input sentence 2-gram and the how question sentence 2-gram is ⁇ 'like to' ⁇ , and the intersection of the input sentence 2-gram and the why question sentence 2-gram is empty. It is a set ⁇ .
  • the priority of the why question (the number of elements in the intersection is 0) is the when question (the number of elements in the intersection is 4) and the how question (the number of elements in the intersection is 4).
  • the number of elements is set higher than the priority of "1").
  • the input sentence includes the named entity "Kyoto" belonging to the class "place name”.
  • the question sentence of why contains the named entity "Japan” belonging to the class "place name”. That is, in this example, the first word "Japan” included in the question sentence and the second word "Kyoto” included in the input sentence, which have the same named entity classification, are extracted. Therefore, since the correction condition described above is satisfied (step S17: YES), the question sentence correction process (step S18) is executed. That is, the question sentence correction unit 25 replaces the first word "Japan” included in the question sentence with the second word "Kyoto". That is, the question sentence correction unit 25 corrects the question sentence "Why are you interested in Japan?" to "Why are you interested in Kyoto?". Subsequently, the corrected question text is presented to the user by the presentation unit 26 (step S19).
  • the model generation device 10 described above includes the pair information acquisition unit 11, the teacher data generation unit 12, and the model generation unit 13 described above. According to the model generation device 10 as described above, a question generation model M that inputs an arbitrary first sentence and generates a question sentence (second sentence) for the first sentence is obtained. In order to generate the question generation model M as described above, it is natural to use a pair of an arbitrary sentence corresponding to the first sentence and a question sentence for the arbitrary sentence as training data. More specifically, since the first sentence input to the question generation model M exists temporally before the second sentence output from the question generation model M, the question generation model M is generated.
  • an arbitrary sentence corresponding to the first sentence is associated with the input data, and a question sentence occurring temporally after the arbitrary sentence (that is, a question sentence for the arbitrary sentence) is associated with the output data.
  • a question sentence occurring temporally after the arbitrary sentence that is, a question sentence for the arbitrary sentence
  • the teacher data in which the temporal context is reversed that is, the question sentence that occurred earlier in time corresponds to the output data, and the answer sentence that occurred later in time corresponds to the output data. teacher data corresponding to the input data) is used. As shown in the example of FIG.
  • the inputted first A question generation model M is obtained that outputs the second sentence, which is the question sentence for the first sentence. That is, a question generation model M capable of generating a second sentence, which is a question sentence related to the content of the first sentence, is obtained. Furthermore, as described above, pair information including a question and an answer to the question can be obtained relatively easily, for example, from example sentences written in commercially available English problem collections. Therefore, according to the model generation device 10, it is possible to efficiently generate a model for generating a question sentence for an input sentence.
  • the teacher data generation unit 12 associates answer sentences with input data, and further associates classification information indicating the classification of question sentences (for example, classification of interrogative words such as 5W1H) with input data. Generate teacher data. Then, the model generation unit 13 generates a question generation model M that inputs an input sentence (first sentence) and arbitrary classification information and outputs a question sentence (second sentence) according to the classification indicated by the arbitrary classification information. do. According to the above configuration, it is possible to obtain a question generation model M that can generate a question sentence regarding the content indicated by the classification information while taking into account the content of the input sentence (see FIG. 2).
  • the teacher data generation unit 12 extracts interrogative words (interrogative words corresponding to 5W1H in this embodiment) included in the question sentence, and uses the extracted interrogative words as classification information. According to the above configuration, it is possible to easily and appropriately extract the classification information based on the interrogative included in the question sentence of the pair information.
  • the question generation device 20 described above includes an analysis unit 22, a question text generation unit 23, and a question text selection unit 24. According to the question generating device 20 as described above, only question sentences corresponding to the second classification other than the first classification specified or presumed to be included in the input sentence among the plurality of predetermined classifications are generated. That is, the process of generating redundant questions about the content (first category) that is likely to be included in the input sentence is omitted, and meaningful questions (that is, first category questions that are likely not to be included in the Only two-class questions) can be generated. Furthermore, the processing load of the question text selection unit 24 is reduced by avoiding the generation of questions related to the first classification and narrowing down the number of question text candidates in advance. Therefore, it is possible to more efficiently generate a question sentence according to the user input sentence.
  • the analysis unit 22 extracts the classification corresponding to the one interrogative word as the first classification. According to the above configuration, it is possible to easily and efficiently extract the first classification by using a known named entity extraction technique.
  • the question text selection unit 24 calculates the degree of similarity between each question text generated by the question text generation unit 23 and the input text, and gives priority to question texts with lower similarities to the input text. Then, it is selected as a question sentence to be presented to the user. According to the above configuration, it is possible to present to the user a question sentence that is not superficially or semantically similar to the input sentence (that is, a question sentence from which redundancy has been eliminated).
  • the number of elements included in the product set of the 2-gram of the input sentence and the 2-gram of each question is used as the indicator of the similarity, but the question sentence selection unit 24
  • a question sentence to be presented to the user may be selected based on a degree of similarity other than the above.
  • the question sentence selection unit 24 vectorizes the input sentence and each question sentence, calculates the cosine similarity between the vectorized input sentence and each vectorized question sentence, and calculates the cosine similarity between the vectorized input sentence and each vectorized question sentence. may be selected as the question to be presented to the user.
  • the question text selection unit 24 may use n-grams (n is an integer of 3 or more) instead of 2-grams, or use 1-grams (bag-of-word, word matching). may
  • the question text selection unit 24 calculates the probability that the content related to each second classification is included in the input text (relevance rate for each second classification), and selects more question texts corresponding to the second classification with the lower probability. It may be selected preferentially as a question sentence to be presented to the user.
  • the question sentence selection unit 24 may input an input sentence to the above-described classification model (see FIG. 5) and calculate the probability that the input sentence includes each classification. Then, the question text selection unit 24 extracts the second class with the lowest probability from among the plurality of second classes, and selects the question text corresponding to the extracted second class as the question text to be presented to the user. good too.
  • the question sentence regarding the content that is highly likely not mentioned in the input sentence in this embodiment, the content of 5W1H that is not included in the input sentence
  • the question sentence regarding the content that is highly likely not mentioned in the input sentence is appropriately selected as the question to be presented to the user. can do.
  • the question generation device 20 includes a question sentence correction unit 25.
  • a question sentence correction unit 25 According to the above configuration, as in the example shown in FIG. 9, part of the description included in the question ("Japan” in the example of FIG. 9) belongs to the same class described in the input sentence ( In the example of FIG. 9, by replacing with "Kyoto"), a more natural question sentence can be generated according to the input sentence. In the example of FIG. 9, by converting "Japan” included in the question sentence into a more specific expression "Kyoto" included in the input sentence, a more natural question sentence corresponding to the input sentence can be obtained.
  • the correction condition for correction by the question sentence correction unit 25 is the condition described in the above embodiment (that the first word and second word belonging to the same named entity class are extracted from the question sentence and the input sentence). ) is not limited to For example, consider a case where the input sentence includes one named entity belonging to the class "place name" and the question text includes two named entities belonging to the class "place name”. In this case, it may not be possible to specify to which of the two named entities included in the question sentence the named entity belonging to the class “place name” included in the input sentence corresponds. Therefore, the question text correcting unit 25 replaces the first word with the second You can replace it with a word.
  • the question generation model M can also generate question sentences corresponding to closed questions. For example, when the second classification is not extracted in the processing by the analysis unit 22, the question sentence generation unit 23 inputs the input sentence and the classification information indicating null to the question generation model M, A question sentence corresponding to the closed question may be generated.
  • an input sentence to the question generation model M and a question sentence output from the question generation model M may be sentences written in a language other than English.
  • a question generation model M corresponding to languages other than English can be obtained.
  • each functional block may be implemented using one device that is physically or logically coupled, or directly or indirectly using two or more devices that are physically or logically separated (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices.
  • a functional block may be implemented by combining software in the one device or the plurality of devices.
  • Functions include judging, determining, determining, calculating, calculating, processing, deriving, investigating, searching, checking, receiving, transmitting, outputting, accessing, resolving, selecting, choosing, establishing, comparing, assuming, expecting, assuming, Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc. can't
  • the model generation device 10 and the question generation device 20 in one embodiment of the present disclosure may function as computers that perform the model generation method and the question generation method of the present disclosure.
  • FIG. 10 is a diagram showing an example of a hardware configuration common to the model generation device 10 and the question generation device 20 according to an embodiment of the present disclosure.
  • Each of the model generation device 10 and the question generation device 20 is physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like. good too.
  • the term "apparatus” can be read as a circuit, device, unit, or the like.
  • the hardware configuration of the model generation device 10 and the question generation device 20 may be configured to include one or more of each device shown in FIG. 10, or may be configured without some devices. .
  • Each function of the model generation device 10 and the question generation device 20 is performed by the processor 1001 performing calculations and communication by the communication device 1004 by loading predetermined software (programs) onto hardware such as the processor 1001 and the memory 1002. and at least one of reading and writing data in the memory 1002 and the storage 1003 .
  • the processor 1001 for example, operates an operating system and controls the entire computer.
  • the processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like.
  • CPU central processing unit
  • the processor 1001 reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 to the memory 1002, and executes various processes according to them.
  • programs program codes
  • the program a program that causes a computer to execute at least part of the operations described in the above embodiments is used.
  • the model generation unit 13 of the model generation device 10 the question generation unit 23 of the question generation device 20, and the like may be stored in the memory 1002 and realized by a control program operating in the processor 1001.
  • Other functional blocks may be realized as well.
  • FIG. Processor 1001 may be implemented by one or more chips.
  • the program may be transmitted from a network via an electric communication line.
  • the memory 1002 is a computer-readable recording medium, and is composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. may be
  • ROM Read Only Memory
  • EPROM Erasable Programmable ROM
  • EEPROM Electrical Erasable Programmable ROM
  • RAM Random Access Memory
  • the memory 1002 may also be called a register, cache, main memory (main storage device), or the like.
  • the memory 1002 can store executable programs (program codes), software modules, etc. for implementing a communication control method according to an embodiment of the present disclosure.
  • the storage 1003 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disc, a magneto-optical disc (for example, a compact disc, a digital versatile disc, a Blu-ray disk), smart card, flash memory (eg, card, stick, key drive), floppy disk, magnetic strip, and/or the like.
  • Storage 1003 may also be called an auxiliary storage device.
  • the storage medium described above may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003 .
  • the communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also called a network device, a network controller, a network card, a communication module, or the like.
  • the input device 1005 is an input device (for example, keyboard, mouse, microphone, switch, button, sensor, etc.) that receives input from the outside.
  • the output device 1006 is an output device (eg, display, speaker, LED lamp, etc.) that outputs to the outside. Note that the input device 1005 and the output device 1006 may be integrated (for example, a touch panel).
  • Each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information.
  • the bus 1007 may be configured using a single bus, or may be configured using different buses between devices.
  • model generation device 10 and the question generation device 20 include a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), etc. hardware, and part or all of each functional block may be realized by the hardware.
  • processor 1001 may be implemented using at least one of these pieces of hardware.
  • Input/output information may be stored in a specific location (for example, memory) or managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.
  • the determination may be made by a value represented by one bit (0 or 1), by a true/false value (Boolean: true or false), or by numerical comparison (for example, a predetermined value).
  • notification of predetermined information is not limited to being performed explicitly, but may be performed implicitly (for example, not notifying the predetermined information). good too.
  • Software whether referred to as software, firmware, middleware, microcode, hardware description language or otherwise, includes instructions, instruction sets, code, code segments, program code, programs, subprograms, and software modules. , applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.
  • software, instructions, information, etc. may be transmitted and received via a transmission medium.
  • the software uses at least one of wired technology (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.) and wireless technology (infrared, microwave, etc.) to website, Wired and/or wireless technologies are included within the definition of transmission medium when sent from a server or other remote source.
  • wired technology coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.
  • wireless technology infrared, microwave, etc.
  • data, instructions, commands, information, signals, bits, symbols, chips, etc. may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of
  • information, parameters, etc. described in the present disclosure may be expressed using absolute values, may be expressed using relative values from a predetermined value, or may be expressed using other corresponding information. may be represented.
  • any reference to elements using the "first,” “second,” etc. designations used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, reference to a first and second element does not imply that only two elements can be employed or that the first element must precede the second element in any way.
  • a and B are different may mean “A and B are different from each other.”
  • the term may also mean that "A and B are different from C”.
  • Terms such as “separate,” “coupled,” etc. may also be interpreted in the same manner as “different.”

Abstract

L'invention concerne un dispositif de génération de modèle (10) qui comprend : une unité d'acquisition d'informations de paire (11) qui acquiert des informations de paire comprenant une phrase de question et une phrase de réponse à la phrase de question ; une unité de génération de données d'apprentissage (12) qui génère des données d'apprentissage dans lesquelles la phrase de réponse incluse dans les informations de paire est associée à des données d'entrée et la phrase de question incluse dans les informations de paire est associée à des données de sortie ; et une unité de génération de modèle (13) qui exécute un apprentissage machine à l'aide des données d'apprentissage pour générer un modèle de génération de questions (M) qui reçoit une première phrase arbitraire et qui délivre en sortie une seconde phrase qui est une phrase de question pour la première phrase.
PCT/JP2021/046088 2021-02-24 2021-12-14 Dispositif de génération de modèle et procédé de génération de modèle WO2022180989A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2023502088A JPWO2022180989A1 (fr) 2021-02-24 2021-12-14

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021027206 2021-02-24
JP2021-027206 2021-02-24

Publications (1)

Publication Number Publication Date
WO2022180989A1 true WO2022180989A1 (fr) 2022-09-01

Family

ID=83048014

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/046088 WO2022180989A1 (fr) 2021-02-24 2021-12-14 Dispositif de génération de modèle et procédé de génération de modèle

Country Status (2)

Country Link
JP (1) JPWO2022180989A1 (fr)
WO (1) WO2022180989A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117574919A (zh) * 2023-08-24 2024-02-20 华东师范大学 基于大型语言模型指令微调的流调问答模板生成方法
CN117574919B (zh) * 2023-08-24 2024-05-17 华东师范大学 基于大型语言模型指令微调的流调问答模板生成方法

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020135456A (ja) * 2019-02-20 2020-08-31 日本電信電話株式会社 生成装置、学習装置、生成方法及びプログラム

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020135456A (ja) * 2019-02-20 2020-08-31 日本電信電話株式会社 生成装置、学習装置、生成方法及びプログラム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YUICHI SASAZAWA, SHO TAKASE, NAOMI OKAZAKI: "Omission completion of interactive question answering", PROCEEDINGS OF TWENTY-FIFTH ANNUAL MEETING OF THE ASSOCIATION FOR NATURAL LANGUAGE PROCESSING, 4 March 2019 (2019-03-04), JP, pages 163 - 166, XP009539403 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117574919A (zh) * 2023-08-24 2024-02-20 华东师范大学 基于大型语言模型指令微调的流调问答模板生成方法
CN117574919B (zh) * 2023-08-24 2024-05-17 华东师范大学 基于大型语言模型指令微调的流调问答模板生成方法

Also Published As

Publication number Publication date
JPWO2022180989A1 (fr) 2022-09-01

Similar Documents

Publication Publication Date Title
CN107908635B (zh) 建立文本分类模型以及文本分类的方法、装置
US11748232B2 (en) System for discovering semantic relationships in computer programs
US10796105B2 (en) Device and method for converting dialect into standard language
CN107644011B (zh) 用于细粒度医疗实体提取的系统和方法
US20180011830A1 (en) Annotation Assisting Apparatus and Computer Program Therefor
CN111680159A (zh) 数据处理方法、装置及电子设备
CN109416705A (zh) 利用语料库中可用的信息用于数据解析和预测
CN110457676B (zh) 评价信息的提取方法及装置、存储介质、计算机设备
US10977155B1 (en) System for providing autonomous discovery of field or navigation constraints
CN110580308B (zh) 信息审核方法及装置、电子设备、存储介质
US20220147835A1 (en) Knowledge graph construction system and knowledge graph construction method
US20230075614A1 (en) Automatically identifying multi-word expressions
CN112749547A (zh) 文本分类器训练数据的产生
US20230161763A1 (en) Systems and methods for advanced query generation
CN111930792A (zh) 数据资源的标注方法、装置、存储介质及电子设备
CN111597800A (zh) 同义句的获取方法及装置、设备及存储介质
WO2022180990A1 (fr) Dispositif de génération de question
US10650195B2 (en) Translated-clause generating method, translated-clause generating apparatus, and recording medium
Chakrawarti et al. Machine translation model for effective translation of Hindi poetries into English
CN113486178A (zh) 文本识别模型训练方法、文本识别方法、装置以及介质
WO2022180989A1 (fr) Dispositif de génération de modèle et procédé de génération de modèle
JP2017021523A (ja) 用語意味コード判定装置、方法、及びプログラム
CN111552780B (zh) 医用场景的搜索处理方法、装置、存储介质及电子设备
Dobreva et al. Improving NER performance by applying text summarization on pharmaceutical articles
Mekki et al. Tokenization of Tunisian Arabic: a comparison between three Machine Learning models

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21928082

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023502088

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21928082

Country of ref document: EP

Kind code of ref document: A1