JP2019082860A - Generation program, generation method and generation device - Google Patents

Abstract

To provide a generation program, a generation method and a generation device for holding characteristics included in a conversion parameter.SOLUTION: The generation program is configured so as to, when receiving words, cause a computer to execute a series of processing to generate a first vector and a second vector corresponding to the word by applying the first conversion parameter and the second conversion parameter to the respective received words. The generation program is also configured to cause the computer to execute a series of processing to generate a new third vector corresponding to the words on the basis of the generated first vector and the second vector.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a generation program, a generation method and a generation apparatus.

  In language processing, techniques are known in which multiple corpuses are used in combination. For example, in machine translation, a technique is known which improves the quality of translation results and shortens the response time of translation by using a plurality of corpuses in combination. The technology uses a plurality of translation dictionaries in which words and phrases belonging to the first language and words and phrases belonging to the second language are stored in association with each other. The technique accepts reference information including one or more words and compares a plurality of translation dictionaries with the reference information to determine the degree of similarity of each translation dictionary with respect to the reference information. The technology specifies the priority in searching each translation dictionary based on the similarity. Moreover, in the said technique, not a dictionary but the corpus which is a collection of sentences may be designated.

JP 2004-355217 A

  In the above-described technique, there arises a problem that any corpus feature is lost. For example, when a sentence is decomposed by morphological analysis using a database storing a plurality of sentences such as FAQ, and a word vector is generated by machine learning, a conversion parameter for converting a word into a vector generated from a corpus is used. Use. In this case, for example, when a transformation parameter generated from one corpus is selected, features of the corpus which are not selected are not reflected in the search results. On the other hand, when generating transformation parameters using a corpus integrating two corpuses, the features included in each corpus before consolidation may be lost.

  In one aspect, it is an object of the present invention to provide a generation program, a generation method, and a generation device capable of holding the feature included in the conversion parameter.

  In one aspect, the generation program receives a word and applies a first conversion parameter and a second conversion parameter to the word to generate a first vector and a second vector according to the word. Make it run on a computer. The generation program causes the computer to execute processing of generating a new third vector according to the word based on the generated first vector and second vector.

  According to one aspect, features included in the transformation parameters can be retained.

FIG. 1 is a diagram illustrating an example of the answer extraction process according to the first embodiment. FIG. 2 is a diagram illustrating an example of the generation device in the first embodiment. FIG. 3 is a diagram illustrating an example of the corpus model in the first embodiment. FIG. 4 is a diagram illustrating an example of a search time learning model in the first embodiment. FIG. 5 is a diagram illustrating an example of the answer storage unit in the first embodiment. FIG. 6 is a diagram illustrating an example of vector combination processing in the first embodiment. FIG. 7 is a flowchart illustrating an example of the answer extraction process according to the first embodiment. FIG. 8 is a diagram showing an example of vector generation processing in the background art. FIG. 9 is a diagram illustrating an example of vector generation processing in the first embodiment. FIG. 10 is a diagram illustrating an example of vector generation processing in the second embodiment. FIG. 11 is a diagram illustrating an example of a generation device in the second embodiment. FIG. 12 is a flowchart illustrating an example of the answer extraction process according to the second embodiment. FIG. 13 is a diagram illustrating another example of the vector generation process according to the second embodiment. FIG. 14 is a diagram illustrating an example of vector combination processing in the third embodiment. FIG. 15 is a diagram illustrating an example of vector generation processing in the third embodiment. FIG. 16 is a diagram illustrating another example of the vector generation process according to the third embodiment. FIG. 17 is a diagram illustrating an example of a hardware configuration.

  Hereinafter, an embodiment of a generation program, a generation method, and a generation device disclosed in the present application will be described in detail based on the drawings. The present invention is not limited by this embodiment. In addition, the embodiments described below may be combined appropriately as long as no contradiction occurs.

  The generation device 10 described later in the first embodiment causes the computer to execute an answer extraction process using a word vector generated using a corpus model. The answer extraction process in the first embodiment is, for example, a process of referring to a question and answer session (FAQ) of a call center and extracting an appropriate response to the input question sentence. The generation device 10 is an example of a computer device such as a server, a personal computer, or a tablet. The corpus model is an example of transformation parameters.

  The response extracting process in the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the answer extraction process according to the first embodiment. As shown in FIG. 1, in the answer extraction process in the first embodiment, a corpus is learned in the prior learning process, whereby a corpus model A 121 a and a corpus model B 121 b to be described later are generated. In the following, when expressing the corpus model A 121 a and the corpus model B 121 b without distinction, the corpus model 121 may be used.

  The corpus in the first embodiment is a group of a plurality of sentences, and is, for example, a text group acquired from an external server such as an encyclopedia site, a newspaper database, or a search engine. Further, the corpus in the first embodiment may be internal information of a company using the generating device 10, such as a customer database or a question and answer log of a call center, for example. In the first embodiment, for example, Corpus A is an encyclopedia site and Corpus B is internal information.

  The corpus model 121 in the first embodiment is generated, for example, by dividing a sentence group included in a corpus into words by morphological analysis and learning a word vector (Bag of words) of a distributed expression for the divided word group. . The distributed representation of words is a multidimensional vector in which each word is expressed by quantified continuous values for a plurality of dimensions which are feature elements between the words. In addition, since the distributed expression of a word can be learned by well-known techniques, such as Word2Vec, for example, detailed explanation is omitted.

  In addition, the corpus model 121 inputs, for example, training data in which a word vector and a correct answer label are associated into a neural network or the like, learns the relationship between the input vector and the correct answer label, and generates a learned model. can get.

  Next, in the inference process, the generation device 10 first receives an input of text such as a question sentence. As shown in FIG. 1, the generation device 10 receives an input text 1000 such as “the power of the PC is not turned on”. Next, the generation device 10 performs morphological analysis on the input text, and divides the input text 1000 into a word group 1001 such as “PC, power supply, enter, not,”.

  Then, the generation device 10 inputs each divided word into the corpus model 121 generated in advance, and acquires a word vector. The generation device 10 acquires the word vector A by, for example, inputting the input word “PC” into the corpus model A 121 a. Similarly, the generation device 10 acquires the word vector B by inputting the input word “PC” into the corpus model B 121 b. As shown in FIG. 1, since the corpus model A 121 a and the corpus model B 121 b are obtained from different corpuses, the acquired word vectors A and B are also different from each other. The word vector is an example of a first vector and a second vector.

  Next, the generation device 10 combines the acquired word vectors A and B. The generation device 10 generates a word vector C by, for example, concatenating the word vector B with the word vector A. Similarly, the generation device 10 obtains a distributed representation corresponding to the input text “the power of the PC can not be turned on” by repeating the inference process and the combining process also for the other divided input words. The word vector C is an example of a third vector.

  On the other hand, the generation apparatus 10 also has a learning model 122 at the time of search, which will be described later, which is obtained by the same process as the prior learning process, also for the answer data to be searched using the input text. As shown in FIG. 1, the search time learning model 122 is obtained by classifying the title stored in the answer storage unit 123 described later by morphological analysis and learning a word vector for the divided word group .

  Then, in the search process, the generation apparatus 10 acquires the answer corresponding to the input text by inputting the distributed expression corresponding to the input text to the learning model at search time 122. For example, the generating device 10 refers to the response storage unit 123, and acquires the response "Corrective action is ~" corresponding to the title "The PC does not start even if the power button is pressed." Then, the generation device 10 outputs the acquired answer.

  Thus, the generation device 10 generates, for the input word, the third vector generated using the first vector generated from the first model and the second vector generated from the second model. Machine learning can be performed using vectors that retain the features of each model.

[Function block]
Next, an example of the generation device 10 in the present embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of the generation device in the first embodiment. As shown in FIG. 2, the generation device 10 in the present embodiment includes a storage unit 120 and a control unit 130.

  The storage unit 120 is an example of a storage device that stores programs and data, and is, for example, a memory or a processor. The storage unit 120 stores a corpus model A 121 a, a corpus model B 121 b, a search time learning model 122, and an answer storage unit 123.

  The corpus model A 121 a and the corpus model B 121 b in the first embodiment are dictionaries that are generated from a corpus and in which words are associated with word vectors in a distributed expression. FIG. 3 is a diagram illustrating an example of the corpus model in the first embodiment. As shown in FIG. 3, the corpus model A 121 a according to the first embodiment stores “words” and “vectors” in association with one another. The corpus model B 121 b also has the same configuration as that of the corpus model A 121 a, and thus detailed description will be omitted.

  The search time learning model 122 is a learning model obtained by classifying the title included in the answer data by morphological analysis and learning a word vector for the divided word group. FIG. 4 is a diagram illustrating an example of a search time learning model in the first embodiment. As shown in FIG. 4, the search-time learning model 122 stores “ID” and “vector” in association with each other.

  In FIG. 4, “ID” stores an identifier (Identifier) that uniquely identifies response data. The "vector" stores a word vector of distributed expression obtained from the response data of the ID.

  As illustrated in FIG. 1, the answer storage unit 123 stores, for example, information acquired in advance from another server or the like (not illustrated). FIG. 5 is a diagram illustrating an example of the answer storage unit in the first embodiment. As shown in FIG. 5, the answer storage unit 123 stores the “title” and the “answer” in association with the “ID”.

  In FIG. 5, "title" stores the question sentence corresponding to the answer data. "Answer" stores the text of the answer data.

  Returning to FIG. 2, the control unit 130 is a processing unit that controls the entire generation device 10 and is, for example, a processor. The control unit 130 includes a reception unit 131, a word extraction unit 132, a vector extraction unit 133, a vector combination unit 134, an answer search unit 135, and an output unit 136. The receiving unit 131, the word extracting unit 132, the vector extracting unit 133, the vector combining unit 134, the answer searching unit 135, and the output unit 136 are an example of an electronic circuit included in the processor and an example of a process executed by the processor.

  The accepting unit 131 accepts an input text and outputs the text to the word extracting unit 132. The accepting unit 131 accepts an input text from the user, for example, through an operation unit of the generation device 10 (not shown). Also, the receiving unit 131 may receive an input text from a terminal operated by the user, for example, through a communication unit of the generating device 10 (not shown).

  The word extraction unit 132 performs morphological analysis on the input text. When receiving the output of the input text from the reception unit 131, the word extraction unit 132 divides the input text into word groups as shown in FIG. 1 by morphological analysis. The word extraction unit 132 outputs the divided word group to the vector extraction unit 133. Note that the word extraction unit 132 may divide the input text into word groups by another method other than morphological analysis.

  The vector extraction unit 133 extracts a plurality of word vectors corresponding to the divided word group for each corpus model 121, and outputs the word vectors to the vector combination unit 134. The vector extraction unit 133 is an example of a first vector processing unit.

  The vector extraction unit 133 extracts a word vector A by inputting a word into the corpus model A 121 a. Similarly, the vector extraction unit 133 extracts a word vector B by inputting a word into the corpus model B 121 b.

  The vector combination unit 134 combines the vectors output from the vector extraction unit 133. The vector combination unit 134 generates a combined word vector C, for example, by concatenating the word vector B with the word vector A. The vector combination unit 134 is an example of a second vector processing unit.

  FIG. 6 is a diagram illustrating an example of vector combination processing in the first embodiment. As shown in FIG. 6, the vector combining unit 134 generates a word vector C by concatenating each element of the word vector B behind each element of the word vector A. In this case, for example, when word vectors A and B are 100-dimensional vectors, word vector C is a 200-dimensional vector.

[Flow of processing]
Next, processing in the present embodiment will be described using FIG. FIG. 7 is a flowchart illustrating an example of the answer extraction process according to the first embodiment. As illustrated in FIG. 7, the reception unit 131 of the generation device 10 waits until an input text is received from a user (not illustrated) (S100: No). When it is determined that the input text is received (S100: Yes), the reception unit 131 outputs the input text to the word extraction unit 132.

  The word extraction unit 132 classifies the outputted input text into words by morphological analysis. Then, the word extraction unit 132 extracts the divided words and outputs the extracted words to the vector extraction unit 133 (S101).

  The vector extraction unit 133 extracts the word vector A by inputting the word into the corpus model A 121a (S110). Similarly, the vector extraction unit 133 extracts a word vector B by inputting a word into the corpus model B 121 b (S 120). Then, the vector extraction unit 133 outputs the extracted word vector A and word vector B to the vector combination unit 134.

  The vector combination unit 134 combines the output word vector A and word vector B to generate a word vector C (S130). Then, the vector combination unit 134 determines whether vectorization has been completed for all the words divided from the input text (S140).

  When it is determined that the vectorization is not completed (S140: No), the vector combining unit 134 returns to S101 and repeats the processing. On the other hand, when it is determined that the vectorization is completed (S140: Yes), the vector combination unit 134 combines the word vectors C generated for each word to vectorize the entire input text (S150), and the answer search unit Output to 135.

  The answer search unit 135 inputs an output vector of the entire input text to the search time learning model 122 to search for an answer similar to the input text (S 160), and outputs a search result to the output unit 136. Then, the output unit 136 outputs the output search result through a communication unit or a display unit (not shown) (S170).

[effect]
As described above, the generation program in this embodiment receives a word, applies the first conversion parameter and the second conversion parameter to the word, respectively, and generates a first vector and a second vector according to the word. Have the computer execute the process of generating The generation program causes the computer to execute processing of generating a new third vector according to the word based on the generated first vector and second vector. Thereby, the features included in the conversion parameters can be held.

  In addition, the generation program uses the learning model in which the learning regarding the features of the words or phrases included in the plurality of sentences to be discriminated is performed, and from the third vector, the received word and the plurality of sentences to be discriminated The computer may be made to execute the process of acquiring the determination result. Thus, in the text search process, the features included in the conversion parameter can be reflected on the search result.

  The effect of the vector generation process in the present embodiment will be described using FIGS. 8 and 9. FIG. 8 is a diagram showing an example of vector generation processing in the background art. In the background art, as shown in FIG. 8, a corpus model is generated from an integrated corpus obtained by combining two corpuses A and B.

  At this time, for example, when there is a bias in the size of two corpuses, features included in any corpus may be lost without being reflected in the corpus model. In this case, the features of each corpus may not be reflected in the word vector obtained by inputting the input word into the corpus model. As a result, even in the answer obtained by inputting the word vector into the learning model, the features of the corpus may be lost.

  On the other hand, in the first embodiment, as shown in FIG. 9, the corpus itself is not combined, but the word vectors obtained from the corpus models A and B generated from the corpuses A and B are combined. FIG. 9 is a diagram illustrating an example of vector generation processing in the first embodiment. As shown in FIG. 9, in the search processing in the first embodiment, a word vector C as shown in FIG. 6 in which the features of the word vector A and the word vector B remain is used. Since the word vector A and the word vector B leave the features of the corpus A and the corpus B, respectively, it is possible to reflect the features of each corpus even in the answer obtained by inputting the word vector C into the learning model. it can.

  Also, the generation program may cause the computer to execute, for example, processing of concatenating the generated first vector and the second vector to generate a third vector. Thus, the elements included in each vector can be left as they are.

  The generation program receives text data including a plurality of words, generates a first vector and a second vector according to the text data, and generates a new third vector according to the text data. It may be executed by a computer. That is, the series of word features may be plural or singular. In this way, it is possible to search using sentences of natural language such as question sentences instead of words. Note that instead of sequentially inputting each word vector generated from each word included in the text data to the neural network, a method may be adopted in which one input data obtained by combining each word vector is input to the neural network. .

  When generating a word vector, the input word may not be included in any corpus, that is, the input word may not be registered in the corpus model generated from the corpus. Therefore, in the second embodiment, a configuration for generating a combined word vector that indicates that there is an input word not registered in any corpus will be described. In addition, the input word which is not registered into any corpus may be described as an "unknown word" below.

  The generation program in the second embodiment combines a word vector indicating that the word is an unknown word with a word vector corresponding to a corpus including the other unknown word. FIG. 10 is a diagram illustrating an example of vector generation processing in the second embodiment. As shown in FIG. 10, in the corpus model B 121 b, the word “notebook PC” included in the corpus model A 121 a is not registered.

  In this case, when the input word is “notebook PC”, the generation program causes the computer to execute a process of extracting the word vector A with reference to the corpus model A 121 a. On the other hand, the generation program can not execute the process of extracting the word vector B because “notebook PC” is not registered in the corpus model B 121 b.

  In this case, the generation program generates a word vector B2 indicating that the input word is an unknown word, and causes the computer to execute a process of combining with the word vector A.

  For example, the generation program causes the computer to execute a process of generating a vector C2 in which a vector in which all elements are 0 is linked to a word vector A. Also, the generation program may cause the computer to execute a process of duplicating the word vector A and generating a vector C3 linked to the word vector A. Furthermore, the generation program may cause the computer to execute a process of generating a vector C4 in which an unknown word vector stored in advance in an unknown word vector storage unit 224 described later is linked to the word vector A.

[Function block]
Next, a generation apparatus that executes the generation program will be described with reference to FIG. FIG. 11 is a diagram illustrating an example of a generation device in the second embodiment. In the following embodiments, the same parts as the parts shown in the above-described drawings are denoted by the same reference numerals, and redundant description will be omitted.

  As illustrated in FIG. 11, the generation device 20 in the present embodiment includes a storage unit 220 and a control unit 230. The storage unit 220 is an example of a storage device that stores programs and data, and is, for example, a memory or a processor. The storage unit 120 further stores an unknown word vector storage unit 224 in addition to the corpus model A 121 a and the corpus model B 121 b, the search time learning model 122 and the answer storage unit 123.

  The unknown word vector storage unit 224 stores an unknown word vector indicating that the input word is an unknown word. In the second embodiment, the unknown word vector is, for example, a random number vector, but may be a vector generated by learning the corpus model A and the corpus model B as described later. The information stored in the unknown word vector storage unit 224 is input in advance by, for example, a manager of the generation device 20 (not shown), or is input by an unknown word vector processing unit 237 described later.

  The control unit 230 is a processing unit that controls the entire generation device 20, and is, for example, a processor. The control unit 230 further includes a vector extraction unit 233 and an unknown word vector processing unit 237 in addition to the reception unit 131, the word extraction unit 132, the vector combination unit 134, the answer search unit 135, and the output unit 136. The vector extraction unit 233 and the unknown word vector processing unit 237 are also an example of an electronic circuit included in the processor and an example of a process executed by the processor.

  Similar to the vector extraction unit 133, the vector extraction unit 233 extracts a plurality of word vectors corresponding to the word group into which the input words are divided for each corpus model, and outputs the word vectors to the vector combination unit 134. At this time, the vector extraction unit 233 determines whether an input word is registered in each corpus model.

  If the vector extraction unit 233 determines that the input word is not registered in the corpus model, the vector extraction unit 233 outputs an unknown word vector acquisition request to the unknown word vector processing unit 237. Then, the vector extraction unit 233 outputs the acquired unknown word vector to the vector combination unit 134 as a word vector corresponding to the corpus model.

  When the unknown word vector processing unit 237 receives an unknown word vector acquisition request from the vector extraction unit 233, the unknown word vector processing unit 237 outputs the unknown word vector to the vector extraction unit 233. As shown in FIG. 10, for example, the unknown word vector processing unit 237 outputs a vector in which all elements are 0 as an unknown word vector.

  Also, the unknown word vector processing unit 237 receives an acquisition request for an unknown word vector including a word vector already extracted from the vector extraction unit 233, for example, and outputs the extracted word vector included in the acquisition request as an unknown word vector May be Furthermore, the unknown word vector processing unit 237 may obtain the unknown word vector stored in the unknown word vector storage unit 224, for example, and output the unknown word vector to the vector extraction unit 233.

[Flow of processing]
Next, processing in the present embodiment will be described using FIG. FIG. 12 is a flowchart illustrating an example of the answer extraction process according to the second embodiment. In the following description, the same reference numerals as in the steps shown in FIG. 7 denote the same steps, so detailed description will be omitted.

  As shown in FIG. 12, the vector extraction unit 233 of the generation device 20 determines whether the word has been registered in the corpus model A 121a (S102). When it is determined that the word has been registered in the corpus model A 121a (S102: Yes), the vector extraction unit 233 extracts the word vector A (S110), and proceeds to S112.

  On the other hand, when determining that the word has not been registered in the corpus model A 121a (S102: No), the vector extraction unit 233 outputs an unknown word vector acquisition request to the unknown word vector processing unit 237.

  When the unknown word vector processing unit 237 receives the output of the unknown word vector acquisition request, the unknown word vector processing unit 237 outputs the unknown word vector to the vector extraction unit 233 (S111), and proceeds to S112.

  Similarly, the vector extraction unit 233 determines whether the word has been registered in the corpus model B 121 b (S112). If the vector extraction unit 233 determines that the word has been registered in the corpus model B 121 b (S112: Yes), it extracts the word vector B (S120), and proceeds to S130.

  On the other hand, when determining that the word is not registered in the corpus model B 121 b (S 112: No), the vector extraction unit 233 outputs an unknown word vector acquisition request to the unknown word vector processing unit 237.

  When the unknown word vector processing unit 237 receives the output of the unknown word vector acquisition request, the unknown word vector processing unit 237 outputs the unknown word vector to the vector extraction unit 233 (S121), and proceeds to S130.

[effect]
As described above, when the generation program in the second embodiment determines that the word is an unknown word not included in the first conversion parameter or the second conversion parameter, it generates a vector indicating an unknown word. Make the computer execute the process to This makes it possible to generate a combined word vector indicating that the input word is an unknown word.

  The generation program may cause the computer to execute a process of generating a zero vector or a random number vector as a vector indicating that the word is an unknown word. Further, the generation program may cause the computer to execute processing of generating a vector obtained by duplicating the first or second vector not determined to be an unknown word as a vector indicating that the word is an unknown word. Thereby, an unknown word vector can be generated in an arbitrary form.

  The unknown word vector processing unit 237 may generate the unknown word vector by learning the corpus model A and the corpus model B as the unknown word vector instead of the random number vector.

  FIG. 13 is a diagram illustrating another example of the vector generation process according to the second embodiment. As shown in FIG. 13, the unknown word vector processing unit 237 learns using the word vector of each word such as “PC”, “smart phone”, “tablet”, etc. commonly included in corpus model A and corpus model B. By doing this, an unknown word vector B3 is generated.

  As described above, the generation program according to the second embodiment executes, on the computer, processing of generating a vector indicating an unknown word by performing learning processing using a combination of the first vector and the second vector. You may This makes it possible to generate an unknown word vector that matches the features of each corpus.

  Although the embodiments of the present invention have been described above, the present invention may be implemented in various different modes other than the above-described embodiments. Therefore, different embodiments will be described below.

  For example, although the configuration in which the vector combining unit 134 links the word vector A and the word vector B has been described, the present invention is not limited thereto, and the configuration may be such that the word vector A and the word vector B are calculated.

  FIG. 14 is a diagram illustrating an example of vector combination processing in the third embodiment. As shown in FIG. 14, the vector combining unit 334 (not shown) in the third embodiment generates a combined word vector C6 by adding the word vector A and the word vector B.

  As described above, the generation program may cause the computer to execute the process of generating the third vector by computing the generated first vector and the generated second vector. Thereby, a plurality of word vectors can be combined without increasing the dimension of the word vector before combining.

  In addition, although the configuration in which the vector combining unit 134 links the word vector A and the word vector B having the same dimension has been described, the configuration is not limited thereto. The configuration is such that the word vector A and the word vector B have different dimensions. It may be

  FIG. 15 is a diagram illustrating an example of vector generation processing in the third embodiment. As shown in FIG. 15, the corpus model A '321a in the third embodiment stores a 200-dimensional word vector. On the other hand, the corpus model B '321b in the third embodiment stores a 100-dimensional word vector. In this case, the vector extraction unit 333 (not shown) may extract a 200-dimensional word vector A and a 100-dimensional word vector B, respectively. The vector combining unit 334 (not shown) may generate a 300-dimensional word vector C by combining the 200-dimensional word vector A and the 100-dimensional word vector B.

  Thus, the generation program may cause the computer to execute a process of generating a second vector having a dimension different from that of the first vector. This makes it possible to change the weight of each word vector corresponding to each corpus model.

  In addition, although the configuration using corpus model A and corpus model B generated from two corpuses has been described, the number of corpus models is not limited to this, and a configuration using three or more corpus models may be used. .

  FIG. 16 is a diagram illustrating another example of the vector generation process according to the third embodiment. As shown in FIG. 16, in the third embodiment, in addition to a corpus model A generated from a corpus A similar to the example shown in FIG. 10 and a corpus model B generated from a corpus B, a corpus N is generated. A corpus model N is further used. In this case, the vector extraction unit 333 (not shown) may further extract the word vector N using the corpus model N in addition to the word vector A and the word vector B. In addition to the word vector A and the word vector B, the vector combining unit 334 (not shown) may further combine the word vector N to generate a word vector C.

  Thus, the generation program may further cause the computer to execute a process of further applying the third conversion parameter to the word to further generate a fourth vector according to the word. In addition, the generation program may cause the computer to execute a process of generating a new third vector according to a word based on the fourth vector in addition to the first vector and the second vector. Thereby, it is possible to use a word vector holding three or more corpus features.

  By the way, when vectors are combined, the dimension of the vectors may be increased, which may require time for the search process. Therefore, any one or a combination of two or more of the first vector, the second vector, and the third vector may be selected. In this case, the learning result corresponding to the selected vector may be used to acquire the discrimination result between the received word and the sentences to be discriminated. Thereby, a vector of appropriate size can be adopted according to the processing load.

  Note that the embodiment is not limited to distributed learning targeting Japanese documents, and documents in other languages such as English and Chinese may be used, for example.

[neural network]
In addition, any neural network such as RNN (Recurrent Neural Network) or CNN (Convolutional Neural Network) can be used for the pre-learning process. Also, as a learning method, various known methods such as an error back propagation (BP) method can be adopted. The neural network has a multistage configuration including, for example, an input layer, an intermediate layer (hidden layer), and an output layer, and each layer has a structure in which a plurality of nodes are connected by an edge. Each layer has a function called "activation function", the edge has "weight", and the value of each node is the value of the node of the previous layer, the value of weight of connection edge, and the activation function of the layer Calculated In addition, about a calculation method, well-known various methods are employable.

[system]
In addition to the above, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each device illustrated is functionally conceptual, and does not necessarily have to be physically configured as illustrated. That is, the specific form of distribution and integration of each device is not limited to the illustrated one. That is, all or part of them can be configured to be functionally or physically dispersed and integrated in arbitrary units in accordance with various loads, usage conditions, and the like. For example, the vector extraction unit 233 and the unknown word vector processing unit 237 shown in FIG. 11 may be integrated. In addition, the response search unit 135 illustrated in FIG. 2 may be realized by a computer different from the generation device 10. That is, the inference process and the combining process may be performed in different cases. Furthermore, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as wired logic hardware.

[Hardware configuration]
FIG. 17 is a diagram illustrating an example of a hardware configuration. As illustrated in FIG. 17, the generation device 10 includes a communication interface 10a, an HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. In addition, although the production | generation apparatus 10 in Example 1 is demonstrated below, the production | generation apparatus in another Example can also be implement | achieved by the same structure.

  The communication interface 10a is a network interface card or the like that controls communication of another device. The HDD 10 b is an example of a storage device that stores programs, data, and the like.

  Examples of the memory 10 c include a random access memory (RAM) such as a synchronous dynamic random access memory (SDRAM), a read only memory (ROM), and a flash memory. Examples of the processor 10 d include a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic device (PLD).

  The generation device 10 also operates as an information processing device that executes a learning method by reading and executing a program. That is, the generation device 10 executes a program that executes the same function as the reception unit 131, the word extraction unit 132, the vector extraction unit 133, the vector combination unit 134, the answer search unit 135, and the output unit 136. As a result, the generation device 10 can execute a process that executes the same function as the reception unit 131, the word extraction unit 132, the vector extraction unit 133, the vector combination unit 134, the answer search unit 135, and the output unit 136. The program referred to in this other embodiment is not limited to being executed by the generation device 10. For example, when the other computer or server executes the program, or when they cooperate to execute the program, the present invention can be applied similarly.

10, 20 generator 120, 220 storage unit 121a corpus model A
121b Corpus Model B
122 Search time learning model 123 Answer storage unit 224 Unknown word vector storage unit 130, 230 Control unit 131 Reception unit 132 Word extraction unit 133, 233 Vector extraction unit 134 Vector combination unit 135 Answer search unit 136 Output unit 237 Unknown word vector processing unit

Claims (14)

Accept words,
Applying a first conversion parameter and a second conversion parameter to the received word to generate a first vector and a second vector according to the word;
A generation program having a computer execute a process of generating a new third vector according to the word based on the generated first vector and the second vector.   The program according to claim 1, wherein the process of generating the third vector generates the third vector by concatenating the generated first vector and the second vector.   The process of generating the third vector is characterized in that the third vector is generated by calculating the generated first vector and the second vector. Generator of.   4. The process according to any one of claims 1 to 3, wherein the process of generating the first vector and the second vector generates the second vector having a dimension different from that of the first vector. Description generator.   The process of generating the first vector and the second vector is the unknown word when it is determined that the word is an unknown word not included in the first conversion parameter or the second conversion parameter. The generator according to any one of claims 1 to 4, characterized by generating a vector indicating that.   The program according to claim 5, wherein the process of generating the first vector and the second vector generates a zero vector or a random number vector as a vector indicating that the word is an unknown word.   In the process of generating the first vector and the second vector, a vector obtained by duplicating the first vector or the second vector not determined to be an unknown word as the vector indicating the unknown word. The generation program according to claim 5, characterized in that:   The computer-implemented method further causes the computer to execute a process of generating a vector indicating the unknown word by performing a learning process using a combination of the first vector and the second vector. The generator described in 5.   Using the learning model in which the learning about the feature of the word or phrase included in the sentences to be discriminated is performed, the discrimination result between the received word and the sentences to be discriminated is obtained from the third vector The generation program according to any one of claims 1 to 8, further causing a computer to execute the processing.   The process of acquiring the discrimination result selects any one or a combination of one or more of the first vector, the second vector, and the third vector, and corresponds to the selected vector. The generation program according to claim 9, wherein a discrimination result between the received word and the plurality of sentences to be discriminated is acquired using the learning model. The accepting process accepts text data including a plurality of the words,
The process of generating the first vector and the second vector generates the first vector and the second vector according to the text data,
The generation program according to any one of claims 1 to 10, wherein the process of generating the third vector generates a new third vector according to the text data. The process of generating the first vector and the second vector further applies a third conversion parameter to the received word to further generate a fourth vector according to the word;
The process of generating the third vector generates a new third vector according to the word based on the fourth vector in addition to the generated first vector and the second vector. The generation program according to any one of claims 1 to 11, wherein: The computer is
Accept words,
Applying a first conversion parameter and a second conversion parameter to the received word to generate a first vector and a second vector according to the word;
A generation method for performing processing of generating a new third vector according to the word based on the generated first vector and the second vector. A reception unit for receiving words,
A first vector processing unit that applies a first conversion parameter and a second conversion parameter to the received word to generate a first vector and a second vector according to the word;
A second vector processing unit that generates a new third vector according to the word based on the generated first vector and the second vector;
A generator having:

Images