JP7012811B1 - Search device, search method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the search efficiency of an answer to a question. A search system 1 includes a learning unit 10 and a search unit 20. The learning unit 10 inputs the text group to be processed, and co-occurrence the topic model and the words of the text group, which are decomposed by NMF based on the matrices D and S representing the importance of the words and part of speech of each text of the text group. Simultaneously learn with a skipgram model based on co-occurrence to generate a model that outputs a variance vector when a document or word is input. The search unit 20 inputs a word or a document into the model, and outputs a word or a document having a variance vector close to the obtained variance vector as a search result. [Selection diagram] Fig. 1

Description

The present invention relates to a search device, a search method, and a program.

The existing search system is an extension of the database search and searches for keyword matches. For example, in a search method using a distributed embedded expression (distributed vector), a word or a document is represented by a distributed vector, and a search is executed using a vector learned so that the closeness of the vector becomes close to the meaning. .. This makes it possible to search for words or documents that are semantically close to the words or documents by vector operation.

International Publication No. 2018/151125

However, since the existing variance vector is learned using only the information before and after the word, the antonym may have a similar variance vector. Also, although a document is composed of words, not all words have the same weight, but the dispersion vector of the document is not learned in consideration of them, so the accuracy of document retrieval is reduced. There is a problem.

The present invention has been made in view of the above, and an object of the present invention is to improve the search efficiency of answers to questions.

In the search device of one aspect of the present invention, a first model in which a text group to be processed is input and a matrix representing the importance of a word in each text of the text group is decomposed by a non-negative matrix factor based on a topic and the text. Obtained by learning a second model based on the co-occurrence of words in a group at the same time and inputting a word or document into the model with a learning unit that generates a model that outputs a dispersion vector when a document or word is input. It is provided with a search unit that outputs a word or document having a distribution vector close to the distribution vector as a search result.

According to the present invention, it is possible to improve the search efficiency of answers to questions.

FIG. 1 is a diagram showing an example of the configuration of the search system of the present embodiment. FIG. 2 is a diagram representing the model of the present embodiment by non-negative matrix factorization. FIG. 3 is a diagram showing the model of the present embodiment by skip-gram. FIG. 4 is a flowchart showing an example of the flow of the learning process of the search system of the present embodiment. FIG. 5 is a flowchart showing an example of the flow of the search process of the search system of the present embodiment.

[System configuration]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of the configuration of the search system of the present embodiment. The search system 1 shown in the figure includes a learning unit 10, a search unit 20, a data storage unit 30, and a calculation result storage unit 40. Each part included in the search system 1 may be configured by a computer provided with an arithmetic processing unit, a storage device, and the like, and the processing of each part may be executed by a program. This program is stored in a storage device included in the search system 1, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or can be provided through a network.

The learning unit 10 inputs a text group to be processed, learns a topic model (Topic model) and an embedded model (Embedding model) at the same time, and when a document set is input, the document included in the document and the dispersion vector of the word are input. Generate. The topic model is a model of the text generation process using the co-occurrence of global words in text units based on the topic. The embedded model is a model of word co-occurrence using local information such as a word set that appears around each word. By linking the topic model and the embedded model, it is possible to generate a model that enjoys the advantages of both models.

More specifically, the learning unit 10 includes a preprocessing unit 11 and a calculation processing unit 12, and expresses a matrix representing the importance of a word based on a topic by using non-negative matrix factorization, and also expresses the word. A model is generated in which the co-occurrence of is expressed in a matrix using skip-gram.

The preprocessing unit 11 divides the text group to be processed into words by morphological analysis, generates matrices D and S representing the importance of the words and parts of each text based on the frequency of occurrence of the words, and creates inter-word intervals. The co-occurrence matrix W of words, the co-occurrence matrix E of parts, and the co-occurrence matrix R of words and parts are generated based on the self-mutual information amount (Pointwise Mutual Information: PMI) between parts and between words and parts.

The calculation processing unit 12 decomposes the matrix D into a document-topic matrix Θ and a topic-word matrix T, decomposes the matrix S into a document-topic matrix Θ and a topic-part-timer matrix G, and decomposes the matrix D into a document-topic matrix Θ and a topic-part-timer matrix G, and then decomposes the matrix D into a document-topic matrix Θ and a topic-word matrix T. While finding the matrix G, the embedded matrix B of the word derived by decomposing the co-occurrence matrix W of the word, the embedded matrix C of the context word, and the embedded matrix Q of the part derived by decomposing the co-occurrence matrix E of the part of the word, and the context. The topic-word matrix T is decomposed and derived by finding the part-word embedded matrix Q and the context word embedded matrix C derived by decomposing the part-word embedded matrix H and the co-occurrence matrix R of the word and part. The embedded matrix A of the topic, the embedded matrix C of the context word, the embedded matrix A of the topic derived by decomposing the topic-participation matrix G, and the embedded matrix H of the part words according to the context are obtained. The details of the proposed model will be described later.

The search unit 20 searches the calculation result storage unit 40 for a variance vector close to the variance vector obtained from the model using the word or document received from the user terminal 5 as a key, and the word or word having the variance vector obtained by the search. The document is acquired from the data storage unit 30 and returned to the user terminal 5 as a search result.

For example, when the search content is input in spoken language from the user terminal 5, the search unit 20 searches for words that are semantically close to each other using the spoken language as a key, and inputs the word obtained in the search into the model as a search keyword. Search results may be obtained.

The data storage unit 30 holds the document and the word to be searched. The document held by the data storage unit 30 is also used for learning the model of the learning unit 10 as a text group to be processed.

The calculation result storage unit 40 holds the dispersion vector of the document and the word. The dispersion vector of the document and the word is obtained by inputting the document and the word into the model generated by the learning unit 10. The calculation result storage unit 40 is an index of the semantic space of a document and a word, and the closeness of the distance in the space is the closeness of the meaning.

[Proposed model]
Here, the model proposed in this embodiment will be described.

FIG. 2 is a diagram showing a matrix representation of a graphical model of each text using non-negative matrix factorization (NMF).

The left side of FIG. 2 is a graphical model of each text. The graphical model consists of nodes and edges, where the nodes are variables and the direction of the edges shows the causal relationship between the nodes. The graphical model of FIG. 2 shows how the part of speech l _ji is stochastically sampled from the topic z _ji in the token i of the document d _j , and the word w _ji is sampled from the topic z _ji and the part of speech l _ji .

The right side of FIG. 2 is a matrix representation of a text set (document set) using NMF. The document x word matrix D is decomposed into the document x topic matrix Θ and the topic x word matrix T. The document × part of speech matrix S is decomposed into the document × topic matrix Θ and the topic × part of speech matrix G. The matrix D and S share the matrix Θ.

FIG. 3 is a matrix representation of the nodes (variables) in which each text appears in the graphical model using skip-gram (window size = 2).

A graphical model of each text is shown on the left side of FIG. 3, and three skip-grams are shown on the right side. The skip-gram on the left predicts the words w _t-2 , w _t-1 , w _{t + 1} , w _{t + 2} , which are within the second before and after the word w _t at the position t. The skip-gram in the middle predicts the part of speech l _t-2 , l _t-1 , l _{t + 1} , l _{t + 2} , which is within the second before and after the part of speech l _t at the position t. The skip-gram on the right predicts the words w _t-2 , w _t-1 , w _{t + 1} , w _{t + 2} , which are within the second before and after the part of speech l _t at position t.

Learning the skip-gram is equivalent to the decomposition of the PMI matrix. In model training, the co-occurrence matrix W, the co-occurrence matrix E, and the shifted positive PMI (SPPMI) of the co-occurrence matrix R are used to decompose the co-occurrence matrix W into a word embedded matrix B and a context word embedded matrix C. Then, the co-occurrence matrix E is decomposed into the embedded matrix Q of the part words and the embedded matrix H of the part words according to the context, and the co-occurrence matrix R is decomposed into the embedded matrix Q of the part words and the embedded matrix C of the context words.

In the present embodiment, the matrix T and G obtained in FIG. 2 are further decomposed into the topic embedded matrix A and the context word embedded matrix C by using SPPMI to decompose the topic × word matrix T into the topic × part words. The matrix G of is decomposed into the embedded matrix A of the topic and the embedded matrix H of the part words according to the context.

The learning unit 10 includes the document-topic matrix Θ, the topic-word matrix T, the topic-part part matrix G, the topic embedding matrix A, the word embedding matrix B, the context word embedding matrix C, and the contextual part words. By finding the embedded matrix H of the above and the embedded matrix Q of the part words, the topic model and skip-gram are linked to learn the dispersion vector of the documents and words included in the text group.

[motion]
Next, the learning process will be described with reference to the flowchart of FIG. The symbols used in the explanation and their explanations are shown in Table 1 below.

In step S11, the learning unit 10 reads the text group to be processed from the data storage unit 30, and divides the text group into words by morphological analysis. The learning unit 10 also acquires the part of speech of the word.

In step S12, the learning unit 10 creates a document × word matrix D and a document × part of speech matrix S. The matrix D is a matrix consisting of V (number of different words) × N (number of documents), and the element d _ji of the matrix uses the tf * idf value of the word j in the document i. The matrix S is a matrix consisting of O (number of different parts of speech) × N (number of documents), and the element s _ji of the matrix uses the tf * idf value of the part of speech j in the document i.

In step S13, the learning unit 10 creates a word co-occurrence matrix W, a part-speech co-occurrence matrix E, and a word-part-speech co-occurrence matrix R.

The (i, j) component of the word co-occurrence matrix W can be obtained by the following equation using SPPMI.

The co-occurrence matrices E and R can be created in the same manner as the co-occurrence matrix W.

In step S14, the learning unit 10 updates the parameters of the model. The derivation of parameters will be described below.

As described with reference to FIG. 2, in the present embodiment, the matrices D and S are interpreted as a topic model using NMF, and D = Θ × T and S = Θ × G. The matrices T, G, Θ can be derived using the objective functions L _D , _LS of the following equation. The matrices D and S can be decomposed by finding the matrices T, G and Θ that minimize the objective functions L _D and L _S.

The second term on the right side is a normalization term for preventing overfitting, and λ _d and λ _s are parameters for adjusting the normalization term. The same applies to the following formulas.

As explained with reference to FIG. 3, learning of skip-gram corresponds to decomposition of co-occurrence matrices W, E, and R created in step S13, and the matrices B, C, Q, and H are objective functions L _W of the following equation. , _LE , L _R can be used for derivation.

Further, by decomposing the matrix T = A × C and the matrix G = A × H, the matrices A, C, and H can be derived using the objective functions LT _{and G} of the following equation.

Summarizing the above objective functions, the objective function of the model is as follows.

Expanding the objective function of the model yields the following equation.

The parameter to be updated by the learning unit 10 is the following equation derived from the objective function of the model.

Subsequently, the search process will be described with reference to the flowchart of FIG.

In step S21, the search unit 20 receives the search content (query) from the user terminal 5. The query can be a word or a document.

In step S22, the search unit 20 queries the calculation result storage unit 40 using the received query as a key, and obtains the corresponding variance vector.

In step S23, the search unit 20 searches the calculation result storage unit 40 for a variance vector close to the variance vector obtained in step S22, and acquires a document or word corresponding to the searched close variance vector from the data storage unit 30. do. The distance between the variance vectors can be measured using the cosine similarity.

In step S24, the search unit 20 returns the word or document acquired in step S23 to the user terminal 5 as a search result.

[inspection]
In order to evaluate the effectiveness of the search system of this embodiment for improving the conformity rate of the QA system, the search functions of the current help desk system and the search system of this embodiment were verified.

About 700 QA data were prepared as search data, and about 150 questions and correct answers to the questions were prepared as search contents. Search data was registered in the current system by batch registration, questions were searched at any time, and up to the fifth output was acquired. The search system of the present embodiment was made to learn the search data, the questions were searched at any time, and up to the fifth output was acquired. Among the 1st to 5th outputs, 5 points, 4 points, 3 points, 2 points, and 1 point were given according to the position of the correct answer to the question. For example, if the first output is the correct answer to the question, 5 points will be given. If the output does not include the correct answer, it will be given 0 points. The verification results are shown in Table 2 below.

It can be said that the search system of the present embodiment has a higher total number of correct answers and a higher matching rate than the current system. Further, it can be said that the search system of the present embodiment has a smaller number of 0 points and a higher reproducibility than the current system. The precision rate is an index related to accuracy, which indicates the degree of agreement with the correct answer data. The reproducibility is an index related to completeness, which indicates the degree of coverage of correct answer data.

As described above, the search system 1 of the present embodiment includes a learning unit 10 and a search unit 20. The learning unit 10 inputs the text group to be processed, and co-occurrence the topic model and the words of the text group, which are decomposed by NMF based on the matrices D and S representing the importance of the words and part of speech of each text of the text group. Simultaneously learns a skipgram model based on co-occurrence to generate a model that outputs a variance vector when a document or word is input. The search unit 20 inputs a word or a document into the model, and outputs a word or a document having a variance vector close to the obtained variance vector as a search result. In this way, the search system 1 can simultaneously learn the dispersion vector of the document and the word, so that the document and the word can be expressed in the same semantic space. As a result, not only the keywords match, but also those with similar meanings can be searched, and improvement in search efficiency can be expected. In addition, when an appropriate keyword cannot be recalled, appropriate search results can be obtained by inputting a document (spoken language) representing the keyword.

The search system 1 of the present embodiment considers the weights of the words constituting the document by generating a matrix representing the importance of the words and parts of speech in each text using the tf * idf values of the words and parts of speech. Therefore, it can be expected that the accuracy of document retrieval will be improved.

Since the search system 1 of the present embodiment learns the model by combining topics and part of speech, it is possible to suppress that the antonyms have similar variance vectors.

1 ... Search system 10 ... Learning unit 11 ... Pre-processing unit 12 ... Calculation processing unit 20 ... Search unit 30 ... Data storage unit 40 ... Calculation result storage unit

Claims (7)

The first model in which the text group to be processed is input and the matrix representing the importance of the words in each text group is decomposed into non-negative matrix factors based on the topic, and the second model based on the co-occurrence of the words in the text group. A learning unit that learns a model at the same time and generates a model that outputs a variance vector when a document or word is input.
A search device including a search unit that inputs a word or a document into the model and outputs a word or a document having a variance vector close to the obtained variance vector as a search result. The search device according to claim 1.
In the first model, the matrix representing the importance of words in each text is decomposed into the document-topic matrix and the topic-word matrix, and the matrix representing the importance of the part of speech of each text is decomposed into the document-topic matrix and topic-part of speech matrix. It is a model disassembled into
The second model is a model in which the co-occurrence of words in the text group is expressed by a co-occurrence matrix of words, a co-occurrence matrix of parts of speech, and a co-occurrence matrix of words and parts of speech.
The learning unit inputs the text group to be processed, and decomposes the document-topic matrix, the topic-word matrix, the topic-participation matrix, and the co-occurrence matrix of the words into a word embedded matrix derived. Embedded matrix of contextual words, embedded matrix of part words derived by decomposing the co-occurrence matrix of the part words, embedded matrix of part words according to the context, embedded matrix of part words derived by decomposing the co-occurrence matrix of the word and part words And the embedded matrix of the context word, the embedded matrix of the topic derived by decomposing the topic-word matrix and the embedded matrix of the context word, the embedded matrix of the topic derived by decomposing the topic-participatory matrix, and the participatory in line with the context. A search device that calculates the embedded matrix of the above and generates the model. The search device according to claim 2.
The learning unit generates a matrix representing the importance of the word of each text based on the frequency of occurrence of the word, and generates a matrix representing the importance of the part of speech of each text based on the frequency of occurrence of the part of speech.
The learning unit generates a co-occurrence matrix of the word based on the self-mutual information between words, generates a co-occurrence matrix of the part based on the self-mutual information between parts, and generates a co-occurrence matrix between words and parts. A search device that generates a co-occurrence matrix of the word and part of the word based on the amount of self-mutual information. The computer
Enter the text group to be processed and
Documents by simultaneously learning the first model, which is a non-negative matrix factor decomposition based on the topic, and the second model, which is based on the co-occurrence of words in the text group. Or generate a model that outputs a variance vector when you enter a word,
A search method in which a word or document is input to the model and a word or document having a variance vector close to the obtained variance vector is output as a search result. The search method according to claim 4.
In the first model, the matrix representing the importance of words in each text is decomposed into the document-topic matrix and the topic-word matrix, and the matrix representing the importance of the part of speech of each text is decomposed into the document-topic matrix and topic-part of speech matrix. It is a model disassembled into
The second model is a model in which the co-occurrence of words in the text group is expressed by a co-occurrence matrix of words, a co-occurrence matrix of parts of speech, and a co-occurrence matrix of words and parts of speech.
The computer inputs the text group to be processed, inputs the document-topic matrix, the topic-word matrix, the topic-participation matrix, and the text group to be processed, and decomposes the co-occurrence matrix of the words. The embedded matrix of the word and the embedded matrix of the context word derived by The embedded matrix of part words and the embedded matrix of the context word derived by decomposition, the embedded matrix of the topic derived by decomposing the topic-word matrix and the embedded matrix of the context word, and the topic derived by decomposing the topic-partial word matrix. A search method that generates the model by finding the embedded matrix of the part of the word and the embedded matrix of the part words according to the context. The search method according to claim 5.
The computer
A matrix representing the importance of the words in each of the texts is generated based on the frequency of the words.
A matrix representing the importance of the part of speech in each of the texts is generated based on the frequency of appearance of the part of speech.
A co-occurrence matrix of the words is generated based on the amount of self-mutual information between words.
A co-occurrence matrix of the part of speech is generated based on the amount of self-mutual information between the part of speech.
A search method that generates a co-occurrence matrix of the word and part of speech based on the amount of self-mutual information between the word and part of speech. A program that operates a computer as each part of the search device according to any one of claims 1 to 3.

Images