KR102655808B1 - A method and apparatus for question-answering using a paraphraser model - Google Patents

Abstract

This specification includes the steps of generating a user query vector for a user query; selecting a similar query based on a similarity analysis result with the user query vector; and determining a response to the similar query as a response to the user query.

Description

Question-answering search method and search apparatus using a paraphraser model {A method and apparatus for question-answering using a paraphraser model}

The present invention relates to a method of retrieving and presenting a response to a user inquiry to the user.

As a method for search engines to provide answers to user queries, a method of providing database search results based on text information that constitutes the user query entered by the user is widely used.

However, since the way of expressing a query is different for each user, the method of providing search results for a user's query based on text information causes search results to vary depending on the search skill.

To solve this problem, semantic labeling is performed by performing linguistic analysis on text documents or user questions in Korea Patent Publication No. 10-2011-0134909, etc., and to find text related to the semantically labeled user question. Methods for searching a database of labeled text documents are presented.

However, the above method has the problem that a large text document database must be searched to find a response corresponding to a user request, and the search time increases as the size of the text document database increases.

Publication of Patent No. 10-2011-0134909 (December 15, 2011)

In this specification, we would like to propose a method for efficiently searching responses to user queries using paraphrase.

A search method performed by an electronic device according to an embodiment of the present invention to solve the above problem includes generating a user query vector for a user query; selecting a similar query based on a similarity analysis result with the user query vector; and determining a response to the similar query as a response to the user query.

The search method further includes selecting a similar query candidate based on similarity with the user query vector, and selecting a similar query candidate based on a similarity analysis result with the user query vector includes: It may include selecting a similar query from among the similar query candidates based on the similarity analysis result.

The similar query candidate is selected based on similarity with the user query vector by a first similarity analysis model, and the similar query is selected based on a similarity analysis result with the user query vector by a second similarity analysis model, , the first similarity analysis model and the second similarity analysis model may be different analysis models.

The second similarity analysis model is trained using first training data and second training data, and the first training data consists of a first query pair and a label indicating that the first query pair is similar, and the second The learning data may consist of a second query pair and a label indicating that the second query pair is dissimilar.

The electronic device may include a database, and the database may store a query and a query vector for the query in association with the query.

The step of selecting a similar query candidate based on the similarity with the user query vector may include selecting a query associated with a query vector whose similarity with the user query vector is higher than a preset standard as the similar query candidate. there is.

The step of selecting a similar query from among the similar query candidates based on the similarity analysis result with the user query vector includes determining a similarity rank of candidate queries belonging to the similar query candidates based on the similarity analysis result with the user query vector. steps; and selecting a predetermined number of candidate queries as similar queries according to the similarity ranking.

Determining the response to the similar query as a response to the user query includes generating a response to the user query such that the similar query and the response to the similar query are provided to the user according to the similarity ranking. can do.

In addition, a search device according to an embodiment of the present invention to solve the above problem includes a processor; and a memory storing instructions executed by the processor, wherein the processor generates a user query vector for a user query by executing the instruction, and based on a result of analysis of similarity with the user query vector. A similar query is selected, and the response to the similar query is determined as a response to the user query.

In addition, a computer program stored in a computer-readable storage medium according to an embodiment of the present invention for solving the above problem includes generating a user query vector for a user query; selecting a similar query based on a similarity analysis result with the user query vector; and determining a response to the similar query as a response to the user query.

The present invention provides the effect of presenting search results for queries similar to the user's query to the user. Furthermore, the present invention provides the effect of enabling a search device to more accurately provide answers to similar queries by using a paraphrase model to select a query with high similarity to the user's query and providing a response to the selected query.

1A is a flowchart explaining a search method performed by a terminal according to an embodiment of the present invention.
Figure 1B is a block diagram illustrating a search system according to an embodiment of the present invention.
2 and 3 are block diagrams showing a terminal and a server according to an embodiment of the present invention.
Figure 4 is a flowchart explaining a search method according to an embodiment of the present invention.
Figures 5 and 6 are diagrams illustrating examples of the process of creating an analysis model by learning a neural network.
Figure 7 is a diagram showing an example of generating a vector by embedding a user query expressed in natural language.
Figure 8 shows the neural network learning process for generating an encoder.
Figure 9 is a diagram explaining the concept of a Siamese neural network.
Figure 10 is a flowchart explaining how the analysis model generation device generates a paraphrase model according to an embodiment of the present invention.
Figure 11 is a diagram explaining the vector-based search method of the server.

The following merely illustrates the principles of the invention. Therefore, a person skilled in the art can invent various devices that embody the principles of the invention and are included in the concept and scope of the invention, although not clearly described or shown herein. In addition, all conditional terms and embodiments listed in this specification are, in principle, expressly intended only for the purpose of ensuring that the inventive concept is understood, and should be understood as not limiting to the embodiments and conditions specifically listed as such. .

The above-mentioned purpose, features and advantages will become clearer through the following detailed description in relation to the attached drawings, and accordingly, those skilled in the art in the technical field to which the invention pertains will be able to easily implement the technical idea of the invention. .

In the specification and claims, terms such as “first,” “second,” “third,” and “fourth,” if any, are used to distinguish between similar elements and, although not necessarily, to distinguish between similar elements in a particular sequence. Or it is used to describe the order of occurrence. It will be understood that the terms so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein may be operated in sequences other than those shown or described herein, for example. Similarly, where a method is described herein as comprising a series of steps, the order of such steps presented herein is not necessarily the order in which such steps may be performed, and any described step may be omitted and/or performed herein. Any other steps not described may be added to the method.

Additionally, terms such as “left”, “right”, “front”, “back”, “top”, “bottom”, “above”, “below”, etc. in the specification and claims are used for descriptive purposes; It is not necessarily intended to describe an immutable relative position. It will be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can be operated in orientations other than those shown or described herein, for example. As used herein, the term “connected” is defined as connected, directly or indirectly, in an electrical or non-electrical manner. Objects described herein as being “adjacent” to each other may be in physical contact with each other, in close proximity to each other, or in the same general extent or area as each other, as appropriate for the context in which the phrase is used. The presence of the phrase “in one embodiment” herein does not necessarily mean the same embodiment.

In addition, in the specification and claims, various variations of these expressions such as 'connected', 'connecting', 'concluded', 'fastened', 'coupled', 'coupled', etc. are used directly with other components. It is used to mean connected or indirectly connected through other components.

In addition, the suffixes “module” and “part” for components used in this specification are given or used interchangeably only considering the ease of writing the specification, and do not have distinct meanings or roles in themselves.

Additionally, the terms used in this specification are for describing embodiments and are not intended to limit the present invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, 'comprise' and/or 'comprising' refers to the presence of one or more other components, steps, operations and/or elements. or does not rule out addition.

Additionally, when describing the invention, if it is determined that a detailed description of known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

FIG. 1A is a flowchart explaining a search method performed by the terminal 100 according to an embodiment of the present invention. First, the terminal 100 obtains a user query and creates a user query vector for the user query (S10). Next, the terminal 100 selects a similar query based on the similarity analysis result with the user query vector. Do it (S20). Next, the terminal 100 determines the response to the similar query as the response to the user query (S30). Next, the terminal 100 generates a response to the user inquiry (S40).

Figure 1B is a block diagram illustrating a search system according to an embodiment of the present invention. A search system according to an embodiment consists of a user terminal 100 and a search server 200. Hereinafter, for convenience of explanation, the user terminal 100 is abbreviated as terminal 100, and the search server 200 is abbreviated as server 200.

The terminal 100 according to one embodiment transmits a query to the server 200 and receives a response to the query from the server 200. The terminal 100 provides an answer to the user's inquiry based on the inquiry response received from the server 200.

The server 200 may be a search engine that performs elastic search based on keywords. In one embodiment, the server 200 may search for similar queries based on vectors. Through this, the server 200 can select a query with high similarity to the user query and provide it to the terminal 100. The server 200 primarily selects a similar query candidate group for the user query, and the terminal 100 performs a similarity evaluation on queries belonging to the similar query candidate group provided by the server 200, so that the terminal 100 allows the user to In order to select similar queries similar to the query, the number of queries whose similarity to the user query must be analyzed can be limited to a certain range.

Figure 2 is a block diagram showing a terminal 100 according to an embodiment of the present invention. The terminal 100 according to an embodiment of the present invention may be composed of an encoder 110, a paraphraser 120, a similar question selection unit 130, a question answering unit 140, and a database 150.

The encoder 110 generates an embedding vector corresponding to a user query expressed in natural language. An embedding vector is a vector that represents the structure and meaning of a user query expressed in natural language. The encoder 110 may be composed of a neural network and may be a neural network model learned to generate an embedding vector.

Paraphraser 120 is a similarity analysis model that analyzes the similarity of multiple queries expressed in natural language. The paraphraser 120 receives two natural language queries as input and provides the similarity value of the queries as output. For example, the paraphraser 120 can vectorize two queries and determine a similarity value based on the distance or cosine similarity in Euclidean space of the vectors for each query. The paraphraser 120 may be configured to include an encoder 110 to vectorize a query expressed in natural language. The paraphraser 120 may be composed of a neural network and may be a neural network model learned to determine similarity between queries.

The similar query selection unit 130 selects a query similar to the user query according to the degree of similarity obtained by the paraphraser 120. Then, the similarity level of similar queries is determined according to the order of similarity to the user query.

The question answering unit 140 obtains a user inquiry from the user. And the question answering unit 140 provides answers to user inquiries to the user. If the input user query and the answer thereto are stored in the database 150, the question answering unit 140 provides the user with an answer to the input user query. Meanwhile, if the input user query is not stored in the database 150, the question answering unit 140 provides a similarity level according to a preset standard according to the similarity level selected by the similar question selection unit 130. Answers to similar questions can be provided to the user as the user's answer.

Meanwhile, the terminal 100 and the server 200 according to an embodiment of the present invention include a processor 101, a memory 102, a communication unit 103, a user interface 104, and a database 105, as shown in FIG. 3. It may be configured to include. The memory 102 may store a program that performs the operation of a search method according to an embodiment of the present invention. The memory 102 may store a neural network model learned to implement the encoder 110 and the paraphraser 120. The processor 101 processes data according to the program and neural network model stored in the memory 102 and controls the communication unit 103, user interface 104, and database 105 to perform a search method according to an embodiment of the present invention. It can be done.

Hereinafter, a search method according to an embodiment of the present invention will be described. Figure 4 is a flowchart explaining a search method according to an embodiment of the present invention. The terminal 100 obtains the user's inquiry based on the user input (S110). The terminal 100 can obtain the user's inquiry through the user's text input through a keyboard or touchpad or voice input through the user's speech. The user query may be a sentence expressed in natural language.

Next, the terminal 100 vectorizes the user query expressed in natural language (S120). The terminal 100 can generate a user query vector by converting the user query into a vector representation using the encoder 110. Next, the terminal 100 transmits the user query vector to the server 200 (S130).

The server 200 searches for similar queries using the user query vector received from the terminal 100 (S140). The server 200 searches for a query having the same or similar vector as the user query vector among queries stored in the database of the server 200. In more detail, the query and the response to the query may be stored in association in the database of the server 200. For example, a (query, response) pair may be stored in a database. Additionally, a query and a vector for the query may be stored in association with each other in the database. For example, pairs (query, response, query vector) may be stored in a database.

In order to search for queries with similar vectors, the server 200 may determine vectors located within a certain distance from the user query vector in Euclidean space as similar vectors. Alternatively, the server 200 may determine the similarity based on cosine similarity. The server 200 generates a group of similar query candidates using the similar query search results. The similar query candidate group may consist of only queries or may include at least one of a question, a response, and a vector. Additionally, the similar query candidate group may further include a similarity value with the user query vector for each query in the candidate group. Furthermore, the similar query candidate group may further include the order of similarity between queries belonging to the candidate group. The server 200 transmits the generated similar query candidate group to the terminal 100 (S150).

Next, the terminal 100 selects a similar query using the similar query candidate information received from the server 200 (S160). The terminal 100 may select a query similar to the user query from among queries belonging to the similar query candidate group. The terminal 100 may use the paraphraser 120 to determine the degree of similarity between each query belonging to the similar query candidate group and the input user query. The terminal 100 can select the query with the highest similarity as the similar query. Alternatively, the terminal 100 may select similar queries by grade by grading the queries belonging to the similar query candidate group in descending order of similarity. The similarity analysis method performed by the paraphraser 120 may be different from the similarity analysis method performed by the server 200. For example, the similarity analysis model of the paraphraser 120 may be a different analysis model from the similarity analysis model of the server 200, and when it is composed of a neural network model, the values of variables constituting the model configuration or hidden layer are different from each other. can be different.

Next, the terminal 100 provides a response according to the selected similar query (S170). The terminal 100 may provide a response corresponding to a similar query with the highest degree of similarity as a response to a user query entered by the user. Alternatively, the terminal 100 may provide a response corresponding to a query belonging to a candidate group of similar queries as a response to the user query entered by the user, depending on the level of the similar query.

The terminal 100 may not receive all responses to similar query candidates from the server 200, but may only receive responses to selected similar queries from the server 200. This means that the similar query candidate group does not include response information, and the terminal 100 transmits a similar query requiring response information to the server 200 and receives a response to the similar query from the server, thereby only providing a response to the similar query. It can be implemented by receiving transmission from the server 200. As a result, traffic between the terminal 100 and the server 200 can be reduced.

A similar query may be a query that has a different meaning from the user query. For example, when a user inputs the query “How much does it take to get to City Hall?”, the terminal 100 asks similar questions such as “How much time does it take to get to City Hall?” and “How much is the distance to get to City Hall?” ' Alternatively, you can select a similar query such as 'How much does it cost to travel to City Hall?' Additionally, in the case of similar queries such as ‘How much time does it take to reach City Hall?’, ‘How much is the transportation cost to reach City Hall by private car?’, ‘How much is the transportation cost to reach City Hall by public transportation?’ Or, similar queries may exist, such as 'How much does it cost to travel to City Hall on foot?'

In this case, if only answers to similar queries are provided to the user, the user may misunderstand the response with a different meaning as an answer to the query entered by the user. Accordingly, when the terminal 100 provides the user with a response corresponding to a similar query, the terminal 100 may provide the user with a response to the similar query along with the similar query.

Meanwhile, in the description referring to FIG. 4, step S130 may be performed by the terminal 100 transmitting the user query itself rather than the user query vector to the server 200. In this case, the server 200 may generate a user query vector for the user query using an encoder. Then, in step S140, the server 200 can search for similar queries by using the generated user query vector.

Each step will be described in more detail below with reference to the drawings. The terminal 100 and the server 200 according to an embodiment of the present invention can analyze natural language to vectorize queries and store a learned neural network model to analyze the similarity between queries at the vector level. A neural network model according to one embodiment may be created at a scale that can operate depending on the application field or the computing power of the device.

In one embodiment, the neural network model may be a deep learning model. Deep learning models can be built with multiple hidden layers. Nodes belonging to a layer can send and receive data according to the connection relationship between nodes. The terminal 100 and the server 200 according to an embodiment of the present invention generate and generate an analysis model by learning a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), and a Bidirectional RecurrentDeep Neural Network (BRDNN). The method according to one embodiment can be performed using the analysis model.

Figure 5 is a flowchart explaining the process of creating an analysis model by learning a neural network. The encoder 110 and the paraphraser 120 according to one embodiment may be created by an analysis model generating device according to the analysis model generation method according to FIG. 5. First, the analysis model creation device acquires learning data (S210). Learning data can be set according to the analysis model to be created.

Next, the analysis model creation device trains a neural network using the acquired training data (S220). This will be described in more detail with reference to FIG. 6. The analysis model creation device can set a neural network model according to the analysis model to be created and determine the predicted value by performing forward propagation. And the analysis model creation device can determine the error value for determining the value of the variable by comparing the predicted value and the actual value using a loss function. The device can adjust the variable values of the neural network model using backward propagation so that the error between the predicted value and the actual value is reduced. As an optimizer used in backpropagation, batch gradient descent, stochastic gradient descent, etc. can be used.

After learning of the neural network progresses, the analysis model creation device acquires test data to check whether the neural network produces valid analysis results (S230) and tests the neural network using the acquired test data (S240). Test data may consist of input values and corresponding actual values. The analysis model creation device terminates learning of the neural network when the predicted value, which is the result of applying the input value of the test data to the neural network, has an error value within the tolerance range from the actual value of the test data, and can use the learned neural network as an analysis model. (S250). Meanwhile, the analysis model generation device can continue learning the neural network when the error value between the predicted value and the actual value according to the input of test data is outside the tolerance range.

Meanwhile, the analysis model generator may apply early termination or dropout to prevent overfitting. In one embodiment, the analysis model generation device monitors the error value between the predicted value and the actual value of the verification data, and if it is found that the accuracy of the verification data continues to decrease, it trains the epoch before the accuracy decreases. Early termination can be implemented by setting it to epoch. Alternatively, the device may prevent overfitting by dropping out nodes selected based on certain criteria that are not used for learning. This allows the device to prevent the neural network from changing depending on a specific node or combination of nodes.

Hereinafter, a method of vectorizing a query performed by the encoder 110 will be described. The encoder 110 can generate a vector by embedding a user query expressed in natural language, as shown in FIG. 7. The encoder 110 according to an embodiment of the present invention is a vector generation model created using a neural network, and can take a query as input and output a vector value corresponding to the query. The encoder 110 according to one embodiment may be a model such as Bag of Words (BOW), TF-IDF, Doc2Vec, Text Embedding Encoder, or Word2Vec.

Figure 8 shows the neural network learning process for generating the encoder 110. The analysis model generation device can generate an encoder 110 model by training a neural network model such as a DNN or RNN model (S320) using preprocessed learning data (S310) and testing the learned model (S330, S340). There is (S350).

For example, the analysis model creation device may preprocess learning data prior to generating the encoder 110. The analysis model creation device can extract words by performing tokenization on the query. In one embodiment, the analysis model creation device may perform tokenization based on spaces, colons, commas, etc. Additionally, the analysis model generation device can perform preprocessing through morpheme analysis and stop word removal.

Next, in one embodiment, the analysis model creation device may learn the Word2Vec model by applying CBOW (Continuous Bag of Words). The analysis model generator applies the score vector according to the one-hot vector of the surrounding words and the one-hot vector of the center word to the cross-entropy function, which is a loss function, and learns so that the value of the loss function approaches 0. The encoder 110 model can be created by proceeding with the process and modifying the variables of the neural network.

Hereinafter, a method for determining the similarity of a query pair performed by the paraphraser 120 will be described. The paraphrase 120 is a paraphrase model that outputs the similarity of a query pair for an input consisting of a query pair. The analysis model creation device can build a paraphrase model by learning a neural network model. The neural network model may be CNN, DNN, RNN, etc.

The paraphrase model according to one embodiment can obtain a vector representation of the query using the encoder 110 described above. The paraphrase model can evaluate the similarity between two queries by evaluating vector similarity. For example, the paraphrase model can generate a vector representation of the query and evaluate the similarity between two queries by calculating the distance between the two query vectors in Euclidean space. The paraphrase model can determine that two query vectors are similar to the extent that their distances are close.

In another embodiment, the paraphrase model may evaluate similarity by calculating the cosine similarity between vector representations of two queries. The paraphrase model can determine that the closer the cosine similarity value is to 1, the more similar it is, and the closer it is to -1, the more dissimilar it is.

On the other hand, when analyzing the similarity of queries in the above manner, the questions can be analyzed as similar even if the form of the question is similar but the query has completely different content due to changes in some words. For example, the query ‘How much is the transportation cost from here to City Hall?’ and the query ‘How far is it from here to City Hall?’ are composed of the same words except for the words ‘transportation cost’ and ‘distance’. Using the similarity analysis method described above, it can be determined that the queries are similar.

In this way, the paraphraser 120 must be able to identify that queries containing similar words but different semantic content are dissimilar to each other. The paraphrase model according to one embodiment uses a Siamese neural network and can be learned through supervised learning.

Figure 9 is a diagram explaining the concept of a Siamese neural network. Siamese neural networks are composed of identical neural networks. The analysis model generation device changes the variables that make up the Siamese neural network so that when queries with the same meaning are input, similar vectors are generated as result values, and when queries with different meanings are input, dissimilar vectors are generated as result values. By training the neural network as you go, you can create a paraphrase model. Thus, the paraphraser 120 can determine the similarity of both queries by analyzing the similarity of vector values generated for each query. Here, the similarity of the vectors may be determined based on distance proximity in Euclidean space or how close the cosine similarity value is to 1.

Figure 10 is a flowchart explaining how the analysis model generation device generates a paraphrase model according to an embodiment of the present invention. In the following description, the query constituting the learning data is described as being expressed as text, but depending on the embodiment, the query constituting the learning data may be expressed as a vector by the encoder described above.

The analysis model generation device according to an embodiment of the present invention generates first learning data composed of queries that are composed of similar words and structures, have similar sentence formats, and have similar meanings (410). The first learning data may also include a pair of queries and a label indicating that the queries have similar meanings. For example, training data for learning a paraphrase model may consist of a query pair and a similarity label for the query pair. For example, the learning data may consist of a query pair consisting of (first query, second query) and a similarity score of the query pair.

Next, the analysis model generating device performs learning of the neural network using the first learning data (S420). The analysis model creation device can check the degree of learning using test data according to the progress of learning and stop learning when overfitting begins to occur.

Next, the analysis model generation device generates second learning data consisting of queries that are composed of similar words and structures and have similar sentence formats, but the meanings of the queries are not the same (430). The second learning data may also include a label indicating that the queries do not have similar meanings along with the query pair.

Next, the analysis model generating device performs learning of the neural network using the second learning data (S440). The analysis model creation device can check the degree of learning using test data according to the progress of learning and stop learning when overfitting begins to occur.

Meanwhile, neural network learning in steps S420 and S440 is performed with learning data with different labels. The analysis model generator can train the neural network without distinguishing between steps S420 and S440 by mixing first and second learning data to increase learning efficiency. The analytical model creation device may use a triplet loss function, etc. to determine the error value between the predicted value and the result value.

Next, the analysis model creation device inputs test data into the learned neural network as above. If the analysis model generating device generates a predicted value within the error range for the test data, it may determine that the learned neural network can be used as a paraphrase model (S450).

Hereinafter, the vector-based search method of the server 200 will be described. The server 200 according to an embodiment of the present invention may store a question-and-answer pair consisting of a query and a response in a database, and may also store a query vector, which is a vector representation of the query, in the database along with the query and response. there is. For example, a pair of (query, response, query vector) may be stored in the database of the server 200. The encoder described above can be used to generate the query vector. The server 200 may generate a query vector for the query using the encoder described above.

Referring to FIG. 11, the vector-based search method of the server 200 will be described in more detail. The server 200 may preprocess the query (S510). In one embodiment, all uppercase letters may be converted to lowercase letters. Stop words can be eliminated. For example, stop words such as a, an, and the can be removed. In addition, you can go through a morphological analysis process.

Next, the server 200 vectorizes the query (S520). The encoder 110 model described above can be used for vectorization. The device may use the encoder 110 model to generate vector values of the query.

Next, the server 200 adds the vector to the database (S530). Accordingly, the database may include queries, responses, and vector values for the queries.

Accordingly, the server 200 can perform a query search based on the query vector (S540). For example, the server 200 may search for the same query based on the text of the query entered by the user, or may present queries containing similar keywords as search results based on the text.

Additionally, the server 200 may vectorize the query entered by the user, search the database based on the generated vector, and present a query with the same or similar vector value to the user. Vector similarity can be judged by distance or cosine similarity in Euclidean space.

Meanwhile, the server 200 may receive a vector value from the terminal 100. In this case, the server 200 may search the database based on the input vector value and provide the terminal 100 with a pair of queries and responses having the same or similar vector values.

The search method according to an embodiment described above may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured depending on the embodiment, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Optical media (magneto-optical media) and a specially configured hardware platform management server to store and execute program instructions such as ROM, RAM, flash memory, etc. may be included. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

Each of the drawings referenced in the description of the preceding embodiment is merely an example shown for convenience of explanation, and the items, content, and images of information displayed on each screen may be modified and displayed in various forms.

The present invention has been described with reference to an embodiment shown in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the attached registration claims.

Claims (9)

In a search method performed by an electronic device,
generating a user query vector for a user query;
Obtaining a group of similar query candidates including at least one similar query candidate selected based on similarity to the user query vector - the similar query candidates are selected as query responses included in the database of the server through a first similarity analysis model. The similarity between the query vector included in the pair and the user query vector is analyzed and selected based on the first similarity analysis result, and the similar query candidate group includes a similar query candidate, a query vector corresponding to the similar query candidate, and a query vector. Includes similarity values between the user query vectors;
Analyzing the similarity between the user query vector and a query vector corresponding to a similar query candidate included in the similar query candidate group through a second similarity analysis model, and determining a similar query based on the second similarity analysis result; and
Receiving a response to the similar query from the server;
determining a response to the user query based on responses to the received similar query; and
Including providing the determined response to the user,
The first similarity analysis model and the second similarity analysis model have different neural network model configurations,
The second similarity analysis model is learned using first training data and second training data,
The first learning data consists of a first query pair and a label indicating that the first query pair is similar, the first query pair is composed of similar words and structures, so that the sentence format is similar to each other, and the meaning of each query consists of similar queries,
The second learning data consists of a second query pair and a label indicating that the second query pair is dissimilar, and the second query pair is composed of similar words and structures so that the sentence format is similar to each other, but the meaning of each query is similar. consists of different queries,
How to search.
According to claim 1,
The above similar query candidates are:
Obtained by determining as the similar query candidate a query vector contained in a database located within a predetermined distance from the user query vector in Euclidean space,
How to search.
According to claim 1,
The above similar query candidates are:
Obtained based on cosine similarity between the user query vector and the query vector included in the database,
How to search.
According to claim 1,
The step of providing the determined response to the user is,
Further comprising providing the user with the determined response and a similar query corresponding to the determined response,
How to search.
According to claim 1,
The electronic device includes a database,
In the database, a query and a query vector for the query are stored in association.
How to search.
According to claim 1,
The above similar query candidates are:
Obtained by determining a query associated with a query vector whose similarity with the user query vector is higher than a preset standard as the similar query candidate,
How to search.
According to claim 1,
The step of determining the similar query is,
determining a similarity ranking of similar query candidates belonging to the similar query candidate group based on a second similarity analysis result between the query vector corresponding to the similar query candidate and the user query vector; and
Comprising the step of selecting a predetermined number of similar query candidates according to the similarity ranking as a similar query,
How to search.
processor; and
Includes a memory that stores instructions executed by the processor,
By executing the instruction, the processor
Generate a user query vector for the user query,
At least one similar query candidate selected based on similarity to the user query vector - the similar query candidate is selected through a first similarity analysis model, and the query vector included in the question-answer pair included in the database of the server and the user query The similarity between vectors is analyzed and selected based on the first similarity analysis result, and the similar query candidate group includes a similar query candidate, a query vector corresponding to the similar query candidate, and a query vector and the user query vector. The similarity value between the two is included - to obtain,
Analyzing the similarity between the user query vector and a query vector corresponding to a similar query candidate included in the similar query candidate group through a second similarity analysis model, and determining a similar query based on the second similarity analysis result,
Receiving a response to the similar query from the server,
Based on the response to the received similar query, determine a response to the user query,
Provide the above-determined response to the user,
The first similarity analysis model and the second similarity analysis model have different neural network model configurations,
The second similarity analysis model is learned using first training data and second training data,
The first learning data consists of a first query pair and a label indicating that the first query pair is similar, the first query pair is composed of similar words and structures, so that the sentence format is similar to each other, and the meaning of each query consists of similar queries,
The second learning data consists of a second query pair and a label indicating that the second query pair is dissimilar, and the second query pair is composed of similar words and structures so that the sentence format is similar to each other, but the meaning of each query is similar. consists of different queries,
Search device.
A computer-readable recording medium recording a program for executing the method according to any one of claims 1 to 7 on a computer.