KR20240025728A - Query processing method and system - Google Patents

Abstract

The present invention relates to a query processing method and system for outputting a response according to the user's query intent. The query processing method according to the present invention includes receiving a user query from a user, using a query normalization model learned to generate a normalized sentence or word using at least one sentence as input, and responding to the user query. It may include generating a normalized query, generating a response corresponding to the normalized query, and providing the response to the user.

Description

Query processing method and system {QUERY PROCESSING METHOD AND SYSTEM}

The present invention relates to a query processing method and system for outputting a response according to the user's query intent.

The dictionary meaning of artificial intelligence is a technology that realizes human learning ability, reasoning ability, perception ability, and natural language understanding ability through computer programs. Artificial intelligence has made rapid progress thanks to deep learning.

In particular, thanks to the development of artificial intelligence, various language models have been developed. These language models not only recognize text and understand its meaning, but also extract information from data containing a large amount of text such as documents, It has reached the level of classifying and directly generating text.

These language models are actively used in various fields, for example, search services, document writing (ex: resume writing, report writing, posting writing, etc.), free conversation on various topics, and data parsing from a given text ( There are a variety of fields that can be performed based on text, such as data summarization, classification, etc.), providing expert knowledge, programming, converting a given sentence into an appropriate style of sentence, etc.

Meanwhile, in the case of a search service, it is important to receive a query from a user and derive an accurate correct answer (or response) that matches the intention of the user's query. In particular, as technology develops, means and situations for receiving user inquiries have also diversified. Recently, many technologies have been developed to specify the user's query, identify the user's query intention, and derive an appropriate answer to the user's query in a conversation situation between the agent and the user using an interactive agent. Research is being done.

At this time, various studies are being conducted to understand the user's query intention, and research is needed on how to identify the user's exact query intention by actively utilizing a language model composed of a large amount of data.

The purpose of the present invention is to provide a query processing method and system that can specify the user's exact query intent.

Furthermore, the purpose of the present invention is to provide a query processing method and system that can determine the user's query intention based on the conversation context between the user and the agent.

Furthermore, the present invention is intended to provide a query processing method and system that can generate a normalized query that reflects the user's query intent.

Furthermore, the present invention is intended to provide a query processing method and system that can generate a normalized query corresponding to a user's query using a language model.

Furthermore, the present invention is intended to provide a method of learning a language model to generate a normalized query according to the user's query intention.

In order to solve the problems described above, the query processing method according to the present invention includes the steps of receiving a user query from a user, and a query normalization model learned to generate one normalized sentence or word using at least one sentence as input. It may include generating a normalized query corresponding to the user query, generating a response corresponding to the normalized query, and providing the response to the user.

The query processing system using a language model according to the present invention uses a query receiver that receives a user query from a user and a query normalization model learned to generate one normalized sentence or word using at least one sentence as input, It may include a query normalization unit that generates a normalized query corresponding to the user query.

A program executed by one or more processes in an electronic device according to the present invention and stored in a computer-readable recording medium includes the steps of: receiving a user inquiry from a user; and instructions for generating a normalized query corresponding to the user query using a query normalization model learned to generate a normalized sentence or word using at least one sentence as input. there is.

The method for providing a search service according to the present invention includes receiving a user voice from a user terminal, obtaining a user query corresponding to the user voice based on STT conversion, and responding to the user query from a pre-learned language model. It may include obtaining a corresponding normalized query and providing a search page containing search results related to the normalized query to the user terminal.

As seen above, the query processing method using the language model according to the present invention can identify the user's query intention and generate a normalized query that reflects the user's query intention. In this way, the query processing method using the language model according to the present invention can search for the correct answer (or response) corresponding to the user's query using the normalized query intent.

1 and 2 are conceptual diagrams for explaining a query processing method and system using a language model according to the present invention.
Figures 3 and 4 are conceptual diagrams for explaining normalized queries.
Figures 5 to 8 are conceptual diagrams for explaining learning data.
11 to 14 are conceptual diagrams illustrating examples of use of the query processing method and system according to the present invention.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings. However, identical or similar components will be assigned the same reference numbers regardless of drawing symbols, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but that other components may exist in between. It should be. On the other hand, when a component is said to be “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

1 and 2 are conceptual diagrams for explaining a query processing system according to the present invention, and FIGS. 3 and 4 are conceptual diagrams for explaining a normalized query. Furthermore, Figures 5 to 8 are conceptual diagrams for explaining learning data.

An agent may receive a query from a user and output an appropriate answer corresponding to the received query to the user.

As shown in FIG. 1, the agent is included as a function of various types of electronic devices 20, or provides various services through which user inquiries can be made, such as search services, consultation services, information provision services, virtual assistant services, etc. It may be included as a function of a provided website, application, or software.

As an example, the agent may receive a query from the user (USER) in the form of a speech through a speaker (20b) equipped with a voice recognition function, and provide the user with an appropriate response to the query. In this case, a conversation 40 in the form of speech may occur between the agent and the user (USER).

As another example, the agent receives an inquiry from the user (USER) through a chat with the user (USER) through an electronic device equipped with a display unit, such as a smartphone (20a), tablet, TV, etc., and responds appropriately to the inquiry. can be provided to the user.

As another example, the agent may receive a query in voice or text format through a website or application that provides a search service, such as Naver or Google. Meanwhile, in order to provide an appropriate response to a user's inquiry, it is important to accurately understand the intention of the user's inquiry. Accordingly, the present invention provides a method for normalizing user queries. In the query processing method and system according to the present invention, normalization can be performed to generate a user's query into a single sentence (or one sentence).

In the query processing method and system according to the present invention, the user query received through the agent is input to the query normalization unit 120 including the query normalization model 130, and as the output of the query normalization unit 120, the user query You can obtain a normalized query corresponding to .

Furthermore, in the query processing method and system according to the present invention, a response (correct answer or search result, etc.) to a normalized query can be obtained from the service server 30, and the obtained response can be provided to the agent. That is, the normalized query can be delivered to the service server 30 through the query processing system 100, and a response to the normalized query can be obtained through the service server 30. Meanwhile, the obtained response can be delivered to the agent and used as the agent's output in response to the user's query.

As shown in Figures 1 and 2, the query processing system 100 according to the present invention may include a query reception unit 110 and a query normalization unit 120.

The query receiving unit 110 of the query processing system 100 may be configured to receive a user query from an agent. Meanwhile, it is possible for user inquiries to be received in at least one of voice and text forms.

Next, the query normalization unit 120 can perform a series of controls to obtain a normalized query by performing normalization on the user query.

The query normalization unit 120 may include a query normalization model 130. The query normalization unit 120 may input a user query into the query normalization model 130. Additionally, the query normalization unit 120 may obtain a normalized query for the user query from the query normalization model 130.

The query normalization model 130, which generates a normalized query for a user's query, is composed of a language model for natural language processing and has no restrictions on its type. The query normalization model 130 is trained to generate one normalized sentence or word using at least one sentence as input.

As shown in FIG. 2, the query normalization model 130 can use a pre-trained language model (140) that has already been trained on various information. At this time, the previously learned language model may be a very large language model in which a deep learning model consisting of hundreds of millions of parameters or more is trained with large-scale data. In this case, the learning data is used to generate normalized queries to improve performance. A prompt encoder (150, Prompt Encoder) can be used for low-fine tuning. The p-tuning method using the prompt encoder will be described later.

The query processing system 100 may use the query normalization model 130 of the query normalization unit 120 to generate a normalized query for the user query by considering the context of the conversation between the user and the agent. Specifically, the query processing system 100 inputs not only the user query that is the target of normalization, but also at least a portion of the conversation conducted through a conversation session formed between the agent and the user into the query normalization model 130 to normalize the user query. You can create a query.

At this time, the user query subject to normalization may be the last sentence of the input. Queries can be divided into a plurality of query types. Among the plurality of query types, the first query type is an interactive query type, and the second query type is a search query type.

The query normalization model 130 is trained based on learning data including examples for each of a plurality of query types, and can generate a normalized query according to the user's query type. That is, the query processing system 100 can generate a normalized query according to the user's query type using the query normalization model 130 learned by learning data including examples of each of a plurality of query types. .

The interactive query type can be understood as a type that performs normalization on user queries by considering the context of the conversation between the agent and the user.

Interactive query types can be divided into multiple utterance types. The first utterance type is a type in which the user's query is related to the content of the previous conversation, and can also be expressed as a “continuous utterance in context.” That is, the first utterance type may be a type in which the last utterance of the user specified by the user query is related to the context of the previous conversation. In this case, the query processing system 100 may generate a normalized query by considering the context of the previous conversation as well as the user query. For example, (a) to (c) of Figures 3 may be a context continuous utterance type corresponding to the first utterance type among the interactive query types corresponding to the first query type.

For example, as shown in Figure 3(a), while a conversation is taking place between a user and an agent about the characteristics of a “turtle” (301), the user asks, “Is it a carnivore?” You can perform the utterance. In this case, the user's query did not specify the subject of the question, that is, “who” the carnivore was. However, in the flow of conversation between the user and the agent, the user would have made the utterance with “turtle” (301) in mind, so the normalized query reflecting the user’s intention would be “Are turtles carnivores?” It can be.

For example, in the user query, at least one of the subject, predicate, object, complement, adjective, and adverb that constitute sentence components may be omitted. The query processing system 100 may supplement at least one of the omitted sentence components using the context of a previous conversation to generate a normalized query capable of generating a response according to the intention of the user's query. As a result, as shown in (a) of Figure 3, the normalized query for the last user query in the conversation between the user and the agent includes the word “turtle” 303 included in the previous conversation content and the user query. May contain the word “carnivore.”

Meanwhile, the contents according to the omitted sentence components include outline or meaning request (ex: What is the content? What does it mean?), hexadecimal principle (ex: who, when, where, what, how, why), comparison (ex: It may be used to specify requests (differences, similarities), type (ex: is there something) request, request for additional information (ex: tell me in more detail, tell me more, tell me more easily, tell me something else, etc.) .

As another example, as shown in (b) of Figure 3, a conversation was held between the user and the agent about “how to make electricity,” “how to generate electricity,” and “wind power plant,” and as the user’s last utterance, “ “Does it look like a fan?” A user inquiry was received. At this time, the query processing system 100 considers the user query's previous conversation context and determines whether the user's intention to ask a question about whether the “wind power plant” (311) looks like an “electric fan” (312), that is, the shape of the wind power generator. You can. As a result, the query processing system 100 may combine the content 314 of the user query and the content 313 of the previous conversation to generate a normalized query, “Does a wind turbine look like a fan?”

As a similar example, for the user query “Please tell me the time of the first bus,” which corresponds to the user’s last utterance in Figure 3 (c), the query processing system 100 considers the context of the previous conversation and asks, “The first bus of Gwangju 1187 bus.” You can create a normalized query like “Tell me the time.”

Meanwhile, among the plurality of utterance types of the interactive query type corresponding to the first query type, the second utterance type is a type in which the user query is not related to the content of the conversation between the user and the agent, and is also called a “context disconnected utterance.” can be expressed. That is, the second utterance type may be a type in which the last utterance of the user specified by the user query is not related to the context of the previous conversation. In this case, the query processing system 100 may consider only the user query and not consider the context of the previous conversation.

For example, as shown in (d) of Figure 3, the user query corresponding to the user's last utterance, “How is the weather today?” Looking at the previous conversation context, a conversation was taking place between the user and the agent about elementary school study methods. In this case, the user query “How is the weather today?” is not relevant to the preceding context. Therefore, in this case, the query processing system 100 may output “How is the weather today?” as a normalized query, which is the same as the user query.

Likewise, as shown in (e) of Figure 3, the user query corresponding to the user's last utterance, “What was the soccer result in our country yesterday?” Looking at the previous conversation context, a conversation was taking place between the user and the agent about space and shooting stars. In this case, the user query is not relevant to the preceding context. Therefore, in this case, the query processing system 100 can output “What was the soccer result in Korea yesterday?” as a normalized query, which is the same as the user query. Alternatively, it can be normalized into a sentence that is more suitable for the service, such as “Please tell me the soccer results of my country yesterday.”

The search-type query type may correspond to a type that changes to a search-type query for a search service. The search-type query type may be divided into a plurality of utterance types, where the first utterance type may correspond to a single utterance for information search, and the second utterance type may correspond to multiple utterance types.

The single utterance type may be a case where a single utterance from a user is received to search for information, as shown in Figures 4 (a) to (c). In this case, the user's utterance may soon become a user query.

For example, as shown in (a) of FIG. 4, in response to a user query, “When do you eat cherries?”, the query processing system 100 generates a normalized query called “cherries in season,” and “currently.” In response to a user query, “Tell me a rib restaurant near the location,” the query processing system 100 may generate a normalized query called “Nearby rib restaurant.” Likewise, in response to a user query, “I want to watch a video, but it doesn’t come out well,” the query processing system 100 may generate a normalized query called “video playback error.” In this way, the query processing system 100 can generate a normalized query centered on nouns, which is a form that can be easily searched.

The multiple utterance type may be a type that generates a normalized query for the user query using the contents of the previous context, similar to the context continuous utterance, which is the first utterance type among the first query types discussed above. The query normalization model 130 can be trained to generate a normalized query by recombining additional words (slots or intents) from the user query appearing in the previous context with the user query.

In this way, if the user's query that is the target of normalization is related to the previous user's query, the query processing system 100 may generate a normalized query using the previous user's query.

For example, as shown in (d) of FIG. 4, the query processing system 100 processes multiple user queries corresponding to multiple utterances: “pension lottery winning date” (401), “tell me the number 62” (402) ), the normalized user query “Pension Lottery 62nd winning number” can be generated.

For another example, as shown in (e) of FIG. 4, the query processing system 100 processes multiple user queries corresponding to multiple utterances: “BMW 520i” (411), “Tell me the price of a used car” (412) Based on this, a normalized user query “BMW 520i used car price” can be created.

The query normalization model 130 is trained to generate normalized queries using learning data for the various query types and utterance types discussed above.

The query normalization model 130 can generate a normalized query using the previous conversation context for a user query corresponding to the context continuous speech type among the interactive query types. In addition, the query normalization model 130 can be learned using learning data configured to generate the user query as a normalized query without considering the previous context for the user query corresponding to the context-disconnected utterance type among the interactive query types. there is.

In addition, the query normalization model 130 can generate a normalized query centered on nouns for a user query corresponding to a single utterance type among search query types. Furthermore, the query normalization model 130 can be trained using learning data configured to generate a normalized query by considering previous user queries for user queries corresponding to multiple utterance types among search query types.

As described above, the query normalization model 130 can use the super-large learning language model 140, and the P-tunning learning method can be used for additional learning based on new data for normalization.

In one embodiment of the present invention, the pre-trained language model 140 is learned using hundreds of billions of token data based on the Transformer structure, and consists of T tokenized consecutive tokens x=(x1, x2,... , xT) may follow a probability distribution as shown in Equation 1 below (θ: parameter of the pre-learning language model 140).

[Equation 1]

The prompt encoder 150 follows Equation 2 below for n prompt tokens p=(p1,...,pn) defined in the template.

[Equation 2]

The vector h = (h1, ..., h2) encoded by the prompt encoder 150 is replaced for each position of the prompt token p = (p1, ..., pn) in the template as an input to the previously learned language model 140. I go in. At this time, ReLU can be used as the activation function.

In the present invention, in order to learn the query normalization model 130, the input is defined by mapping the input value x and the vector h extracted from the prompt encoder 150 to a defined P-tunning template, and K pairs are created For the response token y=(y1, …, yk), follow the objective function as shown in Equation 3 below.

[Equation 3]

At this time, the parameter θ of the previously learned language model 140 is fixed, and only the parameter ø of the prompt encoder 150 is learned.

Meanwhile, a plurality of prompt tokens (60, for example, [PROMPT 1], [PROMPT 2], etc.) may be placed in the prompt input to the query normalization model 130. For example, as shown in FIG. 2, a prompt may consist of a prompt token 60, dialogue examples 510, 520, and a correct answer (normalized query, 530). At this time, the query normalization model 130 is trained to specify the last user utterance 520 of the conversation example as a user query and to generate the correct answer 530 as a normalized query for the user query.

The query normalization model 130 receives a conversation sequence containing at least two or more sentences, considers the context of previous sentences of the last sentence in the conversation sequence, and is trained to generate a normalized sentence for the last sentence. .

The query normalization model 130 can be learned using a learning data set including the conversation sequence and a normalized sentence pair for the last sentence of the conversation sequence.

Meanwhile, the training data set includes at least one of a first type of training data and a second type of training data, and the normalized sentence included in the first type of training data is a component omitted from the last sentence of the conversation sequence. It can be constructed by being filled in using at least some of the previous sentences. At this time, the omitted component is at least one of the subject, object, and predicate indicating the intention of the query.

Furthermore, the normalized sentences included in the second type of learning data may be configured with the last sentence changed to a form corresponding to a specific service.

The first type of learning data is configured so that the contexts are correlated between the last sentence and the previous sentences, and the second type of learning data is configured so that the contexts are correlated between the last sentence and the previous sentences. It can be configured not to do so.

Meanwhile, the query normalization model 130 may perform learning for each query type based on learning data corresponding to each different query type.

For example, the training data includes training data that is a conversation set (or conversation example) according to an interactive query type corresponding to the first query type, and a conversation set (or conversation example) according to a search query type corresponding to the second query type. It may include at least one of the learning data (example).

Furthermore, the learning data according to the interactive query type corresponding to the first query type is the learning data that is a conversation set (or conversation example) corresponding to the first utterance type (context continuous utterance type) and the learning data that is a conversation set (or conversation example) corresponding to the second utterance type (context disconnected utterance type). It may include at least one of the learning data that is a conversation set (or conversation example) corresponding to the type.

Furthermore, the learning data according to the search query type corresponding to the second query type is the learning data that is a conversation set (or conversation example) corresponding to the first utterance type (single utterance type) and the second utterance type (multiple utterance type). It may include at least one of the learning data that is a conversation set (or conversation example) corresponding to .

Here, the conversation set may correspond to the conversation examples 510 and 520 seen above in FIG. 2, and the query set may correspond to the user's last utterance (user query, 520) and the correct answer 530 seen above.

Figure 5 is an example to explain the learning data template structure. The learning data may be configured to include a learning target conversation set (602, 605, ...) and a learning target query set (603, 606, ...) for each conversation example. Additionally, the learning data may be configured to include a plurality of prompt tokens (601, etc.).

The learning target conversation sets 602, 605, ... of the learning data may be configured to include at least one user query and a plurality of sentences each corresponding to a response to the at least one user query. And, the learning target query set (603, 606, ...) is the learning target query that is the target of normalization learning (ex: “What is the most exciting album?”, “Who was the US president at that time?”, etc.) and the learning target query set (603, 606, …). A plurality of sentences each corresponding to a normalized learning target query (ex: “What is the most exciting album among Love poem, Palette, CHAT-SHIRE?”, “Who is the President of the United States in 2018?”, etc.) It may be configured to include.

Meanwhile, as shown, the learning data is configured to include a plurality of prompt tokens, and a portion 601 of the plurality of prompt tokens may be placed above at least one conversation set 602 to be learned. And, another part 604 of the plurality of prompt tokens may be placed in front of the normalization query.

Meanwhile, the example in FIG. 5 is learning data corresponding to the context continuous utterance type. In this case, the learning data may be configured so that the contexts between the sentences constituting the learning target conversation set 602 and the learning target query set 603 are interrelated. Normalized queries (learning target normalized queries) included in the learning target query set 603 (ex: “What is the most exciting album among Love poem, Palette, CHAT-SHIRE?”, “Who is the President of the United States in 2018?”, etc. ) may be configured to include at least some of the words constituting a plurality of sentences included in the learning target conversation set of the corresponding conversation set.

Furthermore, the learning target normalization query corresponding to the first utterance type may further include at least some of the words constituting the sentence corresponding to the learning target query (user query). Through this, the query normalization model 130 considers the context of the conversation before the user query in the case of the first utterance type (contextual continuous utterance type) among the interactive query types corresponding to the first query type, and the contents of the previous conversation. and the contents of the user's query can be combined to learn to generate a normalized query.

Meanwhile, although not shown, the learning data corresponding to the second utterance type (context disconnected utterance type) among the interactive query types corresponding to the first query type includes the learning target conversation set (or conversation example) and the learning target. The context between sentences constituting each query set (or user query) may be configured so that there is no correlation between them. In this case, the learning target normalization query corresponding to the second utterance type may be configured to be the same as or correspond to the sentence corresponding to the learning target query.

The query normalization model 130 is created from learning data corresponding to the second utterance type (context disconnected utterance type) among the interactive query types corresponding to the first query type, the previous conversation before the user query is received, and the context between the user query. You can learn that there is no relevance. And, as a learning result, when a conversation or user query corresponding to a context-disconnected utterance is received, the user query as is or a sentence corresponding to the user query can be output as a normalized query without considering the context according to the content of the conversation. there is.

Next, although not shown, the learning data corresponding to the first utterance type (single utterance type) among the search-type queries corresponding to the second query type include the user query and the words (or similar meanings) included in the user query. It can be configured to include a normalized query consisting of words of . At this time, the normalized query may be configured to include noun-oriented words, and a plurality of prompt tokens may be placed between the user query and the normalized query.

Furthermore, the learning data corresponding to the second utterance type (multiple utterance types) among the search-type query types corresponding to the second query type may be configured as shown in FIG. 6. Like the city, the learning data can be structured to include multiple user queries corresponding to multiple utterances (“The third king of Joseon -> “And then”, 701) and normalized queries for them, and the normalized queries (“The fourth king of Joseon”) may have multiple prompt tokens 702 placed there.

The learning data corresponding to the second utterance type (multiple utterance types) among the search-type queries corresponding to the second query type may be configured to include normalized queries reflecting the contents of a plurality of user queries. Through this, the query normalization model 130 can generate a normalized query by referring to previous user queries when a user query for multiple utterance types is received.

As described above and as shown in FIGS. 7 and 8, the query normalization model 130 according to the present invention includes learning data including the correct answer for various user query situations according to each query type and utterance type. It can be learned through.

As shown in (a) of Figure 7, the learning data (ex: first type of learning data) corresponding to the first utterance type (contextual continuous utterance type) among the interactive query types corresponding to the first query type includes , a normalized query generated considering the context of the conversation can be included as the correct answer.

As shown in (a) of Figure 7, the learning data corresponding to the context-continuous utterance type is structured so that there is a correlation between the context of the conversation between the user and the agent and the user's inquiry (“What is the most exciting album?”) It can be done. At this time, the user query may be the last sentence of the conversation, and the correct answer corresponding to the normalized query may be constructed by taking the context of the conversation into consideration. Even if the last sentence corresponding to the user's query omits at least some of the sentence components, the correct answer may be constructed to fill in the omitted sentence components from the conversation.

The correct answer can be constructed to fill in the sentence components omitted in the user query using at least part of the content of the conversation, and as a result, the normalized query corresponding to the correct answer is “The most exciting among Love poem, Pallette, and CHAT-SHIRE.” It may consist of “What is an album?”

Meanwhile, as shown in (b) of FIG. 7, learning data (ex: learning data of the second type) corresponding to the second utterance type (context disconnected utterance type) among the interactive query types corresponding to the first query type ), the context of the conversation is not considered, and a normalized query for the user's last utterance (user query) may be included as the correct answer.

Learning data corresponding to context-disconnected utterance types may consist of various categories of learning data.

An example of a context-disconnected utterance type is a case in which a response to a different domain is requested from the conversation preceding the user inquiry (category: request for another domain). For example, as shown in (b) of FIG. 7, a user query and a conversation between a user and an agent prior to the user query may be configured to have different domains (or topics). Like the city, the domain of the last sentence of the conversation corresponding to the user query, “Where did people get married in the past?” is “wedding,” and the domain of the content of the conversation before the user query is “traditional customs.” It becomes a context-severe utterance.

As another example, the user's query may correspond to an unhealthy utterance (ex. Unhealthy utterance unrelated to the content of the previous conversation, such as Shut up, are you kidding me?) (Category: Unhealthy utterance).

Another example, another example, corresponds to chat-like utterances such as utterances for responding to an agent's response or for simple conversation (ex: That's fascinating, I see, thank you for letting me know, I know, you're smart, thank you, etc.) This may be the case (Category: Chat). As another example, it may be an utterance to stop a conversation with an agent (ex: stop, stop telling me now, okay, okay, etc.) (category: request to stop).

In the case of this context-disconnected utterance type, the normalized query corresponding to the correct answer may be the same as the user's query, or may be a sentence that is the same or similar to the content of the user's query and is more suitable for the service.

As shown in (a) of Figure 8, the learning data (ex: learning data of the second type) corresponding to the first utterance type (single utterance type) among the search-type query types corresponding to the second query type, includes: A normalized query consisting mainly of nouns may be included as the correct answer to the user's query. And, as shown in (b) of FIG. 8, learning data (ex: first type of learning data) corresponding to the second utterance type (multiple utterance type) among the search-type query types corresponding to the second query type. , a normalized query generated by considering the context of the conversation (the user's multiple questions) may be included as the correct answer.

The query processing system 100 according to the present invention uses the query normalization model 130 for various queries and utterance types, and normalizes the user's query to suit the user's intention in a conversation situation between the user and the agent. can do.

Below, as discussed above, a method of processing user queries in a query processing system that normalizes user queries using the query normalization model 130 will be examined in more detail with the accompanying drawings. Figures 9 to 14 are conceptual diagrams for explaining usage examples of the query processing method and system according to the present invention.

Referring to FIG. 9, in the query processing method according to the present invention, a conversation session is formed between the agent and the user (S1010). Next, in the present invention, a process of receiving a user query corresponding to at least a part of a conversation according to a conversation session is performed in the query reception unit 110 of the query normalization system 100 (S1020). As discussed above, the user's query may be specified as the user's last utterance during a conversation according to the conversation session. For example, as shown in FIG. 10, among the conversations (1111, 1112, 1113, 1114, 1115) between the user (USER) and the agent 20, the user's last utterance (1115, “What about Iron Man?”) , can be specified as a user query.

Next, in the present invention, a process of generating a normalized query corresponding to the user query can be performed using the query normalization model 130 learned based on learning data (S1030). The query processing system 100 specifies the last utterance of the user during a conversation according to the conversation session as the user query, and normalizes the last utterance of the conversation corresponding to the user query into a single sentence, thereby generating the normalized query. can be created.

The query processing system 100 determines that when the last utterance is related to the context of a conversation before the last utterance among conversations according to the conversation session, at least some of the plurality of words constituting the normalized query are before the last utterance. User queries can be normalized so that they relate to the content of the conversation.

More specifically, if the last utterance is related to the context of the conversation before the last utterance among the conversations according to the conversation session, the query processing system 100 selects the part of a plurality of words constituting the conversation before the last utterance and the The normalized query can be generated by combining some of the plurality of words that make up the last utterance.

For example, a conversation like the one shown in Figure 10 is a situation where a question is asked about the rating of a movie. In this situation, the user's last utterance (1115) corresponding to the user query, “What is Iron Man?” includes the movie title. In this case, the query normalization model 130 may perform normalization on the last utterance by considering the context of the previous conversation. In this case, the query processing system 100 uses the query normalization model 130 to form a normalized query, “What is the movie rating for Iron Man?” can be created.

On the other hand, if the last utterance corresponding to the user's query is not related to the context of the conversation before the last utterance among the conversations according to the conversation session, the query processing system 100 normalizes the content related only to the last utterance. You can create queries. In this case, the normalized query may be identical to the user's last utterance.

The query processing system 100 may generate a response corresponding to the normalized query using the service server 30 (see FIG. 1).

For example, if the service server 30 linked to the query processing system 100 corresponds to a search engine, the query processing system 100 may transmit a normalized query to the search engine. Furthermore, when search results are obtained from a search engine, the search results may be transmitted to the agent. The agent may provide search results to the user as a response to a normalized query.

For example, if the agent is a voice-based conversational agent like the city, the response may be output as the agent's utterance 1116. And, if the agent is a chatbot-type agent, the response may be provided to the user's terminal in the form of a chat.

In this way, the query processing system can obtain a response (ex: search result) to a normalized query corresponding to the user query from the service server 30 and process it as a response to the user query. At this time, the response to the normalized query may be output as an utterance of the agent.

Meanwhile, the query processing system 100 may perform additional processing to provide a better response to the user.

As an example, the query processing system 100 may obtain responses to each of a normalized query and a user query (ex: a user query subject to normalization) from the service server 30 and perform scoring. At this time, scoring can be understood as a score indicating which response among the response to the normalized query and the response to the user query better reflects the intention of the user query. Scoring methods can vary, and there are no special restrictions.

The query processing system 100 may determine a response with a higher score among the scoring results, the response to the normalized query, and the response to the user query as the final response. And, the determined final response can be processed as the agent's utterance. That is, the user may be provided with the final, determined response.

As an example different from the above example, the query processing system 100 may generate a plurality of normalized queries for user queries. Additionally, the query processing system 100 may obtain a response from the service server 30 for each of a plurality of normalized queries and perform scoring on the obtained responses. As a result of the scoring, the response with the highest score may be selected as the response to the user inquiry.

As another example, the query processing system 100 obtains responses to user queries as well as responses to each of a plurality of normalized queries from the service server 30, and performs scoring on the obtained responses. You can. As a result of the scoring, the response with the highest score among the responses to each of the plurality of normalized queries and the response to the user query may be selected as the response to the user query.

Through post-processing processes such as the examples examined above, a response that best reflects the intention of the user's inquiry can be provided to the user.

Next, we will look at examples of various services provided by an agent that works with the query processing system 100 according to the present invention.

11 to 14 are examples of a search service that receives a user query through an agent providing a search service, normalizes the user query, and provides a response to the normalized query. As shown in FIGS. 11 to 14, the search service may be provided through pages 1210, 1310, 1410, and 1510 that provide the search service on the display unit of the user terminal. At this time, the user query can be received in various ways, such as through voice recognition or through text input through the query input areas 1220, 1320, 1420, and 1520.

The agent may receive a user query and transmit it to the query processing system 100. Additionally, the query processing system 100 can perform normalization on user queries using a language model learned based on learning data according to the search-type query type.

For example, as shown in (a) of FIG. 11, the query processing system 100 determines that the user's search intention is It can be identified as a “spelling checker” and output as a normalized query. As a result, the search result 1250 according to the user query and the search result 1260 according to the normalized query are different, and the search result according to the normalized query can better reflect the user's intention.

Meanwhile, the agent receives a normalized query from the query processing system 100 and outputs the received normalized query 1240 to the query input area 1220, as shown in (b) of FIG. 11. You can. Accordingly, the user can recognize that a search has been performed for the normalized query 1240.

Meanwhile, the case in which a normalized query is output to the query input area 1220 may vary depending on the situation. The agent may output a normalized query to the query input area 1220 instead of a user query only if the reliability of the normalized query satisfies a preset standard. In this case, the reliability of a normalized query refers to the extent to which the normalized query satisfies the user's query intent, and there are no special limitations on the method of evaluating reliability.

For another example, as shown in (a) of FIG. 12, when “Jewel of October” is received as a user query, the query processing system 100 as shown in (b) of FIG. 12 , this can be normalized as “October birthstone”. The query processing system 100 may determine the query intention of the user's query, determine that the user's search intention is “October birthstone,” and output this as a normalized query. As a result, the search result 1350 according to the user query and the search result 1360 according to the normalized query are different, and the search result according to the normalized query can better reflect the user's intention.

Meanwhile, the agent receives a normalized query from the query processing system 100 and outputs the received normalized query 1340 to the query input area 1320, as shown in (b) of FIG. 13. You can. Accordingly, the user can recognize that a search has been performed for the normalized query 1340.

As seen above, the case in which a normalized query is output to the query input area 1320 may vary depending on the situation, and the agent replaces the user query only if the reliability of the normalized query satisfies preset standards. The normalized query can be output to the query input area 1220.

For example, as shown in (a) of FIG. 13, assume that “board riding in the sea” is input as a user query, and “surfing board” is created as a normalization query for the user query. And, let us assume that the reliability of the normalized query did not meet the preset standards. In this case, the agent may not immediately output the normalized query to the query input area 1420. In this case, the agent may provide a normalized query (“surfboard”) to an area 1440 of the search service page 1410, as shown in (a) of FIG. 13. Through this, the user can recognize that a normalized query exists for the user query. And, when a normalized query is selected by the user, the normalized query 1460 may be output to the query input area 1420, as shown in (b) of FIG. 13. Furthermore, search results 1470 for normalized queries may be provided to the user terminal. The normalized query may be displayed in the form of an icon in the area 1440 and selected by the user.

For another example, as shown in Figure 14, assume that in the previous situation, there was a conversation 1500 between the agent and the user. And, let us assume that in the conversation (1500) there was a question about the reign of the “king.” In this case, the agent may transmit the conversation and user query (1530, “King Sejong”) according to the previous conversation situation to the query processing system 100. Additionally, the query processing system 100 may generate a normalized query 1520 that is relevant to the context of the preceding conversation situation, based on the query normalization model 130 learned according to the learning method discussed above. As a result, a normalized query can be created as “Period of King Sejong’s reign.”

As shown in (a) and (b) of FIGS. 14, the search result 1550 according to the user query 1530 may be at least partially different from the search result 1560 according to the normalized query 1540. The search results may further include a search result 1570 that reflects the user's query intention.

In this way, the query processing method and system using the language model according to the present invention can identify the user's query intention and generate a normalized query that reflects the user's query intention. Additionally, the query processing method and system using a language model according to the present invention can search for the correct answer (or response) corresponding to the user's query using a normalized query. Through this, users can obtain the satisfactory search results they intended simply by talking to the agent.

Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored in a medium (or recording medium) that can be read by such a computer.

Furthermore, the present invention discussed above can be implemented as computer-readable codes or instructions on a program-recorded medium. That is, the present invention may be provided in the form of a program.

Meanwhile, computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is.

Furthermore, the computer-readable medium may be a server or cloud storage that includes storage and can be accessed by electronic devices through communication. In this case, the computer can download the program according to the present invention from a server or cloud storage through wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a CPU (Central Processing Unit), and there is no particular limitation on its type.

Meanwhile, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (12)

Receiving a user query from a user;
Generating a normalized query corresponding to the user query using a query normalization model learned to generate a normalized sentence or word using at least one sentence as input;
generating a response corresponding to the normalized query; and
A query processing method comprising providing the response to the user as a response to the user query. According to paragraph 1,
The query normalization model is,
A query processing method characterized by receiving a conversation sequence containing at least two or more sentences, and learning to generate a normalized sentence for the last sentence by considering the context of previous sentences of the last sentence in the conversation sequence. According to clause 2,
The query normalization model is,
A query processing method, characterized in that it is learned using a learning data set including the conversation sequence and a normalized sentence pair for the last sentence of the conversation sequence. According to paragraph 3,
The training data set includes at least one of a first type of training data and a second type of training data,
The normalized sentences included in the first type of learning data are:
It is constructed by filling in the omitted element in the last sentence of the dialogue sequence using at least some of the previous sentences,
The normalized sentences included in the second type of learning data are:
A query processing method characterized in that the last sentence is changed to a form corresponding to a specific service. According to paragraph 4,
A query processing method, wherein the omitted component is at least one of a subject, an object, and a predicate indicating the intention of the query. According to paragraph 4,
The first type of learning data is configured so that the contexts are correlated between the last sentence and the previous sentences,
The second type of learning data is a query processing method characterized in that the context is not correlated between the last sentence and the previous sentences. Receiving a user voice from a user terminal;
Obtaining a user query corresponding to the user voice based on STT conversion;
Obtaining a normalized query for the user query using a query normalization model learned to generate a normalized sentence or word using at least one sentence as input; and
A method of providing a search service using a language model, comprising providing a search page including search results related to the normalized query to the user terminal. In clause 7,
further comprising providing the normalized query to the search page,
The normalized query is provided in different areas of the search page according to the reliability of the normalized query. According to clause 8,
The search page includes a query input area,
If the reliability of the normalized query satisfies a preset standard, providing the normalized query to the query input area and providing search results related to the normalized query to the search page. How we provide services. According to clause 9,
If the reliability of the normalized query does not meet a preset standard, the normalized query is provided as an icon in an area different from the query input area, and the user query is provided in the query input area,
A search service providing method, wherein search results related to the normalized query are provided on the search page when the icon is selected. In the query processing system,
The system is,
a query receiving unit that receives a user query from a user; and
Query processing comprising a query normalization unit that generates a normalized query corresponding to the user query using a query normalization model learned to generate a normalized sentence or word using at least one sentence as input. system. A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium,
The above program is,
Receiving a user query from a user; and
Characterized by comprising instructions for generating a normalized query corresponding to the user query using a query normalization model learned to generate a normalized sentence or word using at least one sentence as input. A program stored on a recording medium that can be read by a computer.

Images