KR102377156B1 - Method and apparatus for chatbot service providing using responses received from open interface - Google Patents

Abstract

In providing a chatbot service, a method and apparatus for constructing an utterance of a chatbot using an answer received through an open interface are provided. A chatbot service providing method according to an embodiment of the present invention includes the steps of receiving a plurality of candidate answers to a user's utterance, filtering the candidate answers, and providing a chatbot answer to the candidate answers that have passed the filtering performing scoring for selection, and transmitting a final answer selected as a result of the scoring to a chatbot client terminal that has transmitted the user's utterance.

Description

Method and device for providing chatbot service using answers received through public interface

The present invention relates to a method and apparatus for providing a chatbot service. More particularly, in providing a chatbot service, it relates to a method and an apparatus for constructing a chatbot utterance using an answer received through an open interface.

The chatbot service is a service that provides a response automatically generated by software logic to the chatbot client in response to a user's utterance input through the chatbot client. Therefore, the software logic for generating an appropriate answer to the user's utterance is one of the most important elements in the chatbot service.

Preferably, the answer is text in natural language format. Accordingly, the software logic for generating the answer includes a Natural Language Generation module. In addition, in order to generate the answer, various techniques such as rule-based logic, machine learning-based logic, and search logic for an external information source are used.

Although answer generation is performed using various technologies, a technology that allows third parties other than the ^chatbot service provider to participate in answer generation is not yet provided. This seems to be because it is difficult to guarantee the quality of the answers provided by the chatbot service if a third party is involved in answer generation.

US registered patent US 8,738,739

The technical problem to be solved by the present invention is to provide a method for providing a chatbot service using an answer received through an open interface of a third party, and an apparatus for providing the method.

Another technical problem to be solved by the present invention is a method and method for providing a chatbot service that guarantees the quality of an answer actually provided to a chatbot client terminal among answers received through an open interface of a third party to a certain level or higher To provide a device to provide.

Another technical problem to be solved by the present invention is to provide a method for providing a chatbot service as a novel advertising medium and an apparatus for providing the method.

Another technical problem to be solved by the present invention is to provide a method for providing a chatbot service and an apparatus for providing the method, which minimizes the chatbot service user's rejection of the advertisement by exposing the advertisement in a form that helps to improve the quality of the answer. will provide

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above technical problem, a chatbot service providing method according to an embodiment of the present invention includes receiving a plurality of candidate answers to a user's utterance, filtering the candidate answers, and passing the filtering. performing scoring to select a chatbot answer based on candidate answers; and transmitting a final answer selected as a result of the scoring to a chatbot client terminal that has transmitted the user's utterance.

In one embodiment, the filtering includes the steps of first filtering by applying a spam filter to the candidate answers, and secondary filtering through review quality analysis on candidate answers that have passed the first filtering including the steps of In this case, the review quality analysis is characterized by filtering based on the value of ((#E + #O) / LEN). However, (#E = the number of entities included in the candidate answer, #O = the number of opinion expressions included in the candidate answer, LEN = the length of the candidate answer).

In an embodiment, the performing of the scoring includes performing the scoring using a correlation with contents of a chatbot service conversation transmitted and received between the chatbot client terminal and the terminal. In this case, the step of performing the scoring using the relevance to the chatbot service conversation content may include performing the scoring by synthesizing all relevance of the conversation contents for each entity of the candidate answers that have passed the filtering. .

In an embodiment, the receiving of the candidate answers includes receiving the candidate answers through an open interface, and receiving information about payment of a cost for increasing the rate given to the candidate answers together with the candidate answers may include In this case, the performing of the scoring may include performing the scoring using a rate assigned to the candidate answer.

In an embodiment, the performing of the scoring may include performing the scoring using a rate granted to the candidate answer according to payment of a cost of the submitter of the candidate answer. In addition, the performing of the scoring may include performing the scoring using a rate given to all candidate answers of the submitter according to the payment of the cost of the submitter of the candidate answer.

In an embodiment, the receiving of the candidate answers may include receiving the candidate answers through a public interface on condition that payment of a fee for receiving the candidate answers is provided. In this case, the performing of the scoring may include performing the scoring using only the relevance to the chatbot service conversation content transmitted and received between the chatbot client terminal and the chatbot client terminal.

1 is a block diagram showing the structure of a chatbot service providing system according to an embodiment of the present invention.
2 is a diagram for explaining that a candidate answer received from a third party is used when a chatbot service is provided according to some embodiments of the present invention.
3A is a flowchart of a method for providing a chatbot service according to another embodiment of the present invention.
3B is a flowchart of a method for providing a chatbot service according to another embodiment of the present invention.
4 is a conceptual diagram for explaining in detail some operations of FIG. 3A or 3B.
5 is a diagram for explaining a first scenario of a user model and a conversation log referenced in some embodiments of the present invention.
FIG. 6 is a conceptual diagram for explaining in detail some operations of FIG. 3A or 3B based on the first scenario of FIG. 5 .
7 is a diagram for explaining a second scenario of a user model and a conversation log referenced in some embodiments of the present invention.
8A and 8B are conceptual diagrams for explaining in detail some operations of FIGS. 3A or 3B based on the second scenario of FIG. 7 .
9 is a block diagram of a chatbot service providing system according to another embodiment of the present invention.
10 is a hardware configuration diagram of an apparatus for providing a chatbot service according to another embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments published below, but may be implemented in various different forms, and only these embodiments allow the publication of the present invention to be complete, and common knowledge in the technical field to which the present invention pertains It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular. The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase.

Hereinafter, some embodiments of the present invention will be described with reference to the drawings.

1 is a block diagram of a chatbot service providing system according to an embodiment of the present invention. As shown in FIG. 1 , the system according to the present embodiment may include a chatbot service server 100 and a candidate answer reception server 200 . 1, the chatbot service server 100 and the candidate response receiving server 200 are shown as physically separate separate devices, but the chatbot service server 100 and the candidate response receiving server 200 are one server device. Note that each may be implemented as an internal software module. The chatbot client may be user terminals 10a, 10b, and 10c such as a smart phone, tablet, notebook, or PC, but it should be noted that the chatbot client may also be a server device 20 that provides other services.

2 is a diagram for explaining that a candidate answer received from a third party is used when a chatbot service is provided according to some embodiments of the present invention. In some embodiments of the present invention, a chatbot service is provided using candidate answers to specific questions received from a third party.

The third party refers to an entity other than the chatbot service provider, and may be, for example, a chatbot service user or an advertiser who wants to issue advertisements for the chatbot service. The candidate response reception server 200 may receive candidate responses to a specific question through the candidate response reception public interface 210 such as a web document. To this end, the candidate answer reception server 200 may include a web server module (not shown) for servicing the web document.

On the other hand, the candidate response reception public interface 210 may not have a specific limitation in access qualification. For example, the candidate answer reception public interface 210 may submit the candidate answer without a membership registration process, or the candidate answer can be submitted only by going through a membership registration process that does not limit subscription requirements to a chatbot service provider or a related party. It may be something that can be submitted. In this regard, the candidate response reception public interface 210 is an “open” candidate response disclosure channel, and by receiving candidate responses from various entities through this open channel, the answers provided by the chatbot service are enriched. effect can be obtained.

In some embodiments of the present invention, the candidate answers for each question received from the terminals of the third party are filtered out, and then output to the chatbot client 10a as a response to the user's utterance among the candidate answers that have passed the filtering. An answer may be selected. Note that the selection of the answer will be performed by the chatbot service server 100 , but the filtering may be performed by the candidate answer reception server 200 or the chatbot service server 100 .

In some embodiments of the present invention, submission of a candidate answer may require payment of a fee for receiving a candidate answer. That is, in order to have an opportunity to output the answer proposed by the user through the chatbot service by submitting the candidate answer, payment of a predetermined cost to the chatbot service provider or the like may be required.

In this case, all submitted candidate answers may be given the same rate. That is, in this case, all candidate responses submitted will be evaluated only in terms of response suitability in the same position. In view of the suitability of the answer, the candidate answer may be compared with at least one of a user utterance, a conversation log, and a user profile.

In some embodiments of the present invention, the questions to be accepted may be automatically differentiated or merged according to the reception performance of the candidate answers. For example, when a candidate answer exceeding the standard value is received with respect to the question "Please recommend a card" shown in FIG. 2 , the candidate answer reception server 200 automatically responds to the question "Please recommend a card" as "a lot of discount benefits" You can decide to receive candidate answers by dividing the questions to be accepted into detailed sub-concept questions such as “Please recommend a card”, “Please recommend a free card with annual membership fee”, and “Please recommend a card that earns a lot of points” there is. According to the decision of the candidate answer reception server 200 , the candidate response reception public interface 210 will display the differentiated question as a candidate response reception target question.

Next, a chatbot service providing method according to another embodiment of the present invention will be described with reference to FIGS. 3A to 8B . The chatbot service providing method according to the present embodiment may be performed by one or more electronic devices. The electronic device may be, for example, a computing device. The computing device may be, for example, at least one of the chatbot service providing apparatus 100 and the candidate answer receiving server 200 described with reference to FIGS. 1 to 2 . Note that at least some of the operations of the chatbot service server 100 and the candidate answer receiving server 200 described with reference to FIGS. 1 and 2 may be included in the chatbot service providing method.

Hereinafter, a method of providing a chatbot service according to another embodiment of the present invention will be described with reference to FIG. 3A. As shown in FIG. 3A , in the chatbot service providing method according to this embodiment, receiving candidate answers for each question through the candidate answer reception open interface (S101), filtering at least some of the received candidate answers to answer Excluding the selection target (S103), performing scoring to select whether to output the candidate answers that have passed the filtering as answers to the user's utterance (S105), using the scoring result to provide an answer It may include a step of making a final selection (S107) and transmitting the selected answer to the chatbot client (S109). The filtering step (S103) and the scoring step (S105) will be described later in detail with reference to FIGS. 4 to 8B .

The step S109 of transmitting the selected answer to the chatbot client is preferably performed by the chatbot service server 100 . In addition, it is preferable that the candidate answer receiving step S101 for each question is performed by the candidate answer receiving server 200 .

According to an embodiment, as shown in FIG. 3A , candidate answer filtering (S103), scoring (S105), and final selection of answers (S107) may be performed as an answer resource management process (S10). According to the present embodiment, the answer to the specific question is predetermined before the user utterance. That is, a user who receives candidate answers, excludes some of them through filtering, performs scoring on the remaining candidate answers, selects answers to specific questions in advance, and matches the specific questions When there is an utterance, the predetermined answer may be transmitted to the chatbot client. According to this embodiment, it is possible to provide a chatbot service with a fast reaction speed.

According to another embodiment, as shown in FIG. 3B , candidate answer filtering ( S103 ), scoring ( S105 ), and final selection of answers ( S107 ) may be performed as the speech processing processor ( S20 ). According to the present embodiment, an answer to a specific question is determined when there is a user utterance. That is, only the operation of receiving candidate answers and recording the received candidate answers is performed as the answer resource management processor (S10), some of them are primarily excluded through filtering, and the remaining candidate answers are scored by , the operation of selecting an answer to the specific question may be performed when there is a user utterance matching the specific question. According to the present embodiment, in determining the answer to the user's utterance, there is an effect that the answer can be determined by reflecting all candidate answers received up to the time of the user's utterance. In addition, in the scoring process, there is an effect that an answer suitable for the conversation situation can be dynamically determined in consideration of the profile information of the user who has input the user's utterance and the log of the conversation transmitted and received through the chatbot service.

Hereinafter, the filtering step S103 will be described in detail with reference to FIG. 4 .

The filtering step (S103) may include a primary filtering step (S103a) of applying a spam filter and a secondary filtering step (S103b) of performing a review quality analysis on candidate answers that have passed the primary filtering.

The step of applying the spam filter (S103a) is to determine whether the candidate answer is spam based on the degree of including a predetermined spam-related keyword, or whether the candidate answer is spam based on a predetermined spam determination rule, or , it may include judging whether the candidate answer is spam by using the machine-learned spam judgment model for judging whether spam is spam or not. For example, it is shown in FIG. 4 that a total of two candidate answers A4 and A5 among a total of five candidate answers 30a are determined to be spam. By applying the spam filter to the received candidate answers, there is an effect that can fundamentally block the selection of reckless advertising phrases irrelevant to the question as answers to the user's utterances.

The review quality analysis step (S103b) is performed on candidate answers that have passed the spam filter. The review quality analysis, for example, as a method for analyzing the quality of review content for a specific product or a specific service, may be understood as a method of evaluating the fidelity of the content of the review content.

In one embodiment, the review quality analysis is characterized by filtering based on the value of ((#E + #O) / LEN), where #E is the number of entities included in the candidate answer, and #O is It is the number of opinion expressions included in the candidate answer, and LEN may be the length of the candidate answer. In Figure 4, the entity is "shopping discount", "traffic discount", "S card company A card", "K card company", "H card company", etc. pre-registered keywords or entity name recognition (Named-Entity Recognition) ) is the name of the entity extracted using the algorithm, and it can point to the object of giving an opinion. Also, in FIG. 4 , the “expression of opinion” refers to an expression indicating an evaluation of the entity. For example, if there is a candidate answer "K card company's card has low annual fee but few benefits", then the entity is "K card company", "annual fee", and "benefit", and "expression of opinion" is "cheap", " It will be two of "less". Also, the LEN value of the candidate answer will be 6, which is the number of spaces. Therefore, the review quality analysis result of the candidate answer will be determined to be (3 + 2) / 6.

Note that, in some embodiments, the "LEN" value may be set according to various criteria, such as the number of texts of a candidate answer and the number of minimum units divided by morphological analysis.

In this embodiment, by performing the review quality analysis on candidate answers that have passed the spam filter, it is possible to prevent an answer that gives no information to the chatbot service user from being provided. FIG. 4 shows that answer A3 has been determined to have low review quality as a result of the review quality analysis and has been filtered out.

Next, scoring may be performed to evaluate whether the candidate answers that have passed the filtering are answers suitable for the user's utterance. In this regard, it will be described with reference to FIGS. 5 to 6 .

For the above scoring, as shown in FIG. 5 , a log 40 of a conversation transmitted and received between the user and the chatbot is analyzed, and as a result, a user model 50 may be constructed. The user model 50 may represent a score for one or more interests of the user. The interest may be performed through keyword analysis on the user's utterance in the conversation log. 5, as the user model 50, a shopping discount of 1 point and a low annual fee of 1 point are shown.

On the premise of the user model 50 of FIG. 5 , it will be described with reference to FIG. 6 that the dialogue content relevance evaluation ( S105a ) is performed as the scoring method ( S105 ). Conversation relevance evaluation (S105a) may include performing scoring using relevance to chatbot service conversation contents transmitted and received between the chatbot client terminal, and in this case, each of the candidate answers that have passed the filtering The above scoring may be performed by synthesizing all relevance of conversation contents for each entity.

When the conversation content relevance evaluation is performed on the candidate answer A1 shown in FIG. 6, since the entity "S card company A card" and the user model 50 do not have a relationship, no points are obtained from "S card company A card", Since the entity "shopping discount" and the user model 50 are related and the point for "shopping discount" of the user model 50 is 1, 1 point is obtained in the "shopping discount" entity. In the case of candidate answer A2, since neither of the two entities "H card company" and "traffic discount" are registered as interests in the user model 50, no points are obtained. As a result, the candidate answer A2 is rejected from the answer selection because no points are obtained in the conversation relevance evaluation (S105a). 6 shows that one of the two candidate answers A1 and A2 that has passed the review quality analysis (S103b) is dropped (30c). The remaining answer A1 as a result of the scoring (S105) is finally selected as an answer to the user's utterance (S107).

On the other hand, unlike shown in FIG. 5 , as shown in FIG. 7 , as a result of analysis of the conversation log 40 , 1 point for the first interest “shopping discount”, 1 point for the second interest “traffic discount”, and the second interest Assuming that the user model 50 is configured in which 1 point is given to each of the 3 interests "cheap annual fee", as well as the conversation content relevance evaluation (S105a) as a scoring (S105) procedure, rate-based scoring (S105b) What this additionally does is described.

Referring to FIG. 8A , as a conversation content relevance evaluation, the candidate answer A1 and the candidate answer A2 acquire the same points. Therefore, it is difficult to differentiate between the candidate answer A1 and the candidate answer A2 only by evaluating the relevance of the conversation content (30d). This means that even if either the candidate answer A1 or the candidate answer A2 is adopted as the final answer, the chatbot user does not feel much repulsion. In this case, rate-based scoring ( S105b ) may be performed using a rate assigned to all candidate answers submitted by the submitter according to the cost paid by the submitter of each candidate answer. In the case of FIG. 8A , if S card company, the submitter of candidate answer A1, paid 1 million won and obtained a rate increase of 1 for all candidate answers submitted by S card company, the final score of candidate answer A1 is 2 (1+1) will be In addition, if H card company, the submitter of candidate answer A2, paid 2 million won and obtained a rate increase of 2 for all candidate answers submitted by H card company, the final score of candidate answer A2 will be 3 (1+2) . Accordingly, in the rate-based scoring (S105b), A1 is dropped (30e). In this case, the answer selected as the final answer becomes the candidate answer A2.

Meanwhile, in another embodiment, a cost may be paid for each candidate answer, and a rate increase of a received candidate answer may be determined according to the paid cost. Referring to FIG. 8B , as the conversation content relevance evaluation, the candidate answer A1 and the candidate answer A2 acquire the same points, and it is difficult to differentiate between the candidate answer A1 and the candidate answer A2 only by the conversation content relevance evaluation (30d) . At this time, if S card company, the submitter of candidate answer A1, paid 100,000 won for candidate answer A1, and H card company, the submitter of candidate answer A2, paid 200,000 won for candidate answer A2, candidate answer A1 and candidate answer A2 The rate increase will be +1 and +2, respectively. Accordingly, the final score of the candidate answer A1 becomes 2 (1+1), and the final score of the candidate answer A2 becomes 3 (1+2). That is, in the rate-based scoring (S105b), A1 is dropped (30e). In this case, the answer selected as the final answer becomes the candidate answer A2.

Meanwhile, in another embodiment, the candidate answers may be received through the public interface on condition of payment of the candidate answer reception fee, but a separate rate increase may not be recognized for the received candidate answers. In this case, the scoring ( S105 ) may be performed using only the relevance with the chatbot service conversation content transmitted and received between the chatbot client terminal and the chatbot client terminal. In this case, in the situation shown in FIG. 8A or FIG. 8B , any one of the candidate answers A1 or A2 may be selected as the final answer. Of course, in another embodiment, another scoring method not based on rate may be additionally applied to finally select one candidate answer.

The methods according to the embodiments of the present invention described so far may be performed by executing a computer program embodied as computer readable code. The computer program may be transmitted from the first computing device to the second computing device through a network such as the Internet and installed in the second computing device, thereby being used in the second computing device. The first computing device and the second computing device include all of a server device, a physical server belonging to a server pool for a cloud service, and a stationary computing device such as a desktop PC.

The computer program may be stored in a recording medium such as a DVD-ROM or a flash memory device.

Next, a chatbot service providing system 150 according to another embodiment of the present invention will be described with reference to FIG. 9 . The chatbot service providing system 150 of the present embodiment may include a candidate answer receiving unit 151 , a filtering unit 152 , and a scoring unit 153 corresponding to a software or hardware module of a logical unit. The candidate answer reception unit 151 , the filtering unit 152 , the scoring unit 153 , and the chatbot engine 154 may each be configured as a physically separate dedicated computing device, and at least some of them are in the form of software modules or hardware module parts. Note that it may be configured together in one computing device. A plurality of candidate answers to a specific question received through the candidate answer reception unit 151 are filtered through the filtering unit 152, the candidate answers that have passed the filtering process are evaluated through the scoring unit 153, and the scoring unit ( 153), the selected final answer is provided to the chatbot engine 154 . The chatbot engine 154 provides the final answer to the chatbot client 10 .

Next, the configuration and operation of a chatbot service server according to another embodiment of the present invention will be described with reference to FIG. 10 . As shown in FIG. 10 , the chatbot service server 100 according to the present embodiment includes a processor 101 and a memory 102 , and in some embodiments a storage 103 , a network interface 105 and a system It may further include at least one of the buses 104 . One or more instructions loaded and stored in the memory 102 are executed through the processor 101 . The chatbot service server 100 according to the present embodiment relates to the operation related to the chatbot service server 100 described with reference to FIGS. 1 to 2 and the chatbot service providing method described with reference to FIGS. 3A to 8B, even if there is no separate explanation. Note that actions can be performed.

The network interface 105 receives the user utterance data provided through the network from the chatbot client terminal, and the received user utterance data is provided to the chatbot engine 126 .

The candidate answer inquiry instruction 120 receiving the user utterance data from the chatbot engine 126 inquires candidate answers to a question matching the user utterance data from the candidate answer DB 130 stored in the storage 103 .

The filtering instruction 122 filters the inquired candidate answers using a spam filter, review quality analysis, or the like. In addition, the scoring instruction 124 selects a final answer by performing conversation relevance evaluation and rate-based scoring, etc. for candidate answers that have passed filtering, and provides the final answer to the chatbot engine 126 , so that the final answer is transmitted to the chatbot client It can be transmitted to the terminal.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. can understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (8)

A method performed by a computing device, comprising:
receiving a plurality of candidate answers to the user's utterance;
filtering the candidate answers;
performing scoring to select a chatbot answer from the candidate answers that have passed the filtering; and
Comprising the step of transmitting the final answer selected as a result of the scoring to the chatbot client terminal that has transmitted the user's utterance,
The filtering step is
filtering through review quality analysis;
The review quality analysis is characterized in that filtering based on the value of ((#E + #O) / LEN),
#E = number of entities included in the candidate answer,
#O = number of comments included in the candidate answer,
LEN = length of candidate answer,
The entity, as the entity name extracted from the candidate answer, is a pre-registered keyword or is obtained as a result of performing an entity name recognition algorithm applied to the candidate answer,
wherein the expression of opinion is an expression indicating an evaluation of the entity;
How to provide chatbot service. delete According to claim 1,
The step of performing the scoring comprises:
Comprising the step of performing the scoring by using the correlation with the chatbot service conversation content transmitted and received between the chatbot client terminal,
How to provide chatbot service. 4. The method of claim 3,
The step of performing the scoring by using the relevance to the chatbot service conversation content,
Comprising the step of performing the scoring by synthesizing all the relevance of the conversation content for each entity of the candidate answers that have passed the filtering,
How to provide chatbot service. According to claim 1,
The step of receiving the candidate answers includes:
Receiving the candidate answers through an open interface, comprising the step of receiving information about payment of a cost for increasing the rate given to the candidate answers together with the candidate answers,
The step of performing the scoring comprises:
performing the scoring using a rate assigned to the candidate answer;
How to provide chatbot service. According to claim 1,
The step of performing the scoring comprises:
performing the scoring using a rate assigned to the candidate answer according to the payment of the cost of the submitter of the candidate answer;
How to provide chatbot service. According to claim 1,
The step of performing the scoring comprises:
performing the scoring using a rate awarded to all candidate answers of the submitter according to the submitter's payment of the candidate answer;
How to provide chatbot service. According to claim 1,
The step of receiving the candidate answers includes:
receiving the candidate answers through an open interface on condition of payment of the candidate answer reception fee;
The step of performing the scoring comprises:
Comprising the step of performing the scoring using only the relevance with the chatbot service conversation content transmitted and received between the chatbot client terminal,
How to provide chatbot service.

Images