JP7013329B2 - Learning equipment, learning methods and learning programs - Google Patents

Description

Patent Law Article 30, Paragraph 2 Application Proceedings https: // www 2018. thewebconf. org / proceedings / Submitted paper 1 https: // dl. acm. org / citation. cfm? id = 318592 TheWebConference2018 https://www2018. thewebconf. org / Association for Natural Language Processing 24th Annual Conference (NLP2018) http: // www. anlp. jp / nlp2018 / Submitted Paper 2

The present invention relates to a learning device, a learning method and a learning program.

Conventionally, a dialogue system is known in which when a voice or text message is received from a user, a message corresponding to the received message is provided to the user. In such a dialogue system, in response to a message received from a user (hereinafter collectively referred to as "utterance"), a message having various contents corresponding to the utterance (hereinafter collectively referred to as "response") is provided by the user. The technology provided to is applied. Further, in such a dialogue system, a technique for evaluating dialogue based on the flow of utterance and response is known.

Japanese Unexamined Patent Publication No. 2011-253053

However, it may be difficult to improve the accuracy of dialogue with the above-mentioned conventional techniques.

For example, in a dialogue system, a method of learning a model that generates a response from a dialogue based on an evaluation of the dialogue can be considered. However, it is not always easy to improve the dialogue because such a method requires a lot of evaluation.

The present application has been made in view of the above, and an object thereof is to improve the accuracy of dialogue.

The learning device according to the present application includes an acquisition unit that acquires an index utterance that is an index for evaluation of the dialogue service among the utterances for the dialogue model that realizes the dialogue service by generating a response to the input utterance, and the acquisition unit. It is characterized by having a learning unit that performs enhanced learning of the dialogue model by setting a reward based on the index utterance acquired by the unit.

According to one aspect of the embodiment, the accuracy of the dialogue can be improved.

FIG. 1 is a diagram showing an example of processing executed by the information providing device according to the embodiment. FIG. 2 is a diagram showing a configuration example of the information providing device according to the embodiment. FIG. 3 is a diagram showing an example of information registered in the session log database according to the embodiment. FIG. 4 is a diagram showing an example of information registered in the session evaluation database according to the embodiment. FIG. 5 is a diagram showing an example of information registered in the index utterance database according to the embodiment. FIG. 6 is a diagram showing an example of information registered in the engagement database according to the embodiment. FIG. 7 is a diagram showing an example of information registered in the model database according to the embodiment. FIG. 8 is a diagram showing a concept of a process of extracting a new index utterance from a seed index utterance. FIG. 9 is a diagram showing a concept of a process for extracting a new index utterance based on a usage mode in a dialogue service. FIG. 10 is a diagram showing a concept of a process for extracting a new index utterance based on the evaluation result of the session. FIG. 11 is a diagram showing an example of the functional configuration of the index utterance extraction processing unit according to the embodiment. FIG. 12 is a diagram showing a concept of a process of evaluating a session based on an index utterance. FIG. 13 is a diagram showing a concept of a process of evaluating a session based on an index utterance. FIG. 14 is a diagram showing an example of a process of evaluating a session using a ternary evaluation model trained in the characteristics of three types of dialogue. FIG. 15 is a diagram showing an example of a process of evaluating a session based on engagement. FIG. 16 is a diagram showing the concept of a process of evaluating a session based on repeated utterances. FIG. 17 is a diagram showing the concept of a process of evaluating a session using a model trained in the characteristics of automatically generated data. FIG. 18 is a diagram showing the concept of evaluation of repeated utterances using images. FIG. 19 is a diagram illustrating the concept of a process of evaluating a session based on an apology response. FIG. 20 is a diagram showing a concept of a process of evaluating a session by combining a plurality of evaluation methods. FIG. 21 is a diagram illustrating the concept of a process for automatically expanding a labeled session. FIG. 22 is a diagram showing an example of the functional configuration of the session evaluation processing unit according to the embodiment. FIG. 23 is a diagram showing a concept of a process for predicting a future usage mode based on an index utterance. FIG. 24 is a diagram showing a concept of processing for predicting a future usage mode based on the evaluation result. FIG. 25 is a diagram showing an example of the functional configuration of the engagement prediction processing unit according to the embodiment. FIG. 26 is a diagram showing an example of relevance of processing executed by the information providing device according to the embodiment. FIG. 27 is a diagram showing a concept of a process for executing reinforcement learning of a dialogue model. FIG. 28 is a diagram showing an example of the functional configuration of the reinforcement learning processing unit according to the embodiment. FIG. 29 is a flowchart showing an example of a flow of processing executed by the information providing device according to the embodiment based on the usage mode. FIG. 30 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment extracts an index utterance. FIG. 31 is a flowchart showing an example of a process flow in which the information providing device according to the embodiment evaluates a session. FIG. 32 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment evaluates a session based on an image. FIG. 33 is a flowchart showing an example of a process flow in which the information providing device according to the embodiment evaluates a session based on repeated utterances. FIG. 34 is a flowchart showing an example of the flow of reinforcement learning in which the information providing device according to the embodiment executes the utterance as a reward. FIG. 35 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment extracts an index utterance based on co-occurrence. FIG. 36 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment evaluates a session using a ternary evaluation model. FIG. 37 is a flowchart showing an example of a process flow in which the information providing device according to the embodiment evaluates a session using a plurality of models. FIG. 38 is a diagram showing an example of a hardware configuration. FIG. 39 is a diagram showing an example of the result of labeling the session data. FIG. 40 is a diagram showing the results of the first experiment. FIG. 41 is a diagram showing the results of the second experiment. FIG. 42 is a diagram showing the results of Prop_D in the third experiment. FIG. 43 is a diagram showing the results of Prop_S in the fourth experiment. FIG. 44 is a diagram showing the results of Prop_E in the fifth experiment.

Hereinafter, the learning device, the learning method, and the embodiment for implementing the learning program (hereinafter referred to as “the embodiment”) according to the present application will be described in detail with reference to the drawings. It should be noted that this embodiment does not limit the learning device, learning method and learning program according to the present application. Further, in each of the following embodiments, the same parts are designated by the same reference numerals, and duplicate description will be omitted.

[Embodiment]
[1. About the outline of the information providing device]
First, with reference to FIG. 1, an example of processing executed by an information providing device, which is an example of an extraction device, an evaluation device, and a learning device, will be described. FIG. 1 is a diagram showing an example of processing executed by the information providing device according to the embodiment.

In FIG. 1, the information providing device 10 can communicate with the dialogue device 200 via a predetermined network N (see, for example, FIG. 2) such as the Internet. The information providing device 10 may be capable of communicating with the terminal device 100 used by the user. Further, it is assumed that the information providing device 10 can communicate with various external servers other than the terminal device 100 and the dialogue device 200.

The terminal device 100 is a smart device such as a smartphone or tablet, and is a mobile terminal device capable of communicating with an arbitrary server device via a wireless communication network such as 3G (3rd Generation) or LTE (Long Term Evolution). be. The terminal device 100 may be not only a smart device but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC. When the terminal device 100 acquires the voice generated by the user (so-called "utterance"), the acquired voice is transmitted to the dialogue device 200, and when the response transmitted from the dialogue device 200 is acquired, the terminal device 100 acquires the voice. Any device such as a smart speaker can be adopted as long as it is a device that outputs the response as voice.

The dialogue device 200 is an information processing device that executes a dialogue process that realizes a dialogue with a user, and is realized by, for example, a server device or a cloud system. For example, when the dialogue device 200 receives an utterance from the terminal device 100, the dialogue device 200 generates a response corresponding to the received utterance, and transmits data for outputting the generated response by voice to the terminal device 100.

Here, an example of the dialogue processing realized by the dialogue device 200 will be described. The dialogue processing shown below is merely an example, and does not limit the embodiment. For example, the terminal device 100 acquires the voice uttered by the user as an utterance (step S1). To give a more specific example, the terminal device 100 acquires voice data of utterances by using various voice extraction techniques such as beamforming. In such a case, the terminal device 100 transmits the acquired voice data to a predetermined conversion server (not shown). In such a case, the conversion server generates utterance text data from the voice data and transmits the generated text data to the terminal device 100. Then, the terminal device 100 transmits the text data as an utterance to the dialogue device 200 (step S2).

On the other hand, the dialogue device 200 analyzes the text data received as the utterance by using various intention analysis techniques, and estimates the user's intention indicated by the utterance. Then, the dialogue device 200 executes various processes according to the estimated intention, and generates text data indicating the execution result. That is, the dialogue device 200 generates a response (step S3). Then, the dialogue device 200 transmits the text data generated as a response to the terminal device 100 (step S4). As a result, the terminal device 100 provides the user with a response to the utterance by reading out the text data or displaying the text data on the screen (step S5).

In the dialogue processing realized by such a dialogue device 200, it is important to evaluate whether or not the user is satisfied with the content of the response. For example, the information as to whether or not the user is satisfied with a certain response can be adopted as information for improving the accuracy of the dialogue processing executed by the dialogue device 200. However, if the user is inquired whether or not he / she is satisfied with the response, the user experience may be damaged.

Here, in order to appropriately evaluate whether or not the user is satisfied with the dialogue, data of the labeled dialogue with a label indicating whether or not the user is satisfied is prepared and used in various models. A method is conceivable in which the characteristics of the labeled dialogue data are learned by supervised or unsupervised learning, and the dialogue is evaluated using a trained model. However, learning of such a model requires a relatively large amount of data of the labeled dialogue, but there is a problem that it is troublesome to create the data of the labeled dialogue.

For example, to create data for a labeled dialogue, it may be created manually, such as by cloud sourcing. However, it can be difficult to manually label a dialogue because some dialogues (ie, long dialogues) include many utterances and responses. Therefore, if the data of the labeled dialogue used for learning can be automatically acquired, it is considered that the learning can be performed efficiently and the accuracy of the dialogue processing can be improved.

Therefore, the information providing device 10 executes an evaluation process for automatically evaluating whether or not the user is satisfied with the dialogue process by using the utterance and the response of the user in the dialogue process. Further, the information providing device 10 executes an extraction process for extracting utterances used when performing the evaluation process. Further, the information providing device 10 executes a learning device for learning a model for performing a dialogue. Then, the information providing device 10 improves the accuracy of the dialogue processing by the dialogue device 200 based on the extraction device, the evaluation device, and the learning device.

[1-1. An example of the processing executed by the information providing device]
Hereinafter, an example of the processing executed by the information providing device 10 will be described. First, the information providing device 10 acquires a dialogue history, which is a history of utterances and responses in dialogue processing, from the dialogue device 200 (step S6). Then, the information providing device 10 evaluates the dialogue based on the dialogue history (step S7).

More specifically, the information providing device 10 uses a dialogue history including a series of consecutive utterances and a plurality of responses such as an utterance by the user, a response to the utterance, and a user's utterance to the response as a session. Extract. Here, the information providing device 10 may adopt a plurality of utterances and responses defined by arbitrary criteria as one session.

For example, when the information providing device 10 has not spoken from the user for a predetermined time (for example, 30 minutes) or more, the information providing device 10 receives the utterance received from the user and the response provided to the user by 1 It may be an utterance and a response contained in one session. Further, the information providing device 10 may use the utterances and responses acquired from the start of the application for performing the dialogue processing by the user to the sleep or termination of the application as one session. That is, the session may include utterances and responses organized in any unit as long as it is a series of dialogues that can be an evaluation unit by the user.

Then, the information providing device 10 evaluates the dialogue in the session. More specifically, the information providing device 10 estimates whether or not the user is satisfied with the dialogue in the session based on the response or utterance in the session. For example, the information providing device 10 evaluates a session by using the evaluation results of a plurality of models that evaluate the session based on the characteristics of different types among the plurality of features that are indicators of the user's evaluation of the session. conduct.

For example, if the utterances contained in one session contain the same or similar utterances, it is presumed that the user is not satisfied with the dialogue in that session. For example, when a user utters "inn job" with the intention of seeking a job at an inn, the dialogue device 200 erroneously determines that the user intends to search for an accommodation facility from the word "inn". , "I will show you the accommodation" may be returned. When such a response contrary to the intention is output, the user presumes that the voice recognition did not go well and speaks "the work of the inn" again, and the dialogue device 200 again "guides the accommodation facility". It may return a response. As described above, the repetition of similar utterances in one session is considered to indicate that the user is not very satisfied with the session.

Therefore, the information providing device 10 generates a model for evaluating the session based on the repetition of the same or similar utterances (step S7-1). For example, when the information providing device 10 inputs a character string of an utterance or a response included in a session, how unsatisfied the user is with the session based on the repetition of the utterance included in the session. That is, the iterative evaluation model M1 that outputs the first dissatisfaction degree, which is a score indicating the dissatisfaction degree of the user, is generated. For example, in such a repetitive evaluation model, when a session containing the same utterance (or the same character string) or a session containing a similar utterance (or a similar character string) is input, the same utterance (or the same character string) is input. A higher value of the first dissatisfaction is output than when a session that does not include a column) or a session that does not include a session that does not include a similar speech (or a similar character string) is input. Here, the "similar utterance" is an utterance having a similar meaning, such as "work at an inn" and "work at a hotel". The iterative evaluation model M1 evaluates the session based on the characteristics of the image generated from the character string, not the character string itself of the session. Details of such processing will be described later.

In addition, for example, a response indicating an apology such as "I'm sorry" or "I'm sorry" may be provided for an utterance such as "No". When a response indicating such an apology (hereinafter, may be collectively referred to as "apology response") is included in the session, it is presumed that the user is not very satisfied with the session. Therefore, the information providing device 10 generates a model for evaluating the session based on whether or not the apology response is included (step S7-2). For example, the information providing device 10 outputs a higher value of the second dissatisfaction for a session containing more apology responses and a lower value for a session containing less apology responses. Generate an apology evaluation model M2 that outputs the second dissatisfaction of the value.

In addition, the utterance of the user includes an utterance that suggests the user's impression of the session, and the utterance that suggests the user's impression is an utterance that serves as an index for evaluating the session, that is, an index utterance. Can be adopted as. For example, utterances such as "unusable" and "uninstall" by the user are considered to be utterances indicating the user's unfavorable evaluation of the response, that is, unfavorable feedback. It is presumed that the user's satisfaction is low in the session containing many utterances showing such unfavorable evaluation (hereinafter, referred to as "non-favorable index utterance"). In addition, utterances such as "wow" and "well done" by the user are considered to be utterances indicating the user's favorable evaluation of the response, that is, feedback feedback. It is presumed that the user's satisfaction is high in the session containing many utterances showing such a favorable evaluation (hereinafter, referred to as "favorable index utterance").

That is, such favorable index utterances and non-favorable index utterances (hereinafter, may be collectively referred to as "index utterances") can be an index of user satisfaction with the session. Therefore, the information providing device 10 generates a model for evaluating a session based on various index utterances such as favorable index utterances and non-favorable index utterances.

For example, the information providing device 10 generates a model for evaluating a session based on a non-favorable index utterance (step S7-3). For example, the information providing device 10 has a higher value for a session in which more preset non-favorable index utterances such as "unusable" and "uninstall" are included in the utterances included in the session. Generates a non-favorable index utterance evaluation model M3 that outputs a third dissatisfaction and outputs a lower value of the third dissatisfaction for a session that contains less unfavorable index utterances.

Further, the information providing device 10 generates a model for evaluating the session based on the favorable index utterance (step S7-4). For example, the information providing device 10 has a higher value for a session in which the utterances included in the session include more preset favorable index utterances such as "wow" and "well done". A favorable index utterance evaluation model M4 is generated, which outputs the first satisfaction level of the above and outputs a lower value of the first satisfaction level for a session containing less favorable index utterances. The above-mentioned non-favorable index utterances and favorable index utterances may be utterances automatically extracted from the session by a process described later.

The information providing device 10 may learn each model by using any known technique. For example, the information providing device 10 may learn each model by using techniques such as backpropagation, reinforcement learning, and GAN (Generative Adversarial Network).

Then, the information providing device 10 evaluates the input session based on each dissatisfaction degree and satisfaction degree output by each model (step S7-5). For example, the information providing device 10 extracts a session to be evaluated, and repeats the extracted session as an evaluation model M1, an apology evaluation model M2, a non-favorable index utterance evaluation model M3, and a favorable index utterance evaluation model M4, respectively. Enter in. Then, the information providing device 10 calculates a value obtained by subtracting the first satisfaction from the sum of the first dissatisfaction, the second dissatisfaction, and the third dissatisfaction as the dissatisfaction score. That is, the information providing device 10 calculates, as the dissatisfaction score, a score that is so high that the user is not satisfied with the session. In addition, "dissatisfaction" means not only when the user is frustrated with respect to the response in the session, but also when the user is disappointed, and the subjectivity is unfavorable to the session. It is a concept that includes any state that it has.

Further, the information providing device 10 calculates a dissatisfaction score from each session extracted from the dialogue history. Then, the information providing device 10 extracts a predetermined number of sessions from the one with the higher dissatisfaction score among the sessions, and sets the extracted sessions as low satisfaction sessions, which are sessions with low user satisfaction. Further, the information providing device 10 extracts a predetermined number of sessions in order from the one with the lowest dissatisfaction score, and sets the extracted sessions as high-satisfaction sessions, which are sessions with high user satisfaction. In the information providing device 10, a session in which the dissatisfaction score exceeds a predetermined threshold value may be regarded as a low satisfaction session, and a session in which the dissatisfaction score is lower than the predetermined threshold value may be regarded as a high satisfaction session. By executing such a process, the information providing device 10 realizes a process of evaluating whether or not the user is satisfied with the session.

Then, the information providing device 10 updates the dialogue model based on the evaluation result (step S8). For example, when the utterance and the response to the utterance are included in the low satisfaction session, the information providing device 10 lowers the probability of outputting the response to the utterance, and the utterance and the response to the utterance become. If included in a highly satisfied session, update the interaction model to generate the response to increase the probability of outputting the response to the utterance. That is, the information providing device 10 updates the dialogue model so that the user is more satisfied.

As described above, the information providing device 10 acquires a session including the user's utterance and response in the dialogue service, and is based on the characteristics of different types among the plurality of characteristics that are indicators of the user's evaluation of the session. Evaluating a session Evaluate a session using the evaluation results of a plurality of models M1 to M4. As a result, the information providing device 10 comprehensively evaluates the session from a plurality of viewpoints, so that the accuracy of the evaluation can be improved. Further, since the information providing device 10 can automatically realize the evaluation of the session without manually evaluating the session such as cloud sourcing, it is possible to facilitate the evaluation of the dialogue.

[1-2. About the processing executed by the information providing device]
In the above-mentioned example, as an example of the processing executed by the information processing apparatus 10, depending on the presence / absence of repeated utterances included in the session, the presence / absence of an apology response, and the presence / absence of favorable index utterances and non-favorable index utterances. An example of the process of evaluating a session is described. However, the embodiments are not limited to this.

For example, as described below, the information providing device 10 may evaluate the session from various viewpoints. Further, in order to evaluate such a session, the information providing device 10 may automatically extract various index utterances from the utterances included in the session. Further, in order to evaluate such a session, the information providing device 10 may estimate the period and frequency (hereinafter, may be referred to as “engagement”) in which the user uses the dialogue service. Further, the information providing device 10 may learn a dialogue model for generating a response from an utterance based on the history of the dialogue.

That is, the information providing device 10 may realize the evaluation of the session shown in FIG. 1 by combining various processes described below. The various processes described below may be performed in any combination.

[1-3. About user subjectivity]
In the above description, the information providing device 10 evaluates the session by using various index utterances showing a user's favorable or unfavorable impression of the response. Here, the "favorable impression" and "non-favorable impression" are not only the impression that the user likes or dislikes, but also the impression that the response is correct or incorrect, and the user is positive about the response. It is a concept that includes various impressions and negative impressions.

Further, in the above description, the information providing device 10 evaluates whether or not the user is satisfied with the session. Here, the information providing device 10 may perform not only a binary evaluation of whether or not the information is satisfied, but also an evaluation of any aspect. For example, the information providing device 10 may use the value of the dissatisfaction score itself as the evaluation result. Further, the information providing device 10 was learned not to calculate the dissatisfaction level, but to calculate a score in which the higher the user's satisfaction level, the higher the value, and the lower the satisfaction level, the lower the value, that is, the satisfaction level. Models M1 to M3 may be used. Further, the information providing device 10 may calculate a satisfaction score indicating the degree of satisfaction of the user for each session instead of the dissatisfaction score.

In the above description, the information providing device 10 evaluated whether the user was "satisfied" or "dissatisfied" with the session. Here, the concepts of "satisfaction" and "dissatisfaction" include various positive or unfavorable impressions of the user on the session, such as whether or not the user is frustrated with the session. Is.

That is, the above-mentioned statements such as "favorable" and "satisfied" indicate various favorable impressions by the user, and the statements such as "non-favorable" and "dissatisfied" are unfavorable by the user. It shows various impressions. If the information providing device 10 evaluates the session based on various index utterances indicating the user's impression of the response, the information providing device 10 is arbitrary by various processes described in the embodiment regardless of the type of the user's impression. The evaluation based on the impression of is may be realized.

[1-4. Evaluation based on index utterances]
In the above description, the information providing device 10 evaluates whether or not the user is satisfied with the session based on various index utterances such as favorable index utterances and non-favorable index utterances. Here, various index utterances are utterances such as whether or not the user has a favorable impression on the target such as dialogue and response until the index utterance appears, and are indicators for session evaluation. It is a possible utterance.

Here, the index utterance can be an utterance showing the user's impression on any target as long as it can be an index for session evaluation, regardless of the target of the user's impression indicated by the index utterance. For example, the information providing device 10 may use an utterance that indicates the user's impression of a single response (that is, a utterance that serves as feedback) as an index utterance, and an utterance that indicates the user's impression of a series of dialogues or the entire session. It may be adopted as an index utterance. That is, if the information providing device 10 uses an utterance indicating the user's evaluation of the dialogue service for the response as the index utterance, the information providing device 10 adopts the utterance indicating the user's impression of an arbitrary object in the dialogue service as the index utterance. May be good.

[2. Configuration of information providing device]
Hereinafter, an example of the functional configuration of the information providing device 10 that realizes various processes executed by the information providing device 10 will be described. FIG. 2 is a diagram showing a configuration example of the information providing device according to the embodiment. As shown in FIG. 2, the information providing device 10 includes a communication unit 20, a storage unit 30, and a control unit 40.

[2-1. Communication section]
The communication unit 20 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 20 is connected to the network N by wire or wirelessly, and transmits / receives information between the terminal device 100 and the dialogue device 200.

[2-2. About the memory part]
The storage unit 30 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. Further, the storage unit 30 stores the session log database 31, the session evaluation database 32, the index utterance database 33, the engagement database 34, and the model database 35.

Hereinafter, an example of each database 31 to 35 stored in the storage unit 30 will be described with reference to FIGS. 3 to 7. The dialogue history is registered in the session log database 31 for each session. For example, FIG. 3 is a diagram showing an example of information registered in the session log database according to the embodiment. As shown in FIG. 3, information having items such as "session ID (Identifier)", "dialogue time", "user ID", and "dialogue content" is registered in the session log database 31. In addition to the information shown in FIG. 3, any information may be registered in the session log database 31 as long as it is information related to utterances and responses in the dialogue processing.

Here, the "session ID" is information for identifying a session. Further, the "dialogue time" is information indicating the date and time when the dialogue included in the session indicated by the "session ID" was performed, and is, for example, information indicating the date and time when the first utterance included in the session was acquired. .. Further, the "user ID" is an identifier that identifies the user who made the utterance included in the session indicated by the "session ID". Further, the "dialogue content" is information in which the contents of utterances and responses included in the session indicated by the "session ID" are arranged in chronological order.

For example, in the example shown in FIG. 3, the session ID “session # 1”, the dialogue time “dialogue time # 1”, the user ID “user # 1”, and the dialogue content “Usr: inn work sys: accommodation facility guide”. Information such as "..." is registered. For such information, the date and time when the dialogue included in the session indicated by the session ID "session # 1" was performed is "dialogue time # 1", and the user speaking in that session is "user # 1". Indicates that the user is the user. In addition, the session indicated by the session ID "session # 1" includes the user's utterance "work at the inn", and indicates that the response to such utterance was "guidance to the accommodation facility".

In the example shown in FIG. 3, conceptual values such as "session # 1", "dialogue time # 1", and "user # 1" are described, but in reality, a character string or a numerical value that identifies the session is described. , Numerical values indicating the date and time, character strings and numerical values for identifying users, and the like are registered in the session log database 31.

The evaluation result for the session is registered in the session evaluation database 32. For example, FIG. 4 is a diagram showing an example of information registered in the session evaluation database according to the embodiment. As shown in FIG. 4, information having items such as "session ID", "session type", and "dissatisfaction score" is registered in the session evaluation database 32. The information registered in the session evaluation database 32 is not limited to this, and any information may be registered as long as it is information related to the evaluation of the session. Further, the session evaluation database 32 may be integrated with the session log database 31, for example.

Here, the "session type" is information indicating whether or not the user is presumed to be satisfied with the dialogue in the session indicated by the associated "session ID", for example, for the session. The label to be given. For example, if it is estimated that the user is not satisfied with the "session type" (for example, the dissatisfaction score is high), the "low satisfaction session" is satisfied with the user (for example, the dissatisfaction score is high). If it is estimated to be low), then a "high satisfaction session" is adopted, and if it is estimated to be neither (eg, if the dissatisfaction score falls within a given range), then a "neutral session" is adopted. To. Further, the "dissatisfaction score" is a value of the dissatisfaction score given to the session indicated by the associated "session ID".

For example, in the example shown in FIG. 4, the session ID “session # 1”, the session type “low satisfaction session”, and the dissatisfaction score “150” are registered in the session evaluation database 32 in association with each other. Such information indicates that the session type of the session indicated by the session ID "session # 1" is "low satisfaction session" and the dissatisfaction score is "150". If the session has not been evaluated, the session type will be "unevaluated", and if the dissatisfaction score has not been calculated, the dissatisfaction score will be "unevaluated".

Various index utterances are registered in the index utterance database 33. For example, FIG. 5 is a diagram showing an example of information registered in the index utterance database according to the embodiment. As shown in FIG. 5, information having items such as "index utterance", "index type", "index score", and "registration source" is registered in the index utterance database 33. In addition to the information shown in FIG. 5, various information related to the index utterance may be registered in the index utterance database 33.

Here, the "index utterance" is a character string of utterances that is an index of whether or not the user is satisfied with the dialogue in the session including the utterances such as unfavorable index utterances and favorable index utterances. Is. Further, the "index type" is information indicating whether the associated index utterance is a non-favorable index utterance or a favorable index utterance. The "index score" is a score indicating the degree to which the user is not satisfied with the dialogue in the session including the associated "index utterance". Further, the "registration source" is whether the associated index utterance is a preset index utterance, that is, a seed index utterance or an index utterance automatically extracted from the session by the extraction process. In the case of index utterances extracted from the session, the session ID of the session that is the extraction source is registered.

For example, in the example shown in FIG. 5, the index utterance database 33 is registered in association with the index utterance "well done", the index type "favorable", the index score "-150", and the registration source "pre-registration". Has been done. In such information, the character string "well done" is registered as an index utterance, the type of the index utterance is "favorable", that is, the favorable index utterance, and the index score is "-150". Indicates that. Further, such information indicates that the favorable index utterance "well done" is a pre-registered index utterance, that is, a seed index utterance.

Information indicating the usage mode of the dialogue service by the user is registered in the engagement database 34. More specifically, in the engagement database 34, information such as a usage period and usage frequency, that is, engagement information indicating how much the user uses the dialogue service is registered. For example, FIG. 6 is a diagram showing an example of information registered in the engagement database according to the embodiment. As shown in FIG. 6, information having items such as "user ID", "engagement history", and "predicted engagement" is registered in the engagement database 34. In addition to the information shown in FIG. 6, any information may be registered in the engagement database 34 as long as it is information related to engagement.

Here, the "engagement history" is a history period which is a history of the usage period of the dialogue service by the user indicated by the associated "user ID", and a history of the usage frequency of the dialogue service by the user. Information indicating "history frequency". Further, the "predicted engagement" indicates a "predicted period" which is a future usage period by the user indicated by the associated "user ID" and a "predicted frequency" which is a future usage frequency by the user. Information.

Further, the "history period" is information indicating, for example, a period during which a user has continuously used the dialogue service in a predetermined period in the past, for example, without leaving a predetermined period (for example, several days). Information indicating the longest period of time that the user has used the dialogue service. The "history frequency" is information indicating the frequency with which the user used the dialogue service during the history period. For example, the number of times the user used the dialogue service during the history period (for example, of a session). The number) is divided by the history period. Further, the "predicted period" is information indicating a period in which the user is estimated to continuously use the dialogue service in a predetermined period in the future. Further, the "predicted frequency" is information indicating the frequency at which the user is estimated to use the dialogue service in a predetermined period in the future.

For example, in the example shown in FIG. 6, the user ID "user # 1", the engagement history "history period # 1 history frequency # 1", and the predicted engagement "predicted period # 1 predicted frequency # 1" are registered in association with each other. ing. For such information, the user engagement history indicated by the user ID "user # 1" is "history period # 1" and "history frequency # 1", and the predicted engagement is "predicted period # 1" and "predicted period # 1". Prediction frequency # 1 ”.

In the example shown in FIG. 6, conceptual values such as "history period # 1", "history frequency # 1", "prediction period # 1", and "prediction frequency # 1" are described, but in reality, they are described. Numerical values indicating various periods and frequencies will be registered.

Various models are registered in the model database 35. For example, FIG. 7 is a diagram showing an example of information registered in the model database according to the embodiment. As shown in FIG. 7, information having items such as "model ID", "model type", and "model data" is registered in the model database 35. In addition to the information shown in FIG. 7, various information related to the model may be registered in the model database 35.

Here, the "model ID" is an identifier that identifies the model. Further, the "model type" is information indicating what kind of model the model is. The "model data" is model data. In the example shown in FIG. 7, conceptual values such as "model data # 1" are described, but in reality, information such as various parameters constituting the model is registered.

For example, in the example shown in FIG. 7, the model database 35 includes a “repetition evaluation model” that evaluates a session based on whether or not the session contains iterations, and whether or not the session contains an apology response. An "apology evaluation model" that evaluates the session based on, a "non-favorable index utterance evaluation model" that evaluates the session based on whether the session contains non-favorable index utterances, and a favorable index utterance to the session. A "favorable index utterance evaluation model" that evaluates sessions based on whether or not is included is registered.

Further, in the model database 35, a "ternary evaluation model" in which the characteristics of a dialogue presumed to be satisfied by the user, a dialogue presumed to be unsatisfied by the user, and a dialogue that is neither of them is learned. Is registered. Further, in the model database 35, an "engagement session evaluation model" is registered in which the characteristics of the session including the utterance of the user whose usage mode of the dialogue service satisfies a predetermined condition are learned. Further, in the model database 35, a "repetitive utterance evaluation model" which is a pseudo-generated session and learns the characteristics of a session including the same or similar utterances is registered.

Further, in the model database 35, a "dialogue model" for generating a response to an utterance, which is a model used by the dialogue device 200, is registered. Further, in the model database 35, a "user model model" that is a model that simulates the user and outputs an utterance to the response when a response is input is registered.

In the model database 35, information on a model of an arbitrary format is registered as model data of each model. For example, each model may be realized by SVM (Support Vector Machine) or DNN (Deep Neural Network). Here, the DNN may be a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). Further, the RNN may be an LSTM (Long short-term memory) or the like. That is, as each model, any type of model can be adopted. Further, each model may be a model realized by combining a plurality of models, for example, a model in which a CNN and an RNN are combined.

[2-3. About the control unit]
Returning to FIG. 2, the explanation will be continued. The control unit 40 is a controller, and for example, various programs stored in a storage device inside the information providing device 10 by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) are RAM or the like. Is realized by executing as a work area. Further, the control unit 40 is a controller, and may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 2, the control unit 40 includes a session acquisition unit 41, a usage mode specifying unit 42, an index utterance extraction processing unit 50, a session evaluation processing unit 60, an engagement prediction processing unit 70, and a reinforcement learning processing unit 80. ..

The session acquisition unit 41 acquires a session. For example, when the session acquisition unit 41 acquires the dialogue history from the dialogue device 200, the session acquisition unit 41 divides the acquired dialogue history for each session. For example, the session acquisition unit 41 selects a user to be processed and extracts an utterance history including the utterance of the selected user. Subsequently, the session acquisition unit 41 specifies the time between each response and the utterance in the extracted utterance history. Then, when the time from one response to the next utterance is longer than a predetermined period, the session acquisition unit 41 determines that the response is the last response in the session, and uses it based on the determination result. Divide the dialogue with the person into sessions. Then, the session acquisition unit 41 assigns a session ID and a user ID of the user to each session and registers them in the session log database 31.

The usage mode specifying unit 42 specifies the usage mode of the user. For example, the usage mode specifying unit 42 selects a user to be processed and specifies a usage mode of the user's dialogue service within a predetermined period (for example, the past six months). More specifically, the usage mode specifying unit 42 specifies the history period and the history frequency within a predetermined period. Then, the usage mode specifying unit 42 registers the specified history period and history frequency as the engagement history in the engagement database 34 in association with the user ID of the user.

[2-4. Extraction process]
The index utterance extraction processing unit 50 executes an extraction process for extracting index utterances used for session evaluation from the session. Hereinafter, with reference to FIGS. 8 to 11, an example of the concept of the process executed by the index utterance extraction processing unit 50 and the functional configuration of the index utterance extraction processing unit 50 will be described.

[2-4-1. Extraction process based on co-occurrence]
For example, the index utterance extraction processing unit 50 extracts a new index utterance based on the co-occurrence with the index utterance. For example, FIG. 8 is a diagram showing a concept of a process of extracting a new index utterance from a seed index utterance. For example, if a session contains favorable indicator utterances, the same session is likely to contain favorable indicator utterances. Also, if a session contains non-favorable index utterances, it is presumed that the same session is likely to contain non-favorable index utterances. That is, it is considered that favorable index utterances are likely to co-occur and non-favorable index utterances are likely to co-occur within the same session.

Therefore, the index utterance extraction processing unit 50 identifies the utterance that is an index of the user's evaluation of the dialogue service and the preset index utterance from the history of the user's utterance and the response to the utterance in the dialogue service. Then, based on the co-occurrence with the specified index utterance, a new index utterance is extracted from the user's utterance. For example, the index utterance extraction processing unit 50 extracts a new index utterance from each utterance based on the co-occurrence with the seed non-favorable index utterance and the seed favorable index utterance registered in advance, and the index utterance database 33. To register with.

For example, when the character string "wow" is registered in the index utterance database 33 as a seed-friendly index utterance, the index utterance extraction processing unit 50 includes the character string "wow" from the session log database 31 in the utterance. Extract all sessions. Further, the index utterance extraction processing unit 50 counts the number of sessions including the utterance for each utterance included in the extracted session. That is, the index utterance extraction processing unit 50 counts the number of co-occurrence with the seed-favorable index utterance for each utterance.

Further, for example, when the character string "stupid guy" is registered in the index utterance database 33 as a seed non-favorable index utterance, the index utterance extraction processing unit 50 uses the session log database 31 to read the character string "stupid guy". Extracts all sessions included in the utterance. Further, the index utterance extraction processing unit 50 counts the number of sessions including the utterance for each utterance included in the extracted session. That is, the index utterance extraction processing unit 50 counts the number of co-occurrence with the seed non-favorable index utterance for each utterance.

Then, the index utterance extraction processing unit 50 calculates the ratio of the counted number of co-occurrences for each utterance. More specifically, the index utterance extraction processing unit 50 calculates a value obtained by dividing the number of co-occurrence with the seed-favorable index utterance by the number of co-occurrence with the seed-friendly index utterance for each utterance. Then, the index utterance extraction processing unit 50 extracts utterances having a high ratio of N (N is an arbitrary number) as new non-favorable index utterances, and extracts utterances having a low ratio of N as a new favorable index. Extract as an utterance. Further, the index utterance extraction processing unit 50 registers the extracted non-favorable index utterances and favorable index utterances in the index utterance database 33.

Here, the index utterance extraction processing unit 50 may extract a new index utterance from the seed index utterance based on the activity propagation on the co-occurrence network. For example, the index utterance extraction processing unit 50 generates a co-occurrence network of utterances included in the same session as the seed index utterance. For example, the index utterance extraction processing unit 50 uses each utterance as a node and generates a network in which utterances included in the same session are connected by a link. Any known technique can be adopted for the generation of the co-occurrence network.

Subsequently, the index utterance extraction processing unit 50 identifies a node corresponding to the seed index utterance from the co-occurrence network, and for each node, the distance between the seed index utterance and the corresponding node (for example, a hop following a link). Number) is calculated. Then, the index utterance extraction processing unit 50 newly unfavorable the utterance corresponding to the node in which the distance between the seed-favorable index utterance and the corresponding node is short and the distance between the seed-favorable index utterance and the corresponding node is long. The target index utterance. In addition, the index utterance extraction processing unit 50 newly favors the utterance corresponding to the node in which the seed-friendly index utterance is close to the corresponding node and the seed non-favorable index utterance is far from the corresponding node. Use index utterances.

For example, the index utterance extraction processing unit 50 identifies and identifies a node in which the distance from the seed-friendly index utterance is less than a predetermined threshold value and the distance from the seed-non-favorable index utterance is equal to or more than a predetermined threshold value. Extract the utterances corresponding to the nodes as new favorable index utterances. Further, the index utterance extraction processing unit 50 identifies and identifies a node in which the distance from the seed-friendly index utterance is equal to or greater than a predetermined threshold value and the distance from the seed non-favorable index utterance is less than the predetermined threshold value. Extract the utterances corresponding to the nodes as new unfavorable index utterances. Then, the index utterance extraction processing unit 50 registers each extracted index utterance in the index utterance database 33.

The index utterance extraction processing unit 50 uses the index utterance extracted by the above-mentioned extraction process as a new seed index utterance, and executes the above-mentioned extraction process using the new seed index utterance to further execute a new index utterance. May be extracted.

[2-4-2. Extraction process based on usage mode]
Further, for example, the index utterance extraction processing unit 50 extracts a new index utterance based on the usage mode in the dialogue service of the user. For example, FIG. 9 is a diagram showing a concept of a process for extracting a new index utterance based on a usage mode in a dialogue service.

For example, it is presumed that a user who continuously uses the dialogue service, that is, a user who has a long usage period or a user who frequently uses the dialogue service, is satisfied with the dialogue service. Therefore, it is presumed that a user who has a long usage period or a user who frequently uses the user makes a lot of favorable (favorable) remarks, that is, a favorable index utterance. On the other hand, it is presumed that users with a short usage period and users with a short usage frequency often make unfavorable (non-favorable) remarks, that is, non-favorable index utterances.

Therefore, the index utterance extraction processing unit 50 acquires a session including utterances of a user whose usage mode of the dialogue service satisfies a predetermined condition, and evaluates the dialogue service by the user based on the frequency of appearance in the acquired session. Extract the index utterances that serve as the index of. For example, the index utterance extraction processing unit 50 refers to the engagement database 34 and identifies a user whose engagement history satisfies a predetermined condition.

For example, the index utterance extraction processing unit 50 identifies a user whose history period and history frequency exceed a predetermined threshold value as a favorable user, and a user whose history period and history frequency are each less than a predetermined threshold value. As an unfavorable user. Further, the index utterance extraction processing unit 50 identifies a session including utterances of a favorable user and a session including utterances of a non-favorable user, and extracts utterances included in the specified session.

Then, the index utterance extraction processing unit 50 calculates the appearance frequency in the session of the favorable user and the appearance frequency in the session of the non-favorable user for each extracted utterance. For example, the index utterance extraction processing unit 50 divides the number of times the utterance appears in the session including the utterance of the favorable user by the total number of sessions including the utterance of the favorable user, and divides the value in the session of the favorable user. Calculated as the frequency of appearance. Further, for example, the index utterance extraction processing unit 50 divides the number of times the utterance appears in the session including the utterance of the unfavorable user by the total number of sessions including the utterance of the unfavorable user, and divides the value by the total number of sessions including the utterance of the unfavorable user. Calculated as the frequency of appearance in the user's session.

Further, the index utterance extraction processing unit 50 calculates a value obtained by dividing the appearance frequency in the session of the unfavorable user by the appearance frequency in the session of the favorable user, that is, the ratio of the appearance frequency for each utterance. Then, the index utterance extraction processing unit 50 extracts the utterances having the highest calculated ratio of N as new non-favorable index utterances and the utterances of the lower N as new favorable index utterances.

[2-4-3. Extraction process based on evaluation results]
Further, for example, the index utterance extraction processing unit 50 extracts a new index utterance based on the evaluation result of the session. For example, FIG. 10 is a diagram showing the concept of a process for extracting a new index utterance based on the evaluation result of the session.

For example, the session may be given an evaluation result by an evaluation process described later. In addition, the session may be given a manual evaluation result such as cloud sourcing. The evaluation result for such a session can be an index for extracting index utterances from the utterances included in the session.

For example, it is presumed that a session with high user satisfaction (that is, a high satisfaction session) contains a large number of favorable index utterances. On the other hand, it is presumed that the session with low user satisfaction (that is, the low satisfaction session) contains many unfavorable index utterances.

Therefore, the index utterance extraction processing unit 50 favorably evaluates the utterances whose appearance frequency satisfies the predetermined condition among the utterances included in the session whose evaluation satisfies the predetermined condition, and the user's evaluation for the session including the utterance. It is extracted as an index utterance that is an index of whether or not the user is satisfied with the session. For example, the index utterance extraction processing unit 50 extracts utterances included in the high-satisfaction session and the low-satisfaction session, and calculates the appearance frequency in the low-satisfaction session and the appearance frequency in the high-satisfaction session for each extracted utterance.

Then, the index utterance extraction processing unit 50 calculates a value obtained by dividing the appearance frequency in the low satisfaction session by the appearance frequency in the high satisfaction session as the ratio of the appearance frequency. Then, the index utterance extraction processing unit 50 extracts the utterances having the highest calculated ratio of N as new non-favorable index utterances and the utterances of the lower N as new favorable index utterances.

[2-4-4. About the functional configuration of the index utterance extraction processing unit]
Next, an example of the functional configuration of the index utterance extraction processing unit 50 will be described with reference to FIG. FIG. 11 is a diagram showing an example of the functional configuration of the index utterance extraction processing unit according to the embodiment. As shown in FIG. 11, the index utterance extraction processing unit 50 includes an index utterance identification unit 51, an utterance extraction unit 52, and a network generation unit 53.

The index utterance specifying unit 51 identifies a preset index utterance, which is an index of the user's evaluation of the dialogue service, from the history of the user's utterance and the response in the dialogue service. For example, the index utterance specifying unit 51 refers to the index utterance database 33 and identifies the index utterance. The index utterance specifying unit 51 may identify the index utterance registered in advance, or may identify the index utterance extracted from the session.

Then, the index utterance specifying unit 51 refers to the session log database 31 and searches for a session including the identified index utterance. That is, the index utterance specifying unit 51 identifies a session in the session history in which the user's utterance includes a favorable index utterance or a non-favorable index utterance. Then, the index utterance specifying unit 51 notifies the utterance extraction unit 52 of the specified session, that is, the session including the index utterance.

When the index utterance specifying unit 51 identifies the index utterance from the co-occurrence network shown in FIG. 8, the index utterance specifying unit 51 notifies the network generation unit 53 of the session including the index utterance. Then, the index utterance specifying unit 51 is a node corresponding to the identified index utterance among the nodes included in the co-occurrence network generated by the network generation unit 53, that is, a favorable index registered in the index utterance database 33. Identify the node that corresponds to the utterance or unfavorable indicator utterance. Then, the index utterance specifying unit 51 notifies the utterance extraction unit 52 of the co-occurrence network and the specified node.

The utterance extraction unit 52 extracts index utterances from each session. For example, as shown in FIG. 8, the utterance extraction unit 52 extracts a new index utterance from the utterance included in the session history based on the co-occurrence with the seed index utterance.

For example, the utterance extraction unit 52 extracts the session specified by the index utterance identification unit 51, that is, the session including the index utterance from the session log database 31. Then, the utterance extraction unit 52 identifies the co-occurrence with the favorable index utterance for each utterance, and extracts the utterance whose co-occurrence with the favorable index utterance exceeds a predetermined threshold as a new favorable index utterance. do. Further, the utterance extraction unit 52 identifies the co-occurrence with the non-favorable index utterance for each utterance, and extracts the utterance in which the specified co-occurrence exceeds a predetermined threshold value as a new non-favorable index utterance.

For example, the utterance extraction unit 52 counts the number of appearances in the session including the favorable index utterance and the number of appearances in the session including the non-favorable index utterance for each utterance included in the extracted session. That is, the utterance extraction unit 52 counts the number of times the favorable index utterance is included in the same session and the number of times the non-favorable index utterance is included in the same session for each utterance. Then, the utterance extraction unit 52 extracts utterances in which the ratio of each counted number of times satisfies a predetermined condition as a new index utterance. For example, the utterance extraction unit 52 calculates the value obtained by dividing the number of appearances in the session including the unfavorable index utterance by the number of appearances in the session including the favorable index utterance as a ratio, and the ratio is the top N N. The utterance is a new non-favorable index utterance, and the utterance with a lower ratio of N is a new favorable index utterance.

The utterance extraction unit 52 may simply set the utterance in which the number of appearances in the session including the favorable index utterance exceeds a predetermined threshold value as a new favorable index utterance. Further, the utterance extraction unit 52 may use an utterance in which the number of appearances in the session including the unfavorable index utterance exceeds a predetermined threshold value as a new non-favorable index utterance.

Further, when the utterance extraction unit 52 receives the notification between the co-occurrence network and the node corresponding to the index utterance, the utterance extraction unit 52 utters the utterance whose co-occurrence with the index utterance satisfies a predetermined condition based on the co-occurrence network. Extract as. That is, the utterance extraction unit 52 extracts utterances whose distance from the index utterance on the co-occurrence network satisfies a predetermined condition as new index utterances.

For example, the utterance extraction unit 52 may extract an utterance in which the distance from the favorable index utterance on the co-occurrence network is equal to or less than a predetermined threshold value as a new favorable index utterance. Further, for example, the utterance extraction unit 52 may extract utterances in which the distance from the unfavorable index utterance on the co-occurrence network is equal to or less than a predetermined threshold value as the unfavorable index utterance. Further, for example, the utterance extraction unit 52 sets an utterance in which the distance from the favorable index utterance is equal to or less than a predetermined threshold value and the distance from the non-favorable index utterance is equal to or greater than the predetermined threshold value as a new favorable index utterance. It may be extracted. Further, for example, the utterance extraction unit 52 uses a new non-favorable index utterance for an utterance in which the distance from the unfavorable index utterance is equal to or less than a predetermined threshold value and the distance from the favorable index utterance is equal to or greater than the predetermined threshold value. It may be extracted as.

For example, the utterance extraction unit 52 specifies the distance between the favorable index utterance and the corresponding node and the distance between the non-favorable index utterance and the corresponding node for each node included in the co-occurrence network. Then, the utterance extraction unit 52 corresponds to a node in which the distance between the non-favorable index utterance and the corresponding node is equal to or less than a predetermined threshold value and the distance between the favorable index utterance and the corresponding node exceeds the predetermined threshold value. Make the utterance a new non-favorable index utterance. Further, the utterance extraction unit 52 corresponds to a node in which the distance between the favorable index utterance and the corresponding node is equal to or less than a predetermined threshold value and the distance between the non-favorable index utterance and the corresponding node exceeds the predetermined threshold value. Make the utterance a new favorable index utterance.

Further, when the utterance extraction unit 52 executes the process shown in FIG. 9, the utterance extraction unit 52 acquires a session including the utterance and the response of the user whose usage mode of the dialogue service satisfies a predetermined condition, and determines the frequency of appearance in the acquired session. Based on, the index utterances are extracted. For example, the utterance extraction unit 52 refers to the engagement database 34 and identifies the user's engagement history within a predetermined period. To give a more specific example, the utterance extraction unit 52 specifies the processing date and time, and the history period and history frequency in the last few months.

Then, the utterance extraction unit 52 identifies a user whose history period is longer than a predetermined threshold value and whose history frequency is higher than a predetermined threshold value as a favorable user. Further, the utterance extraction unit 52 identifies a user whose history period is shorter than a predetermined threshold value and whose history frequency is lower than a predetermined threshold value as an unfavorable user. The utterance extraction unit 52 may specify a favorable user or a non-favorable user based on whether or not the prediction period and the prediction frequency registered as the prediction engagement exceed a predetermined threshold value.

Then, the utterance extraction unit 52 calculates a value obtained by dividing the appearance frequency in the session of the unfavorable user by the appearance frequency in the session of the favorable user for each utterance. Then, the utterance extraction unit 52 extracts the utterances having the highest calculated value of N as new non-favorable index utterances, and the utterances having the calculated values of the lower N as new favorable index utterances.

That is, the utterance extraction unit 52 extracts utterances whose appearance frequency exceeds a predetermined threshold value in a session including the utterances of a user whose usage mode satisfies a predetermined condition, that is, a favorable user, as a new favorable index utterance. do. Further, the utterance extraction unit 52 uses a new non-favorable index for utterances in which the frequency of appearance in a session including the utterances of a user whose usage mode does not satisfy a predetermined condition, that is, a non-favorable user exceeds a predetermined threshold value. Extract as an utterance.

Further, when the process shown in FIG. 10 is executed, the utterance extraction unit 52 extracts utterances whose appearance frequency satisfies a predetermined condition from the sessions whose evaluation satisfies a predetermined condition as index utterances. For example, the utterance extraction unit 52 refers to the session evaluation database 32 and identifies a high-satisfaction session and a low-satisfaction session. Further, the utterance extraction unit 52 refers to the session log database 31 and extracts high-satisfaction sessions and low-satisfaction sessions.

Then, the utterance extraction unit 52 calculates the ratio of the appearance frequency in the low satisfaction session and the appearance frequency in the high satisfaction session for each utterance, and extracts the utterance in which the ratio satisfies a predetermined condition as a new index utterance. For example, the utterance extraction unit 52 calculates a value obtained by dividing the frequency of appearance in a low-satisfaction session by the frequency of appearance in a high-satisfaction session, and calculates the top N utterances as new unfavorable index utterances. The utterances with the lower N values are extracted as new favorable index utterances.

The utterance extraction unit 52 registers the index utterances extracted by the various processes described above in the index utterance database 33. In this way, the index utterances registered in the index utterance database 33 are used as index utterances for evaluating the seed index utterances and sessions in the post-registration process.

The network generation unit 53 generates a co-occurrence network for connecting utterances or responses having co-occurrence from the history of utterances and responses. For example, the network generation unit 53 refers to the session log database 31 and identifies each utterance included in the session. Then, the network generation unit 53 sets a node corresponding to each utterance, and generates a co-occurrence network connecting each utterance and the corresponding node based on the co-occurrence of each utterance. Then, the network generation unit 53 provides the co-occurrence network to the index utterance identification unit 51.

[2-5. Evaluation process]
Returning to FIG. 2, the explanation will be continued. The session evaluation processing unit 60 executes an evaluation process for evaluating a session. Hereinafter, the concept of evaluation executed by the session evaluation processing unit 60 and an example of the functional configuration of the session evaluation processing unit 60 will be described with reference to FIGS. 12 to 21.

[2-5-1. Evaluation processing based on index utterances]
For example, the session evaluation processing unit 60 evaluates the session based on the index utterance included in the session. For example, FIG. 12 is a diagram showing the concept of a process of evaluating a session based on an index utterance. For example, when a session contains many favorable index utterances, it is estimated that the user's impression of each response included in the session is good, and therefore the user's satisfaction with the session is estimated to be high. In addition, when there are many unfavorable index utterances in a session, it is estimated that the user's impression of each response included in the session is bad, and therefore the user's satisfaction with the session is estimated to be low.

Therefore, the session evaluation processing unit 60 evaluates the session based on the appearance frequency of the non-favorable index utterance included in the session and the appearance frequency of the favorable index utterance. For example, the session evaluation processing unit 60 refers to the session evaluation database 32 and selects a session that has not been evaluated as an evaluation target. The session evaluation processing unit 60 may select the session for which evaluation has been performed as an evaluation target. For example, the session evaluation processing unit 60 may reselect a session for which a predetermined period has passed since the evaluation was performed as an evaluation target.

Subsequently, the session evaluation processing unit 60 extracts the session to be evaluated from the session log database 31, and refers to the index utterance database 33 to specify the index utterance included in the extracted session. Then, the session evaluation processing unit 60 calculates the ratio between the total appearance frequency of the non-favorable index utterances and the total appearance frequency of the favorable index utterances. For example, the session evaluation processing unit 60 identifies all the utterances registered as non-favorable index utterances among the utterances included in the session to be processed, and sets the total number of the specified utterances as the appearance frequency of the non-favorable index utterances. Specified as the sum of. Further, the session evaluation processing unit 60 identifies all the utterances registered as the favorable index utterances among the utterances included in the session to be processed, and the total number of the specified utterances is the total of the appearance frequencies of the favorable index utterances. Identify as.

Then, the session evaluation processing unit 60 calculates a value obtained by dividing the total appearance frequency of the unfavorable index utterances by the total appearance frequency of the favorable index utterances as a ratio. Further, the session evaluation processing unit 60 calculates the ratio for other sessions in the same manner, extracts the sessions with the highest N calculated values as low satisfaction sessions, and extracts the sessions with the calculated values of the lower N. Extract as a highly satisfied session.

Here, it is considered that there is a difference in the degree of user satisfaction suggested by each index utterance. For example, it is considered that the user who made the utterance "Wow" is more satisfied with the dialogue service than the user who made the utterance "Like". Therefore, the session evaluation processing unit 60 may evaluate the session in consideration of the degree of user satisfaction suggested by the index utterance.

For example, the index utterance database 33 is given an index score indicating how satisfied or unsatisfied the user who uttered the corresponding index utterance is with the dialogue service (see, for example, FIG. 5). ). Such an index score may be, for example, manually provided by cloud sourcing, and is automatically calculated, for example, from the relationship between the dissatisfaction score given to the session and the frequency of appearance of each index utterance. It may be the one. For example, the information providing device 10 may calculate an index score having a higher value as the non-favorable index utterance is included in a session having a higher dissatisfaction score, and the favor is included in a session having a lower dissatisfaction score. An index score with a higher value may be calculated as the target index utterance is made.

When such an index score is given, the session evaluation processing unit 60 executes the following processing. First, the session evaluation processing unit 60 extracts the session to be evaluated from the session log database 31, and refers to the index utterance database 33 to specify the index utterance included in the extracted session.

Then, the session evaluation processing unit 60 calculates the ratio between the total index score of the non-favorable index utterance included in the session to be processed and the total index score of the favorable index utterance. For example, the session evaluation processing unit 60 calculates a value obtained by dividing the total of the index scores of the unfavorable index utterances included in the session to be processed by the total of the index scores of the favorable index utterances as a ratio. The session evaluation processing unit 60 may calculate the ratio from the total of the absolute values of the index scores of each index utterance. Further, the session evaluation processing unit 60 calculates the ratio for other sessions in the same manner, and sets the sessions with the highest N calculated values as low satisfaction sessions and the sessions with the calculated values of the lower N low satisfaction. Let's call it a session. After that, the session evaluation processing unit 60 registers the evaluation result in the session evaluation database 32.

[2-5-2. Evaluation processing based on index utterances]
Further, for example, the session evaluation processing unit 60 evaluates the session using a model for evaluating the session based on the index utterance included in the session. For example, FIG. 13 is a diagram showing the concept of a process of evaluating a session based on an index utterance.

For example, the session evaluation processing unit 60 refers to the session log database 31 and extracts the highly satisfied session and the session other than the highly satisfied session as learning data. For example, the session evaluation processing unit 60 extracts a session evaluated as a highly satisfied session from the session log database 31 based on the evaluation result of the processing shown in FIG. Further, the session evaluation processing unit 60 extracts sessions other than the sessions evaluated as highly satisfied sessions from the session log database 31.

Then, the session evaluation processing unit 60 learns the favorable index utterance evaluation model M4 using the extracted learning data. For example, the session evaluation processing unit 60 outputs a first satisfaction level higher than a predetermined threshold value when inputting a character string of an utterance and a response included in a high satisfaction session, and outputs a first satisfaction level higher than a predetermined threshold value to a session other than the high satisfaction session. When the character strings of the included utterances and responses are input, the favorable index utterance evaluation model M4 is trained so as to output the first satisfaction level having a value lower than a predetermined threshold value. The session evaluation processing unit 60 outputs a value indicating that the session is highly satisfied when the character strings of the utterance and the response included in the highly satisfied session are input, and the utterance and the utterance included in the session other than the highly satisfied session are output. The favorable index utterance evaluation model M4 may be trained so that when the response character string is input, a value indicating that the session is not highly satisfied is output.

It is presumed that the high-satisfaction session contains many favorable index utterances. Therefore, when the above-mentioned learning is performed, the favorable index utterance evaluation model M4 outputs a higher value of the first satisfaction level as the target session to be evaluated contains more favorable index utterances and evaluates it. The lower the value of the first satisfaction is, the lower the target utterance is not included in the target session.

Further, the session evaluation processing unit 60 refers to the session log database 31 and extracts the low-satisfaction session and the session other than the low-satisfaction session as learning data. Then, the session evaluation processing unit 60 learns the non-favorable index utterance evaluation model M3 using the extracted learning data. For example, the session evaluation processing unit 60 outputs a third dissatisfaction degree higher than a predetermined threshold value when inputting a character string of an utterance and a response included in the low satisfaction session, and outputs a third dissatisfaction degree higher than a predetermined threshold value to a session other than the low satisfaction session. The non-favorable index utterance evaluation model M3 is trained so as to output a third dissatisfaction level having a value lower than a predetermined threshold value when the included utterance and response character strings are input. The session evaluation processing unit 60 outputs a value indicating that the session is low satisfaction when the character strings of the utterance and the response included in the low satisfaction session are input, and the utterance and the speech included in the session other than the low satisfaction session are output. The non-favorable index utterance evaluation model M3 may be trained so as to output a value indicating that the session is not a low satisfaction session when the response character string is input.

It is presumed that the low-satisfaction session contains many unfavorable index utterances. Therefore, when the above-mentioned learning is performed, the non-favorable index utterance evaluation model M3 outputs a higher value of the third dissatisfaction as the target session to be evaluated contains more non-favorable index utterances. , The lower the third dissatisfaction level is output so that the target session to be evaluated does not include the unfavorable index utterance.

Then, the session evaluation processing unit 60 evaluates the session using the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4. For example, the session evaluation processing unit 60 inputs the utterance and response character strings of the target session to be evaluated into the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4, and inputs the third dissatisfaction degree and the first. Get satisfaction. Then, the session evaluation processing unit 60 evaluates the session based on the third dissatisfaction degree and the first satisfaction degree.

For example, the session evaluation processing unit 60 may evaluate the target session as a low satisfaction session when the third dissatisfaction degree exceeds a predetermined threshold value. Further, the session evaluation processing unit 60 may evaluate the target session as a highly satisfied session when the first satisfaction level exceeds a predetermined threshold value. Further, the session evaluation processing unit 60 may evaluate the target session as a low satisfaction session when the third dissatisfaction level exceeds a predetermined threshold value and the first satisfaction level is less than the predetermined threshold value. Further, the session evaluation processing unit 60 may evaluate the target session as a highly satisfied session when the third dissatisfaction level is less than the predetermined threshold value and the first satisfaction level is equal to or higher than the predetermined threshold value. Further, the session evaluation processing unit 60 may evaluate the target session by the process shown in FIG.

[About evaluation processing based on 2-5-3.3 value learning]
Here, in various dialogues not limited to the session, the dialogue conducted prior to the favorable index utterance has a high degree of user satisfaction, and the dialogue conducted prior to the non-favorable index utterance is the user's dialogue. It is estimated that the satisfaction level is low. Further, it is considered that the user's impression indicated by the index utterance is not the dialogue after the index utterance but the dialogue immediately before the index utterance.

Therefore, the session evaluation processing unit 60 may extract a series of dialogues corresponding to the index utterances and use the extracted dialogues to learn a model for estimating the user's evaluation for the input dialogues. For example, the session evaluation processing unit 60 may extract utterances and responses exchanged prior to the index utterance, train the characteristics of the extracted utterances in a model, and evaluate the session using the trained model. good. Further, the session evaluation processing unit 60 may train the model to match the characteristics of a neutral dialogue in which the impression by the user is neither favorable nor unfavorable (that is, neither satisfied nor unsatisfied). good.

For example, FIG. 14 is a diagram showing an example of a process of evaluating a session using a ternary evaluation model trained in the characteristics of three types of dialogue. For example, the session evaluation processing unit 60 refers to the session log database 31 and identifies a session including a non-favorable index utterance. Subsequently, the session evaluation processing unit 60 extracts from the session the non-favorable index utterance and a predetermined number of utterances and responses made immediately before the non-favorable index utterance as a non-favorable dialogue example UD. For example, the session evaluation processing unit 60 extracts two utterances made immediately before the non-favorable index utterance "I do not understand the meaning" and the response to these utterances, and the non-favorable index "I do not understand the meaning". Extracted as an example of unfavorable dialogue with utterance.

Further, the session evaluation processing unit 60 refers to the session log database 31 and identifies a session including a favorable index utterance. Subsequently, the session evaluation processing unit 60 extracts the favorable index utterance and a predetermined number of utterances and responses made immediately before the favorable index utterance from the session as a favorable dialogue example FD. For example, the session evaluation processing unit 60 extracts two utterances made immediately before the favorable index utterance of "smart" and the response to these utterances, and is favorable together with the favorable index utterance of "smart". Dialogue example Extract as FD.

Further, the session evaluation processing unit 60 selects one utterance that is not an index utterance as a neutral utterance, and from the session including the selected neutral utterance, a predetermined number of utterances and responses performed immediately before the neutral utterance. Is extracted from the session as an example of neutral dialogue ND. For example, the session evaluation processing unit 60 extracts two utterances made immediately before the neutral utterance "tomorrow's weather" and the response to these utterances, and neutralizes them together with the neutral utterance "tomorrow's weather". Dialogue example Extract as ND.

Then, the session evaluation processing unit 60 causes the ternary evaluation model M5 to learn the characteristics of the non-favorable dialogue example UD, the neutral dialogue example ND, and the favorable dialogue example FD. For example, when the session evaluation processing unit 60 inputs a character string of an utterance and a response included in a non-favorable dialogue example UD, the session evaluation processing unit 60 outputs a value indicating that the input utterance and response are an unfavorable dialogue example. Then, when the character string of the utterance and the response included in the neutral dialogue example ND is input, the value indicating that the input utterance and the response is the neutral dialogue example is output and included in the favorable dialogue example FD. When the character strings of the utterances and responses to be input are input, the ternary evaluation model M5 is trained so as to output a value indicating that the input utterances and responses are examples of favorable dialogue. That is, the session evaluation processing unit 60 determines whether the input dialogue is satisfied by the user (whether it is a favorable dialogue), is not satisfied (whether it is a non-favorable dialogue), or other than that. Learn a model that is classified into three categories (whether it is a neutral dialogue).

Then, the session evaluation processing unit 60 evaluates the target session using the generated ternary evaluation model M5. For example, the session evaluation processing unit 60 inputs the character strings of the utterance and the response included in the target session into the trivalue evaluation model M5, and the user is satisfied with the target session based on the output of the trivalue evaluation model M5. Evaluate whether it is a highly satisfied session (ie, a highly satisfied session), whether the user is not satisfied (ie, a low satisfaction session), or otherwise (ie, a neutral session).

In addition, when extracting each dialogue example, the session evaluation processing unit 60 may extract an arbitrary number of utterances and responses made immediately before the index utterance or the neutral utterance. In this way, by having the model learn the characteristics of the utterances and responses made immediately before the index utterances and the neutral utterances, it is possible to improve the accuracy of the evaluation of the dialogue by the model and the evaluation of the session.

[2-5-4. About engagement-based evaluation processing]
FIG. 15 is a diagram showing an example of a process of evaluating a session based on engagement. For example, it is considered that a user who is highly satisfied with the dialogue has a high usage period and frequency of use, and a user who is not satisfied with the dialogue has a low usage period and frequency of use. Therefore, it is presumed that the session including the utterance of the user with high engagement such as the usage period and the frequency of use has a high satisfaction level, and the session including the utterance of the user with low engagement has a low satisfaction level. Therefore, the session evaluation processing unit 60 evaluates the session based on the engagement of the user who made the utterance included in the session.

For example, the session evaluation processing unit 60 refers to the engagement database 34 and identifies a user whose history period and history frequency within a predetermined period exceed a predetermined threshold value. Subsequently, the session evaluation processing unit 60 extracts a session including the utterance of the specified user from the session log database 31, and sets the extracted session as the session of the user with high engagement.

On the other hand, the session evaluation processing unit 60 refers to the engagement database 34 and identifies a user whose history period and history frequency within a predetermined period are less than a predetermined threshold value. Subsequently, the session evaluation processing unit 60 extracts a session including the utterance of the specified user from the session log database 31, and sets the extracted session as the session of the user with low engagement.

Then, the session evaluation processing unit 60 causes the engagement session evaluation model M6 to learn the characteristics of the session of the user with high engagement and the session of the user with low engagement. For example, when the session evaluation processing unit 60 inputs an utterance and a response included in the session of a user with high engagement, the session evaluation processing unit 60 outputs a value indicating that the user is highly satisfied with the session, and the session of the user with low engagement. When the utterance and the response included in the above are input, the engagement session evaluation model M6 is learned so as to output a value indicating that the user's satisfaction with the session is low.

For example, when the session evaluation processing unit 60 inputs an utterance and a response included in the session of a user with high engagement, the session evaluation processing unit 60 outputs an evaluation score having a value higher than a predetermined threshold value and is included in the session of a user with low engagement. When an utterance and a response are input, the engagement session evaluation model M6 is trained so as to output an evaluation score having a value lower than a predetermined threshold value. That is, the session evaluation processing unit 60 uses the learning data extracted based on the engagement to classify the input session into a session with high user satisfaction or a session with low user satisfaction. The evaluation model M6 is learned.

Then, the session evaluation processing unit 60 evaluates the target session using the engagement session evaluation model M6. For example, the session evaluation processing unit 60 is satisfied with the target session based on the evaluation score output by the engagement session evaluation model M6 when the character strings of the utterance and the response included in the target session are input. Evaluate whether or not it is.

[2-5-5. Evaluation processing based on repeated utterances]
Further, FIG. 16 is a diagram showing a concept of a process of evaluating a session based on repeated utterances. For example, if the same or similar utterances are repeated in a session, the user's satisfaction with the session is low, and if the same or similar utterances are not repeated in the session, the user's satisfaction with the session. The degree is estimated to be high. Therefore, the session evaluation processing unit 60 may evaluate the session based on the repeated utterances included in the session.

For example, the session evaluation processing unit 60 counts the number of pairs having similar notations as the number of notation-similar pairs from the utterances included in the target session. For example, in the session evaluation processing unit 60, the number of pairs whose similarity exceeds a predetermined threshold such as Jaccard coefficient, Dice coefficient, Simpson coefficient, etc., and the similarity based on various editing distances such as Levenshtein distance exceed a predetermined threshold. Count the number of pairs. Further, the session evaluation processing unit 60 may calculate the number of pairs whose semantic similarity exceeds a predetermined threshold value as the number of semantic similarity pairs by using word2vec, GloVe, or the like.

Then, when the value of the number of notation-like pairs or the number of semantic-similar pairs extracted from the target session exceeds a predetermined threshold value, the session evaluation processing unit 60 evaluates the target session as a low-satisfaction session and extracts it from the target session. If the value of the number of similar notation pairs or the number of semantically similar pairs is below a predetermined threshold, the target session is evaluated as a highly satisfied session.

Further, the session evaluation processing unit 60 may learn the repetitive utterance evaluation model M7 that evaluates the target session by letting the model learn the characteristics of the session including the repetitive utterance. For example, when the session evaluation processing unit 60 inputs a session including repeated utterances (for example, a session in which the number of pairs exceeds a predetermined threshold), the session evaluation processing unit 60 outputs that the input session is a low satisfaction session. The iterative speech evaluation model M7 may be trained. Further, the session evaluation processing unit 60 may evaluate the target session by using such a repetitive utterance evaluation model M7.

[2-5-6. About automatic generation of repeated utterances]
Here, the session evaluation processing unit 60 may automatically generate a session including repeated utterances and let the repeated utterance evaluation model M7 learn the characteristics of the generated session. For example, FIG. 17 is a diagram showing the concept of a process of evaluating a session using a model trained in the characteristics of automatically generated data. For example, the session evaluation processing unit 60 refers to the session log database 31 and extracts any utterance as a seed utterance. For example, the session evaluation processing unit 60 extracts an utterance such as "Today's weather is" as a seed utterance.

Subsequently, the session evaluation processing unit 60 generates a plurality of utterances similar to the seed utterances from the extracted seed utterances. For example, in order to imitate a voice recognition error, a typo, or the like, the session evaluation processing unit 60 randomly deletes words or characters included in a character string of a seed utterance such as "Today's weather is", or sounds similar. Generate multiple utterances replaced with words or strings. To give a more specific example, the session evaluation processing unit 60 uses the character string of the seed utterance "Today's weather is" to "Today's electricity", "Industry's electricity", and "Today's electricity". Is generated as a duplicate utterance.

Further, the session evaluation processing unit 60 extracts one or a plurality of utterances different from the seed utterances as mixed utterances from the session log database 31. For example, the session evaluation processing unit 60 extracts utterances such as "Aries' fortune" as mixed utterances. Then, the session evaluation processing unit 60 randomly rearranges the seed utterance, the plurality of duplicate utterances generated from the seed utterance, and the mixed utterance, and generates a session consisting of pseudo utterances as a repetitive session RS. For example, in the example shown in FIG. 17, the session evaluation processing unit 60 includes the character string “Today's utterance”, which is a seed utterance, as the user's utterance “Usr: Today's utterance”, and is a duplicate utterance “Today's utterance”. The character string "Today's electricity is" is included as the user's utterance "Usr: Today's electricity is", and the character string "Aries' fortune", which is a mixed utterance, is included in the user's utterance "Usr: Aries". Included as "fortune" and re-included the seed utterance "Today's weather" as the user's utterance "Usr: Today's weather", and the duplicate utterance "Business weather" and "Business weather" Generates a repetitive session RS that includes the user's utterances "Usr: Business weather" and "Usr: Today's electricity".

That is, the session evaluation processing unit 60 generates a session including pseudo-repeated utterances. The session evaluation processing unit 60 may extract a plurality of seed utterances and generate duplicate utterances from each seed utterance.

Further, the session evaluation processing unit 60 refers to the session log database 31 and randomly extracts a plurality of utterances. Then, the session evaluation processing unit 60 generates a session in which the extracted utterances are randomly rearranged as a non-repetitive session NRS. Then, the session evaluation processing unit 60 learns the iterative utterance evaluation model M7 using the iterative session RS and the non-repetitive session NRS as learning data. For example, the session evaluation processing unit 60 outputs a score indicating that the user is not satisfied when the iterative session RS is input, and the user is satisfied when the non-repeated session NRS is input. The iterative utterance evaluation model M7 is trained so as to output a score indicating that the score is correct.

Then, the session evaluation processing unit 60 evaluates the target session using the iterative utterance evaluation model M7. For example, the session evaluation processing unit 60 evaluates whether or not the user is satisfied with the target session based on the value of the score output by the repeated utterance evaluation model M7 when the target session is input.

[2-5-7. Evaluation of repeated utterances using images]
Here, the session evaluation processing unit 60 may evaluate repeated utterances using images. For example, FIG. 18 is a diagram showing the concept of evaluation of repeated utterances using images. For example, set a square image with the same number of areas in the vertical and horizontal directions, and map each character in the spoken string contained in a session to the vertical and horizontal directions of the image, vertically and horizontally. When an image is generated in which an area to which the same characters are associated in the direction is filled, such an image becomes a distribution image showing the distribution of the same characters included in the character string of the utterance. Such a distribution image includes an area filled to form a diagonal line when the utterance string contains repetitions. That is, if utterances similar to a session are included, diagonal lines will appear in the distribution image.

In recent years, advances in image analysis technology have improved the accuracy of determining similar images using a model using DNN or the like. Therefore, it is considered that learning the characteristics of the distributed image can more accurately learn the characteristics of the session including the repeated utterances than simply learning the characteristics of the character string of the utterance included in the session.

Therefore, the session evaluation processing unit 60 may make the iterative evaluation model M1 learn the features of the distribution image and evaluate the target session using the iterative evaluation model M1. For example, as shown in FIG. 16, the session evaluation processing unit 60 extracts from the session log database 31 a session in which the number of notation-like pairs and semantic-similar pairs exceeds a predetermined threshold value as a session including repeated utterances. The session evaluation processing unit 60 may acquire the iterative session RS generated by the method shown in FIG. 17 as a session including repeated utterances. Then, the session evaluation processing unit 60 generates a distribution image from each utterance of the session including the repeated utterances.

On the other hand, as shown in FIG. 16, the session evaluation processing unit 60 extracts from the session log database 31 a session in which the number of notation-like pairs and semantic-similar pairs is less than a predetermined threshold value as a session that does not include repeated utterances. The session evaluation processing unit 60 may acquire the non-repetitive session NRS generated by the method shown in FIG. 17 as a session that does not include repeated utterances. Then, the session evaluation processing unit 60 generates a distribution image from each utterance of the session that does not include repeated utterances.

Then, the session evaluation processing unit 60 repeatedly causes the evaluation model M1 to learn the features of the generated distribution image. For example, when the distribution image generated from each utterance of the session including the repeated utterance is input, the session evaluation processing unit 60 outputs a score indicating that the user is not satisfied, and the session does not include the repeated utterance. When the distribution image generated from each utterance of is input, the evaluation model M1 is repeatedly trained so as to output a score indicating that the user is satisfied.

Subsequently, the session evaluation processing unit 60 generates a distribution image from each utterance included in the target session, and repeatedly inputs the generated distribution image into the evaluation model M1. Then, the session evaluation processing unit 60 evaluates whether or not the user is satisfied with the target session based on the output score of the iterative evaluation model M1.

[2-5-8. Evaluation based on apology response]
Further, FIG. 19 is a diagram showing a concept of a process of evaluating a session based on an apology response. As shown in FIG. 19, it is estimated that the user's satisfaction with the session including the apology response is low, and the user's satisfaction with the session without the apology response is high. Therefore, the session evaluation processing unit 60 may evaluate the session based on the apology response.

For example, the session evaluation processing unit 60 refers to the session log database 31 and extracts a session including a predetermined apology response registered in advance. Further, the session evaluation processing unit 60 extracts a session that does not include a predetermined apology response. Then, the session evaluation processing unit 60 causes the apology evaluation model M2 to learn the characteristics of the extracted session. For example, the session evaluation processing unit 60 outputs a score indicating that the user is not satisfied when a session including an apology response is input, and uses the session when a session not including an apology response is input. The apology evaluation model M2 is trained so as to output a score indicating that the person is satisfied.

Subsequently, the session evaluation processing unit 60 inputs the target session into the apology evaluation model M2. Then, the session evaluation processing unit 60 evaluates whether or not the user is satisfied with the target session based on the output score of the apology evaluation model M2.

[2-5-9. About the combination of each evaluation standard]
The session evaluation processing unit 60 may evaluate the target session by combining the above-mentioned evaluation methods. For example, FIG. 20 is a diagram showing a concept of a process of evaluating a session by combining a plurality of evaluation methods. As shown in FIG. 20, the session evaluation processing unit 60 includes a repetitive evaluation model M1, an apology evaluation model M2, a non-favorable index speech evaluation model M3, a favorable index speech evaluation model M4, a three-value evaluation model M5, and an engagement session evaluation. A model M6 and a repetitive speech evaluation model M7 are generated. Then, the session evaluation processing unit 60 inputs the target session and the distribution image generated from the target session into each of the models M1 to M7, and evaluates the target session.

For example, the iterative evaluation model M1 outputs an evaluation that the user's satisfaction is low when the distributed image of the target session is similar to the distributed image of the repeated session, that is, when the target session has repeated utterances. Further, the apology evaluation model M2 outputs an evaluation that the user's satisfaction is low when the target session includes an apology utterance. Further, the non-favorable index utterance evaluation model M3 outputs an evaluation that the user's satisfaction is low when the target session includes the non-favorable index utterance. Further, the favorable index utterance evaluation model M4 outputs an evaluation that the user's satisfaction is high when the favorable index utterance is included in the target session.

Further, the ternary evaluation model M5 outputs an evaluation that the user's satisfaction is high when the target session includes a dialogue having similar characteristics to the dialogue that the user is satisfied with. Further, in the engagement session evaluation model M6, when the characteristics of the target session are similar to the session including the utterance of the user with high engagement, that is, when the characteristics are similar to the session including the utterance of the user with high satisfaction. , Outputs an evaluation that the user's satisfaction is high. Further, the repeated utterance evaluation model M7 outputs an evaluation indicating that the user's satisfaction level is low when the target session has repeated utterances.

The session evaluation processing unit 60 evaluates the target session based on the combination of the evaluation results of each of the models M1 to M7. For example, the session evaluation processing unit 60 counts the number of models determined to have high user satisfaction with the target session and the number of models determined to have low user satisfaction, and determines that the satisfaction level is high. If the number of determined models is larger than the number of models determined to be low in satisfaction, it may be determined that the user's satisfaction with the target session is high. That is, the session evaluation processing unit 60 may evaluate the target session by majority vote.

Further, the session evaluation processing unit 60 may evaluate the target session by using the evaluation results of some of the models M1 to M7. Further, for example, the session evaluation processing unit 60 may integrate the scores output by the models M1 to M7. For example, the session evaluation processing unit 60 calculates a score obtained by subtracting the scores of the models M4 to M6 from the sum of the scores of the models M1, M2, M3, and M7, and evaluates the target session based on the calculated score values. You may.

Further, the session evaluation processing unit 60 may further learn the evaluation results using the models M1 to M7. For example, the session evaluation processing unit 60 causes a model to learn a session evaluated to have high user satisfaction and a session evaluated to have low user satisfaction as learning data, and uses such a model. Then, the target session may be evaluated. Further, the index utterance extraction processing unit 50 may extract the index utterance based on the evaluation result by the session evaluation processing unit 60. For example, the index utterance extraction processing unit 50 extracts favorable index utterances from a session evaluated as having high satisfaction by the session evaluation processing unit 60, that is, a highly satisfied session, and the session evaluation processing unit 60 has low satisfaction. The unfavorable index utterances may be extracted from the sessions evaluated as, that is, the low satisfaction sessions.

[2-5-10. About automatic expansion of labeled data]
Here, the session evaluation processing unit 60 may automatically expand the session data used for learning the models M1 to M7. For example, FIG. 21 is a diagram illustrating the concept of a process for automatically expanding a labeled session.

For example, the session evaluation processing unit 60 extracts each utterance included in a session with low user satisfaction, that is, a low satisfaction session, and extracts other utterances similar to each extracted utterance from the session log database 31. Search for. Further, the session evaluation processing unit 60 extracts each response included in the low satisfaction session, and searches the session log database 31 for other responses similar to each extracted response.

Then, the session evaluation processing unit 60 pseudo-generates a new session by combining the searched utterances and responses, and treats the generated new session as a low-satisfaction session for learning during learning of the models M1 to M7. It may be adopted as data.

[2-5-11. About the functional configuration of the session evaluation processing unit]
Next, an example of the functional configuration of the session evaluation processing unit 60 will be described with reference to FIG. 22. FIG. 22 is a diagram showing an example of the functional configuration of the session evaluation processing unit according to the embodiment. As shown in FIG. 22, the session evaluation processing unit 60 includes a learning data extraction unit 61, a learning unit 62, an image generation unit 63, an utterance group generation unit 64, and an evaluation unit 65.

The learning data extraction unit 61 extracts a session to be learning data. For example, as shown in FIG. 12, the learning data extraction unit 61 sets a session in which the frequency of appearance of favorable index utterances and the total of the index scores of favorable index utterances satisfy a predetermined condition in the favorable index utterance evaluation model M4. Extract as a highly satisfying session that will be the learning data of. Further, the learning data extraction unit 61 extracts the sessions other than the high satisfaction session as the sessions other than the high satisfaction session which is the learning data of the favorable index speech evaluation model M4.

Further, for example, as shown in FIG. 12, the learning data extraction unit 61 unfavorable a session in which the frequency of appearance of non-favorable index utterances and the total of the index scores of non-favorable index utterances satisfy a predetermined condition. It is extracted as a highly satisfying session that is the learning data of the index utterance evaluation model M3. Further, the learning data extraction unit 61 extracts the sessions other than the low satisfaction session as the sessions other than the low satisfaction session which is the learning data of the unfavorable index speech evaluation model M3.

The learning data extraction unit 61 refers to the session evaluation database 32, identifies the high-satisfaction session and the low-satisfaction session, and identifies the specified high-satisfaction session and low-satisfaction session as the non-favorable index speech evaluation model M3 and the favorable index. It may be extracted from the session log database 31 as the training data of the utterance evaluation model M4.

Further, as shown in FIG. 15, the learning data extraction unit 61 may extract a session to be learning data of the engagement session evaluation model M6 based on the engagement of the user. For example, the learning data extraction unit 61 refers to the engagement database 34 and specifies the usage mode of each user, that is, the history period and the history frequency. Then, the learning data extraction unit 61 identifies a user whose specified usage mode satisfies a predetermined condition, and extracts a session including the utterance of the specified user from the session log database 31.

For example, the learning data extraction unit 61 identifies a user whose history period or history frequency exceeds a predetermined threshold value, and sets a session including the utterance of the specified user as a session of a user with high engagement from the session log database 31. Extract. Further, the learning data extraction unit 61 identifies a user whose history period or history frequency is below a predetermined threshold value, and sets a session including the utterance of the specified user as a session of a user with low engagement from the session log database 31. Extract. Then, the learning data extraction unit 61 may use each extracted session as the learning data of the engagement session evaluation model M6.

That is, the learning data extraction unit 61 extracts the session including the utterance of the user whose usage mode satisfies the predetermined condition as the learning data having the first attribute that the user's satisfaction is high. Further, the learning data extraction unit 61 extracts a session including an utterance of a user whose usage mode does not satisfy a predetermined condition as learning data having a second attribute that the user's satisfaction level is low.

Further, as shown in FIG. 16, the learning data extraction unit 61 extracts from the session log database 31 a session having a large number of notation-like pairs and a large number of semantic-similar pairs as a low-satisfaction session, and the number of notation-like pairs and the number of notation-like pairs. Meaning Sessions with a small number of similar pairs are extracted as highly satisfied sessions. Then, the learning data extraction unit 61 may use the extracted session as a session that is the source of the learning data of the iterative evaluation model M1 and the iterative utterance evaluation model M7.

Further, as shown in FIG. 19, the learning data extraction unit 61 extracts from the session log database 31 a session including an apology response and a session not including an apology response as training data of the apology evaluation model M2. ..

In addition to the above-mentioned processing, the learning data extraction unit 61 may extract sessions useful for learning each of the models M1 to M7 as learning data. For example, the learning data extraction unit 61 may extract a high-satisfaction session or a low-satisfaction session as learning data of each model M1 to M7 based on the evaluation result of the session registered in the session evaluation database 32.

The learning unit 62 learns each model M1 to M7. For example, as shown in FIG. 13, the learning unit 62 learns the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4 as the learning data extracted by the learning data extraction unit 61.

For example, as shown in FIG. 13, the learning unit 62 causes the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4 to learn the characteristics of the session in which the evaluation satisfies a predetermined condition. For example, the learning unit 62 learns the favorable index speech evaluation model M4 by using the highly satisfied session extracted by the learning data extraction unit 61 and the sessions other than the highly satisfied session as learning data. That is, the learning unit 62 learns the favorable index utterance evaluation model M4 that classifies the input target session into a highly satisfied session and another session.

Further, the learning unit 62 learns the unfavorable index speech evaluation model M3 using the low-satisfaction session extracted by the learning data extraction unit 61 and the sessions other than the low-satisfaction session as learning data. That is, the learning unit 62 learns the non-favorable index utterance evaluation model M3 that classifies the input target session into a low-satisfaction session and a non-satisfaction session.

Further, the learning unit 62 learns the ternary evaluation model M5 as learning data of the dialogue example generated by the utterance group generation unit 64, as shown in FIG. That is, the learning unit 62 learns the ternary evaluation model M5 using a series of dialogues corresponding to the index utterance as learning data. For example, the utterance group generation unit 64 extracts a predetermined number of utterances and responses made immediately before the unfavorable index utterance as a non-favorable dialogue example UD, and determines a predetermined number of utterances and responses performed immediately before the favorable index utterance. A number of utterances and responses are extracted as a favorable dialogue example FD, and a predetermined number of utterances and responses made immediately before the neutral utterance are extracted as a neutral dialogue example ND. In such a case, the learning unit 62 learns the ternary evaluation model M5 that evaluates the target session using the non-favorable dialogue example UD, the favorable dialogue example FD, and the neutral dialogue example ND as learning data. That is, the learning unit 62 learns a model that estimates the user's evaluation of the dialogue in the target session.

For example, the learning unit 62 uses the non-favorable dialogue example UD as the learning data of the dialogue for which the evaluation by the user is unfavorable. Further, the learning unit 62 uses the favorable dialogue example FD as the learning data of the dialogue for which the evaluation by the user is favorable. Further, the learning unit 62 uses the neutral dialogue example ND as the learning data of the dialogue whose evaluation by the user is neutral. Then, the learning unit 62 trains the characteristics of the learning data in the ternary evaluation model M5, so that the evaluation by the user is input into a non-favorable dialogue, a favorable dialogue, or a neutral dialogue. Realize a model that classifies dialogues.

Further, as shown in FIG. 15, the learning unit 62 learns the engagement session evaluation model M6 by using the sessions of users with high engagement and the sessions of users with low engagement extracted by the learning data extraction unit 61 as learning data. .. That is, the learning unit 62 resembles the input target session to a session similar to a session by a user with high engagement (hereinafter, referred to as "session of the first attribute") or a session by a user with low engagement. Engagement session evaluation model M6 is trained to be classified into sessions (hereinafter referred to as "second attribute sessions").

Here, a session by a user with high engagement is estimated to have a high degree of satisfaction by the user, and a session by a user with low engagement is estimated to have a low degree of satisfaction by the user. Therefore, the session of the first attribute is a session in which the evaluation by the user is favorable, that is, a highly satisfied session, and the session of the second attribute is a session in which the evaluation by the user is unfavorable, that is, a low satisfaction. It can be said that it is a session. Therefore, the learning unit 62 can learn the engagement session evaluation model M6 for determining whether the user's impression of the target content is favorable or unfavorable.

Further, as shown in FIGS. 17 and 18, the learning unit 62 learns the repetitive utterance evaluation model M7 and the repetitive evaluation model M1 using the session including the repetitive utterance as learning data. For example, when the learning unit 62 acquires the iterative session RS generated from the seed utterance and the mixed utterance and the non-repetitive session NRS by the utterance group generation unit 64 by the method shown in FIG. 17, the acquired iterative session RS and the non-repetitive session RS are acquired. The iterative utterance evaluation model M7 is trained using the session NRS as training data. That is, the learning unit 62 learns the repetitive utterance evaluation model M7 that classifies the target session into a session including repeated utterances or a session not including repeated utterances.

Here, it is presumed that the user's impression is unfavorable for the session including the repeated utterance, and the user's impression is favorable for the session not including the repeated utterance. Therefore, the learning unit 62 can learn the iterative utterance evaluation model M7 that determines whether the user's impression of the target content is favorable or unfavorable.

Further, the learning unit 62 acquires the distribution image generated by the image generation unit 63 from the session including the repeated utterances and the distribution image generated by the image generation unit 63 from the session not including the repeated utterances as learning data. In such a case, the learning unit 62 learns the iterative evaluation model M1 using these distribution images as learning data. More specifically, the learning unit 62 converts the input image into an image similar to a distribution image generated from a session including repeated utterances, or an image similar to a distribution image generated from a session not including repeated utterances. The iterative evaluation model M1 to be classified is trained.

Here, diagonal lines are likely to appear in the distribution image of the session including the repeated utterances, and the diagonal lines are less likely to appear in the distribution image of the sessions not including the repeated utterances. Further, it is presumed that the user's impression is unfavorable for the session including the repeated utterance, and the user's impression is favorable for the session not including the repeated utterance. Therefore, the learning unit 62 determines whether the user's impression of the target session is favorable or unfavorable by classifying the distribution image in which the utterance character strings of the target session are associated vertically and horizontally. The iterative evaluation model M1 can be learned.

Further, as shown in FIG. 19, the learning unit 62 learns the apology evaluation model M2 using the session extracted by the learning data extraction unit 61 as learning data. For example, the learning unit 62 learns the apology evaluation model M2 that classifies the input target session into a session similar to a session including an apology response or a session similar to a session not including an apology response.

Here, it is presumed that the user's impression is unfavorable for the session including the apology response, and the user's impression is favorable for the session not including the apology response. Therefore, the learning unit 62 can learn the apology evaluation model M2 for determining whether the user's impression of the target session is favorable or unfavorable based on the apology response.

As shown in FIG. 21, the learning unit 62 may generate a new low-satisfaction session from the low-satisfaction session acquired as learning data and add it to the learning data. For example, the learning unit 62 extracts each utterance and a response similar to each response included in the session acquired as learning data from the session log database 31, and combines the extracted utterances and responses for new learning. Data may be generated.

Here, the learning unit 62 may give the same label as the learning data that is the generation source to the generation of the new learning data. For example, the learning unit 62 has a user's impression from a dialogue example in which the user's impression is favorable, such as a session with many favorable index utterances, a session of a user with high engagement, a favorable dialogue example FD, and a non-repetitive session NRS. May generate a new favorable dialogue example and use it as learning data. Further, for example, the learning unit 62 is based on dialogue examples in which the user's impression is unfavorable, such as a session with many unfavorable index utterances, a session of a user with low engagement, a non-favorable dialogue example ND, and a repetitive session RS. A dialogue example in which the user's impression is unfavorable may be newly generated and used as learning data.

The image generation unit 63 generates a distribution image. For example, the image generation unit 63 acquires a session including repeated utterances extracted by the learning data extraction unit 61 and a session not including repeated utterances. Further, the image generation unit 63 acquires the repetitive session RS and the non-repetitive session NRS generated as shown in FIG. 17 by the utterance group generation unit 64. As shown in FIG. 21, the image generation unit 63 may generate a new session from the acquired session.

Then, the image generation unit 63 generates a distribution image indicating whether or not the utterance characters included in the acquired session are the same as each other. That is, the image generation unit 63 generates an image showing the distribution of the same characters in the session. For example, when the image generation unit 63 associates the utterance character string included in the session with each area in the vertical direction and each area in the horizontal direction of the image, the same characters correspond to each other in the vertical direction and the horizontal direction. Generates an image with a predetermined color for the attached area.

The utterance group generation unit 64 extracts the non-favorable dialogue example UD, the favorable dialogue example FD, and the neutral dialogue example ND. That is, the utterance group generation unit 64 identifies the index utterance from the history in the dialogue service, and extracts a series of dialogues corresponding to the specified index utterance as the dialogue group. For example, the utterance group generation unit 64 extracts a predetermined number of previous responses and utterances from the index utterance as learning data. It should be noted that the number of responses and utterances to be extracted as dialogue examples from the responses and utterances prior to the index utterance can be arbitrarily set.

For example, the utterance group generation unit 64 refers to the session log database 31 and identifies a session including a favorable index utterance. Subsequently, the utterance group generation unit 64 extracts a predetermined number of responses and utterances preceding the favorable index utterance as a favorable dialogue example FD. Similarly, the utterance group generation unit 64 identifies a session including a non-favorable index utterance, and extracts a predetermined number of responses and utterances before the non-favorable index utterance as a non-favorable dialogue example UD. Further, the utterance group generation unit 64 identifies a session including a neutral utterance, and extracts a predetermined number of responses and utterances before the neutral utterance as a neutral dialogue example ND.

In addition, the utterance group generation unit 64 generates a repetitive session RS (that is, the first utterance group) and a non-repetitive session NRS (that is, the second utterance group). For example, the utterance group generation unit 64 selects one of the utterances from the history in the dialogue service, and generates an utterance group including a plurality of utterances that are the same as or similar to the selected utterance.

To give a more specific example, the utterance group generation unit 64 extracts the seed utterance from the session log database 31 and generates a plurality of duplicate utterances similar to the seed utterance. Further, the utterance group generation unit 64 extracts mixed utterances from the session log database 31. Then, the utterance group generation unit 64 randomly rearranges the seed utterance, the plurality of duplicate utterances, and the mixed utterances, and generates a session consisting of pseudo utterances as a repetitive session RS. Further, the utterance group generation unit 64 refers to the session log database 31, randomly extracts a plurality of utterances, and randomly rearranges the extracted utterances to generate a session as a non-repetitive session NRS.

The utterance group generation unit 64 is an utterance and response similar to the utterance and response included in the non-favorable dialogue example UD, the favorable dialogue example FD, or the neutral dialogue example ND (hereinafter collectively referred to as “dialogue example”). May be generated and a new dialogue example may be generated by combining the extracted utterances and responses. For example, the utterance group generation unit 64 may generate a new non-favorable dialogue example UD from the non-favorable dialogue example UD by the method shown in FIG. 21, and a new favorable dialogue example from the favorable dialogue example FD. An FD may be generated, or a new neutral dialogue example ND may be generated from the neutral dialogue example ND. Further, the utterance group generation unit 64 may generate a new iterative session RS from the iterative session RS, or may generate a new non-repetitive session NRS from the non-repetitive session NRS.

The evaluation unit 65 evaluates the session. For example, the evaluation unit 65 selects a target session to be evaluated from the session log database 31. Then, the evaluation unit 65 refers to the index utterance database 33, and evaluates the target session based on the appearance frequency of the index utterances extracted by the index utterance extraction processing unit 50 and the index utterances as seeds.

For example, the evaluation unit 65 evaluates the target session based on whether or not a non-favorable index utterance or a favorable index utterance is included. For example, as shown in FIG. 12, the evaluation unit 65 determines whether the target session is a low-satisfaction session based on the ratio of the frequency of occurrence of non-favorable index utterances to the frequency of occurrence of favorable index utterances. Evaluate whether it is a highly satisfying session. Further, the evaluation unit 65 determines whether the target session is a low-satisfaction session or a high-satisfaction session based on the ratio of the total index score of the non-favorable index utterance and the total index score of the favorable index utterance. May be evaluated.

Here, the index utterance used for the evaluation of the session includes the index utterance extracted by the index utterance extraction processing unit 50, that is, the index utterance extracted by co-occurrence with the seed index utterance. Therefore, in the evaluation unit 65, for example, the frequency of appearance of utterances in which co-occurrence with unfavorable index utterances satisfies a predetermined condition, and the appearance of utterances in which co-occurrence with favorable index utterances satisfy a predetermined condition The target session will be evaluated based on the frequency.

Further, the index utterance used for the evaluation of the session includes the index utterance extracted by the index utterance extraction processing unit 50 based on the user's engagement. That is, the index utterance is an index utterance extracted based on the frequency of appearance in the session of the user whose usage mode satisfies the predetermined condition and the frequency of appearance in the session of the user whose usage mode does not satisfy the predetermined condition. Is included. Therefore, the evaluation unit 65 evaluates the target session based on the utterance of the user whose usage mode satisfies the predetermined condition and the utterance of the user whose usage mode does not satisfy the predetermined condition.

In addition, the index utterances used to evaluate the session include index utterances extracted based on the ratio of the frequency of appearance in the low-satisfaction session to the frequency of appearance in the high-satisfaction session. Therefore, the evaluation unit 65 evaluates the target session based on the utterances in which the appearance frequency in the low satisfaction session satisfies a predetermined condition and the utterances in which the appearance frequency in the high satisfaction session satisfies a predetermined condition.

Further, as shown in FIG. 13, the evaluation unit 65 may evaluate the target session by using the favorable index utterance evaluation model M4 that has learned the characteristics of the highly satisfied session. Further, the evaluation unit 65 may evaluate the target session by using the non-favorable index utterance evaluation model M3 that has learned the characteristics of the low satisfaction session. For example, the evaluation unit 65 reads the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4 from the model database 35, and sets the target session in the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4. input. Then, the evaluation unit 65 may evaluate the target session based on the score output by the non-favorable index utterance evaluation model M3 or the favorable index utterance evaluation model M4. For example, when the evaluation unit 65 is classified by the non-favorable index utterance evaluation model M3 as being similar to a session including a non-favorable index utterance, the evaluation unit 65 evaluates the target session as a low-satisfaction session and evaluates the favorable index utterance. If the model M4 classifies the session as similar to a session containing a favorable index utterance, the subject session may be evaluated as a highly satisfied session.

Further, as shown in FIG. 14, the evaluation unit 65 uses a ternary evaluation model M5 that has learned the characteristics of the non-favorable dialogue example UD, the neutral dialogue example ND, and the favorable dialogue example FD, and uses the target session. Evaluation may be performed. For example, the evaluation unit 65 may input the target session into the ternary evaluation model M5 read from the model database 35 and evaluate the target session based on the classification result by the ternary evaluation model M5. For example, when the evaluation unit 65 classifies the target session as similar to the non-favorable dialogue example UD by the trivalent evaluation model M5, the evaluation unit 65 evaluates the target session as a low-satisfaction session and resembles the favorable dialogue example FD. If classified, the target session may be evaluated as a highly satisfied session.

Further, as shown in FIG. 15, the evaluation unit 65 may evaluate the target session based on the similarity with the session including the utterance of the user whose usage mode satisfies a predetermined condition. For example, the evaluation unit 65 inputs a target session into the engagement session evaluation model M6 that has learned the characteristics of the session of the user with high engagement and the session of the user with low engagement, and based on the classification result of the engagement session evaluation model M6. , The target session may be evaluated. For example, the evaluation unit 65 may evaluate the target session as a highly satisfied session when the target session is classified by the engagement session evaluation model M6 as being similar to the session of a user with high engagement.

Further, as shown in FIGS. 16 to 18, the evaluation unit 65 may evaluate the target session based on repeated utterances. For example, when the same or similar character string appears repeatedly in the target session, the evaluation unit 65 evaluates the target session as a low satisfaction session, and the same or similar character string does not repeatedly appear in the target session. In that case, the target session may be evaluated as a highly satisfied session.

Further, the evaluation unit 65 may evaluate the target session based on the number of pairs of utterances in which the similarity of the character strings and the semantic similarity exceeds a predetermined threshold value. For example, as shown in FIG. 16, when the number of notation-like pairs or the number of semantic-similar pairs included in the target session exceeds a predetermined threshold value, the evaluation unit 65 evaluates the target session as a low-satisfaction session and notates it. If the number of similar pairs or the number of semantically similar pairs is below a predetermined threshold, the target session may be evaluated as a highly satisfied session.

Further, as shown in FIG. 17, the evaluation unit 65 may evaluate the target session by using the repetitive utterance evaluation model M7 that has learned the characteristics of the session including the repetitive utterance. For example, when the evaluation unit 65 inputs the target session into the iterative utterance evaluation model M7 and the iterative utterance evaluation model M7 classifies the target session into a session similar to the iterative session RS, the evaluation unit 65 evaluates the target session as a low satisfaction session. However, when the target session is classified into a session similar to the non-repetitive session NRS, the target session may be evaluated as a highly satisfied session.

Further, as shown in FIG. 18, the evaluation unit 65 uses the iterative evaluation model M1 that learns the characteristics of the session including the repeated utterances by learning the characteristics of the distribution image generated from the utterances in the session including the repeated utterances. The target session may be evaluated. For example, the image generation unit 63 generates a distribution image of utterances included in the target session. Then, the image generation unit 63 evaluates the target session based on the distribution image.

For example, when the evaluation unit 65 inputs the target image into the iterative evaluation model M1 and the iterative evaluation model M1 classifies the distributed image of the target session into an image similar to the distributed image of the session including repeated utterances, the target. The session may be evaluated as a low satisfaction session. Further, for example, when the evaluation unit 65 inputs the target image into the iterative evaluation model M1 and the iterative evaluation model M1 classifies the distribution image of the target session into an image similar to the distribution image of the session that does not include repeated utterances. May evaluate the target session as a highly satisfied session.

The evaluation unit 65 uses a known image analysis technique to determine whether or not the distribution image of the target session contains diagonal lines, and if it is determined that the diagonal lines are included, the target session is set to low. It may be evaluated as a satisfaction session. That is, the evaluation unit 65 determines whether or not the target session contains repeated utterances from the distribution image of the target session without using the repeated evaluation model M1, and evaluates the target session based on the determination result. You may.

In addition, the evaluation unit 65 may evaluate the target session based on whether or not the apology response is included. For example, the evaluation unit 65 may evaluate the target session as a low satisfaction session when the number of apology responses included in the target session exceeds a predetermined threshold value. Further, as shown in FIG. 19, the evaluation unit 65 inputs the target session into the apology evaluation model M2, and when the apology evaluation model M2 classifies the target session into a session similar to the session including the apology response, the evaluation unit 65 is the target. The session may be evaluated as a low satisfaction session. Further, when the apology evaluation model M2 classifies the target session into a session similar to a session that does not include an apology response, the evaluation unit 65 may evaluate the target session as a highly satisfied session.

Further, the evaluation unit 65 may evaluate the target session using a plurality of models. For example, as shown in FIG. 20, the evaluation unit 65 evaluates the evaluation results of a plurality of models M1 to M7 that evaluate the session based on the characteristics of different types among the plurality of features that are indicators of the user's evaluation of the session. May be used to evaluate the target session.

In this way, when evaluating a session using the evaluation results of a plurality of models M1 to M7, any combination of models can be adopted. For example, as shown in FIG. 1, the evaluation unit 65 has a target session (and a target session) for each of the repeated evaluation model M1, the apology evaluation model M2, the non-favorable index speech evaluation model M3, and the favorable index speech evaluation model M4. The target image) is input, and the value obtained by subtracting the value of the score output by the favorable index speech evaluation model M4 from the sum of the scores output by each model M1 to M3 is calculated. Then, the evaluation unit 65 evaluates a predetermined number of sessions from the one with the highest calculated value as a low satisfaction session, and evaluates a predetermined number of sessions from the one with the lowest calculated value as a high satisfaction session. You may.

When the value obtained by subtracting the value of the score output by the favorable index speech evaluation model M4 from the sum of the scores output by each model M1 to M3 is calculated, the user is satisfied with the target session. Is calculated as a negative value, and if the user is not satisfied with the target session, a positive value is calculated. Therefore, the evaluation unit 65 may evaluate the target session depending on whether the calculated value is a positive value or a negative value.

Further, the evaluation unit 65 at least evaluates the evaluation result by the repeated evaluation model M1 or the repeated utterance evaluation model M7 that evaluates the session based on whether or not the same or similar utterances are repeatedly included in the utterances included in the session. You may use it. Further, the evaluation unit 65 may use at least the evaluation result of the apology evaluation model M2 that evaluates the session based on whether or not the response included in the session includes a predetermined response indicating an apology.

Further, the evaluation unit 65 may use at least the evaluation result of the non-favorable index utterance evaluation model M3 that evaluates the session based on whether or not the utterance included in the session includes the non-favorable index utterance. Further, the evaluation unit 65 may use at least the evaluation result of the favorable index utterance model M4 that evaluates the session based on whether or not the utterance included in the session includes the favorable index utterance.

Here, in the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4, it is necessary to set or extract the index utterance in advance when extracting the session to be the learning data. However, it is troublesome to set and extract the index utterance in advance in this way. Therefore, the evaluation unit 65 may evaluate the target content only based on the evaluation results of the iterative evaluation model M1 and the apology evaluation model M2. For example, the evaluation unit 65 calculates the sum of the score of the iterative evaluation model M1 and the score of the apology evaluation model M2, and evaluates a predetermined number of target sessions as low-satisfaction sessions in order from the one with the largest sum value. A predetermined number of target sessions may be evaluated as highly satisfied sessions in order from the one with the smallest sum value.

When evaluating the target session using a plurality of models M1 to M7, the evaluation unit 65 may set a predetermined weight to the evaluation result of each model M1 to M7. For example, the evaluation unit 65 may integrate a predetermined coefficient for each model M1 to M7 into the value of the score output from each model M1 to M7, and integrate the value of each score obtained by integrating the coefficients. When the evaluation unit 65 evaluates the target session, the evaluation unit 65 registers the evaluation result in the session evaluation database 32. Such an evaluation result may be used, for example, for extracting index utterances, or may be used for evaluation of other sessions.

[2-6. About prediction processing]
Returning to FIG. 2, the explanation will be continued. The engagement prediction processing unit 70 executes prediction processing for predicting the usage mode of the dialogue service of future users. Hereinafter, with reference to FIGS. 23 to 25, the concept of the processing executed by the engagement prediction processing unit 70 and an example of the functional configuration of the engagement prediction processing unit 70 will be described.

[2-6-1. Prediction processing based on index utterances]
For example, the engagement prediction processing unit 70 performs prediction processing based on the index utterance included in the session of the user (hereinafter referred to as “target user”) who is the prediction target of the usage mode of the future dialogue service. Execute. For example, FIG. 23 is a diagram showing a concept of a process for predicting a future usage mode based on an index utterance.

As shown in FIG. 23, a user who has a lot of favorable index utterances is considered to have a favorable impression of the dialogue service. In this way, it is presumed that a user who has a favorable impression of the dialogue service will naturally become a user who has a high engagement such as the period of use and the frequency of use as a result of using the dialogue service. On the other hand, users who often speak unfavorable indicators are considered to have an unfavorable impression of the dialogue service. In this way, it is presumed that users who have an unfavorable impression of the dialogue service will stop using the dialogue service and become users with low engagement.

Therefore, the engagement prediction processing unit 70 predicts the future engagement of the user based on the appearance frequency of the index utterance in the session including the user's utterance. For example, the engagement prediction processing unit 70 is a session including utterances of the target user, and includes a dialogue performed within a predetermined period (hereinafter, referred to as “observation period”) from the time of execution of the prediction processing. Extract the session.

Subsequently, the engagement prediction processing unit 70 extracts the favorable index utterances and the non-favorable index utterances included in the session during the observation period, the total frequency of appearances of the favorable index utterances during the observation period, and the non-favorable index utterances during the observation period. Favor index Calculate the total frequency of utterances. Then, the engagement prediction processing unit 70 calculates a score obtained by dividing the total frequency of appearance of favorable index utterances in the observation period by the total frequency of appearance of non-favorable index utterances in the observation period, and the higher the calculated score, the higher the score. Judge that the user has high engagement in the future.

The engagement prediction processing unit 70 may consider the value of the index score of the favorable index utterance or the non-favorable index utterance. For example, the engagement prediction processing unit 70 calculates the total of the index scores of the favorable index utterances in the observation period and the total of the index scores of the non-favorable index utterances in the observation period. Then, the engagement prediction processing unit 70 calculates a score obtained by dividing the total of the index scores of the favorable index utterances in the observation period by the total of the index scores of the non-favorable index utterances in the observation period, and the higher the calculated score, the higher the score. It may be determined that the user has high engagement in the future. Then, the engagement prediction processing unit 70 registers the prediction result in the engagement database 34.

The future engagement predicted by such processing is used for extracting index utterances and evaluating sessions. For example, in the process shown in FIG. 9, the index utterance extraction processing unit 50 regards a user with high future engagement predicted by the engagement prediction processing unit 70 as a favorable user, and the engagement prediction processing unit 70 predicts the future. By setting users with low engagement as unfavorable users, index utterances may be extracted from the utterances of each user. Further, for example, in the process shown in FIG. 15, the session evaluation processing unit 60 evaluates the engagement session from the sessions of users with high future engagement predicted by the engagement prediction processing unit 70 and the sessions of other users. The model M6 may be learned and the learned engagement session evaluation model M6 may be used to evaluate the target session.

[2-6-2. About prediction processing based on evaluation results]
Further, for example, the engagement prediction processing unit 70 may execute the prediction processing based on the evaluation result of the session. For example, FIG. 24 is a diagram showing a concept of processing for predicting a future usage mode based on the evaluation result.

For example, as shown in FIG. 24, a session evaluated to be highly satisfied by the evaluation process, that is, a user having many highly satisfied sessions is presumed to have a favorable impression on the dialogue service. Engagement in the future is expected to be higher. On the other hand, sessions evaluated as having low satisfaction by the evaluation process, that is, users with many low-satisfaction sessions, are presumed to have an unfavorable impression of the dialogue service, and therefore, it is considered that the engagement in the future will be low. Be done.

Therefore, the engagement prediction processing unit 70 predicts the future engagement of the target user based on the evaluation result for the session including the utterance of the target user. For example, the engagement prediction processing unit 70 extracts a session including a dialogue performed during the observation period, which is a session including the utterance of the target user. Subsequently, the engagement prediction processing unit 70 acquires the evaluation result for the extracted session. For example, the engagement prediction processing unit 70 refers to the session evaluation database 32 and acquires the evaluation result by the session evaluation processing unit 60.

Then, the engagement prediction processing unit 70 predicts the engagement of the target user in the prediction period based on the statistical value based on the evaluation result. For example, the engagement prediction processing unit 70 calculates the ratio of the high-satisfaction session and the ratio of the low-satisfaction session among the extracted sessions. Then, the engagement prediction processing unit 70 may determine that the target user is a user with high engagement in the future as the ratio of the high satisfaction session is high. Further, for example, the engagement prediction processing unit 70 may determine that the target user is a user with high engagement in the future when the number of highly satisfied sessions among the extracted sessions exceeds a predetermined threshold value.

Also, for example, the engagement prediction processing unit 70 assigns a positive score to a high satisfaction session and a negative score to a low satisfaction session, and sets a larger coefficient for a newer session. Then, the engagement prediction processing unit 70 may estimate the future engagement of the target user based on whether or not the score obtained by summing the product of the score and the coefficient of each session exceeds a predetermined threshold value. good. The engagement prediction processing unit 70 assigns a negative score to a high-satisfaction session and a positive score to a low-satisfaction session, sets a larger coefficient for a newer session, and sets a value of the product of the score and the coefficient of each session. The future engagement of the target user may be estimated based on whether or not the total score of the above is equal to or less than a predetermined threshold.

[2-6-3. About the functional configuration of the engagement prediction processing unit]
Next, an example of the functional configuration of the engagement prediction processing unit 70 will be described with reference to FIG. 25. FIG. 25 is a diagram showing an example of the functional configuration of the engagement prediction processing unit according to the embodiment. As shown in FIG. 25, the engagement prediction processing unit 70 has an estimation unit 71.

The estimation unit 71 estimates the future usage mode of the dialogue service by the target user based on the evaluation result for the session including the utterance of the target user. For example, the estimation unit 71 calculates the total of the index scores of the favorable index utterances during the observation period, that is, the favorable score indicating how favorable the evaluation by the user is. In addition, the estimation unit 71 calculates the total of the index scores of the non-favorable index utterances during the observation period, that is, the non-favorable score indicating how unfavorable the evaluation by the user is. Then, the estimation unit 71 estimates the future engagement of the target user, that is, the usage mode, based on the ratio of the favorable favorable score and the non-favorable score.

Further, the estimation unit 71 identifies whether each session in the observation period of the target user is a high-satisfaction session or a low-satisfaction session. Then, the estimation unit 71 estimates the future usage mode of the target user based on the number and ratio of the high-satisfaction session and the low-satisfaction session.

[2-7. About the relevance of each process]
Here, with reference to FIG. 26, the relationship between the extraction process executed by the index utterance extraction processing unit 50, the evaluation process executed by the session evaluation processing unit 60, and the prediction processing executed by the engagement prediction processing unit 70 will be described. FIG. 26 is a diagram showing an example of relevance of processing executed by the information providing device according to the embodiment.

First, the relationship between the extraction process and the evaluation process will be described. For example, it is presumed that a session with a high evaluation is likely to include a favorable index utterance, and a session with a low evaluation is likely to include a non-favorable index utterance. Therefore, the evaluation result of the session can be used for the extraction process of the index utterance. More specifically, the information providing device 10 can improve the extraction amount and the extraction accuracy of the index utterance by advancing the evaluation of the session by the evaluation process described above. On the other hand, a session containing a favorable index utterance is presumed to have a high evaluation, and a session containing a non-favorable index utterance is presumed to have a low evaluation. Therefore, the extraction result of the index utterance can be used for the evaluation process of the session. More specifically, the information providing device 10 can improve the evaluation accuracy of the session by extracting the index utterance by the above-mentioned extraction process.

Next, the relationship between the extraction process and the prediction process will be described. For example, it is presumed that users with high engagement have a high frequency of uttering favorable index utterances, and users with low engagement have a low frequency of uttering non-favorable index utterances. Therefore, the prediction result of engagement can be used for the extraction process of the index utterance. More specifically, the information providing device 10 can improve the extraction amount and the extraction accuracy of the index utterance by predicting the engagement. On the other hand, it is presumed that users with many favorable index utterances have high engagement, and users with many non-favorable index utterances have low engagement. Therefore, the extraction result of the index utterance can be used for the prediction processing of engagement. More specifically, the information providing device 10 can improve the prediction accuracy of engagement by extracting index utterances.

Next, the relationship between the evaluation process and the prediction process will be described. For example, it is presumed that the engagement of users with many high-rated sessions is high, and the engagement of users with many low-rated sessions is low. Therefore, the evaluation result of the session can be used for the prediction processing of engagement. More specifically, the information providing device 10 can improve the prediction accuracy of engagement by evaluating the session. On the other hand, it is presumed that users with high engagement have a high evaluation of the session, and users with low engagement have a low evaluation of the session. Therefore, the prediction result of engagement can be used for the evaluation process of the session. More specifically, the information providing device 10 can improve the evaluation accuracy of the session by predicting the engagement.

As described above, the various extraction processes, evaluation processes, and prediction processes described above have a bootstrap relationship with each other. Therefore, the information providing device 10 can gradually improve the accuracy of each process by repeatedly executing each process in order. For example, the information providing device 10 may alternately execute the extraction process and the prediction process, the extraction process and the evaluation process, or the evaluation process and the prediction process. Further, the information providing device 10 may repeatedly execute each process in order.

[2-8. About reinforcement learning process]
Returning to FIG. 2, the explanation will be continued. The reinforcement learning processing unit 80 executes reinforcement learning of the dialogue model used by the dialogue device 200.

[2-8-1. About an example of reinforcement learning]
For example, FIG. 27 is a diagram showing the concept of a process for executing reinforcement learning of a dialogue model. The dialogue device 200 uses the dialogue model M10 to generate a response to the user's utterance. Here, in the dialogue service, the user may utter a completely new content (sentence different from the previous sentence) in response to the response, or may utter an utterance including evaluation of the response. For example, the above-mentioned favorable index utterances and non-favorable index utterances are considered to be evaluations of the response made immediately before. Then, when the dialogue model M10 is regarded as an agent in reinforcement learning and the user as an environment by reinforcement learning, various index utterances spoken by the user can be regarded as rewards to the agent.

Therefore, the reinforcement learning processing unit 80 realizes reinforcement learning of the dialogue model M10 by setting a reward based on the index utterance for the dialogue model M10. More specifically, the reinforcement learning processing unit 80 acquires an index utterance that is an index of evaluation for the dialogue service among the utterances to the response model M10, and sets a reward based on the acquired index utterance to obtain a response model. Reinforcement learning of M10 is performed.

For example, the reinforcement learning processing unit 80 refers to the session log database 31 and identifies the favorable index utterance and the non-favorable index utterance included in the session. Then, when the favorable index utterance is specified, the reinforcement learning processing unit 80 sets a positive reward as a reward for the response immediately before the specified favorable index utterance, and learns the dialogue model M10. Further, when the non-favorable index utterance is specified, the reinforcement learning processing unit 80 sets a negative reward as a reward for the response immediately before the specified non-favorable index utterance, and learns the dialogue model M10. ..

Further, the reinforcement learning processing unit 80 generates a user model model M20 that imitates the user's utterance, causes the user model model M20 and the dialogue model M10 to perform a virtual dialogue, and the user model model M20 performs a virtual dialogue. Reinforcement learning of the dialogue model M10 may be performed by setting a reward according to the output index utterance. For example, the reinforcement learning processing unit 80 generates a user model model M20 using a predetermined dialogue rule and a session registered in the session log database 31 as learning data. Subsequently, the reinforcement learning processing unit 80 inputs the utterance generated by the user model model M20 into the dialogue model M10, and inputs the response output by the dialogue model M10 into the user model model M20.

Further, the reinforcement learning processing unit 80 repeatedly executes a virtual dialogue between the dialogue model M10 and the user model model M20. Then, when the user model model M20 outputs an index utterance as an utterance, the reinforcement learning processing unit 80 sets a reward according to the index utterance, and performs reinforcement learning of the dialogue model M10 based on the set reward. ..

[2-8-2. An example of the functional configuration of the reinforcement learning processing unit]
Next, an example of the functional configuration of the reinforcement learning processing unit 80 will be described with reference to FIG. 28. FIG. 28 is a diagram showing an example of the functional configuration of the reinforcement learning processing unit according to the embodiment. As shown in FIG. 28, the reinforcement learning processing unit 80 has an index utterance acquisition unit 81, a reward setting unit 82, and a reinforcement learning unit 83.

The index utterance acquisition unit 81 acquires the index utterance that is an index for evaluation of the dialogue service among the utterances for the dialogue model M10. More specifically, the index utterance acquisition unit 81 acquires favorable index utterances and non-favorable index utterances from the utterances output by the user and the user model model M20. For example, the index utterance acquisition unit 81 refers to the session log database 31 and acquires the index utterance. Further, the index utterance acquisition unit 81 acquires the index utterance among the utterances output by the user model model M20.

The reward setting unit 82 sets a reward according to the index utterance acquired by the index utterance acquisition unit 18. For example, the reward setting unit 82 sets a reward according to the acquired index score of the index utterance. More specifically, the reward setting unit 82 sets a positive reward when the acquired index utterance is a favorable index utterance, and sets a positive reward when the acquired index utterance is a non-favorable index utterance. Set a negative reward.

The reinforcement learning unit 83 performs reinforcement learning of the dialogue model M10 using the reward set by the reward setting unit 82. For example, when the index utterance acquisition unit 81 extracts a certain index utterance from the session, the reinforcement learning unit 83 identifies the response immediately before the index utterance, and the reward when the dialogue model M10 outputs the specified response is the reward. Reinforcement learning of the dialogue model M10 is performed as a reward based on the index utterance.

Further, for example, the reinforcement learning unit 83 uses an utterance model that outputs an index utterance in response to the response generated by the dialogue model M10, that is, a reward based on the index utterance output by the user model model M20, and the dialogue model M10. Reinforcement learning. For example, the reinforcement learning unit 83 learns the user model model M20 by using the contents of each session registered in the session log database 31 and the dialogue rules. For such a user model model M20, for example, a learning method similar to the learning of the dialogue model M10 can be adopted.

Subsequently, the reinforcement learning unit 83 causes the user model model M20 and the dialogue model M10 to execute a virtual dialogue. Then, when the user model model M20 outputs the index utterance, the reinforcement learning unit 83 executes the reinforcement learning of the dialogue model M10 based on the reward set by the reward setting unit 82.

[3. An example of the flow of processing executed by the information providing device]
Subsequently, an example of the flow of each process executed by the information providing apparatus 10 will be described with reference to FIGS. 29 to 37.

[3-1. Example of processing flow based on usage mode]
First, with reference to FIG. 29, an example of the flow of processing executed by the information providing device 10 based on the usage mode will be described. FIG. 29 is a flowchart showing an example of a flow of processing executed by the information providing device according to the embodiment based on the usage mode.

First, the information providing device 10 specifies a usage mode of the dialogue service of each user (step S101). Subsequently, the information providing device 10 identifies a user whose specific result satisfies a predetermined condition (step S102). Then, the information providing device 10 extracts a session including the utterance of the specified user (step S103). Further, the information providing device 10 learns the engagement session evaluation model M6 using the session extracted in this way as learning data (step S104), and ends the process.

The information providing device 10 may evaluate the extracted session (step S105). For example, when the information providing device 10 extracts the sessions of users with high engagement, those sessions may be evaluated as highly satisfying sessions. Further, the information providing device 10 may extract the index utterance from the session extracted in this way (step S106). For example, the information providing device 10 extracts utterances in which the frequency of appearance in a session of a user with high engagement satisfies a predetermined condition as a favorable index utterance, and the frequency of appearance in a session of a user with low engagement satisfies a predetermined condition. The utterance may be extracted as a non-favorable index utterance.

[3-2. An example of the process flow for extracting index utterances]
Next, with reference to FIG. 30, an example of the flow of the process in which the information providing device 10 extracts the index utterance will be described. FIG. 30 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment extracts an index utterance.

First, the information providing device 10 acquires a session of a user whose usage mode satisfies a predetermined condition (step S201), and extracts index utterances based on the appearance frequency in the acquired session (step S202). Then, the information providing device 10 further extracts a new index utterance based on the co-occurrence with the index utterance (step S203), and ends the process.

The information providing device 10 may evaluate the session based on the extracted index utterance (step S204). For example, the information providing device 10 may evaluate a session containing a large number of favorable index utterances as a high-satisfaction session and a session containing a large number of non-favorable index utterances as a low-satisfaction session. Further, the information providing device 10 may estimate the usage mode of the user based on the appearance frequency of the index utterance (step S205). For example, the information providing device 10 estimates that a user who utters a lot of favorable index utterances is a user with high engagement, and a user who includes a lot of non-favorable index utterances is estimated as a user with low engagement. May be good.

[3-3. An example of the process flow for evaluating a session]
Next, an example of the flow of the process in which the information providing device 10 evaluates the session will be described with reference to FIG. 31. FIG. 31 is a flowchart showing an example of a process flow in which the information providing device according to the embodiment evaluates a session.

First, the information providing device 10 acquires a session to be evaluated (step S301), and evaluates the session based on the appearance frequency of the index utterance in the acquired session (step S302). For example, the information providing device 10 evaluates a session containing a large number of favorable index utterances as a high-satisfaction session, and evaluates a session containing a large number of non-favorable index utterances as a low-satisfaction session. Then, the information providing device 10 learns the characteristics of the session as the learning data to which the label corresponding to the evaluation result is attached (step S303), and ends the process. For example, the information providing device 10 causes the models M3 and M4 to learn the characteristics of the high-satisfaction session and the low-satisfaction session.

The information providing device 10 may extract a new index utterance from the session based on the evaluation result of the session (step S304). For example, the information providing device 10 may extract utterances having a high frequency of appearance in a high-satisfaction session as favorable index utterances and utterances having a high frequency of appearance in a low-satisfaction session as non-favorable index utterances. Further, the information providing device 10 may estimate the usage mode of the user based on the evaluation result of the session (step S305). For example, the information providing device 10 may presume that a user with many high-satisfaction sessions is a user with high engagement and a user with many low-satisfaction sessions is a user with low engagement.

[3-4. An example of the process flow for evaluating a session based on an image]
Next, with reference to FIG. 32, an example of the flow of the process in which the information providing device 10 evaluates the session based on the image will be described. FIG. 32 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment evaluates a session based on an image.

First, the information providing device 10 acquires a session to be evaluated (step S401), and generates an image showing whether or not the utterance characters included in the acquired session are the same as each other, that is, a distribution image (step). S402). Then, the information providing device 10 evaluates the session based on the generated distribution image (step S403). For example, when the distribution image includes diagonal lines, the information providing device 10 evaluates the target session as a low-satisfaction session when it is classified by the iterative evaluation model M1 as containing the iterations. Then, the information providing device 10 learns the characteristics of the session as the learning data to which the label corresponding to the evaluation result is attached (step S404), and ends the process.

The information providing device 10 may extract a new index utterance from the session based on the evaluation result of the session (step S405). Further, the information providing device 10 may estimate the usage mode of the user based on the evaluation result of the session (step S406).

[3-5. An example of the process flow that evaluates a session based on repeated utterances]
Next, with reference to FIG. 33, an example of a process flow in which the information providing device 10 evaluates a session based on repeated utterances will be described. FIG. 33 is a flowchart showing an example of a process flow in which the information providing device according to the embodiment evaluates a session based on repeated utterances.

First, the information providing device 10 acquires a session to be evaluated (step S501), and evaluates the session based on the repetition of the utterance character string included in the acquired session (step S502). For example, the information providing device 10 evaluates a low satisfaction session when the number of utterances having the same or similar character strings or the number of utterances having the same or similar meaning exceeds a predetermined threshold value. Then, the information providing device 10 learns the characteristics of the session as the learning data to which the label corresponding to the evaluation result is attached (step S503), and ends the process.

The information providing device 10 may extract a new index utterance from the session based on the evaluation result of the session (step S504). Further, the information providing device 10 may estimate the usage mode of the user based on the evaluation result of the session (step S505).

[3-6. An example of the flow of reinforcement learning processing using utterances as a reward]
Next, with reference to FIG. 34, an example of the flow of reinforcement learning in which the information providing device 10 executes an utterance as a reward will be described. FIG. 34 is a flowchart showing an example of the flow of reinforcement learning in which the information providing device according to the embodiment executes the utterance as a reward.

First, the information providing device 10 inputs an utterance to the dialogue model M10 (step S601), and inputs the response output by the dialogue model M10 to the user model model M20 to acquire an index utterance for the response (step S601). Step S602). Then, the information providing device 10 executes reinforcement learning of the dialogue model M10 using the index utterance acquired from the user model model M20 as a reward (step S603), and ends the process.

[3-7. An example of the process flow for extracting index utterances based on co-occurrence]
Next, with reference to FIG. 35, an example of the flow of the process in which the information providing device 10 extracts the index utterance based on the co-occurrence will be described. FIG. 35 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment extracts an index utterance based on co-occurrence.

First, the information providing device 10 identifies the index utterance from the session history (step S701), and specifies the co-occurrence with the index utterance in each session for each utterance (step S703). Then, the information providing device 10 extracts a new index utterance based on the specified co-occurrence (step S703). For example, the information providing device 10 extracts utterances that are highly co-occurring with favorable index utterances as new favorable index utterances, and newly unfavorable utterances that are highly co-occurring with non-favorable index utterances. Extract as an index utterance.

The information providing device 10 may further extract a new index utterance based on the co-occurrence with the index utterance newly extracted in this way (step S704). Further, the information providing device 10 may evaluate the session based on the index utterance newly extracted in this way (step S705), and the usage mode of the user is determined based on the appearance frequency of the index utterance. It may be estimated (step S706).

[An example of the process flow for evaluating a session using a 3-8.3 value evaluation model]
Next, with reference to FIG. 36, an example of the flow of the process of evaluating the session using the ternary evaluation model M5 will be described. FIG. 36 is a flowchart showing an example of a flow of processing in which the information providing device according to the embodiment evaluates a session using a ternary evaluation model.

First, the information providing device 10 identifies favorable index utterances, non-favorable index utterances, and neutral utterances from the session history (step S801), and learns a predetermined number of previous utterances and responses from each index utterance. (Step S802). That is, the information providing device 10 generates a favorable dialogue example FD, a neutral dialogue example ND, and a non-favorable dialogue example UD. Further, the information providing device 10 assigns a label according to the index utterance included in the extracted learning data (step S803), and learns the trivalue evaluation model M5 using the learning data (step S804). After that, the information providing device 10 evaluates the session using the ternary evaluation model M5 (step S805), and ends the process.

[3-9. An example of the process flow for evaluating a session using multiple models]
Next, with reference to FIG. 37, an example of a process flow for evaluating a session using a plurality of models M1 to M7 will be described. FIG. 37 is a flowchart showing an example of a process flow in which the information providing device according to the embodiment evaluates a session using a plurality of models.

For example, the information providing device 10 acquires a session to be evaluated (step S901), and causes a plurality of models M1 to M7 that have learned different characteristics to evaluate the session (step S902). Then, the information providing device 10 integrates the evaluation results of the models M1 to M7, evaluates the session (step S903), and ends the process.

[4. others〕
Further, among the processes described in the above-described embodiment, all or a part of the processes described as being automatically performed can be manually performed, and conversely, the processes described as being manually performed. It is also possible to automatically perform all or part of the above by a known method. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown in the figure.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in any unit according to various loads and usage conditions. Can be integrated and configured.

In addition, the above-described embodiments can be appropriately combined as long as the processing contents do not contradict each other.

[5. program〕
Further, the information providing device 10 according to the above-described embodiment is realized by, for example, a computer 1000 having a configuration as shown in FIG. 38. FIG. 38 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to the output device 1010 and the input device 1020, and the arithmetic unit 1030, the primary storage device 1040, the secondary storage device 1050, the output IF (Interface) 1060, the input IF 1070, and the network IF 1080 are connected by the bus 1090. Have.

The arithmetic unit 1030 operates based on a program stored in the primary storage device 1040 or the secondary storage device 1050, a program read from the input device 1020, or the like, and executes various processes. The primary storage device 1040 is a memory device that temporarily stores data used by the arithmetic unit 1030 for various operations such as RAM. Further, the secondary storage device 1050 is a storage device in which data used by the calculation device 1030 for various calculations and various databases are registered, such as a ROM (Read Only Memory), an HDD (Hard Disk Drive), and a flash memory. Is realized by.

The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various information such as a monitor and a printer. For example, USB (Universal Serial Bus), DVI (Digital Visual Interface), and the like. It is realized by a connector of a standard such as HDMI (registered trademark) (High Definition Multimedia Interface). Further, the input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, a scanner, and the like, and is realized by, for example, USB.

The input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like. Further, the input device 1020 may be an external storage medium such as a USB memory.

The network IF 1080 receives data from another device via the network N and sends it to the arithmetic unit 1030, and also transmits the data generated by the arithmetic unit 1030 to the other device via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 10, the arithmetic unit 1030 of the computer 1000 realizes the function of the control unit 40 by executing the program loaded on the primary storage device 1040. Further, for example, when the computer 1000 functions as the terminal device 100, the arithmetic unit 1030 of the computer 1000 realizes the function of the control unit 40 by executing the program loaded on the primary storage device 1040.

[6. About experimental results]
Next, an example of the experimental result of the above-mentioned information providing device 10 will be described with reference to FIGS. 39 to 47.

[6-1. About data]
First, an example of session data used in the experiment of the information providing device 10 will be described. In the following experiments, 4000 sessions containing at least two or more utterances and responses were labeled by cloud sourcing. Specifically, 2000 training sessions (Training), 1000 adjustment sessions (Validation), and 1000 test sessions (Test) were prepared. In addition, each session is presented to 7 judges, and if it is judged that the user is frustrated, "Yes", and if it is judged that the user is not frustrated, it is judged that the user is not frustrated. , "No" was given to each session.

For example, FIG. 39 is a diagram showing an example of the result of labeling the session data. For example, in the example shown in FIG. 39, a session in which at least one judge is labeled with “Yes” is defined as FL (Frustration Level) 1, and a session in which at least two judges are labeled with “Yes”. FL2, a session in which at least three judges labeled "Yes" was described as FL3, and a session in which at least four judges labeled "Yes" was described as FL4. In addition, FIG. 39 shows the number of sessions classified into FL1 to FL4 among training sessions, adjustment sessions, and test sessions.

[6-2. About the first experiment]
Subsequently, the contents of the first experiment using the above-mentioned session data will be described. In the first experiment, the method shown by Prop_D and the method shown by Prop_U were set as the experimental targets, and the methods shown by BL_S1, BL_S2, and BL_SS were set as the baselines to be compared.

Here, as shown in FIG. 1, Prop_D is a determination method using a repeated evaluation model M1, an apology evaluation model M2, a non-favorable index utterance evaluation model M3, and a favorable index utterance evaluation model M4. For example, in Prop_D, a test session was input to each model M1 to M4 to calculate a dissatisfaction score. Further, in Prop_D, 500,000 sessions are determined to be low-satisfaction sessions (that is, the label is “Yes”) in order from the one with the highest calculated dissatisfaction score value, and the one with the highest dissatisfaction score value is used first. In turn, 500,000 sessions were determined to be highly satisfied sessions (ie, the label is "No").

Here, in Prop_D, when learning the iterative evaluation model M1, the learning data is generated by using the method shown in FIG. 17, and as shown in FIG. 18, the distribution image of the generated learning data has. The features were repeatedly trained by the evaluation model M1. Further, in Prop_D, when learning the apology evaluation model M2, the learning data was generated by using the method shown in FIG. 19, and the features of the learning data were learned by the apology evaluation model M2. Further, in Prop_D, the favorable index utterance evaluation model M3 and the non-favorable index utterance evaluation model M4 were learned by using the method shown in FIG. Further, in Prop_D, a model for classifying images was adopted as the iterative evaluation model M1, and a model for classifying input information using LSTM was adopted as models M2 to M4.

Further, Prop_U is a determination method using only the repeated evaluation model M1 and the apology evaluation model M2 among the models M1 to M4 shown in FIG. Since such a determination method does not require index utterance for model learning, it is not necessary to manually create data in advance, and it is an Unsupervised method. For example, in Prop_U, the target session was repeatedly input to the evaluation model M1 and the apology evaluation model M2, and the sum of the satisfactions output by the models M1 and M2 was calculated as the dissatisfaction score. In Prop_U, 500,000 sessions are determined to be low-satisfaction sessions (that is, the label is "Yes") in order from the one with the highest calculated dissatisfaction score value, and in order from the one with the highest dissatisfaction score value. The 500,000 sessions were determined to be highly satisfied sessions (ie, the label is "No").

In addition, BL_S1 used a model in which learning was performed using the above-mentioned training session and adjustment was performed using the adjustment session. Further, in BL_S1, a model for classifying input information using LSTM is adopted as in the case of Prop_D and Prop_U described above.

Further, in BL_S2, an SVM in which learning was performed using a training session and adjustment was performed using an adjustment session was used. Further, in BL_S2, the output of the LSTM layer of the model used in BL_S1 described above was input to the SVM, and the output of the SVM was adopted as the classification result.

In addition, in BL_SS, in addition to the training session, a transductive SVM (TSVM) in which learning was performed using one million unlabeled sessions was used. In addition, the adjustment session and the test session were used for learning as unlabeled sessions by ignoring the labels.

Here, in the first experiment, the sessions classified into FL1 to FL4 are classified by the various methods described above, and the accuracy (accuracy), precision (matching rate), recall (recall rate), and recall of the classification results are performed. And the F1 value which is the harmonic mean of the precision rate and the recall rate was calculated. FIG. 40 is a diagram showing the results of the first experiment. In the example shown in FIG. 40, an asterisk was added to the method having a significant difference from Prop_D, and a sharpening was added to the method having a significant difference from Prop_U. As shown in FIG. 40, when the F1 value is compared, the accuracy of the classification result by Prop_D is the best in the classification result of the session data classified into FL1, FL2, and FL3.

[6-3. About the second experiment]
Subsequently, in order to determine how much each of the above-mentioned models contributes to the accuracy, the following second experiment was performed. In the second experiment, w / o-Rep in which the iterative evaluation model M1 was removed in Prop_D, w / o-Apo in which the apology evaluation model M2 was removed in Prop_D, and w in which the non-favorable index speech evaluation model M3 was removed in Prop_D. In / o-Unf and Prop_D, a method called w / o-Fav was set, in which the favorable index utterance evaluation model M4 was removed.

Further, in the second experiment, a method called w / o-RAUF was set in which the four models M1 to M4 shown in FIG. 1 were not used. In w / o-RAUF, 500,000 high-satisfaction sessions and 500,000 low-satisfaction sessions are used as learning data, and the input sessions are either high-satisfaction sessions (that is, sessions with the label "No") or low satisfaction. We trained a model to classify into sessions (that is, sessions with the label "Yes"), and classified sessions using the trained model.

Further, in the second experiment, a method called w / o-CandRet was set. In w / o-CandRet, the dissatisfaction score is calculated using each of the above-mentioned models M1 to M4 for the session history acquired in advance, and 500,000 sessions are labeled in order from the one with the highest calculated score. The sessions were extracted as "Yes" sessions, and 500,000 sessions were extracted as the sessions with the label "No" in order from the one with the lowest calculated score. Then, in w / o-CandRet, the model for classifying the input session was trained using the generated learning data, and the session was classified using the trained model.

Then, in the second experiment as well, the sessions classified into FL1 to FL4 are classified by the various methods described above, and the accuracy (accuracy), precision (correction rate), recall (recall rate), and recall (recall rate) of the classification results are performed. The F1 value was calculated. FIG. 41 is a diagram showing the results of the second experiment. As shown in FIG. 41, it was found that the accuracy of Prop_D was the highest when the F1 value was used as a comparison target, and the accuracy was lowered when any of the models M1 to M4 was not used. As a result, it was found that all of the models M1 to M4 contributed to the improvement of the session classification accuracy.

[6-4. About the third experiment]
Subsequently, in the above-mentioned Prop_D, the following third experiment was performed in order to determine the contribution of the amount of training data to the classification accuracy. In the following description, the amount of training data used is indicated by the subscript k. For example, Prop_D_k learns each model M1 to M4 using k / 2 high-satisfaction sessions and k / 2 low-satisfaction sessions, and classifies test sessions using each of the learned models M1 to M4. Indicates that In the third experiment, the learning data was changed by changing the value of k to 500,000 (500k), 1 million (1M), 2 million (2M), 3 million (3M), and 4 million (4M). The contribution of the amount of to accuracy in each method was investigated.

FIG. 42 is a diagram showing the results of Prop_D in the third experiment. In FIG. 42, an asterisk was added to the result having a significant difference from the case where k was set to 1M. As shown in FIG. 42, it was found that the accuracy of Prop_D was the highest when k = 2M regardless of the classification of the test session. It is also considered that this is because there is room for improvement in accuracy when the training data is less than 2 million, and overfitting occurs when the training data exceeds 2 million.

[6-5. About the 4th experiment]
Here, in the above-mentioned Prop_D, a neural network is used as a model for performing evaluation based on repeated utterances, apologies, and index utterances. Here, in order to examine the effect of using the neural network for each recognition, the following fourth experiment was performed. For example, in the fourth experiment, in the Prop_D method, instead of the neural network, the session is evaluated based on repeated utterances using the editing distance, and the evaluation based on repeated utterances and index utterances is performed by character string matching. Run as.

FIG. 43 is a diagram showing the results of Prop_S in the fourth experiment. In addition, in FIG. 43, the results having a significant difference from Prop_D were obtained in FL2, FL3, and FL4. Comparing the results of Prop_D shown in FIGS. 40 and 41 with the results of Prop_S shown in FIG. 43, in Prop_S, the conformance rate is higher than that of Prop_D, and the recall rate and the F1 value are lower. That is, it is considered that Prop_S is less accurate than Prop_D. Therefore, in Prop_D, the accuracy can be improved by using the neural network when evaluating the session.

[6-6. About the 5th experiment]
Next, a fifth experiment was conducted in which the session was evaluated using a model in which the outputs of each model used for evaluation in Prop_D were fused and the generalization ability (prediction ability for unlearned data) was improved. More specifically, in the fifth experiment, a method in which the accuracy of Prop_D was further improved by using the method of the ensemble was experimented as Prop_E.

FIG. 44 is a diagram showing the results of Prop_E in the fifth experiment. In FIG. 44, the result of Prop_E and the result of Prop_D_2M are shown side by side. As shown in FIG. 44, the result of Prop_E has a higher F1 value than the result of Prop_D_2M. Therefore, by using an ensemble method such as Prop_E, the accuracy of evaluating the session can be further improved.

[7. About the effect of the embodiment]
Hereinafter, an example of the effect of the information providing device 10 for executing each of the above-mentioned processes will be described.

[7-1. Extraction of learning data based on usage mode]
As described above, the information providing device 10 specifies the usage mode of the dialogue service by the user. Then, the information providing device 10 records the session including the utterance of the user whose specified usage mode satisfies a predetermined condition as the learning data for evaluating the session including the utterance and the response in the dialogue service. Extract from.

In this way, the information providing device 10 extracts learning data for learning a model for evaluating a session based on the engagement of each user. Therefore, the information providing device 10 can automatically extract, for example, learning data for learning a model for determining whether or not the user is satisfied with the session without human intervention. As a result of such processing, the information providing device 10 can easily prepare a large amount of training data, and as a result, the accuracy of the model can be improved, and as a result, the evaluation accuracy of the session can be improved.

Further, the information providing device 10 specifies the usage period of the dialogue service by the user as a usage mode, and extracts the session including the utterance of the user whose usage period satisfies a predetermined condition as learning data. Further, the information providing device 10 specifies the frequency of use of the dialogue service by the user as a usage mode, and extracts a session including the utterance of the user whose usage frequency satisfies a predetermined condition as learning data. Therefore, for example, when the information providing device 10 extracts learning data for the purpose of extracting a session with low user satisfaction, the information providing device 10 is a user with high engagement, that is, a user with a high evaluation for the dialogue service. Sessions are extracted as an example, and sessions of users with low engagement, that is, users with a low evaluation of dialogue services, are extracted as positive examples, so that data suitable for learning can be easily prepared. As a result, sessions The evaluation accuracy of can be improved.

Further, the information providing device 10 causes the model to learn the characteristics of the extracted session. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 extracts the session including the utterance of the user whose usage mode satisfies the predetermined condition as learning data having the first attribute, and also extracts the utterance of the user whose usage mode does not satisfy the predetermined condition. The included session is extracted as learning data having the second attribute. Then, the information providing device 10 learns a model that classifies the input session into either the data having the first attribute or the data having the second attribute by using the extracted learning data.

For example, the information providing device 10 extracts a session including an utterance of a user whose usage mode satisfies a predetermined condition as learning data whose evaluation by the user is favorable, and the usage mode does not satisfy the predetermined condition. Sessions containing user utterances are extracted as learning data that the user's evaluation is unfavorable. Then, the information providing device 10 uses the extracted learning data to learn a model for determining whether the user's evaluation of the input session is favorable or unfavorable. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

In addition, the information providing device 10 evaluates a session including a user's utterance based on the specified usage mode. Then, the information providing device 10 extracts a session whose evaluation satisfies a predetermined condition as learning data. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 extracts an utterance similar to the utterance included in the session whose evaluation satisfies a predetermined condition, or a response similar to the response included in the session from the history of the dialogue service, and the extracted utterance or the extracted utterance. A new session using the response is generated as a session in which the evaluation meets a predetermined condition. Then, the information providing device 10 causes the model to learn the characteristics of the generated new session. As described above, the information providing device 10 generates a similar session as learning data from a session whose usage mode satisfies a predetermined condition, and trains the model using the generated learning data. Therefore, the information providing device 10 can improve the evaluation accuracy of the session as a result of improving the determination accuracy of the model.

Further, the information providing device 10 extracts a session whose evaluation satisfies a predetermined condition from the history of the dialogue service, and includes the utterance of the utterance included in the extracted session whose appearance frequency satisfies the predetermined condition. Extract as an index utterance that is an index of whether or not the user's evaluation of the session is favorable. For example, the information providing device 10 extracts an utterance in which the ratio of the appearance frequency in a session in which the evaluation satisfies a predetermined condition to the appearance frequency in a session in which the evaluation does not satisfy the predetermined condition satisfies the predetermined condition as an index utterance. Therefore, the information providing device 10 can extract the index utterance based on the usage mode of the user. It should be noted that such index utterances can be used for session evaluation. Therefore, the information providing device 10 can improve the evaluation accuracy of the session based on the index utterance.

Further, the information providing device 10 evaluates the session based on whether or not a predetermined response indicating an apology is included in the session in addition to the usage mode of the user. Further, the information providing device 10 includes, in addition to the usage mode of the user, an index utterance as an index of whether or not the user's evaluation of the session is favorable or not, the session. To evaluate. Therefore, the information providing device 10 can further improve the evaluation accuracy of the session.

Further, the information providing device 10 identifies the usage mode of the dialogue service by the user, and evaluates the session including the utterance of the user based on the specified usage mode. Therefore, the information providing device 10 can appropriately evaluate the session without human intervention.

[7-2. Extraction of index utterances based on usage mode]
Further, the information providing device 10 acquires a session including an utterance of a user whose usage mode of the dialogue service satisfies a predetermined condition and a response to the utterance. Then, the information providing device 10 extracts an index utterance that is an index of the user's evaluation of the dialogue service based on the frequency of appearance in the acquired session. Such index utterances can be used to evaluate the session. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 acquires a session including an utterance of a user whose usage period of the dialogue service satisfies a predetermined condition. Further, the information providing device 10 acquires a session including the utterance of a user whose frequency of use of the dialogue service satisfies a predetermined condition. Then, the information providing device 10 has a ratio of the appearance frequency in the session including the utterance of the user whose usage mode satisfies the predetermined condition to the appearance frequency in the session including the utterance of the user whose usage mode does not satisfy the predetermined condition. Extracts utterances that satisfy a predetermined condition as index utterances.

Further, the information providing device 10 indicates that the user favorably evaluates the dialogue service for the utterances whose appearance frequency exceeds the predetermined threshold value in the session including the utterances of the user whose usage mode satisfies the predetermined condition. Favorable index to show Extract as utterance. Further, in the information providing device 10, the user unfavorably evaluates the utterance in which the appearance frequency exceeds a predetermined threshold value in the session including the utterance of the user whose usage mode does not satisfy the predetermined condition. Extract as a non-favorable index utterance indicating the effect. As a result of these processes, the information providing device 10 can appropriately extract the index utterance from the utterance of the user, so that the evaluation accuracy of the session can be improved.

Further, the information providing device 10 evaluates the session based on the appearance frequency of the extracted index utterances. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 satisfies the utterances whose appearance frequency exceeds a predetermined threshold value in the session including the utterances of the user whose usage mode satisfies the predetermined condition for the session to be evaluated, and the usage mode satisfies the predetermined condition. The session to be evaluated is evaluated based on the utterances in which the frequency of appearance in the session including the utterances of no user exceeds a predetermined threshold value. Therefore, the information providing device 10 can appropriately evaluate the session.

Further, the information providing device 10 causes the model to learn the characteristics of the session whose evaluation satisfies a predetermined condition. Therefore, the information providing device 10 can realize a model for evaluating a session.

Further, the information providing device 10 identifies the index utterance extracted from the history in the dialogue service, and extracts a series of dialogues corresponding to the specified index utterance as learning data. Then, the information providing device 10 causes the model to learn the characteristics of the extracted learning data. For example, the information providing device 10 causes the model to learn the characteristics of the non-favorable dialogue example UD and the favorable dialogue example FD. Therefore, the information providing device 10 can realize a model for evaluating a session.

[7-3. About session evaluation based on index utterance]
Further, the information providing device 10 acquires a session including the user's utterance and response in the dialogue service. Then, the information providing device 10 evaluates the acquired session based on the index utterance which is an index of the user's evaluation for the dialogue service. Therefore, the information providing device 10 can automatically and accurately learn the session.

Further, the information providing device 10 utters a favorable index indicating that the user's evaluation of the dialogue service is favorable, and a non-favorable index utterance indicating that the user's evaluation of the dialogue service is unfavorable. Evaluate the session based on. Then, for example, the information providing device 10 evaluates the session based on the ratio of the appearance frequency of the favorable index utterance to the appearance frequency of the non-favorable index utterance. Further, for example, the information providing device 10 has a favorable score indicating how favorable the user's evaluation of the dialogue service is in a session including a favorable index utterance, and a session including a non-favorable index utterance. Evaluate the session based on the ratio to the non-favorable score, which indicates how unfavorable the user's rating for the dialogue service is. Therefore, the information providing device 10 can automatically and accurately learn the session.

Further, the information providing device 10 evaluates whether or not the evaluation by the user is a favorable and highly satisfying session. Then, the information providing device 10 causes the model to learn the characteristics of the highly satisfied session. Further, the information providing device 10 evaluates whether or not the evaluation by the user is an unfavorable low satisfaction session. Then, the information providing device 10 causes the model to learn the characteristics of the low satisfaction session.

Further, the information providing device 10 identifies the index utterance from the history in the dialogue service, and extracts a predetermined number of previous responses and utterances from the specified index utterance as learning data. Then, the information providing device 10 causes the model to learn the characteristics of the extracted learning data. For example, the information providing device 10 extracts a predetermined number of previous responses and utterances as a favorable dialogue example from a favorable index utterance indicating that the user's evaluation of the dialogue service is favorable, and uses the dialogue service for the dialogue service. A non-favorable index indicating that a person's evaluation is unfavorable Extracts a predetermined number of previous responses and utterances as non-favorable dialogue examples from the utterances, and the user's evaluation of the dialogue service is favorable or unfavorable. A predetermined number of previous responses and utterances are extracted as examples of neutral dialogue from neutral utterances indicating that they are not targeted. Then, the information providing device 10 causes the model to learn the characteristics of the extracted favorable dialogue example, the non-favorable dialogue example, and the neutral dialogue example.

In addition, the information providing device 10 evaluates the session based on whether or not the session includes an index utterance and a predetermined response indicating an apology. As a result of the various processes described above, the information providing device 10 can learn a model for appropriately evaluating the session from various viewpoints. As a result, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 estimates the future usage mode of the dialogue service by the user based on the evaluation result for the session including the utterance of the user. For example, the information providing device 10 calculates, as an evaluation, a favorable score indicating how favorable the evaluation by the user is, or a non-favorable score indicating how unfavorable the evaluation by the user is. Then, based on the ratio of the favorable score and the non-favorable score, the future usage mode of the dialogue service by the user is estimated. Therefore, the information providing device 10 can appropriately estimate the usage mode of the user.

Further, the information providing device 10 extracts utterances whose appearance frequency satisfies a predetermined condition as index utterances from a session whose evaluation satisfies a predetermined condition. For example, the information providing device 10 evaluates whether the user's evaluation of the session is favorable or unfavorable, and the frequency of appearance and use in the session in which the user's evaluation is evaluated to be favorable. The utterances whose ratio with the frequency of appearance in the session evaluated as unfavorable by the person satisfies the predetermined condition are extracted as the index utterances. Therefore, the information providing device 10 can appropriately extract new index utterances.

[7-4. About session evaluation based on distribution images]
Further, the information providing device 10 acquires a session including the utterance of the user in the dialogue service and the response to the utterance. Further, the information providing device 10 generates an image showing whether or not the utterance characters included in the session are the same as each other. Then, the information providing device 10 evaluates the acquired session based on the generated image.

Here, if the session repeatedly contains the same or similar utterances, the above-mentioned image will include a predetermined pattern. Then, when it is estimated that the image contains a predetermined pattern, it can be estimated that the session contains the same or similar utterances repeatedly. As described above, a session in which the same or similar utterances are repeatedly included is presumed to have a low evaluation by the user. Therefore, the information providing device 10 can appropriately evaluate the acquired session based on the generated image.

Further, the information providing device 10 generates an image showing the distribution of the same characters in the session as an image. For example, when the information providing device 10 associates the utterance character string included in the session with each area in the vertical direction and each area in the horizontal direction of the image, the same characters correspond to each other in the vertical direction and the horizontal direction. Generates an image with a predetermined color for the attached area. Then, the information providing device 10 determines whether or not the same or similar character string appears repeatedly in the session based on the image, and evaluates the session based on the determination result. For example, when the same or similar character string appears repeatedly in a session, the information providing device 10 gives an evaluation to the effect that the evaluation by the user is unfavorable to the session. Therefore, the information providing device 10 can appropriately evaluate the session.

Further, the information providing device 10 causes the model to learn the characteristics of the image generated from the session in which the same or similar character strings repeatedly appear. Then, the information providing device 10 evaluates the session from which the image is based from the image by using the trained model. Here, it is known that the model in which the image is trained is easier to improve the determination accuracy than the model in which the character is trained. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 selects one of the utterances from the history in the dialogue service, and generates an utterance group including a plurality of utterances that are the same as or similar to the selected utterance. Then, the information providing device 10 generates an image from the generated utterance group, and causes the model to learn the features of the generated image. Therefore, the information providing device 10 can improve the accuracy of the model as a result of increasing the learning data of the model.

[7-5. Evaluation of sessions based on repeated utterances]
Further, the information providing device 10 acquires a session including the utterance of the user in the dialogue service and the response to the utterance. Then, the information providing device 10 evaluates the acquired session based on the repetition of the character string of the utterance included in the session. Therefore, the information providing device 10 can accurately evaluate the session without human intervention.

For example, the information providing device 10 extracts a pair of utterances in which the similarity of the character string exceeds a predetermined threshold value from the session, and evaluates the session based on the number of the extracted pairs. Further, the information providing device 10 extracts utterance pairs whose semantic similarity exceeds a predetermined threshold value from the session, and evaluates the session based on the number of the extracted pairs. Then, when the number of pairs extracted from the session exceeds a predetermined threshold value, the information providing device 10 gives an evaluation to the effect that the evaluation by the user is unfavorable to the session. Therefore, the information providing device 10 can appropriately evaluate the session.

Further, the information providing device 10 evaluates a session by using a model that has learned the characteristics of a session in which the same or similar character strings repeatedly appear. For example, the information providing device 10 selects one of the utterances from the history in the dialogue service, and generates an utterance group including a plurality of utterances that are the same as or similar to the selected utterance. Further, the information providing device 10 causes the model to learn the characteristics of the generated utterance group. Then, the information providing device 10 evaluates the session using the trained model.

Further, the information providing device 10 generates a first utterance group including a plurality of utterances that are the same as or similar to the selected utterance, selects a plurality of different utterances from the history in the dialogue service, and is composed of the selected utterances. An utterance group is generated, and the model learns the characteristics of the first utterance group and the characteristics of the second utterance group. Further, the information providing device 10 learns the model for determining whether or not the evaluation by the user is unfavorable for the input session by letting the model learn the characteristics of the utterance group. Therefore, the information providing device 10 can accurately evaluate the session.

Further, the information providing device 10 extracts utterances whose appearance frequency satisfies a predetermined condition from a session whose evaluation satisfies a predetermined condition as an index utterance which is an index for the evaluation of the user for the dialogue service. Therefore, the information providing device 10 can automatically extract a new index utterance.

[7-6. Reinforcement learning with utterances as a reward]
Further, the information providing device 10 acquires an index utterance that is an index of evaluation for the dialogue service among the utterances for the dialogue model that realizes the dialogue service by generating a response to the input utterance, and the acquired index. By setting rewards based on utterances, enhanced learning of the dialogue model is performed. Therefore, the information providing device 10 can realize the learning of the dialogue model that realizes the dialogue service without human intervention, and can improve the accuracy of the dialogue model.

Further, the information providing device 10 utters a favorable index indicating that the user's evaluation of the dialogue service is favorable, or a non-favorable index indicating that the user's evaluation of the dialogue service is unfavorable. Get utterances and set a positive reward for the dialogue model if a favorable indicator utterance is obtained, and set a negative reward for the dialogue model if a non-favorable indicator utterance is obtained do. Therefore, the information providing device 10 can appropriately realize reinforcement learning of the dialogue model.

Further, the information providing device 10 acquires the index utterance output by the utterance model that outputs the index utterance in response to the response generated by the dialogue model. For example, the information providing device 10 acquires the index utterance output by the model that has learned the characteristics of the history of the dialogue between the user and the dialogue model in the dialogue service as the utterance model. Therefore, the information providing device 10 can realize reinforcement learning of the dialogue model even if there is no session history, for example.

[7-7. Extraction of index utterances based on co-occurrence]
Further, the information providing device 10 specifies an utterance that is an index of the user's evaluation of the dialogue service and a preset index utterance from the history of the user's utterance in the dialogue service and the response to the utterance. .. Then, the information providing device 10 extracts a new index utterance from the utterance included in the history based on the co-occurrence with the specified index utterance. As mentioned above, such index utterances can be used to evaluate the session. As a result, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 identifies a favorable index utterance indicating that the user's evaluation of the dialogue service is favorable, and newly utters a utterance in which the co-occurrence with the favorable index utterance exceeds a predetermined threshold. Favorable index Extract as utterance. Further, the information providing device 10 identifies an unfavorable index utterance indicating that the user's evaluation of the dialogue service is unfavorable, and the co-occurrence with the non-favorable index utterance exceeds a predetermined threshold value. Is extracted as a new unfavorable index utterance.

Further, in the information providing device 10, the number of times the favorable index utterance indicating that the user's evaluation of the dialogue service is favorable is included in the same session, and the user's evaluation of the dialogue service is unfavorable. The number of times a non-favorable index utterance indicating a certain point is included in the same session is counted for each user's utterance, and the utterance in which the ratio of the counted times satisfies a predetermined condition is extracted as a new index utterance. ..

Further, the information providing device 10 generates a co-occurrence network connecting utterances or responses having co-occurrence from the history of the utterance and the response, identifies the index utterance from the co-occurrence network, and indicates the index on the co-occurrence network. An utterance whose distance from the utterance satisfies a predetermined condition is extracted as a new index utterance. For example, the information providing device 10 identifies a favorable index utterance indicating that the user's evaluation of the dialogue service is favorable from the co-occurrence network, and a predetermined distance from the favorable index utterance on the co-occurrence network. The utterances below the threshold are extracted as new favorable index utterances. Further, the information providing device 10 is a co-occurrence network with the non-favorable index utterance indicating that the distance from the favorable index utterance is equal to or less than a predetermined threshold value and the user's evaluation of the dialogue service is unfavorable. The utterances whose distance above is equal to or greater than a predetermined threshold are extracted as new favorable index utterances.

Further, the information providing device 10 identifies a non-favorable index utterance indicating that the user's evaluation of the dialogue service is unfavorable from the co-occurrence network, and the distance from the non-favorable index utterance on the co-occurrence network. The utterances in which is less than or equal to a predetermined threshold are extracted as new unfavorable index utterances. For example, the information providing device 10 is on a co-occurrence network with a favorable index utterance indicating that the distance from the non-favorable index utterance is equal to or less than a predetermined threshold value and the user's evaluation of the dialogue service is favorable. The utterances in which the distance in the above is equal to or greater than a predetermined threshold value are extracted as new unfavorable index utterances. As a result of these processes, the information providing device 10 can accurately extract the index utterance indicating the impression of the user.

Further, the information providing device 10 evaluates a session including the utterance and the response in the dialogue service based on whether or not the extracted index utterance is included. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

[7-8. About session evaluation using dialogue examples]
Further, the information providing device 10 extracts a series of dialogues corresponding to the index utterances, which is an index of the user's evaluation of the dialogue service, from the history of the dialogue service. That is, the information providing device 10 extracts various dialogue examples. For example, the information providing device 10 extracts utterances and responses that are predetermined by a predetermined number from the index utterances as dialogue examples. Then, the information providing device 10 learns a model for estimating the user's evaluation for the input dialogue by using the extracted dialogue.

In this way, the information providing device 10 extracts various dialogue examples instead of the session, and causes the model to learn the characteristics of the extracted dialogue examples. Extraction of such dialogue examples can be realized more easily than extraction of sessions if there are various index utterances. As a result, the information providing device 10 can easily realize the evaluation of the session.

Further, the information providing device 10 extracts a predetermined number of previous utterances and responses from the favorable index utterances indicating that the user's evaluation of the dialogue service is favorable, and uses the extracted utterances and responses. The model is learned as the learning data of the dialogue for which the evaluation by the person is favorable. Further, the information providing device 10 extracts a predetermined number of previous utterances and responses from the non-favorable index utterances indicating that the user's evaluation of the dialogue service is unfavorable, and extracts the extracted utterances and responses. , The model is learned as the learning data of the dialogue whose evaluation by the user is unfavorable. Further, the information providing device 10 extracts a predetermined number of previous utterances and responses from the neutral index utterances indicating that the user's evaluation of the dialogue service is neutral, and the extracted utterances and responses are used by the user. The model is trained as the training data of the dialogue for which the evaluation by is neutral. Therefore, the information providing device 10 can improve the evaluation accuracy of the session by the model.

Further, the information providing device 10 acquires a session including the user's utterance and response in the dialogue service. Then, the information providing device 10 evaluates the acquired session using the trained model. Therefore, the information providing device 10 can realize the evaluation of the session.

Further, the information providing device 10 extracts utterances having an appearance frequency satisfying a predetermined condition as index utterances from a session in which the evaluation satisfies a predetermined condition. For example, the information providing device 10 evaluates whether the user's evaluation of the session is favorable or unfavorable, and the frequency of appearance and use in the session in which the user's evaluation is evaluated to be favorable. The utterances whose ratio with the frequency of appearance in the session evaluated as unfavorable by the person satisfies the predetermined condition are extracted as the index utterances. Therefore, the information providing device 10 can automatically extract new index utterances.

[7-9. About session evaluation using multiple models]
Further, the information providing device 10 acquires a session including the user's utterance and response in the dialogue service. Then, the information providing device 10 evaluates the session by using the evaluation results of a plurality of models that evaluate the session based on the characteristics of different types among the plurality of features that are indicators of the user's evaluation of the session. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 uses at least the evaluation result of the model for evaluating the session based on whether or not the same or similar utterances are repeatedly included in the utterances included in the session. Further, the information providing device 10 uses at least the evaluation result of the model for evaluating the session based on whether or not the response included in the session includes a predetermined response indicating an apology. Further, the information providing device 10 holds a session based on whether or not the utterance included in the session includes a non-favorable index utterance indicating that the user is unfavorably evaluating the dialogue service. At least the evaluation result of the model to be evaluated is used. Further, the information providing device 10 evaluates the session based on whether or not the utterance included in the session includes a favorable index utterance indicating that the user favorably evaluates the dialogue service. At least the evaluation result of the model is used. Therefore, the information providing device 10 can improve the evaluation accuracy of the session.

Further, the information providing device 10 apologizes for the evaluation value output by the model that evaluates the session based on whether or not the same or similar utterances are repeatedly included in the utterances included in the session, and the response included in the session. Evaluate the session based on whether or not the given given response is included Evaluate the session based on the sum of the evaluation values output by the model. Therefore, the information providing device 10 can evaluate the session using a plurality of models that can evaluate the session without human intervention.

Further, the information providing device 10 apologizes for the evaluation value output by the model that evaluates the session based on whether or not the same or similar utterances are repeatedly included in the utterances included in the session, and the response included in the session. The evaluation value output by the model that evaluates the session based on whether or not the predetermined response is included, and the non-conformity that the user evaluates the dialogue service unfavorably for the utterance contained in the session. Favorable index From the sum of the evaluation value output by the model that evaluates the session based on whether or not the utterance is included, it is shown that the user favorably evaluates the dialogue service for the utterance included in the session. Favorable index to evaluate the session based on whether or not the utterance is included. The session is evaluated based on the value obtained by subtracting the evaluation value output by the model. For example, the information providing device 10 evaluates a session depending on whether the value is a positive value or a negative value. Therefore, the information providing device 10 can evaluate the session with high accuracy.

Although some of the embodiments of the present application have been described in detail with reference to the drawings, these are examples, and various modifications are made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure column of the invention. It is possible to carry out the present invention in other modified forms.

Further, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the extraction unit can be read as an extraction means or an extraction circuit.

10 Information provider 20 Communication unit 30 Storage unit 31 Session log database 32 Session evaluation database 33 Index utterance database 34 Engagement database 35 Model database 40 Control unit 41 Session acquisition unit 42 Usage specification unit 50 Index utterance extraction processing unit 51 Index utterance identification Part 52 Utterance extraction part 53 Network generation part 60 Session evaluation processing part 61 Learning data extraction part 62 Learning part 63 Image generation part 64 Utterance group generation part 65 Evaluation part 70 Engagement prediction processing part 71 Estimating part 80 Enhanced learning processing part 81 Index speech Acquisition unit 82 Reward setting unit 83 Reinforcement learning unit 100 Terminal device 200 Dialogue device

Claims (5)

Among the utterances for the dialogue model that realizes the dialogue service by generating the response to the input utterance, the acquisition unit that acquires the index utterance that is the index of the evaluation for the dialogue service, and the acquisition unit.
With the learning unit that performs reinforcement learning of the dialogue model by setting a reward based on the index utterance acquired by the acquisition unit.
Have,
The acquisition unit acquires the index utterance output by the utterance model that outputs the index utterance in response to the response generated by the dialogue model.
A learning device characterized by that. The acquisition unit utters a favorable index indicating that the user's evaluation of the dialogue service is favorable, or a non-favorable index utterance indicating that the user's evaluation of the dialogue service is unfavorable. To get,
The learning unit sets a positive reward for the dialogue model when the favorable index utterance is acquired, and is negative for the dialogue model when the non-favorable index utterance is acquired. The learning device according to claim 1, wherein the reward is set. Claim 1 or 2 is characterized in that the acquisition unit acquires, as the utterance model, an index utterance output by a model that has learned the characteristics of the history of dialogue between the user and the dialogue model in the dialogue service. The learning device described in. It is a learning method executed by the learning device.
Among the utterances for the dialogue model that realizes the dialogue service by generating the response to the input utterance, the acquisition process for acquiring the index utterance that is the index of the evaluation for the dialogue service, and the acquisition process.
With a learning process that performs reinforcement learning of the dialogue model by setting a reward based on the index utterance acquired by the acquisition process.
Have,
The acquisition step acquires the index utterance output by the utterance model that outputs the index utterance in response to the response generated by the dialogue model.
A learning method characterized by including things . Of the utterances for the dialogue model that realizes the dialogue service by generating the response to the input utterance, the acquisition procedure for acquiring the index utterance that is the index of the evaluation for the dialogue service, and the acquisition procedure.
A learning procedure for performing reinforcement learning of the dialogue model by setting a reward based on the index utterance acquired by the acquisition procedure.
Have,
The acquisition procedure acquires the index utterance output by the utterance model that outputs the index utterance in response to the response generated by the dialogue model.
A learning program that lets a computer do things .

Images