JP2019215483A - Leaning device, learning method, and learning program - Google Patents

Abstract

To improve accuracy of interactions.SOLUTION: A learning device according to the present application comprises: an acquisition unit configured to acquire an indicator utterance that acts as an evaluation indicator of an interactive service among utterances to an interactive model providing the interactive service by generating a response to an input utterance; and a learning unit configured to perform reinforcement leaning for the interactive model by setting reward based on the indicator utterance acquired by the acquisition unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a learning device, a learning method, and a learning program.

  2. Description of the Related Art Conventionally, there has been known an interactive system in which, when a voice or text message is received from a user, a message corresponding to the received message is provided to the user. In such an interactive system, in response to a message received from the user (hereinafter collectively referred to as "utterance"), a message having various contents corresponding to the utterance (hereinafter collectively referred to as "response") is used. The technology provided for is applied. Further, in such a dialog system, a technique for evaluating a dialog based on a flow of an utterance and a response is known.

JP 2011-253053 A

  However, in the above-described related art, it may be difficult to improve the accuracy of the conversation.

  For example, in a dialog system, a method of learning a model that generates a response from the dialog based on the evaluation of the dialog can be considered. However, such a method requires a lot of evaluations, so that it is not always easy to improve the dialogue.

  The present application has been made in view of the above, and has as its object to improve the accuracy of dialogue.

  A learning device according to the present application, an acquisition unit that acquires an index utterance that is an index of evaluation of the interactive service among utterances of an interactive model that realizes an interactive service by generating a response to an input utterance, A learning unit that performs reinforcement learning of the dialog model by setting a reward based on the index utterance obtained by the unit.

  According to one aspect of the embodiment, the accuracy of the interaction can be improved.

FIG. 1 is a diagram illustrating an example of a process performed by the information providing device according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the information providing apparatus according to the embodiment. FIG. 3 is a diagram illustrating an example of information registered in the session log database according to the embodiment. FIG. 4 is a diagram illustrating an example of information registered in the session evaluation database according to the embodiment. FIG. 5 is a diagram illustrating an example of information registered in the index utterance database according to the embodiment. FIG. 6 is a diagram illustrating an example of information registered in the engagement database according to the embodiment. FIG. 7 is a diagram illustrating an example of information registered in the model database according to the embodiment. FIG. 8 is a diagram illustrating a concept of a process of extracting a new index utterance from an index utterance serving as a seed. FIG. 9 is a diagram illustrating a concept of a process of extracting a new index utterance based on a use mode in the interactive service. FIG. 10 is a diagram illustrating a concept of a process of extracting a new index utterance based on a session evaluation result. FIG. 11 is a diagram illustrating an example of a functional configuration of the index utterance extraction processing unit according to the embodiment. FIG. 12 is a diagram illustrating the concept of processing for evaluating a session based on an index utterance. FIG. 13 is a diagram illustrating a concept of a process of evaluating a session based on an index utterance. FIG. 14 is a diagram illustrating an example of a process of evaluating a session using a ternary evaluation model in which features of three types of conversations are learned. FIG. 15 is a diagram illustrating an example of a process of evaluating a session based on engagement. FIG. 16 is a diagram illustrating the concept of processing for evaluating a session based on repeated utterances. FIG. 17 is a diagram illustrating a concept of a process of evaluating a session using a model in which a feature of automatically generated data is learned. FIG. 18 is a diagram illustrating the concept of evaluating repeated utterances using images. FIG. 19 is a diagram illustrating a concept of a process of evaluating a session based on an apology response. FIG. 20 is a diagram illustrating a concept of a process of evaluating a session by combining a plurality of evaluation methods. FIG. 21 is a diagram illustrating the concept of the process of automatically expanding a labeled session. FIG. 22 is a diagram illustrating an example of a functional configuration of the session evaluation processing unit according to the embodiment. FIG. 23 is a diagram illustrating a concept of a process of predicting a future use mode based on an index utterance. FIG. 24 is a diagram illustrating a concept of a process of predicting a future use mode based on an evaluation result. FIG. 25 is a diagram illustrating an example of a functional configuration of an engagement prediction processing unit according to the embodiment. FIG. 26 is a diagram illustrating an example of the relevance of the process performed by the information providing device according to the embodiment. FIG. 27 is a diagram illustrating a concept of a process of executing reinforcement learning of a dialog model. FIG. 28 is a diagram illustrating an example of a functional configuration of the reinforcement learning processing unit according to the embodiment. FIG. 29 is a flowchart illustrating an example of a flow of a process performed by the information providing apparatus according to the embodiment based on a use mode. FIG. 30 is a flowchart illustrating an example of a flow of a process in which the information providing apparatus according to the embodiment extracts an index utterance. FIG. 31 is a flowchart illustrating an example of the flow of a process in which the information providing apparatus according to the embodiment evaluates a session. FIG. 32 is a flowchart illustrating an example of a flow of processing in which the information providing apparatus according to the embodiment evaluates a session based on an image. FIG. 33 is a flowchart illustrating an example of a flow of processing in which the information providing apparatus according to the embodiment evaluates a session based on repetition of speech. FIG. 34 is a flowchart illustrating an example of the flow of reinforcement learning in which the information providing apparatus according to the embodiment executes an utterance as a reward. FIG. 35 is a flowchart illustrating an example of a flow of a process in which the information providing apparatus according to the embodiment extracts an index utterance based on co-occurrence. FIG. 36 is a flowchart illustrating an example of a flow of processing in which the information providing apparatus according to the embodiment evaluates a session using a ternary evaluation model. FIG. 37 is a flowchart illustrating an example of a flow of a process in which the information providing apparatus according to the embodiment evaluates a session using a plurality of models. FIG. 38 is a diagram illustrating an example of a hardware configuration. FIG. 39 is a diagram illustrating an example of a result of labeling session data. FIG. 40 is a diagram showing the results of the first experiment. FIG. 41 is a diagram showing the results of the second experiment. FIG. 42 is a diagram showing the result of Prop_D in the third experiment. FIG. 43 is a diagram showing the result of Prop_S in the fourth experiment. FIG. 44 is a diagram showing the result of Prop_E in the fifth experiment.

  Hereinafter, an embodiment for implementing a learning device, a learning method, and a learning program according to the present application (hereinafter, referred to as “embodiment”) will be described in detail with reference to the drawings. The embodiment does not limit the learning device, the learning method, and the learning program according to the present application. In the following embodiments, the same portions are denoted by the same reference numerals, and overlapping description will be omitted.

[Embodiment]
[1. Outline of information providing device]
First, an example of a process executed by an information providing device that is an example of an extraction device, an evaluation device, and a learning device will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a process performed by the information providing device according to the embodiment.

  In FIG. 1, the information providing apparatus 10 can communicate with the interactive apparatus 200 via a predetermined network N such as the Internet (for example, see FIG. 2). The information providing device 10 may be able to communicate with the terminal device 100 used by the user. It is assumed that the information providing apparatus 10 can communicate with various external servers in addition to the terminal apparatus 100 and the interactive apparatus 200.

  The terminal device 100 is a smart device such as a smartphone or a tablet, and is a portable terminal device that can communicate with an arbitrary server device via a wireless communication network such as 3G (3rd Generation) or LTE (Long Term Evolution). is there. Note that the terminal device 100 is not limited to a smart device, and may be an information processing device such as a desktop PC (Personal Computer) or a notebook PC. In addition, when the terminal device 100 acquires the voice generated by the user (so-called “utterance”), the terminal device 100 transmits the acquired uttered voice to the interactive device 200, and acquires the response transmitted from the interactive device 200, and acquires the response. Any device such as a smart speaker can be adopted as long as the device outputs a response as voice.

  The interactive device 200 is an information processing device that executes an interactive process for realizing an interaction with a user, and is implemented, for example, by a server device or a cloud system. For example, when the utterance is received from the terminal device 100, the interactive device 200 generates a response according to the received utterance, and transmits data for outputting the generated response as voice to the terminal device 100.

  Here, an example of the interactive processing realized by the interactive device 200 will be described. Note that the following interactive processing is merely an example, and does not limit the embodiment. For example, the terminal device 100 acquires a voice uttered by the user as an utterance (step S1). To give a more specific example, the terminal device 100 acquires speech data of an utterance by using various speech extraction techniques such as beamforming. In such a case, the terminal device 100 transmits the acquired audio data to a predetermined conversion server (not shown). In such a case, the conversion server generates text data of the utterance from the voice data, and transmits the generated text data to the terminal device 100. Then, the terminal device 100 transmits the text data as an utterance to the interactive device 200 (Step S2).

  On the other hand, the interaction device 200 analyzes the text data received as the utterance using various intention analysis techniques, and estimates the user's intention indicated by the utterance. Then, the interactive device 200 executes various processes according to the estimated intention, and generates text data indicating the execution result. That is, the interaction device 200 generates a response (step S3). Then, the interactive device 200 transmits the generated text data to the terminal device 100 as a response (step S4). As a result, the terminal device 100 provides a response to the utterance to the user by reading out the text data or displaying the text data on the screen (step S5).

  In the dialogue processing realized by such a dialogue apparatus 200, it is important to evaluate whether the user is satisfied with the contents of the response. For example, information indicating whether or not the user is satisfied with a certain response can be adopted as information for improving the accuracy of the interactive processing performed by the interactive device 200. However, if the user is inquired as to whether he or she is satisfied with the response, there is a risk that the user experience will be impaired.

  Here, in order to appropriately evaluate whether or not the user is satisfied with the dialogue, prepare data of a labeled dialogue with a label indicating whether or not the user is satisfied, and prepare data for various models. A method is conceivable in which the features of the data of the labeled dialogue are learned by supervised or unsupervised learning, and the dialogue is evaluated using the trained model. However, learning such a model requires a relatively large amount of data of a labeled dialogue, but there is a problem that creating data of a labeled dialogue is troublesome.

  For example, in order to create data of a dialogue with a label, the data may be created by a manual operation such as crowd sowing. However, since some dialogues include many utterances and responses (that is, long dialogues), it may be difficult to manually apply labels. Therefore, it is considered that if the data of the labeled dialogue used for the learning can be automatically acquired, the learning can be performed efficiently, and the accuracy of the dialogue processing can be improved.

  Thus, the information providing apparatus 10 executes an evaluation process for automatically evaluating whether or not the user is satisfied with the interaction process, using the utterance and response of the user in the interaction process. In addition, the information providing apparatus 10 executes an extraction process of extracting an utterance used when performing the evaluation process. The information providing device 10 executes a learning device that learns a model for performing a conversation. Then, the information providing device 10 improves the accuracy of the interactive processing by the interactive device 200 based on the extraction device, the evaluation device, and the learning device.

[1-1. Example of processing executed by information providing apparatus]
Hereinafter, an example of a process performed by the information providing apparatus 10 will be described. First, the information providing apparatus 10 acquires a dialog history from the dialog apparatus 200, which is a history of an utterance and a response in the dialog processing (step S6). Then, the information providing apparatus 10 evaluates the conversation based on the conversation history (Step S7).

  More specifically, the information providing apparatus 10 sets, as a session, a dialogue history including a series of multiple utterances such as a user's utterance, a response to the utterance, and a user's utterance to the response, and a plurality of responses. Extract. Here, the information providing apparatus 10 may employ, as one session, a plurality of utterances and responses determined by an arbitrary standard.

  For example, when a state in which there is no utterance from the user has elapsed for a predetermined time (for example, 30 minutes), the information providing apparatus 10 transmits the utterance received from the user and the response provided to the user up to that time. The utterance and response included in one session may be used. Further, the information providing apparatus 10 may use the utterance and the response acquired from the time when the user activates the application for performing the interactive processing until the time when the application sleeps or ends, as one session. That is, the session may include a speech and a response grouped in arbitrary units as long as the series of conversations can be a unit of evaluation by the user.

  Then, the information providing apparatus 10 evaluates the dialogue in the session. More specifically, the information providing apparatus 10 estimates whether or not the user is satisfied with the conversation in the session based on the response and the utterance in the session. For example, the information providing apparatus 10 evaluates a session by using evaluation results of a plurality of models that evaluate a session based on characteristics of different types from among a plurality of characteristics that are indexes of a user's evaluation of the session. Do.

  For example, when utterances included in one session include utterances having the same or similar contents, it is estimated that the user is not satisfied with the dialogue in the session. For example, when the user utters “work of an inn” with the intention of seeking a job at an inn, the interactive device 200 erroneously determines that the user intends to search for accommodation from the word “inn”. , "A guide to accommodation facilities" may be returned. When a response contrary to the intention is output, the user presumes that the speech recognition did not work well, and speaks again “work of the inn”, and the interactive device 200 again “guides the accommodation”. A response may be returned. Thus, repetition of similar utterances in one session is considered to indicate that the user is not very satisfied with the session.

  Therefore, the information providing apparatus 10 generates a model for evaluating a session based on repetition of the same or similar utterance (step S7-1). For example, when the information providing apparatus 10 inputs a character string of an utterance or a response included in the session, based on the repetition of the utterance included in the session, the user is not satisfied with the session. That is, the repetition evaluation model M1 that outputs the first degree of dissatisfaction, which is a score indicating the degree of dissatisfaction of the user, is generated. For example, such a repetition evaluation model uses the same utterance (or the same character string) when a session including the same utterance (or the same character string) or a session including a similar utterance (or a similar character string) is input. The first dissatisfaction degree is output at a higher value than when a session that does not include a session that does not include a column or a session that does not include a session that does not include a similar utterance (or a similar character string) is input. Here, “similar utterance” is an utterance having a similar meaning, such as “work of an inn” and “work of a hotel”. The repetition evaluation model M1 evaluates a session based not on the character string itself of the session but on the characteristics of an image generated from the character string. The details of such processing will be described later.

  In addition, for example, a response indicating an apology such as “sorry” or “sorry” may be provided for an utterance such as “no”. When a response indicating such an apology (hereinafter sometimes collectively referred to as an “apology response”) is included in a session, it is estimated that the user is not very satisfied with the session. Therefore, the information providing apparatus 10 generates a model for evaluating the session based on whether or not an apology response is included (step S7-2). For example, the information providing apparatus 10 outputs a higher second dissatisfaction degree for a session including more apology responses, and outputs a lower value for a session including no more apology responses. An apology evaluation model M2 that outputs the second degree of dissatisfaction of the value is generated.

  Further, the utterance of the user includes an utterance that suggests the user's impression of the session, and the utterance that suggests the impression of the user is an utterance that is an index of the evaluation of the session, that is, an index utterance. Can be adopted as For example, an utterance such as “unusable” or “uninstall” of the user is considered to be an utterance indicating the user's unfavorable evaluation of the response, that is, unfabricable feedback. It is estimated that a session including many utterances indicating such unfavorable evaluations (hereinafter referred to as “unfavorable index utterances”) has low user satisfaction. In addition, the user's utterance such as “awesome” or “good” is considered to be an utterance indicating a favorable evaluation of the user for the response, that is, a favorable feedback. A session including many utterances indicating such favorable evaluations (hereinafter referred to as “favorable index utterances”) is estimated to have high user satisfaction.

  That is, such favorable index utterances and unfavorable index utterances (hereinafter sometimes collectively referred to as “index utterances”) can be an index of the user's satisfaction with the session. Therefore, the information providing apparatus 10 generates a model for evaluating a session based on various index utterances such as a favorable index utterance and an unfavorable index utterance.

  For example, the information providing apparatus 10 generates a model for evaluating the session based on the unfavorable index utterance (Step S7-3). For example, the information providing apparatus 10 may set a higher value for a session in which the utterance included in the session includes a larger number of unfavorable index utterances such as “unusable” and “uninstall”. The third dissatisfaction degree is output, and the unfavorable index utterance evaluation model M3 that outputs a lower third dissatisfaction degree is generated for a session that does not include the unfavorable index utterance.

  Further, the information providing device 10 generates a model for evaluating the session based on the favorable index utterance (Step S7-4). For example, the information providing apparatus 10 may set a higher value for a session in which the utterance included in the session includes more preset utterances of a favorable index such as “awesome” or “good”. , And generates a favorable index utterance evaluation model M4 that outputs a lower first satisfaction degree for a session that does not include more favorable index utterances. Note that the above-mentioned unfavorable index utterance or favorable index utterance may be an utterance automatically and appropriately extracted from a session by a process described later.

  Note that the information providing apparatus 10 may learn each model using any known technique. For example, the information providing apparatus 10 may learn each model using a technique such as back propagation, reinforcement learning, and GAN (Generative Adversarial Network).

  Then, the information providing apparatus 10 evaluates the input session based on each dissatisfaction degree and satisfaction degree output by each model (step S7-5). For example, the information providing apparatus 10 extracts a session to be evaluated, and repeats the extracted session for each of the evaluation model M1, the apology evaluation model M2, the unfavorable index utterance evaluation model M3, and the favorable index utterance evaluation model M4. To enter. Then, the information providing apparatus 10 calculates, as the dissatisfaction score, a value obtained by subtracting the first satisfaction from the sum of the first dissatisfaction, the second dissatisfaction, and the third dissatisfaction. That is, the information providing apparatus 10 calculates, as the dissatisfaction degree score, a score that is so high that the user is not satisfied with the session. Note that “dissatisfied” means not only that the user is frustrated with the response in the session, but also that the user is uncomfortable with the session, such as when the user is amazed. It is a concept that includes any state that has.

  Further, the information providing apparatus 10 calculates a dissatisfaction score from each session extracted from the conversation history. Then, the information providing apparatus 10 extracts a predetermined number of sessions from each session having a higher dissatisfaction score, and sets the extracted sessions as low-satisfaction sessions in which the user has a lower degree of satisfaction. The information providing apparatus 10 extracts a predetermined number of sessions in descending order of the dissatisfaction score, and sets the extracted sessions as high-satisfaction sessions, which are sessions with a high degree of user satisfaction. Note that the information providing apparatus 10 may determine a session in which the dissatisfaction score exceeds a predetermined threshold as a low-satisfaction session, and a session in which the dissatisfaction score is lower than a predetermined threshold as a high-satisfaction session. By performing such a process, the information providing apparatus 10 realizes a process of evaluating whether or not the user is satisfied with the session.

  Then, the information providing device 10 updates the interaction model based on the evaluation result (Step S8). For example, when a certain utterance and a response to the utterance are included in the low-satisfaction session, the information providing apparatus 10 reduces the probability of outputting the response to the utterance, and the certain utterance and the response to the utterance If included in the high satisfaction session, the dialogue model for generating the response is updated so as to increase the probability of outputting the response to the utterance. That is, the information providing apparatus 10 updates the interaction model so that the user is more satisfied.

  As described above, the information providing apparatus 10 acquires the session including the utterance and the response of the user in the interactive service, and based on the different types of features among the plurality of features serving as indexes of the user's evaluation of the session. The session is evaluated using the evaluation results of the plurality of models M1 to M4 for evaluating the session. As a result, the information providing apparatus 10 comprehensively evaluates the session from a plurality of viewpoints, so that the accuracy of the evaluation can be improved. Further, the information providing apparatus 10 can automatically realize the session evaluation without manually evaluating the session such as crowd sourcing, and thus can easily evaluate the dialogue.

[1-2. Processing performed by information providing device]
In the above-described example, as an example of the process performed by the information processing apparatus 10, according to the presence / absence of repetition of the utterance included in the session, the presence / absence of an apology response, and the presence / absence of a favorable index utterance or an unfavorable index utterance An example of the process for evaluating a session has been described. However, embodiments are not limited to this.

  For example, as described below, the information providing apparatus 10 may evaluate a session from various viewpoints. Further, the information providing apparatus 10 may automatically extract various index utterances from utterances included in the session in order to evaluate such a session. In order to evaluate such a session, the information providing apparatus 10 may estimate a period and a frequency (hereinafter, sometimes referred to as “engagement”) in which the user uses the interactive service. In addition, the information providing apparatus 10 may learn a dialog model for generating a response from an utterance based on the history of the dialog.

  That is, the information providing apparatus 10 may realize the evaluation of the session illustrated in FIG. 1 by combining various processes described below. Note that the various processes described below may be performed in any combination.

[1-3. About user's subjectivity)
In the above description, the information providing apparatus 10 evaluates the session using various index utterances indicating a favorable impression or a non-favorable impression of the response to the user. Here, “favorable impressions” and “unfavorable impressions” include not only impressions that the user likes or dislike, but also positive impressions of the user, such as impressions that the response is correct or incorrect. It is a concept that includes various impressions and negative impressions.

  Further, in the above description, the information providing apparatus 10 evaluated whether or not the user was satisfied with the session. Here, the information providing apparatus 10 may perform not only a binary evaluation of whether or not the user is satisfied, but also an evaluation of an arbitrary mode. For example, the information providing apparatus 10 may use the value of the dissatisfaction score itself as the evaluation result. In addition, the information providing apparatus 10 was trained to calculate a score in which the value is higher as the user's satisfaction is higher, and the value is lower as the user's satisfaction is lower, that is, the satisfaction is not the dissatisfaction. Models M1 to M3 may be used. In addition, the information providing apparatus 10 may calculate, for each session, a satisfaction score indicating the degree of satisfaction of the user instead of the dissatisfaction score.

  In the above description, the information providing apparatus 10 evaluates whether the user is “satisfied” or “unsatisfied” with the session. Here, the concept of “satisfied” or “unsatisfied” is a concept including various favorable impressions or unfavorable impressions of the user on the session, such as whether the user is frustrated with the session. It is.

  That is, the description such as “favorable” or “satisfied” indicates various favorable impressions by the user, and the description “unfavorable” or “unsatisfied” It shows various impressions. If the information providing apparatus 10 evaluates the session based on various index utterances indicating the user's impression of the response, the information providing apparatus 10 can perform arbitrary processing by various processing described in the embodiment regardless of the type of the user's impression. The evaluation based on the impression of may be realized.

[1-4. Evaluation based on index utterance]
In the above description, the information providing apparatus 10 evaluates whether the user is satisfied with the session based on various index utterances such as favorable index utterances and unfavorable index utterances. Here, the various index utterances are utterances indicating whether or not the user has a favorable impression on an object such as a dialogue or a response until the index utterance appears. This is a possible utterance.

  Here, as the index utterance, an utterance indicating an impression of the user with respect to an arbitrary target can be adopted as long as the utterance can be an index of the evaluation of the session regardless of the target of the user's impression indicated by the index utterance. For example, the information providing apparatus 10 may use, as an index utterance, an utterance indicating the user's impression of a single response (that is, an utterance that serves as a feedback), and generate an utterance indicating the user's impression of a series of dialogues or the entire session. It may be adopted as an index utterance. That is, if the utterance indicating the user's evaluation of the dialogue service in response to the response is used as the index utterance, the information providing apparatus 10 adopts the utterance indicating the user's impression of any object in the dialogue service as the index utterance. Is also good.

[2. Configuration of Information Provision Device)
Hereinafter, an example of a functional configuration of the information providing apparatus 10 that realizes various processes executed by the information providing apparatus 10 will be described. FIG. 2 is a diagram illustrating a configuration example of the information providing apparatus according to the embodiment. As shown in FIG. 2, the information providing device 10 includes a communication unit 20, a storage unit 30, and a control unit 40.

[2-1. About communication department)
The communication unit 20 is realized by, for example, a NIC (Network Interface Card). The communication unit 20 is connected to the network N by wire or wirelessly, and transmits and receives information between the terminal device 100 and the interactive device 200.

[2-2. About storage unit)
The storage unit 30 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 30 stores a session log database 31, a session evaluation database 32, an index utterance database 33, an engagement database 34, and a model database 35.

  Hereinafter, an example of each of the databases 31 to 35 stored in the storage unit 30 will be described with reference to FIGS. A conversation history is registered in the session log database 31 for each session. For example, FIG. 3 is a diagram illustrating an example of information registered in the session log database according to the embodiment. As shown in FIG. 3, information having items such as “session ID (Identifier)”, “dialogue time”, “user ID”, and “dialogue content” is registered in the session log database 31. In addition, any information other than the information shown in FIG. 3 may be registered in the session log database 31 as long as the information is related to an utterance and a response in the interactive processing.

  Here, the “session ID” is information for identifying a session. The “dialogue time” is information indicating the date and time when the dialogue included in the session indicated by the “session ID” is performed, and is, for example, information indicating the date and time when the first utterance included in the session is acquired. . The “user ID” is an identifier for identifying a user who has made an utterance included in the session indicated by the “session ID”. Further, the “interaction content” is information in which the contents of utterances and responses included in the session indicated by the “session ID” are arranged in chronological order.

  For example, in the example shown in FIG. 3, the session ID “session # 1”, the conversation time “dialog time # 1”, the user ID “user # 1”, the conversation content “Usr: work at an inn sys: guide the accommodation facility” ... (omitted) ... ". In such information, the date and time when the conversation included in the session indicated by the session ID “session # 1” was performed is “dialogue time # 1”, and the user who spoke in the session is “user # 1”. Indicates that the user is the indicated user. The session indicated by the session ID “session # 1” includes the user's utterance “work of an inn”, and indicates that the response to such utterance is “guide the accommodation facility”.

  In the example shown in FIG. 3, conceptual values such as “session # 1”, “interaction time # 1”, and “user # 1” are described. , A numerical value indicating the date and time, a character string or numerical value for identifying the user, and the like are registered in the session log database 31.

  In the session evaluation database 32, the evaluation result for the session is registered. For example, FIG. 4 is a diagram illustrating an example of information registered in the session evaluation database according to the embodiment. As shown in FIG. 4, information having items such as “session ID”, “session type”, and “satisfaction score” is registered in the session evaluation database 32. The information registered in the session evaluation database 32 is not limited to this, and any information may be registered as long as the information is related to the evaluation of the session. Further, the session evaluation database 32 may be integrated with the session log database 31, for example.

  Here, the “session type” is information indicating whether or not it is estimated that the user has been satisfied with the dialogue in the session indicated by the associated “session ID”. This is the label to be given. For example, if the “session type” is estimated that the user is not satisfied (for example, the dissatisfaction score is high), the “low-satisfaction session” indicates that the user is satisfied (for example, the dissatisfaction score is high). If it is estimated to be "low", then "high satisfaction session" is assumed to be neither (for example, if the dissatisfaction score falls within a predetermined range), then "neutral session" is adopted. You. The “dissatisfaction score” is the value of the dissatisfaction score assigned to the session indicated by the associated “session ID”.

  For example, in the example illustrated in FIG. 4, the session ID “session # 1”, the session type “low satisfaction session”, and the dissatisfaction score “150” are registered in the session evaluation database 32 in association with each other. Such information indicates that the session type of the session indicated by the session ID “session # 1” is “low satisfaction session” and the dissatisfaction score is “150”. When the session is not evaluated, the session type is “unrated”, and when the dissatisfaction score is not calculated, the dissatisfaction score is “unrated”.

  Various index utterances are registered in the index utterance database 33. For example, FIG. 5 is a diagram illustrating an example of information registered in the index utterance database according to the embodiment. As shown in FIG. 5, information having items such as “index utterance”, “index type”, “index score”, and “registration source” is registered in the index utterance database 33. Note that, in addition to the information illustrated in FIG. 5, various kinds of information regarding the index utterance may be registered in the index utterance database 33.

  Here, the “index utterance” is a character string of an utterance that is an index of whether or not the user is satisfied with a dialogue in a session including the utterance, such as an unfavorable index utterance or a favorable index utterance. It is. The “index type” is information indicating whether the associated index utterance is an unfavorable index utterance or a favorable index utterance. The “index score” is a score indicating the degree to which the user is not satisfied with the dialogue in the session including the associated “index utterance”. In addition, “registration source” means whether the associated index utterance is a preset index utterance, that is, an index utterance to be a seed, or an index utterance automatically extracted from a session by an extraction process. If the utterance is an index utterance extracted from a session, the session ID of the session from which the session was extracted is registered.

  For example, in the example illustrated in FIG. 5, the index utterance database 33 registers the index utterance “good”, the index type “favorable”, the index score “−150”, and the registration source “pre-registration” in association with each other. Have been. In such information, a character string “good” is registered as an index utterance, and the type of the index utterance is “favorable”, that is, a favorable index utterance, and the index score is “−150”. Is shown. Such information indicates that the favorable index utterance “good” is an index utterance registered in advance, that is, an index utterance to be a seed.

  In the engagement database 34, information indicating the manner in which the user uses the interactive service is registered. More specifically, information such as a use period and a use frequency, that is, engagement information indicating how much the user uses the interactive service is registered in the engagement database 34. For example, FIG. 6 is a diagram illustrating an example of information registered in the engagement database according to the embodiment. As shown in FIG. 6, information having items such as “user ID”, “engagement history”, and “predicted engagement” is registered in the engagement database 34. Any information other than the information shown in FIG. 6 may be registered in the engagement database 34 as long as the information is related to the engagement.

  Here, the “engagement history” is a history period that is a history of the use period of the interactive service by the user indicated by the associated “user ID”, and a history of the use frequency of the interactive service by the user. This is information indicating “history frequency”. Further, the “prediction engagement” indicates a “prediction period” that is a future use period by the user indicated by the associated “user ID” and a “prediction frequency” that is a future use frequency by the user. Information.

  The “history period” is, for example, information indicating a period during which the user has continuously used the interactive service in a predetermined period in the past. For example, the “history period” is a time period without a predetermined period (for example, several days). This is information indicating the longest period during which the user has used the interactive service. The “history frequency” is information indicating the frequency at which the user has used the interactive service during the history period. For example, the number of times the user has used the interactive service during the history period (for example, Number) divided by the history period. The “prediction period” is information indicating a period in which the user is expected to use the interactive service continuously in a predetermined period in the future. The “predicted frequency” is information indicating a frequency at which the user is assumed to use the interactive service in a predetermined period in the future.

  For example, in the example illustrated in FIG. 6, the user ID “user # 1”, the engagement history “history period # 1 history frequency # 1”, and the prediction engagement “prediction period # 1 prediction frequency # 1” are registered in association with each other. ing. Such information includes that the engagement history of the user indicated by the user ID “user # 1” is “history period # 1” and “history frequency # 1”, and the prediction engagement is “prediction period # 1” and “history frequency # 1”. Predicted frequency # 1. "

  In the example illustrated in FIG. 6, conceptual values such as “history period # 1”, “history frequency # 1”, “prediction period # 1”, and “prediction frequency # 1” are described. Numerical values indicating various periods and frequencies are registered.

  Various models are registered in the model database 35. For example, FIG. 7 is a diagram illustrating an example of information registered in the model database according to the embodiment. As shown in FIG. 7, information having items such as “model ID”, “model type”, and “model data” is registered in the model database 35. Note that, in addition to the information shown in FIG. 7, various types of information regarding the model may be registered in the model database 35.

  Here, the “model ID” is an identifier for identifying a model. The “model type” is information indicating what kind of model the model is. “Model data” is model data. Note that, in the example shown in FIG. 7, a conceptual value such as “model data # 1” is described. However, information such as various parameters constituting the model is actually registered.

  For example, in the example illustrated in FIG. 7, the model database 35 includes a “repetition evaluation model” that evaluates a session based on whether or not the session includes a repetition, and whether or not an apology response is included in the session. "Apology evaluation model" that evaluates sessions based on, "Unfavorable index utterance evaluation model" that evaluates sessions based on whether or not the session contains unfavorable index utterances, and "Indicator utterances that are favorable to sessions" A “favorable index utterance evaluation model” that evaluates a session based on whether or not is included is registered.

  In addition, the model database 35 includes a “ternary evaluation model” that learns features of a dialogue that is estimated to be satisfied by the user, a dialogue that is estimated to be not satisfied by the user, and a dialogue that is neither of them. Is registered. In the model database 35, an "engagement session evaluation model" in which the feature of the session including the utterance of the user in which the use mode of the interactive service satisfies the predetermined condition is registered. In the model database 35, a “repeated utterance evaluation model” that is a pseudo-generated session and that has learned the features of the session including the same or similar utterance is registered.

  In the model database 35, a "dialog model" for generating a response to the utterance, which is a model used by the dialog device 200, is registered. In the model database 35, a "user model model" that is a model that simulates a user and that outputs an utterance in response to a response when the response is input is registered.

  In the model database 35, model information of an arbitrary format is registered as model data of each model. For example, each model may be realized by an SVM (Support Vector Machine) or a DNN (Deep Neural Network). Here, the DNN may be a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). Further, the RNN may be an LSTM (Long Short-Term Memory) or the like. That is, each model can adopt a model of an arbitrary format. In addition, each model may be a model realized by combining a plurality of models, such as a model combining CNN and RNN.

[2-3. About control unit)
Returning to FIG. 2, the description will be continued. The control unit 40 is a controller. For example, various programs stored in a storage device inside the information providing apparatus 10 by a processor such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit) are stored in a RAM or the like. As a work area. The control unit 40 is a controller, and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  As illustrated in FIG. 2, the control unit 40 includes a session acquisition unit 41, a use mode specifying unit 42, an index utterance extraction processing unit 50, a session evaluation processing unit 60, an engagement prediction processing unit 70, and a reinforcement learning processing unit 80. .

  The session acquisition unit 41 acquires a session. For example, when acquiring the conversation history from the conversation device 200, the session acquisition unit 41 divides the acquired conversation history for each session. For example, the session acquisition unit 41 selects a user to be processed and extracts an utterance history including the utterance of the selected user. Subsequently, the session acquisition unit 41 specifies a time between each response and the utterance in the extracted utterance history. If the time from one response to the next utterance is longer than a predetermined period, the session acquisition unit 41 determines that the response is the last response in the session, and uses the response based on the determination result. The conversation with the person into sessions. Then, the session acquisition unit 41 assigns a session ID and a user ID of the user to each session, and registers the session ID in the session log database 31.

  The usage mode specifying unit 42 specifies the usage mode of the user. For example, the use mode specifying unit 42 selects a user to be processed, and specifies a use mode of the user's interactive service within a predetermined period (for example, the past six months). More specifically, the usage mode specifying unit 42 specifies a history period and a history frequency within a predetermined period. Then, the usage mode specifying unit 42 registers the specified history period and history frequency in the engagement database 34 as an engagement history in association with the user ID of the user.

[2-4. About extraction processing)
The index utterance extraction processing unit 50 executes an extraction process of extracting an index utterance used for evaluating the session from the session. Hereinafter, the concept of the process executed by the index utterance extraction processing unit 50 and an example of the functional configuration of the index utterance extraction processing unit 50 will be described with reference to FIGS.

[2-4-1. About extraction processing based on co-occurrence]
For example, the index utterance extraction processing unit 50 extracts a new index utterance based on co-occurrence with the index utterance. For example, FIG. 8 is a diagram illustrating a concept of a process of extracting a new index utterance from an index utterance serving as a seed. For example, if a certain session includes a favorable index utterance, the same session is likely to include a favorable index utterance. Further, when a certain session includes an unfavorable index utterance, it is estimated that the same session is likely to include an unfavorable index utterance. That is, in the same session, favorable index utterances are likely to co-occur and non-favorable index utterances are likely to co-occur.

  Therefore, the index utterance extraction processing unit 50 identifies the utterance which is an index of the evaluation of the user with respect to the interactive service from the history of the utterance of the user in the interactive service and the response to the utterance, and specifies a preset index utterance. Then, a new index utterance is extracted from the user's utterance based on the co-occurrence with the specified index utterance. For example, the index utterance extraction processing unit 50 extracts a new index utterance from each utterance based on co-occurrence with a seed non-favorable index utterance or a seed-friendly index utterance registered in advance, and generates an index utterance database 33. Register with.

  For example, when the character string “Wow” is registered in the index utterance database 33 as the seed-friendly index utterance, the index utterance extraction processing unit 50 includes the character string “Wow” from the session log database 31 in the utterance. Extract all sessions. Further, for each utterance included in the extracted session, the index utterance extraction processing unit 50 counts the number of sessions including the utterance. That is, the index utterance extraction processing unit 50 counts the number of co-occurrences with the seed favorable index utterance for each utterance.

  Further, for example, when a character string “Idiot” is registered in the index utterance database 33 as a seed unfavorable index utterance, the index utterance extraction processing unit 50 reads the character string “Idiot” from the session log database 31. Extracts all sessions included in the utterance. Further, for each utterance included in the extracted session, the index utterance extraction processing unit 50 counts the number of sessions including the utterance. That is, the index utterance extraction processing unit 50 counts the number of co-occurrences with the seed unfavorable index utterance for each utterance.

  Then, the index utterance extraction processing unit 50 calculates a ratio of the counted number of times of co-occurrence for each utterance. More specifically, the index utterance extraction processing unit 50 calculates, for each utterance, a value obtained by dividing the number of times of co-occurrence with the seed unfavorable index utterance by the number of times of co-occurrence with the seed favorable index utterance. Then, the index utterance extraction processing unit 50 extracts the utterances having the higher N ratio (N is an arbitrary number) as new unfavorable index utterances, and extracts the utterances having the lower N ratios as the new favorable index. Extract as utterance. In addition, the index utterance extraction processing unit 50 registers the extracted unfavorable index utterance or favorable index utterance in the index utterance database 33.

  Here, the index utterance extraction processing unit 50 may extract a new index utterance from the seed index utterance based on the activity propagation on the co-occurrence network. For example, the index utterance extraction processing unit 50 generates a co-occurrence network of utterances included in the same session as the seed index utterance. For example, the index utterance extraction processing unit 50 generates a network in which each utterance is a node and the utterances included in the same session are connected by a link. It should be noted that any known technique can be employed for generating the co-occurrence network.

  Subsequently, the index utterance extraction processing unit 50 specifies a node corresponding to the seed index utterance from the co-occurrence network and, for each node, a distance between the seed index utterance and the corresponding node (for example, a hop following a link). Number). Then, the index utterance extraction processing unit 50 generates a new unfavorable utterance corresponding to a node in which the distance between the node corresponding to the seed unfavorable index utterance and the corresponding node is short and the distance between the node corresponding to the seed unfavorable index utterance and the corresponding node is long. The utterance is a target index utterance. In addition, the index utterance extraction processing unit 50 generates a new favorable utterance corresponding to a node in which the distance between the node corresponding to the seed favorable index utterance and the corresponding node is short and the distance between the node corresponding to the seed unfavorable index utterance and the corresponding node is long. This is an index utterance.

  For example, the index utterance extraction processing unit 50 specifies and identifies a node whose distance from the seed favorable index utterance is less than a predetermined threshold and whose distance from the seed unfavorable index utterance is equal to or greater than a predetermined threshold. The utterance corresponding to the node is extracted as a new favorable index utterance. In addition, the index utterance extraction processing unit 50 specifies and identifies a node whose distance from the seed favorable index utterance is equal to or greater than a predetermined threshold and whose distance from the seed unfavorable index utterance is less than a predetermined threshold. The utterance corresponding to the node is extracted as a new unfavorable index utterance. Then, the index utterance extraction processing unit 50 registers each extracted index utterance in the index utterance database 33.

  Note that the index utterance extraction processing unit 50 uses the index utterance extracted by the above-described extraction processing as a new seed index utterance, and executes the above-described extraction processing using the new seed index utterance, thereby obtaining a new index utterance. May be extracted.

[2-4-2. About extraction processing based on usage mode)
In addition, for example, the index utterance extraction processing unit 50 extracts a new index utterance based on a use mode of the user in the interactive service. For example, FIG. 9 is a diagram illustrating a concept of a process of extracting a new index utterance based on a use mode in the interactive service.

  For example, a period during which the interactive service is continuously used, that is, a user who has a long usage period or a user who frequently uses the interactive service is estimated to be satisfied with the interactive service. For this reason, it is presumed that a user who has been using for a long period of time or a user who has been used frequently has a favorable (or favorable) utterance, that is, utters a lot of favorable index utterances. On the other hand, it is estimated that a user with a short usage period or a user with a short usage frequency will utter unfavorable (unfavorable) utterances, that is, many unfavorable index utterances.

  Therefore, the index utterance extraction processing unit 50 obtains a session including a user's utterance in which the usage mode of the interactive service satisfies a predetermined condition, and evaluates the user for the interactive service based on the appearance frequency in the obtained session. An index utterance to be an index of is extracted. For example, the index utterance extraction processing unit 50 refers to the engagement database 34 and specifies a user whose engagement history satisfies a predetermined condition.

  For example, the index utterance extraction processing unit 50 identifies a user whose history period and history frequency each exceed a predetermined threshold as a favorable user, and a user whose history period and history frequency are each less than a predetermined threshold. Are identified as unfavorable users. Further, the index utterance extraction processing unit 50 specifies a session including the utterance of the favorable user and a session including the utterance of the unfavorable user, and extracts the utterance included in the specified session.

  Then, the index utterance extraction processing unit 50 calculates, for each extracted utterance, an appearance frequency in a session of a favorable user and an appearance frequency in a session of a non-favorable user. For example, the index utterance extraction processing unit 50 calculates a value obtained by dividing the number of times the utterance appeared in the session including the utterance of the favorable user by the total number of sessions including the utterance of the favorable user in the session of the favorable user. It is calculated as an appearance frequency. In addition, for example, the index utterance extraction processing unit 50 calculates a value obtained by dividing the number of times the utterance appeared in the session including the utterance of the unfavorable user by the total number of sessions including the utterance of the unfavorable user, It is calculated as the frequency of appearance in the user's session.

  Further, the index utterance extraction processing unit 50 calculates a value obtained by dividing the frequency of appearance in the session of the unfavorable user by the frequency of appearance in the session of the favorable user, that is, the ratio of the frequency of appearance for each utterance. Then, the index utterance extraction processing unit 50 extracts the upper N utterances having the calculated ratio as new unfavorable index utterances and extracts the lower N utterances as new favorable index utterances.

[2-4-3. Extraction processing based on evaluation results]
Further, for example, the index utterance extraction processing unit 50 extracts a new index utterance based on the evaluation result of the session. For example, FIG. 10 is a diagram illustrating a concept of a process of extracting a new index utterance based on an evaluation result of a session.

  For example, an evaluation result may be given to a session by an evaluation process described later. In addition, a session may be provided with a manual evaluation result such as crowd sourcing. An evaluation result for such a session can be an index for extracting an index utterance from the utterance included in the session.

  For example, it is estimated that a session with a high degree of satisfaction by the user (that is, a session with high satisfaction) contains many favorable index utterances. On the other hand, it is estimated that a session with a low degree of satisfaction by the user (that is, a session with low satisfaction) contains a lot of unfavorable index utterances.

  Therefore, the index utterance extraction processing unit 50 favorably evaluates the utterance whose appearance frequency satisfies the predetermined condition among the utterances included in the session whose evaluation satisfies the predetermined condition, with respect to the session including the utterance. , Ie, extracted as an index utterance that is an index of whether the user is satisfied with the session. For example, the index utterance extraction processing unit 50 extracts the utterances included in the high satisfaction session and the low satisfaction session, respectively, and calculates the appearance frequency in the low satisfaction session and the appearance frequency in the high satisfaction session for each extracted utterance.

  Then, the index utterance extraction processing unit 50 calculates a value obtained by dividing the appearance frequency in the low satisfaction session by the appearance frequency in the high satisfaction session as a ratio of the appearance frequency. Then, the index utterance extraction processing unit 50 extracts the upper N utterances having the calculated ratio as new unfavorable index utterances and extracts the lower N utterances as new favorable index utterances.

[2-4-4. Function configuration of index utterance extraction processing unit]
Next, an example of a functional configuration of the index utterance extraction processing unit 50 will be described with reference to FIG. FIG. 11 is a diagram illustrating an example of a functional configuration of the index utterance extraction processing unit according to the embodiment. As illustrated in FIG. 11, the index utterance extraction processing unit 50 includes an index utterance identification unit 51, an utterance extraction unit 52, and a network generation unit 53.

  The index utterance specifying unit 51 specifies, from the history of the user's utterance and response in the interactive service, an utterance serving as an index of the user's evaluation of the interactive service, which is a preset index utterance. For example, the index utterance identification unit 51 refers to the index utterance database 33 and identifies the index utterance. Note that the index utterance identification unit 51 may identify an index utterance registered in advance, or may identify an index utterance extracted from a session.

  Then, the index utterance specifying unit 51 refers to the session log database 31 and searches for a session including the identified index utterance. That is, the index utterance specifying unit 51 specifies a session in which a user utterance includes a favorable index utterance or a non-favorable index utterance in the session history. Then, the index utterance specifying unit 51 notifies the utterance extracting unit 52 of the specified session, that is, the session including the index utterance.

  When specifying the index utterance from the co-occurrence network illustrated in FIG. 8, the index utterance specifying unit 51 notifies the network generation unit 53 of the session including the index utterance. Then, the index utterance identification unit 51, among the nodes included in the co-occurrence network generated by the network generation unit 53, the node corresponding to the identified index utterance, that is, the favorable index registered in the index utterance database 33. The node corresponding to the utterance or the unfavorable index utterance is specified. Then, the index utterance specifying unit 51 notifies the utterance extracting unit 52 of the co-occurrence network and the specified node.

  The utterance extraction unit 52 extracts an index utterance from each session. For example, as illustrated in FIG. 8, the utterance extracting unit 52 extracts a new index utterance from the utterance included in the session history based on the co-occurrence with the index utterance serving as a seed.

  For example, the utterance extracting unit 52 extracts, from the session log database 31, the session specified by the index utterance specifying unit 51, that is, the session including the index utterance. Then, the utterance extraction unit 52 specifies, for each utterance, co-occurrence with the favorable index utterance, and extracts an utterance whose co-occurrence with the favorable index utterance exceeds a predetermined threshold as a new favorable index utterance. I do. Further, the utterance extracting unit 52 specifies, for each utterance, co-occurrence with the unfavorable index utterance, and extracts an utterance in which the specified co-occurrence exceeds a predetermined threshold as a new unfavorable index utterance.

  For example, for each utterance included in the extracted session, the utterance extraction unit 52 counts the number of appearances in the session including the favorable index utterance and the number of appearances in the session including the unfavorable index utterance. That is, the utterance extraction unit 52 counts, for each utterance, the number of times that the favorable index utterance is included in the same session and the number of times that the unfavorable index utterance is included in the same session. Then, the utterance extraction unit 52 extracts, as a new index utterance, an utterance in which the ratio of the counted times satisfies a predetermined condition. For example, the utterance extraction unit 52 calculates, as a ratio, a value obtained by dividing the number of appearances in the session including the unfavorable index utterance by the number of appearances in the session including the favorable index utterance, and calculates the N highest ratios. The utterance is defined as a new unfavorable index utterance, and the utterances having the lower N ratios are defined as new favorable index utterances.

  Note that the utterance extraction unit 52 may simply set an utterance whose appearance frequency in a session including the favorable index utterance exceeds a predetermined threshold value as a new favorable index utterance. In addition, the utterance extraction unit 52 may set, as a new unfavorable index utterance, an utterance in which the number of appearances in a session including the unfavorable index utterance exceeds a predetermined threshold.

  In addition, when the utterance extraction unit 52 receives a notification of the node corresponding to the co-occurrence network and the index utterance, the utterance extraction unit 52 determines, based on the co-occurrence network, an utterance whose co-occurrence with the index utterance satisfies a predetermined condition. Extract as That is, the utterance extracting unit 52 extracts, as a new index utterance, an utterance whose distance from the index utterance on the co-occurrence network satisfies a predetermined condition.

  For example, the utterance extracting unit 52 may extract, as a new favorable index utterance, an utterance whose distance from the favorable index utterance on the co-occurrence network is equal to or less than a predetermined threshold. Further, for example, the utterance extracting unit 52 may extract, as a non-favorable index utterance, an utterance whose distance from the non-favorable index utterance on the co-occurrence network is equal to or less than a predetermined threshold. In addition, for example, the utterance extraction unit 52 sets, as a new favorable index utterance, an utterance whose distance from the favorable index utterance is equal to or less than a predetermined threshold and whose distance from the unfavorable index utterance is equal to or greater than a predetermined threshold. May be extracted. In addition, for example, the utterance extraction unit 52 converts the utterance whose distance to the unfavorable index utterance is equal to or less than a predetermined threshold value and whose distance from the favorable utterance index is equal to or more than a predetermined threshold value to a new unfavorable index utterance. May be extracted.

  For example, the utterance extraction unit 52 specifies, for each node included in the co-occurrence network, the distance between the node corresponding to the favorable index utterance and the distance between the node corresponding to the unfavorable index utterance. Then, the utterance extracting unit 52 corresponds to a node in which the distance between the non-favorable index utterance and the corresponding node is equal to or less than a predetermined threshold and the distance between the favorable index utterance and the corresponding node exceeds the predetermined threshold. Let the utterance be a new unfavorable index utterance. In addition, the utterance extraction unit 52 corresponds to a node in which the distance between the favorable index utterance and the corresponding node is equal to or less than a predetermined threshold and the distance between the non-favorable index utterance and the corresponding node exceeds the predetermined threshold. Let the utterance be a new favorable index utterance.

  When executing the processing illustrated in FIG. 9, the utterance extraction unit 52 acquires a session including an utterance and a response of a user whose use mode of the interactive service satisfies a predetermined condition, and determines the appearance frequency in the acquired session. Based on this, an index utterance is extracted. For example, the utterance extraction unit 52 refers to the engagement database 34 and specifies the user's engagement history within a predetermined period. To give a more specific example, the utterance extraction unit 52 specifies the processing date and time and the history period and history frequency in the latest several months.

  Then, the utterance extraction unit 52 specifies a user whose history period is longer than the predetermined threshold and whose history frequency is higher than the predetermined threshold as a favorable user. Further, the utterance extraction unit 52 specifies a user whose history period is shorter than a predetermined threshold and whose history frequency is lower than the predetermined threshold as a non-favorable user. The utterance extraction unit 52 may specify a favorable user or a non-favorable user based on whether the prediction period and the prediction frequency registered as the prediction engagement exceed a predetermined threshold.

  Then, the utterance extracting unit 52 calculates, for each utterance, a value obtained by dividing the frequency of appearance of the session of the unfavorable user by the frequency of appearance of the session of the favorable user. Then, the utterance extraction unit 52 extracts the utterances whose calculated values are higher N as new unfavorable index utterances, and extracts the utterances whose calculated values are lower N as new favorable index utterances.

  That is, the utterance extraction unit 52 extracts, as a new favorable index utterance, a user whose usage mode satisfies a predetermined condition, that is, an utterance whose appearance frequency in a session including an utterance of a favorable user exceeds a predetermined threshold. I do. Further, the utterance extraction unit 52 determines that a user whose usage mode does not satisfy the predetermined condition, that is, an utterance whose appearance frequency in a session including the utterance of the unfavorable user exceeds a predetermined threshold value is a new unfavorable index. Extract as utterance.

  In addition, when executing the process illustrated in FIG. 10, the utterance extraction unit 52 extracts, as an index utterance, an utterance whose appearance frequency satisfies a predetermined condition from a session whose evaluation satisfies a predetermined condition. For example, the utterance extraction unit 52 refers to the session evaluation database 32 and specifies a high satisfaction session and a low satisfaction session. Further, the utterance extraction unit 52 refers to the session log database 31 and extracts a high satisfaction session and a low satisfaction session.

  Then, the utterance extracting unit 52 calculates, for each utterance, the ratio between the frequency of appearance in the low-satisfaction session and the frequency of appearance in the high-satisfaction session, and extracts the utterance whose ratio satisfies a predetermined condition as a new index utterance. For example, the utterance extracting unit 52 calculates a value obtained by dividing the frequency of appearance in the low-satisfaction session by the frequency of appearance in the high-satisfaction session, and calculates the utterances having the highest N values as new unfavorable index utterances. The lower N utterances are extracted as new favorable index utterances.

  Note that the utterance extraction unit 52 registers the index utterance extracted by the various processes described above in the index utterance database 33. As described above, the index utterance registered in the index utterance database 33 is used as an index utterance for evaluating a seed index utterance or a session in a process after registration.

  The network generation unit 53 generates a co-occurrence network connecting utterances or responses having co-occurrence from the history of the utterance and the response. For example, the network generation unit 53 refers to the session log database 31 and specifies each utterance included in the session. Then, the network generation unit 53 sets a node corresponding to each utterance, and generates a co-occurrence network connecting the nodes corresponding to each utterance based on the co-occurrence of each utterance. Then, the network generation unit 53 provides the co-occurrence network to the index utterance identification unit 51.

[2-5. Evaluation process)
Returning to FIG. 2, the description will be continued. The session evaluation processing unit 60 executes an evaluation process for evaluating a session. Hereinafter, the concept of the evaluation performed by the session evaluation processing unit 60 and an example of the functional configuration of the session evaluation processing unit 60 will be described with reference to FIGS.

[2-5-1. Evaluation process based on index utterance]
For example, the session evaluation processing unit 60 evaluates the session based on the index utterance included in the session. For example, FIG. 12 is a diagram illustrating a concept of a process of evaluating a session based on an index utterance. For example, when a session includes many favorable index utterances, it is estimated that the user has a good impression of each response included in the session, and thus the user's satisfaction with the session is estimated to be high. In addition, when there are many unfavorable index utterances in a certain session, it is estimated that the user's impression of each response included in the session is bad, and thus the user's satisfaction with the session is estimated to be low.

  Therefore, the session evaluation processing unit 60 evaluates the session based on the appearance frequency of the unfavorable index utterance included in the session and the appearance frequency of the favorable index utterance. For example, the session evaluation processing unit 60 refers to the session evaluation database 32 and selects a session that has not been evaluated as an evaluation target. Note that the session evaluation processing unit 60 may select the evaluated session as an evaluation target. For example, the session evaluation processing unit 60 may reselect a session for which a predetermined period has elapsed since the evaluation was performed as an evaluation target.

  Subsequently, the session evaluation processing unit 60 extracts the session to be evaluated from the session log database 31 and refers to the index utterance database 33 to specify the index utterance included in the extracted session. Then, the session evaluation processing unit 60 calculates the ratio of the total appearance frequency of unfavorable index utterances to the total appearance frequency of favorable index utterances. For example, the session evaluation processing unit 60 specifies all the utterances registered as unfavorable index utterances among the utterances included in the session to be processed, and determines the total number of specified utterances as the frequency of occurrence of the unfavorable index utterance. Specified as the sum of Further, the session evaluation processing unit 60 specifies all utterances registered as favorable index utterances among the utterances included in the session to be processed, and calculates the total number of the specified utterances as the sum of the appearance frequencies of the favorable index utterances. To be specified.

  Then, the session evaluation processing unit 60 calculates, as a ratio, a value obtained by dividing the total appearance frequency of unfavorable index utterances by the total appearance frequency of favorable index utterances. In addition, the session evaluation processing unit 60 similarly calculates the ratio for the other sessions, extracts the top N sessions whose calculated value is the low-satisfied session, and assigns the calculated value to the bottom N sessions. Extract as high satisfaction sessions.

  Here, it is considered that there is a difference in the degree of user satisfaction indicated by each index utterance. For example, it can be considered that a user who utters “awesome” is more satisfied with the interactive service than a user who utters “like”. Thus, the session evaluation processing unit 60 may evaluate the session in consideration of the degree of user satisfaction indicated by the index utterance.

  For example, the index utterance database 33 is provided with an index score indicating how much the user who has uttered the corresponding index utterance is satisfied or not satisfied with the interactive service (for example, see FIG. 5). ). Such an index score may be provided manually by, for example, crowdsourcing.For example, the index score is automatically calculated from the relationship between the dissatisfaction score given to the session and the appearance frequency of each index utterance. May be used. For example, the information providing apparatus 10 may calculate an index score having a higher value as the unfavorable index utterance is more included in a session with a higher dissatisfaction score, and a favorable score is more often included in a session with a lower dissatisfaction score. An index score of a higher value may be calculated for a target index utterance.

  When such an index score is given, the session evaluation processing unit 60 executes the following processing. First, the session evaluation processing unit 60 extracts a session to be evaluated from the session log database 31 and refers to the index utterance database 33 to specify an index utterance included in the extracted session.

  Then, the session evaluation processing unit 60 calculates the ratio of the sum of the index scores of the unfavorable index utterances included in the session to be processed to the sum of the index scores of the favorable index utterances. For example, the session evaluation processing unit 60 calculates, as a ratio, a value obtained by dividing the sum of the index scores of the unfavorable index utterances included in the session to be processed by the sum of the index scores of the favorable index utterances. Note that the session evaluation processing unit 60 may calculate the ratio from the sum of the absolute values of the index scores of the respective index utterances. The session evaluation processing unit 60 also calculates the ratio for other sessions in the same manner, and sets the calculated value to the top N sessions as the low-satisfaction session and sets the calculated value to the lower N sessions as the high-satisfaction session. Session. Thereafter, the session evaluation processing unit 60 registers the evaluation result in the session evaluation database 32.

[2-5-2. Evaluation process based on index utterance]
Further, for example, the session evaluation processing unit 60 evaluates the session based on the index utterance included in the session, using a model for evaluating the session. For example, FIG. 13 is a diagram illustrating a concept of a process of evaluating a session based on an index utterance.

  For example, the session evaluation processing unit 60 refers to the session log database 31 and extracts a high satisfaction session and a session other than the high satisfaction session as learning data. For example, the session evaluation processing unit 60 extracts, from the session log database 31, a session that has been evaluated as a highly satisfactory session, based on the evaluation result obtained by the processing illustrated in FIG. In addition, the session evaluation processing unit 60 extracts from the session log database 31 sessions other than the session that has been evaluated as a highly satisfactory session.

  Then, the session evaluation processing unit 60 learns the favorable index utterance evaluation model M4 using the extracted learning data. For example, when the utterance and response character strings included in the high-satisfaction session are input, the session evaluation processing unit 60 outputs a first degree of satisfaction higher than a predetermined threshold, and outputs the first satisfaction degree to a session other than the high-satisfaction session. The learning of the favorable index utterance evaluation model M4 is performed so as to output a first degree of satisfaction lower than a predetermined threshold when a character string of the included utterance and response is input. Note that, when the utterance and response character strings included in the high-satisfaction session are input, the session evaluation processing unit 60 outputs a value indicating that the session is a high-satisfaction session. The learning of the favorable index utterance evaluation model M4 may be performed such that when a response character string is input, a value indicating that the session is not a high satisfaction session is output.

  It is presumed that the high satisfaction session includes many favorable index utterances. Therefore, when the above-described learning is performed, the favorable index utterance evaluation model M4 outputs a higher value of the first satisfaction degree as the target session to be evaluated includes more favorable index utterances, and The lower the value of the first degree of satisfaction is, the less the favorable index utterance is included in the target session.

  Further, the session evaluation processing unit 60 refers to the session log database 31 and extracts a low satisfaction session and a session other than the low satisfaction session as learning data. Then, the session evaluation processing unit 60 learns the unfavorable index utterance evaluation model M3 using the extracted learning data. For example, when the utterance and response character strings included in the low-satisfaction session are input, the session evaluation processing unit 60 outputs a third degree of dissatisfaction that is higher than a predetermined threshold, and outputs The learning of the unfavorable index utterance evaluation model M3 is performed so as to output a third dissatisfaction degree lower than a predetermined threshold when a character string of the included utterance and response is input. Note that, when the utterance and response character strings included in the low-satisfaction session are input, the session evaluation processing unit 60 outputs a value indicating that the session is a low-satisfaction session. When a response character string is input, learning of the unfavorable index utterance evaluation model M3 may be performed such that a value indicating that the session is not a low satisfaction session is output.

  It is estimated that the low satisfaction session includes many unfavorable index utterances. Therefore, when the learning described above is performed, the unfavorable index utterance evaluation model M3 outputs a higher value of the third dissatisfaction degree as the target session to be evaluated includes more unfavorable index utterances. The lower the value of the third degree of dissatisfaction is, the less the unfavorable index utterance is included in the target session to be evaluated.

  Then, the session evaluation processing unit 60 evaluates the session using the unfavorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4. For example, the session evaluation processing unit 60 inputs the utterance or response character string of the target session to be evaluated to the unfavorable index utterance evaluation model M3 or the favorable index utterance evaluation model M4, and outputs the third dissatisfaction degree and the first Get satisfaction. Then, the session evaluation processing unit 60 evaluates the session based on the third degree of dissatisfaction or the first degree of satisfaction.

  For example, when the third degree of dissatisfaction exceeds a predetermined threshold, the session evaluation processing unit 60 may evaluate the target session as a low-satisfaction session. When the first degree of satisfaction exceeds a predetermined threshold, the session evaluation processing unit 60 may evaluate the target session as a high satisfaction session. Further, when the third degree of dissatisfaction exceeds a predetermined threshold and the first degree of satisfaction becomes less than the predetermined threshold, the session evaluation processing unit 60 may evaluate the target session as a low-satisfaction session. When the third degree of dissatisfaction is less than a predetermined threshold and the first degree of satisfaction is equal to or more than a predetermined threshold, the session evaluation processing unit 60 may evaluate the target session as a high satisfaction session. Further, the session evaluation processing unit 60 may evaluate the target session by the processing shown in FIG.

[About evaluation processing based on 2-5-3.3 value learning]
Here, not only in the session but also in various dialogues, the dialogue performed prior to the favorable index utterance has a high degree of user satisfaction, and the dialogue performed prior to the unfavorable index utterance is the user's It is estimated that satisfaction is low. Also, it is considered that the user's impression indicated by the index utterance is not a dialog after the index utterance but a dialog immediately before the index utterance.

  Thus, the session evaluation processing unit 60 may extract a series of dialogues corresponding to the index utterance, and may use the extracted dialogues to learn a model for estimating a user's evaluation of the input dialogue. For example, the session evaluation processing unit 60 may extract the utterance and response exchanged prior to the index utterance, cause the model to learn features of the extracted utterance, and evaluate the session using the trained model. Good. Also, the session evaluation processing unit 60 may cause the model to learn the characteristics of the neutral dialogue in which the impression by the user is neither favorable nor unfavorable (that is, neither satisfied nor unsatisfied). Good.

  For example, FIG. 14 is a diagram illustrating an example of a process of evaluating a session using a ternary evaluation model in which three types of dialogue features are learned. For example, the session evaluation processing unit 60 refers to the session log database 31 and specifies a session including the unfavorable index utterance. Subsequently, the session evaluation processing unit 60 extracts a non-favorable index utterance and a predetermined number of utterances and responses immediately before the non-favorable index utterance from the session as a non-favorable dialogue example UD. For example, the session evaluation processing unit 60 extracts two utterances made immediately before the unfavorable index utterance “I do not know the meaning” and the responses to these utterances, and extracts the unfavorable index “I do not understand the meaning”. It is extracted as an unfavorable dialogue example UD together with the utterance.

  In addition, the session evaluation processing unit 60 refers to the session log database 31 and specifies a session including a favorable index utterance. Subsequently, the session evaluation processing unit 60 extracts a favorable index utterance and a predetermined number of utterances and responses performed immediately before the favorable index utterance from the session as a favorable dialogue example FD. For example, the session evaluation processing unit 60 extracts two utterances made immediately before the favorable index utterance “smart” and responses to these utterances, and extracts the favorable utterance together with the favorable index utterance “smart”. Extracted as a dialog example FD.

  In addition, the session evaluation processing unit 60 selects one utterance that is not an index utterance as a neutral utterance, and performs a predetermined number of utterances and responses performed immediately before the neutral utterance from the session including the selected neutral utterance. Are extracted from the session as a neutral dialogue example ND. For example, the session evaluation processing unit 60 extracts two utterances made immediately before the neutral utterance of “tomorrow's weather” and responses to these utterances, and extracts the neutral utterance together with the neutral utterance of “tomorrow's weather”. Extracted as a dialog example ND.

  Then, the session evaluation processing unit 60 causes the ternary evaluation model M5 to learn features of the unfavorable dialogue example UD, the neutral dialogue example ND, and the favorable dialogue example FD. For example, when the utterance and response character strings included in the unfavorable dialogue example UD are input, the session evaluation processing unit 60 outputs a value indicating that the input utterance and response are unfavorable dialogue examples. Then, when a character string of an utterance and a response included in the neutral dialogue example ND is input, a value indicating that the input utterance and response is a neutral dialogue example is output and included in the favorable dialogue example FD. When a character string of the utterance and response to be input is input, learning of the ternary evaluation model M5 is performed so that a value indicating that the input utterance and response is a favorable dialogue example is output. That is, the session evaluation processing unit 60 determines whether the user is satisfied with the input dialogue (whether it is a favorable dialogue), is not satisfied (whether it is a non-favorable dialogue), or otherwise. (Neutral dialogue) is learned.

  Then, the session evaluation processing unit 60 evaluates the target session using the generated ternary evaluation model M5. For example, the session evaluation processing unit 60 inputs the utterance and response character strings included in the target session to the ternary evaluation model M5, and based on the output of the ternary evaluation model M5, the user is satisfied with the target session. Whether the user is not satisfied (ie, a low-satisfaction session) or not (ie, a neutral session).

  When extracting each dialogue example, the session evaluation processing unit 60 may extract an arbitrary number of utterances and responses performed immediately before the index utterance and the neutral utterance. In this way, by making the model learn the features of the utterance and response performed immediately before the index utterance and the neutral utterance, it is possible to improve the evaluation of the dialogue by the model, and the accuracy of the evaluation of the session.

[2-5-4. Evaluation process based on engagement)
FIG. 15 is a diagram illustrating an example of a process of evaluating a session based on engagement. For example, it is considered that a user who has a high degree of satisfaction with the dialogue has a high use period and frequency of use, and a user who has a low degree of satisfaction with the dialogue has a low use period and frequency of use. For this reason, it is estimated that a session including an utterance of a user with high engagement such as a use period and a use frequency has a high degree of satisfaction, and a session including an utterance of a user with low engagement has a low degree of satisfaction. Therefore, the session evaluation processing unit 60 evaluates the session based on the engagement of the user who has made the utterance included in the session.

  For example, the session evaluation processing unit 60 refers to the engagement database 34 and specifies a user whose history period and history frequency within a predetermined period exceed a predetermined threshold. Subsequently, the session evaluation processing unit 60 extracts a session including the utterance of the specified user from the session log database 31, and sets the extracted session as a session of a user with high engagement.

  On the other hand, the session evaluation processing unit 60 refers to the engagement database 34 and specifies a user whose history period and history frequency within a predetermined period are less than a predetermined threshold. Subsequently, the session evaluation processing unit 60 extracts a session including the utterance of the specified user from the session log database 31, and sets the extracted session as a session of a user with low engagement.

  Then, the session evaluation processing unit 60 causes the engagement session evaluation model M6 to learn the features of the session of the user with high engagement and the session of the user with low engagement. For example, when an utterance and a response included in a session of a user with a high engagement are input, the session evaluation processing unit 60 outputs a value indicating that the user's satisfaction with the session is high, and a session of a user with a low engagement is When the utterance and response included in are input, the engagement session evaluation model M6 is learned so as to output a value indicating that the user's satisfaction with the session is low.

  For example, when an utterance and a response included in a session of a user with high engagement are input, the session evaluation processing unit 60 outputs an evaluation score of a value higher than a predetermined threshold, and is included in a session of a user with low engagement. When an utterance and a response are input, the engagement session evaluation model M6 is learned so as to output an evaluation score lower than a predetermined threshold. That is, the session evaluation processing unit 60 uses the learning data extracted based on the engagement to classify the input session into a session with high user satisfaction or a session with low user satisfaction. The evaluation model M6 is learned.

  Then, the session evaluation processing unit 60 evaluates the target session using the engagement session evaluation model M6. For example, when a character string of an utterance and a response included in the target session is input, the session evaluation processing unit 60 determines whether the user is satisfied with the target session based on the evaluation score output by the engagement session evaluation model M6. Evaluate whether or not

[2-5-5. Evaluation process based on repeated speech)
FIG. 16 is a diagram illustrating a concept of a process of evaluating a session based on repeated utterances. For example, if the same or similar utterance is repeated in a session, the user's satisfaction with the session is low, and if the same or similar utterance is not repeated in the session, the user's satisfaction with the session is low. The degree is estimated to be high. Therefore, the session evaluation processing unit 60 may evaluate the session based on the repeated utterance included in the session.

  For example, the session evaluation processing unit 60 counts, from the utterances included in the target session, the number of pairs with similar notations as the number of notation-similar pairs. For example, the session evaluation processing unit 60 determines whether the similarity based on various editing distances such as the Jaccard coefficient, the Dice coefficient, the Simpson coefficient, and the like exceeds a predetermined threshold, and the similarity based on various editing distances such as the Levenshtein distance exceeds the predetermined threshold. Count the number of pairs. In addition, the session evaluation processing unit 60 may calculate the number of pairs whose similarity in meaning exceeds a predetermined threshold as the number of meaning-similar pairs using word2vec, GloVe, or the like.

  Then, when the value of the number of notation similar pairs or the number of similar meaning pairs extracted from the target session exceeds a predetermined threshold, the session evaluation processing unit 60 evaluates the target session as a low-satisfaction session, and is extracted from the target session. When the value of the number of notation similar pairs or the number of meaning similar pairs is less than a predetermined threshold, the target session is evaluated as a high satisfaction session.

  In addition, the session evaluation processing unit 60 may learn a repetitive utterance evaluation model M7 for evaluating a target session by causing a model to learn features of a session including repetitive utterances. For example, when a session including a repeated utterance (for example, a session in which the number of pairs exceeds a predetermined threshold) is input, the session evaluation processing unit 60 outputs a message indicating that the input session is a low-satisfaction session. The repetitive utterance evaluation model M7 may be learned. Further, the session evaluation processing unit 60 may evaluate the target session using such a repeated utterance evaluation model M7.

[2-5-6. Automatic generation of repeated speech)
Here, the session evaluation processing unit 60 may automatically generate a session including a repeated utterance, and cause the repeated utterance evaluation model M7 to learn features of the generated session. For example, FIG. 17 is a diagram illustrating the concept of a process of evaluating a session using a model that has learned the characteristics of automatically generated data. For example, the session evaluation processing unit 60 refers to the session log database 31 and extracts any utterance as a seed utterance. For example, the session evaluation processing unit 60 extracts an utterance such as "Today's weather" as a seed utterance.

  Subsequently, the session evaluation processing unit 60 generates a plurality of utterances similar to the seed utterance from the extracted seed utterance. For example, the session evaluation processing unit 60 randomly deletes words or characters included in the character string of the seed utterance such as “Today's weather” to mimic a speech recognition mistake or a typo, or sounds similar. Generate multiple utterances replaced with words or character strings. To give a more specific example, the session evaluation processing unit 60 determines from the character string of the seed utterance "Today's weather is" that "Today's electricity is", "Evening electricity is", and "Today is electricity". Is generated as a duplicate utterance.

  Further, the session evaluation processing unit 60 extracts one or a plurality of utterances different from the seed utterance from the session log database 31 as mixed utterances. For example, the session evaluation processing unit 60 extracts an utterance such as “fortune of Aries” as a mixed utterance. Then, the session evaluation processing unit 60 randomly rearranges the seed utterance, the plurality of duplicate utterances generated from the seed utterance, and the mixed utterance, and generates a session including a pseudo utterance as the repetitive session RS. For example, in the example illustrated in FIG. 17, the session evaluation processing unit 60 includes the character string “Today's weather”, which is a seed utterance, as the user's utterance “Usr: Today's weather”, and is a duplicate utterance. The user's utterance "Usr: Aries" includes a character string "Usr: Today's electricity" as a user's utterance "Usr: Today's electricity". The character string "Today's weather", which is a seed utterance, is included again as the user's utterance "Usr: Today's weather", and the character string "Trade weather is", which is a duplicate utterance, and " A repetitive session RS is generated that includes the user's utterances “Usr: Electricity Today” as “Usr: Business Weather” and “Usr: Electricity Today”.

  That is, the session evaluation processing unit 60 generates a session including a pseudo repeated speech. Note that the session evaluation processing unit 60 may extract a plurality of seed utterances and generate a duplicate utterance from each seed utterance.

  Further, the session evaluation processing unit 60 refers to the session log database 31 and randomly extracts a plurality of utterances. Then, the session evaluation processing unit 60 generates a session in which the extracted utterances are randomly rearranged as a non-repetitive session NRS. Then, the session evaluation processing unit 60 learns the repetitive utterance evaluation model M7 using the repetitive session RS and the non-repetitive session NRS as learning data. For example, the session evaluation processing unit 60 outputs a score indicating that the user is not satisfied when the repetitive session RS is input, and outputs a score indicating that the user is not satisfied when the non-repetitive session NRS is input. The learning of the repetitive utterance evaluation model M7 is performed so as to output a score indicating that the repetition utterance is evaluated.

  Then, the session evaluation processing unit 60 evaluates the target session using the repetitive utterance evaluation model M7. For example, when the target session is input, the session evaluation processing unit 60 evaluates whether or not the user is satisfied with the target session based on the value of the score output by the repetitive utterance evaluation model M7.

[2-5-7. Evaluation of repeated speech using images)
Here, the session evaluation processing unit 60 may evaluate a repeated utterance using an image. For example, FIG. 18 is a diagram illustrating the concept of evaluating repeated utterances using images. For example, a square image in which the same number of regions are set in the vertical and horizontal directions is set, and each character of the utterance character string included in a certain session is associated with the vertical and horizontal directions of the image, and the vertical and horizontal directions are set. When an image is generated in which an area in which the same character is associated in the direction is generated, the image is a distribution image indicating the distribution of the same character included in the character string of the utterance. When a character string of an utterance includes repetition, such a distribution image includes a region that is filled so as to form a diagonal line. That is, when an utterance similar to a certain session is included, a hatched line appears in the distribution image.

  In recent years, with the progress of image analysis technology, the accuracy of determining similar images using a model using DNN or the like has been improved. For this reason, it is considered that learning the features of the distribution image can more accurately learn the features of the session including the repeated utterances than simply learning the features of the character strings of the utterances included in the session.

  Therefore, the session evaluation processing unit 60 may cause the repetition evaluation model M1 to learn the features of the distribution image, and may evaluate the target session using the repetition evaluation model M1. For example, as illustrated in FIG. 16, the session evaluation processing unit 60 extracts, from the session log database 31, a session in which the number of notation-similar pairs and meaning-similar pairs exceeds a predetermined threshold, as a session including repeated utterances. Note that the session evaluation processing unit 60 may acquire the repetitive session RS generated by the method illustrated in FIG. 17 as a session including repeated utterances. Then, the session evaluation processing unit 60 generates a distribution image from each utterance of the session including the repeated utterance.

  On the other hand, as shown in FIG. 16, the session evaluation processing unit 60 extracts from the session log database 31 a session in which the number of notation-similar pairs and meaning-similar pairs falls below a predetermined threshold as a session that does not include utterances repeatedly. Note that the session evaluation processing unit 60 may acquire the non-repetitive session NRS generated by the method illustrated in FIG. 17 as a session that does not include repeated utterances. Then, the session evaluation processing unit 60 generates a distribution image from each utterance of the session that does not include the repeated utterance.

  Then, the session evaluation processing unit 60 causes the evaluation model M1 to repeatedly learn features of the generated distribution image. For example, when a distribution image generated from each utterance of a session including a repeated utterance is input, the session evaluation processing unit 60 outputs a score indicating that the user is not satisfied, and outputs a score indicating that the user does not include the repeated utterance. When the distribution image generated from each of the utterances is input, learning of the evaluation model M1 is repeatedly performed so as to output a score indicating that the user is satisfied.

  Subsequently, the session evaluation processing unit 60 generates a distribution image from each utterance included in the target session, and repeatedly inputs the generated distribution image to the evaluation model M1. Then, the session evaluation processing unit 60 evaluates whether the user is satisfied with the target session based on the output score of the repetition evaluation model M1.

[2-5-8. About evaluation based on apology response)
FIG. 19 is a diagram showing the concept of processing for evaluating a session based on an apology response. As shown in FIG. 19, it is estimated that the user's satisfaction with the session including the apology response is low, and the user's satisfaction with the session including no apology response is high. Therefore, the session evaluation processing unit 60 may evaluate the session based on the apology response.

  For example, the session evaluation processing unit 60 refers to the session log database 31 and extracts a session including a predetermined apology response registered in advance. Further, the session evaluation processing unit 60 extracts a session that does not include a predetermined apology response. Then, the session evaluation processing unit 60 causes the apology evaluation model M2 to learn features of the extracted session. For example, when a session including an apology response is input, the session evaluation processing unit 60 outputs a score indicating that the user is not satisfied. The apology evaluation model M2 is learned so as to output a score indicating that the person is satisfied.

  Subsequently, the session evaluation processing unit 60 inputs the target session into the apology evaluation model M2. Then, the session evaluation processing unit 60 evaluates whether the user is satisfied with the target session based on the output score of the apology evaluation model M2.

[2-5-9. Combination of evaluation criteria)
The session evaluation processing unit 60 may evaluate the target session by combining the above-described evaluation methods. For example, FIG. 20 is a diagram illustrating a concept of a process of evaluating a session by combining a plurality of evaluation methods. As shown in FIG. 20, the session evaluation processing unit 60 includes a repetition evaluation model M1, an apology evaluation model M2, an unfavorable index utterance evaluation model M3, a favorable index utterance evaluation model M4, a ternary evaluation model M5, an engagement session evaluation. A model M6 and a repeated speech evaluation model M7 are generated. Then, the session evaluation processing unit 60 inputs the target session and the distribution image generated from the target session to each of the models M1 to M7, and evaluates the target session.

  For example, when the distribution image of the target session is similar to the distribution image of the repetition session, that is, when the target session has repeated utterances, the repetition evaluation model M1 outputs an evaluation indicating that the user satisfaction is low. In addition, when the apology utterance is included in the target session, the apology evaluation model M2 outputs an evaluation indicating that the degree of user satisfaction is low. Further, the unfavorable index utterance evaluation model M3 outputs an evaluation indicating that the degree of user satisfaction is low when the target session includes the unfavorable index utterance. Further, the favorable index utterance evaluation model M4 outputs an evaluation indicating that the degree of user satisfaction is high when the target session includes a favorable index utterance.

  Further, the ternary evaluation model M5 outputs an evaluation indicating that the degree of user satisfaction is high when the target session includes a dialogue having similar characteristics to the dialogue satisfied by the user. Further, the engagement session evaluation model M6 indicates that the feature of the target session is similar to the session including the utterance of the user with high engagement, that is, the feature of the target session is similar to the session including the utterance of the user with high satisfaction. Then, the evaluation that the user's satisfaction is high is output. In addition, the repetitive utterance evaluation model M7 outputs an evaluation indicating that the satisfaction of the user is low when the target session includes repeated utterances.

  The session evaluation processing unit 60 evaluates the target session based on a combination of the evaluation results of the models M1 to M7. For example, the session evaluation processing unit 60 counts the number of models determined to have a high degree of user satisfaction with the target session and the number of models determined to have a low degree of user satisfaction with the target session. If the number of determined models is larger than the number of models determined to have a low degree of satisfaction, it may be determined that the degree of user satisfaction with the target session is high. That is, the session evaluation processing unit 60 may evaluate the target session by majority decision.

  Further, the session evaluation processing unit 60 may evaluate the target session by using the evaluation results of some of the models M1 to M7. Further, for example, the session evaluation processing unit 60 may integrate the scores output from the models M1 to M7. For example, the session evaluation processing unit 60 calculates a score obtained by subtracting the scores of the models M4 to M6 from the sum of the scores of the models M1, M2, M3, and M7, and evaluates the target session based on the calculated score value. May be.

  Further, the session evaluation processing unit 60 may further learn evaluation results using the models M1 to M7. For example, the session evaluation processing unit 60 causes a model to learn, as learning data, a session evaluated as having high user satisfaction and a session evaluated as having low user satisfaction, and uses the model. Then, the target session may be evaluated. In addition, the index utterance extraction processing unit 50 may extract the index utterance based on the evaluation result by the session evaluation processing unit 60. For example, the index utterance extraction processing unit 50 extracts a favorable index utterance from a session evaluated as having a high degree of satisfaction by the session evaluation processing unit 60, that is, a session with high satisfaction, and the session evaluation processing unit 60 has a low degree of satisfaction. An unfavorable index utterance may be extracted from a session evaluated as, that is, a low satisfaction session.

[2-5-10. Automatic expansion of labeled data)
Here, the session evaluation processing unit 60 may automatically extend session data used for learning each of the models M1 to M7. For example, FIG. 21 is a diagram illustrating the concept of a process of automatically expanding a labeled session.

  For example, the session evaluation processing unit 60 extracts each utterance included in a session with a low user satisfaction, that is, each utterance included in the low-satisfaction session, and extracts another utterance similar to each extracted utterance from the session log database 31. Search for. In addition, the session evaluation processing unit 60 extracts each response included in the low satisfaction session, and searches the session log database 31 for another response similar to the extracted response.

  Then, the session evaluation processing unit 60 generates a new session in a pseudo manner by combining the searched utterance and the response, and sets the generated new session as a low-satisfaction session to perform learning at the time of learning the models M1 to M7. It may be adopted as data.

[2-5-11. Functional configuration of session evaluation processing unit]
Next, an example of a functional configuration of the session evaluation processing unit 60 will be described with reference to FIG. FIG. 22 is a diagram illustrating an example of a functional configuration of the session evaluation processing unit according to the embodiment. As shown in FIG. 22, the session evaluation processing unit 60 includes a learning data extraction unit 61, a learning unit 62, an image generation unit 63, an utterance group generation unit 64, and an evaluation unit 65.

  The learning data extraction unit 61 extracts a session serving as learning data. For example, as illustrated in FIG. 12, the learning data extraction unit 61 determines a session in which the appearance frequency of the favorable index utterance and the sum of the index scores of the favorable index utterance satisfy a predetermined condition, by using the favorable index utterance evaluation model M4. Is extracted as a high-satisfaction session that becomes learning data for. Further, the learning data extracting unit 61 extracts sessions other than the high-satisfaction session as sessions other than the high-satisfaction session serving as learning data of the favorable index utterance evaluation model M4.

  Further, for example, as shown in FIG. 12, the learning data extracting unit 61 determines a session in which the appearance frequency of unfavorable index utterances and the sum of the index scores of the unfavorable index utterances satisfy a predetermined condition, as shown in FIG. The session is extracted as a high satisfaction session serving as learning data for the index utterance evaluation model M3. Further, the learning data extracting unit 61 extracts sessions other than the low-satisfaction session as sessions other than the low-satisfaction session serving as learning data of the unfavorable index utterance evaluation model M3.

  The learning data extraction unit 61 refers to the session evaluation database 32, specifies a high satisfaction session and a low satisfaction session, and identifies the specified high satisfaction session and low satisfaction session as the unfavorable index utterance evaluation model M3 and the favorable index. The learning data of the utterance evaluation model M4 may be extracted from the session log database 31.

  Further, as shown in FIG. 15, the learning data extracting unit 61 may extract a session serving as learning data of the engagement session evaluation model M6 based on the user's engagement. For example, the learning data extraction unit 61 refers to the engagement database 34 and specifies a usage mode of each user, that is, a history period and a history frequency. Then, the learning data extraction unit 61 specifies a user whose specified use mode satisfies a predetermined condition, and extracts a session including the utterance of the specified user from the session log database 31.

  For example, the learning data extraction unit 61 specifies a user whose history period or history frequency exceeds a predetermined threshold, and sets a session including the utterance of the specified user as a session of a user with high engagement from the session log database 31. Extract. Further, the learning data extraction unit 61 specifies a user whose history period or history frequency is lower than a predetermined threshold, and sets a session including an utterance of the specified user as a session of a user with low engagement from the session log database 31. Extract. Then, the learning data extraction unit 61 may use each extracted session as learning data of the engagement session evaluation model M6.

  That is, the learning data extraction unit 61 extracts a session including a speech of a user whose use mode satisfies the predetermined condition as learning data having the first attribute of high user satisfaction. Further, the learning data extracting unit 61 extracts a session including an utterance of a user whose use mode does not satisfy a predetermined condition, as learning data having a second attribute of low user satisfaction.

  Further, as shown in FIG. 16, the learning data extraction unit 61 extracts, from the session log database 31, a session having a large number of notation similar pairs and a large number of meaning similar pairs as a low-satisfaction session. A session with a small number of semantically similar pairs is extracted as a highly satisfactory session. Then, the learning data extracting unit 61 may set the extracted session as a session that is a source of learning data of the repetition evaluation model M1 and the repetition utterance evaluation model M7.

  Further, as shown in FIG. 19, the learning data extracting unit 61 extracts, from the session log database 31, a session including an apology response and a session not including an apology response as learning data of the apology evaluation model M2. .

  The learning data extraction unit 61 is not limited to the above-described processing, and may extract sessions useful for learning the models M1 to M7 as learning data. For example, the learning data extracting unit 61 may extract a high satisfaction session or a low satisfaction session as learning data of each of the models M1 to M7 based on the evaluation result of the session registered in the session evaluation database 32.

  The learning unit 62 learns each of the models M1 to M7. For example, as illustrated in FIG. 13, the learning unit 62 learns the unfavorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4 as the learning data extracted by the learning data extraction unit 61.

  For example, as illustrated in FIG. 13, the learning unit 62 causes the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4 to learn features of a session whose evaluation satisfies a predetermined condition. For example, the learning unit 62 learns the favorable index utterance evaluation model M4 using the high satisfaction session extracted by the learning data extraction unit 61 and a session other than the high satisfaction session as learning data. That is, the learning unit 62 learns a favorable index utterance evaluation model M4 that classifies the input target session into a high satisfaction session and other sessions.

  The learning unit 62 learns the unfavorable index utterance evaluation model M3 using the low satisfaction session extracted by the learning data extraction unit 61 and sessions other than the low satisfaction session as learning data. That is, the learning unit 62 learns the unfavorable index utterance evaluation model M3 that classifies the input target session into low-satisfaction sessions and other sessions.

  Further, the learning unit 62 learns the ternary evaluation model M5 as shown in FIG. 14 using the dialogue example generated by the utterance group generation unit 64 as learning data. That is, the learning unit 62 learns the ternary evaluation model M5 using a series of dialogues corresponding to the index utterance as learning data. For example, the utterance group generation unit 64 extracts a predetermined number of utterances and responses performed immediately before the unfavorable index utterance as the unfavorable dialogue example UD, and extracts the predetermined number of utterances and responses performed immediately before the favorable index utterance. A number of utterances and responses are extracted as favorable dialogue examples FD, and a predetermined number of utterances and responses performed immediately before the neutral utterance are extracted as neutral dialogue examples ND. In such a case, the learning unit 62 learns the ternary evaluation model M5 that evaluates the target session using the unfavorable dialogue example UD, the favorable dialogue example FD, and the neutral dialogue example ND as learning data. That is, the learning unit 62 learns a model for estimating the user's evaluation on the dialogue in the target session.

  For example, the learning unit 62 sets the unfavorable dialogue example UD as learning data of a dialogue whose evaluation by the user is unfavorable. In addition, the learning unit 62 sets the favorable conversation example FD as learning data of the conversation in which the evaluation by the user is favorable. Further, the learning unit 62 sets the neutral dialogue example ND as learning data of a dialogue whose evaluation by the user is neutral. Then, the learning unit 62 causes the ternary evaluation model M5 to learn the features of the learning data, so that the evaluation by the user is input into an unfavorable conversation, a favorable conversation, or a neutral conversation. Implement a model for classifying dialogue.

  Further, as shown in FIG. 15, the learning unit 62 learns the engagement session evaluation model M6 using the session of the user with high engagement and the session of the user with low engagement extracted by the learning data extraction unit 61 as learning data. . That is, the learning unit 62 sets the input target session similar to a session similar to a session by a user with high engagement (hereinafter, referred to as a “first attribute session”) or a session by a user with low engagement. The engagement session evaluation model M6, which is classified as a session to be performed (hereinafter, referred to as a “second attribute session”), is learned.

  Here, a session by a user with high engagement is estimated to have a high degree of user satisfaction, and a session by a user with low engagement is estimated to have a low degree of user satisfaction. For this reason, the session of the first attribute is a session in which the evaluation by the user is favorable, that is, a high satisfaction session, and the session of the second attribute is a session in which the evaluation by the user is unfavorable, that is, low satisfaction. It can be said that it is a session. For this reason, the learning unit 62 can learn the engagement session evaluation model M6 that determines whether the user's impression of the target content is favorable or unfavorable.

  Further, as illustrated in FIGS. 17 and 18, the learning unit 62 learns the repetitive utterance evaluation model M7 and the repetition evaluation model M1 using the session including the repetitive utterance as learning data. For example, when the learning unit 62 acquires the repetitive session RS and the non-repetitive session NRS generated from the seed utterance and the mixed utterance by the utterance group generation unit 64 by the method illustrated in FIG. Learning of the repetitive utterance evaluation model M7 is performed using the session NRS as learning data. That is, the learning unit 62 learns a repetitive utterance evaluation model M7 that classifies the target session into a session including repetitive utterance or a session not including repetitive utterance.

  Here, it is presumed that a session including a repeated utterance has a negative impression of the user, and a session not including a repeated utterance has a favorable impression of the user. For this reason, the learning unit 62 can learn the repetitive utterance evaluation model M7 that determines whether the user's impression of the target content is favorable or unfavorable.

  Further, the learning unit 62 acquires, as learning data, a distribution image generated from the session including the repeated utterance by the image generation unit 63 and a distribution image generated from the session not including the repeated utterance by the image generation unit 63. In such a case, the learning unit 62 learns the repetition evaluation model M1 using these distribution images as learning data. More specifically, the learning unit 62 converts the input image into an image similar to a distribution image generated from a session including repetitive utterance, or an image similar to a distribution image generated from a session including no repetitive utterance. The learning of the repeated evaluation model M1 to be classified is performed.

  Here, a hatched line is likely to appear in a distribution image of a session including a repeated utterance, and a hatched line is unlikely to appear in a distribution image of a session not including a repeated utterance. Further, it is presumed that a session including a repeated utterance has a negative impression of the user, and a session not including a repeated utterance has a favorable impression of the user. For this reason, the learning unit 62 determines whether the user's impression of the target session is favorable or unfavorable by classifying the distribution image in which the utterance character strings of the target session are vertically and horizontally associated. The repetition evaluation model M1 can be learned.

  Further, as shown in FIG. 19, the learning unit 62 learns the apology evaluation model M2 using the session extracted by the learning data extraction unit 61 as learning data. For example, the learning unit 62 learns an apology evaluation model M2 that classifies the input target session into a session similar to a session including an apology response or a session similar to a session not including an apology response.

  Here, it is presumed that the session including the apology response has the unfavorable impression of the user, and the session not including the apology response has the favorable impression of the user. For this reason, the learning unit 62 can learn the apology evaluation model M2 that determines whether the user's impression of the target session is favorable or unfavorable based on the apology response.

  The learning unit 62 may generate a new low-satisfaction session from the low-satisfaction sessions acquired as the learning data and add the new low-satisfaction session to the learning data as shown in FIG. For example, the learning unit 62 extracts, from the session log database 31, utterances and responses similar to the utterances and responses included in the session acquired as the learning data, and combines the extracted utterances and responses to generate new learning. Data may be generated.

  Here, the learning unit 62 may assign the same label as the generation source learning data to the generation of new learning data. For example, the learning unit 62 extracts a user's impression from a conversation example in which the user's impression is favorable, such as a session with a lot of favorable index utterances, a session of a user with high engagement, a favorable dialogue example FD, and a non-repetitive session NRS. May generate a new favorable dialogue example and use it as learning data. In addition, for example, the learning unit 62 may determine a session in which the impression of the user is unfavorable, such as a session with many unfavorable indicator utterances, a session of a user with low engagement, a non-favorable dialogue example ND, and a repetitive session RS. A dialog example in which the user's impression is unfavorable may be newly generated and used as learning data.

  The image generation unit 63 generates a distribution image. For example, the image generation unit 63 acquires a session including a repeated utterance extracted by the learning data extraction unit 61 and a session not including a repeated utterance. In addition, the image generation unit 63 acquires the repetition session RS and the non-repetition session NRS generated by the utterance group generation unit 64 as illustrated in FIG. Note that the image generating unit 63 may generate a new session from the acquired session, as shown in FIG.

  Then, the image generation unit 63 generates a distribution image indicating whether or not the characters of the utterance included in the acquired session are the same as each other. That is, the image generation unit 63 generates an image indicating the distribution of the same character in the session. For example, when the image generation unit 63 associates the character string of the utterance included in the session with each of the vertical region and the horizontal region of the image, the same character corresponds to the vertical direction and the horizontal direction. An image in which a predetermined color is applied to the attached area is generated.

  The utterance group generation unit 64 extracts an unfavorable dialogue example UD, a favorable dialogue example FD, and a neutral dialogue example ND. That is, the utterance group generation unit 64 specifies the index utterance from the history in the dialog service, and extracts a series of dialogs corresponding to the specified index utterance as the dialog group. For example, the utterance group generation unit 64 extracts responses and utterances that are a predetermined number before the index utterance as learning data. Note that it is possible to arbitrarily set how many responses and utterances are extracted as examples of dialogues from responses and utterances before the index utterance.

  For example, the utterance group generation unit 64 refers to the session log database 31 and specifies a session including a favorable index utterance. Subsequently, the utterance group generation unit 64 extracts a predetermined number of responses and utterances before the favorable index utterance as a favorable dialogue example FD. Similarly, the utterance group generation unit 64 specifies a session that includes the unfavorable index utterance, and extracts a predetermined number of responses and utterances before the unfavorable index utterance as the unfavorable dialogue example UD. Further, the utterance group generation unit 64 specifies a session including a neutral utterance, and extracts a predetermined number of responses and utterances before the neutral utterance as a neutral dialogue example ND.

  Further, the utterance group generation unit 64 generates a repetition session RS (that is, a first utterance group) and a non-repetition session NRS (that is, a second utterance group). For example, the utterance group generation unit 64 selects any utterance from the history in the interactive service, and generates an utterance group including a plurality of utterances that are the same as or similar to the selected utterance.

  As a more specific example, the utterance group generation unit 64 extracts a seed utterance from the session log database 31 and generates a plurality of duplicate utterances similar to the seed utterance. In addition, the utterance group generation unit 64 extracts mixed utterances from the session log database 31. Then, the utterance group generation unit 64 randomly rearranges the seed utterance, the plurality of duplicate utterances, and the mixed utterance, and generates a session including a pseudo utterance as the repetitive session RS. In addition, the utterance group generation unit 64 refers to the session log database 31, extracts a plurality of utterances at random, and generates a session in which the extracted utterances are randomly rearranged as a non-repetitive session NRS.

  The utterance group generation unit 64 generates the utterance and the response similar to the utterance and the response included in the unfavorable dialogue example UD, the favorable dialogue example FD, or the neutral dialogue example ND (hereinafter, collectively referred to as “dialogue example”). May be extracted and a new dialog example may be generated by combining the extracted utterance and response. For example, the utterance group generation unit 64 may generate a new unfavorable dialogue example UD from the unfavorable dialogue example UD by the method shown in FIG. 21, and may generate a new favorable dialogue example from the favorable dialogue example FD. The FD may be generated, or a new neutral dialogue example ND may be generated from the neutral dialogue example ND. In addition, the utterance group generation unit 64 may generate a new repetition session RS from the repetition session RS, or may generate a new non-repetition session NRS from the non-repetition session NRS.

  The evaluation unit 65 evaluates the session. For example, the evaluation unit 65 selects a target session to be evaluated from the session log database 31. Then, the evaluation unit 65 refers to the index utterance database 33 and evaluates the target session based on the appearance frequency of the index utterance extracted by the index utterance extraction processing unit 50 or the index utterance serving as a seed.

  For example, the evaluation unit 65 evaluates the target session based on whether or not an unfavorable index utterance or a favorable index utterance is included. For example, as shown in FIG. 12, the evaluation unit 65 determines whether the target session is a low-satisfaction session based on the ratio between the frequency of appearance of the unfavorable index utterance and the frequency of appearance of the favorable index utterance for the target session. Evaluate if it is a high satisfaction session. In addition, the evaluation unit 65 determines whether the target session is a low-satisfaction session or a high-satisfaction session based on the ratio of the total of the index scores of the unfavorable index utterances to the total of the index scores of the favorable index utterances. May be evaluated.

  Here, the index utterance used for the evaluation of the session includes the index utterance extracted by the index utterance extraction processing unit 50, that is, the index utterance extracted by co-occurrence with the seed utterance. For this reason, the evaluation unit 65 determines, for example, the appearance frequency of the utterance whose co-occurrence with the unfavorable index utterance satisfies a predetermined condition, and the appearance frequency of the utterance whose co-occurrence with the favorable index utterance satisfies a predetermined condition. The target session will be evaluated based on the frequency.

  The index utterance used for evaluating the session includes the index utterance extracted by the index utterance extraction processing unit 50 based on the user's engagement. That is, in the index utterance, the index utterance extracted based on the appearance frequency in the session of the user whose use mode satisfies the predetermined condition or the appearance frequency in the session of the user whose use mode does not satisfy the predetermined condition is used. Is included. Therefore, the evaluation unit 65 evaluates the target session based on the utterance of the user whose use mode satisfies the predetermined condition and the utterance of the user whose use mode does not satisfy the predetermined condition.

  The index utterance used for evaluating the session includes an index utterance extracted based on the ratio of the frequency of appearance in a low-satisfaction session to the frequency of appearance in a high-satisfaction session. Therefore, the evaluation unit 65 evaluates the target session based on the utterance whose appearance frequency in the low satisfaction session satisfies the predetermined condition and the utterance whose appearance frequency in the high satisfaction session satisfies the predetermined condition.

  In addition, as shown in FIG. 13, the evaluation unit 65 may evaluate the target session by using a favorable index utterance evaluation model M4 that has learned the features of the high satisfaction session. In addition, the evaluation unit 65 may evaluate the target session using the unfavorable index utterance evaluation model M3 that has learned the features of the low satisfaction session. For example, the evaluation unit 65 reads the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4 from the model database 35, and stores the target session in the non-favorable index utterance evaluation model M3 and the favorable index utterance evaluation model M4. input. Then, the evaluation unit 65 may evaluate the target session based on the score output by the unfavorable index utterance evaluation model M3 or the favorable index utterance evaluation model M4. For example, when the evaluation unit 65 is classified by the unfavorable index utterance evaluation model M3 as being similar to a session including the unfavorable index utterance, the evaluation unit 65 evaluates the target session as a low-satisfaction session, and evaluates the favorable index utterance. If the model M4 classifies the target session as similar to the session including the favorable index utterance, the target session may be evaluated as a high satisfaction session.

  In addition, as shown in FIG. 14, the evaluation unit 65 uses the ternary evaluation model M5 that has learned the characteristics of the unfavorable dialogue example UD, the neutral dialogue example ND, and the favorable dialogue example FD to determine the target session. An evaluation may be performed. For example, the evaluation unit 65 may input the target session into the ternary evaluation model M5 read from the model database 35, and evaluate the target session based on the classification result by the ternary evaluation model M5. For example, when the target session is classified by the ternary evaluation model M5 as being similar to the unfavorable dialogue example UD, the evaluation unit 65 evaluates the target session as a low-satisfaction session and determines that the target session is similar to the favorable dialogue example FD. If classified, the target session may be evaluated as a high satisfaction session.

  In addition, as shown in FIG. 15, the evaluation unit 65 may evaluate the target session based on the similarity with the session including the utterance of the user whose use mode satisfies the predetermined condition. For example, the evaluation unit 65 inputs the target session to the engagement session evaluation model M6 that has learned the characteristics of the session of the user with high engagement and the session of the user with low engagement, and based on the classification result of the engagement session evaluation model M6. Alternatively, the target session may be evaluated. For example, if the target session is classified by the engagement session evaluation model M6 as being similar to a session of a user with high engagement, the evaluation unit 65 may evaluate the target session as a high satisfaction session.

  In addition, the evaluation unit 65 may evaluate the target session based on repeated utterances, as shown in FIGS. For example, when the same or similar character string repeatedly appears in the target session, the evaluation unit 65 evaluates the target session as a low satisfaction session, and the same or similar character string does not repeatedly appear in the target session. In this case, the target session may be evaluated as a highly satisfactory session.

  Further, the evaluation unit 65 may evaluate the target session based on the number of utterance pairs in which the similarity of the character strings or the semantic similarity exceeds a predetermined threshold. For example, as shown in FIG. 16, when the number of notation similar pairs or the number of meaning similar pairs included in the target session exceeds a predetermined threshold, the evaluation unit 65 evaluates the target session as a low-satisfaction session, and If the number of similar pairs or the number of semantic similar pairs is below a predetermined threshold, the target session may be evaluated as a highly satisfactory session.

  In addition, as shown in FIG. 17, the evaluation unit 65 may evaluate the target session using a repetitive utterance evaluation model M7 that has learned features of a session including repetitive utterances. For example, the evaluation unit 65 inputs the target session into the repetitive utterance evaluation model M7, and when the repetitive utterance evaluation model M7 classifies the target session into a session similar to the repetitive session RS, the evaluation unit 65 evaluates the target session as a low satisfaction session. However, when the target session is classified into a session similar to the non-repetitive session NRS, the target session may be evaluated as a high-satisfaction session.

  Also, as shown in FIG. 18, the evaluation unit 65 learns the characteristics of the distribution image generated from the utterance in the session including the repeated utterance, and uses the repetition evaluation model M1 that has learned the characteristics of the session including the repeated utterance. Then, the target session may be evaluated. For example, the image generation unit 63 generates an utterance distribution image included in the target session. Then, the image generation unit 63 evaluates the target session based on the distribution image.

  For example, the evaluation unit 65 inputs the target image to the repetition evaluation model M1, and when the repetition evaluation model M1 classifies the distribution image of the target session into an image similar to the distribution image of the session including repetitive utterances, A session may be rated as a low satisfaction session. Further, for example, when the evaluation unit 65 inputs the target image to the repetition evaluation model M1, and the repetition evaluation model M1 classifies the distribution image of the target session into an image similar to the distribution image of the session that does not include the repeated utterance. May evaluate the target session as a high satisfaction session.

  The evaluation unit 65 determines whether or not the distribution image of the target session includes a diagonal line by using a known image analysis technique. You may be evaluated as a satisfactory session. That is, the evaluation unit 65 determines whether or not the target session includes repeated utterances from the distribution image of the target session without using the repetition evaluation model M1, and evaluates the target session based on the determination result. May be.

  Further, the evaluation unit 65 may evaluate the target session based on whether or not an apology response is included. For example, when the number of apology responses included in the target session exceeds a predetermined threshold, the evaluation unit 65 may evaluate the target session as a low satisfaction session. Also, as shown in FIG. 19, the evaluation unit 65 inputs the target session to the apology evaluation model M2, and if the apology evaluation model M2 classifies the target session into a session similar to the session including the apology response, the evaluation unit 65 A session may be rated as a low satisfaction session. When the apology evaluation model M2 classifies the target session into a session similar to a session that does not include an apology response, the evaluation unit 65 may evaluate the target session as a high-satisfaction session.

  The evaluation unit 65 may evaluate the target session using a plurality of models. For example, as illustrated in FIG. 20, the evaluation unit 65 evaluates a plurality of models M1 to M7 that evaluate a session based on different types of characteristics among a plurality of characteristics serving as indexes of the user's evaluation of the session. May be used to evaluate the target session.

  When a session is evaluated using evaluation results of a plurality of models M1 to M7, any combination of models can be adopted. For example, as shown in FIG. 1, the evaluation unit 65 assigns the target session (and the target session) to each of the repetition evaluation model M1, the apology evaluation model M2, the unfavorable index utterance evaluation model M3, and the favorable index utterance evaluation model M4. Is calculated, and a value obtained by subtracting the value of the score output by the favorable index utterance evaluation model M4 from the sum of the scores output by the models M1 to M3 is calculated. Then, the evaluation unit 65 evaluates a predetermined number of sessions from a target session having a higher calculated value as a low satisfaction session, and evaluates a predetermined number of sessions from a lower calculated value as a high satisfaction session. May be.

  When a value obtained by subtracting the value of the score output by the favorable index utterance evaluation model M4 from the sum of the scores output by the models M1 to M3 is calculated, and the user is satisfied with the target session. Is calculated as a negative value, and if the user is not satisfied with the target session, a positive value is calculated. Therefore, the evaluation unit 65 may evaluate the target session according to whether the calculated value is a positive value or a negative value.

  In addition, the evaluation unit 65 determines at least the evaluation result by the repetition evaluation model M1 or the repetition utterance evaluation model M7 that evaluates the session based on whether or not the same or similar utterance is repeatedly included in the utterance included in the session. May be used. Further, the evaluation unit 65 may use at least the evaluation result of the apology evaluation model M2 that evaluates the session based on whether or not the response included in the session includes a predetermined response indicating an apology.

  Further, the evaluation unit 65 may use at least the evaluation result of the unfavorable index utterance evaluation model M3 that evaluates the session based on whether or not the utterance included in the session includes the unfavorable index utterance. Further, the evaluation unit 65 may use at least the evaluation result of the favorable index utterance model M4 that evaluates the session based on whether or not the utterance included in the session includes the favorable index utterance.

  Here, the index utterance needs to be set or extracted in advance in the unfavorable index utterance evaluation model M3 or the favorable index utterance evaluation model M4 when a session serving as learning data is extracted. However, it is troublesome to set and extract the index utterance in advance in this way. Therefore, the evaluation unit 65 may evaluate the target content based on only the evaluation results of the repetition evaluation model M1 and the apology evaluation model M2. For example, the evaluation unit 65 calculates the sum of the score of the repetition evaluation model M1 and the score of the apology evaluation model M2, and evaluates a predetermined number of target sessions as low-satisfaction sessions in descending order of the sum, A predetermined number of target sessions may be evaluated as high satisfaction sessions in ascending order of sum.

  When the target session is evaluated using the plurality of models M1 to M7, the evaluation unit 65 may set a predetermined weight to the evaluation result of each of the models M1 to M7. For example, the evaluation unit 65 may integrate a predetermined coefficient for each of the models M1 to M7 into a score value output from each of the models M1 to M7, and integrate the score values obtained by integrating the coefficients. When the evaluation unit 65 evaluates the target session, the evaluation unit registers the evaluation result in the session evaluation database 32. Such an evaluation result may be used, for example, to extract an index utterance, or may be used to evaluate another session.

[2-6. About prediction processing)
Returning to FIG. 2, the description will be continued. The engagement prediction processing unit 70 executes a prediction process of predicting a future use mode of the user's interactive service. Hereinafter, the concept of the process executed by the engagement prediction processing unit 70 and an example of the functional configuration of the engagement prediction processing unit 70 will be described with reference to FIGS.

[2-6-1. Prediction process based on index utterance]
For example, the engagement prediction processing unit 70 performs a prediction process based on an index utterance included in a session of a user (hereinafter, referred to as a “target user”) that is a prediction target of a usage mode of a future interactive service. Execute. For example, FIG. 23 is a diagram illustrating a concept of a process of predicting a future use mode based on an index utterance.

  As shown in FIG. 23, it is considered that a user who frequently gives a favorable index utterance has a favorable impression on the interactive service. As described above, it is presumed that a user who has a favorable impression of the interactive service naturally uses the interactive service and, as a result, becomes a user who is highly engaged in the use period, the use frequency, and the like. On the other hand, it is considered that a user who has many unfavorable index utterances has an unfavorable impression on the interactive service. As described above, it is presumed that a user who has an unfavorable impression of the interactive service stops using the interactive service and becomes a user with low engagement.

  Therefore, the engagement prediction processing unit 70 predicts the future engagement of the user based on the frequency of appearance of the index utterance in the session including the utterance of the user. For example, the engagement prediction processing unit 70 is a session including the utterance of the target user, and includes a dialog performed within a predetermined period (hereinafter, referred to as an “observation period”) from the time of executing the prediction processing. Extract sessions.

  Subsequently, the engagement prediction processing unit 70 extracts the favorable index utterance and the unfavorable index utterance included in the session during the observation period, and sums the appearance frequency of the favorable index utterance during the observation period and the non-favorable index utterance during the observation period. Calculate the total appearance frequency of the favorable index utterances. Then, the engagement prediction processing unit 70 calculates a score obtained by dividing the total appearance frequency of favorable index utterances in the observation period by the total appearance frequency of non-favorable index utterances in the observation period. It is determined that the user has high engagement in the future.

  Note that the engagement prediction processing unit 70 may consider the value of the index score of a favorable index utterance or an unfavorable index utterance. For example, the engagement prediction processing unit 70 calculates the sum of index scores of favorable index utterances during the observation period and the sum of index scores of unfavorable index utterances during the observation period. Then, the engagement prediction processing unit 70 calculates a score obtained by dividing the total of the index scores of the favorable index utterances in the observation period by the total of the index scores of the non-favorable index utterances in the observation period. It may be determined that the user has high engagement in the future. Then, the engagement prediction processing unit 70 registers the prediction result in the engagement database 34.

  The future engagement predicted by such processing is used for extracting an index utterance and evaluating a session. For example, in the processing illustrated in FIG. 9, the index utterance extraction processing unit 50 regards a user whose future engagement predicted by the engagement prediction processing unit 70 is high as a favorable user, and sets the future prediction predicted by the engagement prediction processing unit 70. The index utterance may be extracted from the utterance of each user by setting the user with low engagement as the unfavorable user. In addition, for example, in the processing illustrated in FIG. 15, the session evaluation processing unit 60 evaluates an engagement session evaluation from a session of a user with a high future engagement predicted by the engagement prediction processing unit 70 or a session of another user. The target session may be evaluated by learning the model M6 and using the learned engagement session evaluation model M6.

[2-6-2. About prediction processing based on evaluation results]
Further, for example, the engagement prediction processing unit 70 may execute a prediction process based on the evaluation result of the session. For example, FIG. 24 is a diagram illustrating a concept of a process of predicting a future use mode based on an evaluation result.

  For example, as shown in FIG. 24, a session evaluated as having a high degree of satisfaction by the evaluation process, that is, a user having many high-satisfaction sessions is estimated to have a favorable impression on the interactive service. Engagement in the future is expected to increase. On the other hand, sessions evaluated as having a low degree of satisfaction by the evaluation process, that is, users who have many low-satisfaction sessions, are presumed to have an unfavorable impression on the dialogue service, and therefore are considered to have low future engagement. Can be

  Therefore, the engagement prediction processing unit 70 predicts the future engagement of the target user based on the evaluation result of the session including the utterance of the target user. For example, the engagement prediction processing unit 70 extracts a session that includes the utterance of the target user and that includes a dialog performed during the observation period. Subsequently, the engagement prediction processing unit 70 acquires an evaluation result for the extracted session. For example, the engagement prediction processing unit 70 acquires the evaluation result by the session evaluation processing unit 60 with reference to the session evaluation database 32.

  Then, the engagement prediction processing unit 70 predicts the engagement of the target user in the prediction period based on the evaluation result based on the statistical value. For example, the engagement prediction processing unit 70 calculates a ratio of a high satisfaction session and a ratio of a low satisfaction session among the extracted sessions. Then, the engagement prediction processing unit 70 may determine that the target user is a user having a high future engagement as the ratio of the high satisfaction sessions is higher. Further, for example, when the number of high-satisfaction sessions among the extracted sessions exceeds a predetermined threshold, the engagement prediction processing unit 70 may determine that the target user is a user with high future engagement.

  Further, for example, the engagement prediction processing unit 70 assigns a positive score to a high satisfaction session and a negative score to a low satisfaction session, and sets a larger coefficient to a newer session. Then, the engagement prediction processing unit 70 estimates the future engagement of the target user based on whether or not the score obtained by summing the product of the score of each session and the coefficient exceeds a predetermined threshold. Good. The engagement prediction processing unit 70 assigns a negative score to a high satisfaction session and a positive score to a low satisfaction session, sets a larger coefficient to a newer session, and sets the value of the product of the score and the coefficient for each session. The engagement of the target user in the future may be estimated based on whether or not the score obtained by summing is equal to or smaller than a predetermined threshold.

[2-6-3. Functional configuration of the engagement prediction processing unit]
Next, an example of a functional configuration of the engagement prediction processing unit 70 will be described with reference to FIG. FIG. 25 is a diagram illustrating an example of a functional configuration of an engagement prediction processing unit according to the embodiment. As shown in FIG. 25, the engagement prediction processing unit 70 has an estimation unit 71.

  The estimating unit 71 estimates a future use mode of the interactive service by the target user based on the evaluation result of the session including the utterance of the target user. For example, the estimating unit 71 calculates a total of the index scores of the favorable index utterances during the observation period, that is, a favorable score indicating how favorable the evaluation by the user is. Further, the estimating unit 71 calculates the total of the index scores of the unfavorable index utterances during the observation period, that is, the unfavorable score indicating how unfavorable the evaluation by the user is. Then, the estimating unit 71 estimates the future engagement of the target user, that is, the usage mode, based on the ratio between the favorable / favorable score and the non-favorable score.

  Further, the estimating unit 71 specifies whether each session during the observation period of the target user is a high satisfaction session or a low satisfaction session. Then, the estimating unit 71 estimates the future use mode of the target user based on the number and ratio of the high satisfaction sessions and the low satisfaction sessions.

[2-7. Relevance of each process)
Here, the relevance of the extraction processing executed by the index utterance extraction processing unit 50, the evaluation processing executed by the session evaluation processing unit 60, and the prediction processing executed by the engagement prediction processing unit 70 will be described with reference to FIG. FIG. 26 is a diagram illustrating an example of the relevance of processing performed by the information providing device according to the embodiment.

  First, the relationship between the extraction processing and the evaluation processing will be described. For example, it is estimated that a session with a high evaluation is likely to include a favorable index utterance, and a session with a low evaluation is likely to include an unfavorable index utterance. Therefore, the evaluation result of the session can be used for the extraction processing of the index utterance. More specifically, the information providing apparatus 10 can improve the extraction amount and extraction accuracy of the index utterance by performing the evaluation of the session by the above-described evaluation processing. On the other hand, a session including a favorable index utterance is estimated to have a high evaluation, and a session including a non-favorable index utterance is estimated to have a low evaluation. For this reason, the extraction result of the index utterance can be used for the session evaluation processing. More specifically, the information providing apparatus 10 can improve the evaluation accuracy of the session by extracting the index utterance by the above-described extraction processing.

  Next, the relationship between the extraction processing and the prediction processing will be described. For example, it is estimated that a user with a high engagement has a high frequency of uttering a favorable index utterance, and a user with a low engagement has a low frequency of uttering an unfavorable index utterance. For this reason, the engagement prediction result can be used for the extraction processing of the index utterance. More specifically, the information providing apparatus 10 can improve the extraction amount and extraction accuracy of the index utterance by predicting the engagement. On the other hand, it is presumed that a user with many favorable index utterances has a high engagement, and a user with many non-favorable index utterances has a low engagement. For this reason, the extraction result of the index utterance can be used for engagement prediction processing. More specifically, the information providing apparatus 10 can improve the prediction accuracy of the engagement by extracting the index utterance.

  Next, the relationship between the evaluation processing and the prediction processing will be described. For example, it is estimated that a user with many sessions with high evaluation has a high engagement, and a user with many sessions with low evaluation has a low engagement. For this reason, the session evaluation result can be used for engagement prediction processing. More specifically, the information providing apparatus 10 can improve the prediction accuracy of the engagement by evaluating the session. On the other hand, it is estimated that a user with a high engagement has a high evaluation of the session, and a user with a low engagement has a low evaluation of the session. For this reason, the prediction result of the engagement can be used for the session evaluation processing. More specifically, the information providing apparatus 10 can improve the evaluation accuracy of the session by predicting the engagement.

  As described above, the above-described various kinds of extraction processing, evaluation processing, and prediction processing have a bootstrap relationship with each other. Therefore, the information providing apparatus 10 can gradually improve the accuracy of each process by repeatedly executing each process in order. For example, the information providing apparatus 10 may alternately execute an extraction process and a prediction process, an extraction process and an evaluation process, or an evaluation process and a prediction process. Further, the information providing apparatus 10 may repeatedly execute each process in order.

[2-8. About reinforcement learning processing)
Returning to FIG. 2, the description will be continued. The reinforcement learning processing unit 80 executes reinforcement learning of the dialog model used by the dialog device 200.

[2-8-1. About an example of reinforcement learning)
For example, FIG. 27 is a diagram illustrating a concept of a process of executing reinforcement learning of a dialog model. The interaction device 200 generates a response to the utterance of the user using the interaction model M10. Here, in the interactive service, the user may utter an entirely new content (a sentence different from the previous sentence) in response to the response, or may utter an utterance including evaluation of the response. For example, the above-mentioned favorable index utterance and unfavorable index utterance are considered to be evaluations of the response performed immediately before. Then, when the dialogue model M10 is regarded as an agent in reinforcement learning and the user as an environment based on reinforcement learning, various index utterances spoken by the user can be regarded as rewards for the agent.

  Thus, the reinforcement learning processing unit 80 realizes reinforcement learning of the dialog model M10 by setting a reward based on the index utterance for the dialog model M10. More specifically, the reinforcement learning processing unit 80 obtains an index utterance that is an index of the evaluation of the dialogue service among the utterances of the response model M10, and sets a reward based on the obtained index utterance, thereby obtaining a response model. Perform M10 reinforcement learning.

  For example, the reinforcement learning processing unit 80 refers to the session log database 31 and specifies a favorable index utterance and a non-favorable index utterance included in the session. Then, when a favorable index utterance is specified, the reinforcement learning processing unit 80 sets a positive reward as a reward for a response immediately before the specified favorable index utterance, and learns the interaction model M10. Further, when the unfavorable index utterance is specified, the reinforcement learning processing unit 80 sets a negative reward as a reward for a response immediately before the specified unfavorable index utterance, and learns the dialog model M10. .

  Further, the reinforcement learning processing unit 80 generates a user model model M20 that imitates the utterance of the user, causes the user model model M20 and the dialog model M10 to perform virtual dialogue, and the user model model M20 A reward according to the output index utterance may be set, and reinforcement learning of the dialog model M10 may be performed. For example, the reinforcement learning processing unit 80 generates the user model model M20 using predetermined dialog rules and sessions registered in the session log database 31 as learning data. Subsequently, the reinforcement learning processing unit 80 inputs the utterance generated by the user model model M20 to the dialog model M10, and inputs the response output by the dialog model M10 to the user model model M20.

  Further, the reinforcement learning processing unit 80 repeatedly executes a virtual dialogue between the dialogue model M10 and the user model model M20. Then, when the user model model M20 outputs an index utterance as an utterance, the reinforcement learning processing unit 80 sets a reward according to the index utterance, and performs reinforcement learning of the dialog model M10 based on the set reward. .

[2-8-2. Example of Functional Configuration of Reinforcement Learning Processing Unit]
Next, an example of a functional configuration of the reinforcement learning processing unit 80 will be described with reference to FIG. FIG. 28 is a diagram illustrating an example of a functional configuration of the reinforcement learning processing unit according to the embodiment. As illustrated in FIG. 28, the reinforcement learning processing unit 80 includes an index utterance acquisition unit 81, a reward setting unit 82, and a reinforcement learning unit 83.

  The index utterance obtaining unit 81 obtains an index utterance, which is an index of evaluation of a dialog service, among the utterances of the dialog model M10. More specifically, the index utterance obtaining unit 81 obtains a favorable index utterance and a non-favorable index utterance from the utterance output by the user or the user model model M20. For example, the index utterance obtaining unit 81 obtains an index utterance by referring to the session log database 31. In addition, the index utterance obtaining unit 81 obtains an index utterance among the utterances output by the user model model M20.

  The reward setting unit 82 sets a reward according to the index utterance obtained by the index utterance obtaining unit 18. For example, the reward setting unit 82 sets a reward according to the index score of the obtained index utterance. More specifically, the reward setting unit 82 sets a positive reward when the obtained index utterance is a favorable index utterance, and sets a positive reward when the obtained index utterance is a non-favorable index utterance. Set a negative reward.

  The reinforcement learning unit 83 performs reinforcement learning of the interactive model M10 using the reward set by the reward setting unit 82. For example, when the index utterance acquisition unit 81 extracts a certain index utterance from the session, the reinforcement learning unit 83 specifies the response immediately before the index utterance, and the reward when the specified response is output by the dialog model M10 is: The reinforcement learning of the dialog model M10 is performed as a reward based on the index utterance.

  Further, for example, the reinforcement learning unit 83 uses the utterance model that outputs an index utterance in response to the response generated by the dialogue model M10, that is, a reward based on the index utterance output by the user model model M20, and uses the dialogue model M10. Perform reinforcement learning. For example, the reinforcement learning unit 83 learns the user model model M20 using the contents of each session registered in the session log database 31 and the dialog rules. For such a user model model M20, for example, a learning method similar to the learning of the interactive model M10 can be adopted.

  Subsequently, the reinforcement learning unit 83 causes the user model model M20 and the dialog model M10 to execute a virtual dialog. Then, when the user model model M20 outputs the index utterance, the reinforcement learning unit 83 executes the reinforcement learning of the dialog model M10 based on the reward set by the reward setting unit 82.

[3. Example of flow of processing executed by information providing apparatus]
Subsequently, an example of the flow of each process executed by the information providing apparatus 10 will be described with reference to FIGS.

[3-1. Example of processing flow based on usage mode)
First, an example of a flow of processing executed by the information providing apparatus 10 based on the usage mode will be described with reference to FIG. FIG. 29 is a flowchart illustrating an example of a flow of a process performed by the information providing apparatus according to the embodiment based on a use mode.

  First, the information providing apparatus 10 specifies a usage mode of each user's interactive service (step S101). Subsequently, the information providing apparatus 10 specifies a user whose specified result satisfies a predetermined condition (step S102). Then, the information providing apparatus 10 extracts a session including the utterance of the specified user (Step S103). Further, the information providing apparatus 10 learns the engagement session evaluation model M6 using the session thus extracted as learning data (step S104), and ends the process.

  Note that the information providing apparatus 10 may evaluate the extracted session (step S105). For example, when extracting sessions of users with high engagement, the information providing apparatus 10 may evaluate those sessions as high-satisfaction sessions. Further, the information providing apparatus 10 may extract the index utterance from the session thus extracted (step S106). For example, the information providing apparatus 10 extracts, as a favorable index utterance, an utterance whose appearance frequency in a session of a user with high engagement satisfies a predetermined condition, and an appearance frequency in a session of a user with low engagement satisfies a predetermined condition. The utterance may be extracted as an unfavorable index utterance.

[3-2. Example of processing flow for extracting index utterance]
Next, an example of a flow of a process in which the information providing apparatus 10 extracts the index utterance will be described with reference to FIG. FIG. 30 is a flowchart illustrating an example of a flow of a process in which the information providing apparatus according to the embodiment extracts an index utterance.

  First, the information providing apparatus 10 acquires a session of a user whose use mode satisfies a predetermined condition (step S201), and extracts an index utterance based on the appearance frequency in the acquired session (step S202). Then, the information providing apparatus 10 further extracts a new index utterance based on the co-occurrence with the index utterance (step S203), and ends the process.

  Note that the information providing apparatus 10 may evaluate the session based on the extracted index utterance (Step S204). For example, the information providing apparatus 10 may evaluate a session including many favorable index utterances as a high-satisfaction session, and evaluate a session including many unfavorable index utterances as a low-satisfaction session. Further, the information providing apparatus 10 may estimate the usage mode of the user based on the appearance frequency of the index utterance (Step S205). For example, the information providing apparatus 10 estimates a user who utters a lot of favorable index utterances as a user with high engagement, and estimates a user who contains many unfavorable index utterances as a user with low engagement. Is also good.

[3-3. Example of processing flow for evaluating session]
Next, an example of a flow of a process in which the information providing apparatus 10 evaluates a session will be described with reference to FIG. FIG. 31 is a flowchart illustrating an example of the flow of a process in which the information providing apparatus according to the embodiment evaluates a session.

  First, the information providing apparatus 10 acquires a session to be evaluated (step S301), and evaluates the session based on the appearance frequency of the index utterance in the acquired session (step S302). For example, the information providing apparatus 10 evaluates a session including many favorable index utterances as a high satisfaction session, and evaluates a session including many unfavorable index utterances as a low satisfaction session. Then, the information providing apparatus 10 learns the characteristics of the session as learning data to which a label corresponding to the evaluation result is added (step S303), and ends the processing. For example, the information providing apparatus 10 causes the models M3 and M4 to learn the features of the high satisfaction session and the low satisfaction session.

  Note that the information providing apparatus 10 may extract a new index utterance from the session based on the evaluation result of the session (step S304). For example, the information providing apparatus 10 may extract an utterance having a high appearance frequency in a high satisfaction session as a favorable index utterance, and may extract an utterance having a high appearance frequency in a low satisfaction session as a non-favorable index utterance. Further, the information providing apparatus 10 may estimate a usage mode of the user based on the evaluation result of the session (Step S305). For example, the information providing apparatus 10 may estimate a user with many high-satisfaction sessions as a user with high engagement and a user with many low-satisfaction sessions as a user with low engagement.

[3-4. Example of process flow for evaluating session based on image)
Next, an example of the flow of a process in which the information providing apparatus 10 evaluates a session based on an image will be described with reference to FIG. FIG. 32 is a flowchart illustrating an example of a flow of processing in which the information providing apparatus according to the embodiment evaluates a session based on an image.

  First, the information providing apparatus 10 acquires a session to be evaluated (step S401), and generates an image indicating whether or not the characters of the utterances included in the acquired session are mutually the same, that is, a distribution image (step S401). S402). Then, the information providing apparatus 10 evaluates the session based on the generated distribution image (Step S403). For example, the information providing apparatus 10 evaluates the target session as a low-satisfaction session when the distribution image includes a diagonal line, and when the distribution image is classified as including a repetition by the repetition evaluation model M1. Then, the information providing apparatus 10 learns the characteristics of the session as learning data to which a label corresponding to the evaluation result is added (step S404), and ends the processing.

  The information providing apparatus 10 may extract a new index utterance from the session based on the evaluation result of the session (step S405). Further, the information providing apparatus 10 may estimate a usage mode of the user based on the evaluation result of the session (Step S406).

[3-5. Example of processing flow for evaluating session based on repetition of utterance]
Next, an example of the flow of a process in which the information providing apparatus 10 evaluates a session based on repetition of speech will be described with reference to FIG. FIG. 33 is a flowchart illustrating an example of a flow of processing in which the information providing apparatus according to the embodiment evaluates a session based on repetition of speech.

  First, the information providing apparatus 10 acquires a session to be evaluated (step S501), and evaluates the session based on repetition of a character string of an utterance included in the acquired session (step S502). For example, if the number of utterances having the same or similar character strings or the number of utterances having the same or similar meaning exceeds a predetermined threshold, the information providing apparatus 10 evaluates the session as a low-satisfaction session. Then, the information providing apparatus 10 learns the characteristics of the session as learning data to which a label corresponding to the evaluation result is added (step S503), and ends the processing.

  The information providing apparatus 10 may extract a new index utterance from the session based on the evaluation result of the session (step S504). Further, the information providing apparatus 10 may estimate a usage mode of the user based on the evaluation result of the session (Step S505).

[3-6. Example of reinforcement learning process flow with utterance as reward)
Next, an example of a flow of reinforcement learning in which the information providing apparatus 10 executes the utterance as a reward will be described with reference to FIG. FIG. 34 is a flowchart illustrating an example of the flow of reinforcement learning in which the information providing apparatus according to the embodiment executes an utterance as a reward.

  First, the information providing apparatus 10 inputs an utterance to the dialogue model M10 (step S601), and obtains an index utterance for the response by inputting the response output by the dialogue model M10 to the user model model M20 (step S601). Step S602). Then, the information providing apparatus 10 executes the reinforcement learning of the interactive model M10 using the index utterance acquired from the user model model M20 as a reward (step S603), and ends the process.

[3-7. An example of the flow of processing for extracting an index utterance based on co-occurrence]
Next, an example of a flow of a process in which the information providing apparatus 10 extracts an index utterance based on co-occurrence will be described with reference to FIG. FIG. 35 is a flowchart illustrating an example of a flow of a process in which the information providing apparatus according to the embodiment extracts an index utterance based on co-occurrence.

  First, the information providing apparatus 10 specifies an index utterance from the history of the session (step S701), and specifies co-occurrence with the index utterance in each session for each utterance (step S703). Then, the information providing apparatus 10 extracts a new index utterance based on the specified co-occurrence (step S703). For example, the information providing apparatus 10 extracts an utterance having a high co-occurrence with the favorable index utterance as a new favorable index utterance, and extracts an utterance having a high co-occurrence with the unfavorable index utterance as a new unfavorable index utterance. Extract as an index utterance.

  The information providing apparatus 10 may further extract a new index utterance based on the co-occurrence with the newly extracted index utterance (step S704). In addition, the information providing apparatus 10 may evaluate the session based on the index utterance newly extracted in this manner (step S705), and determine the usage mode of the user based on the appearance frequency of the index utterance. It may be estimated (step S706).

[3-8.3 Example of Process Flow for Evaluating Session Using Three-Value Evaluation Model]
Next, an example of the flow of processing for evaluating a session using the ternary evaluation model M5 will be described with reference to FIG. FIG. 36 is a flowchart illustrating an example of a flow of processing in which the information providing apparatus according to the embodiment evaluates a session using a ternary evaluation model.

  First, the information providing apparatus 10 specifies a favorable index utterance, an unfavorable index utterance, and a neutral utterance from the history of the session (step S801), and outputs a predetermined number of utterances and responses from each index utterance as learning data. (Step S802). That is, the information providing apparatus 10 generates a favorable interaction example FD, a neutral interaction example ND, and a non-favorable interaction example UD. Further, the information providing apparatus 10 gives a label corresponding to the index utterance included in the extracted learning data (step S803), and learns the ternary evaluation model M5 using the learning data (step S804). Thereafter, the information providing apparatus 10 evaluates the session using the ternary evaluation model M5 (Step S805), and ends the processing.

[3-9. Example of process flow for evaluating session using multiple models]
Next, an example of a flow of a process of evaluating a session using a plurality of models M1 to M7 will be described with reference to FIG. FIG. 37 is a flowchart illustrating an example of a flow of a process in which the information providing apparatus according to the embodiment evaluates a session using a plurality of models.

  For example, the information providing apparatus 10 acquires a session to be evaluated (step S901), and causes a plurality of models M1 to M7 that have learned different characteristics to evaluate the session (step S902). Then, the information providing apparatus 10 integrates the evaluation results of the models M1 to M7, evaluates the session (step S903), and ends the processing.

[4. Others)
Further, among the processes described in the above embodiment, all or some of the processes described as being performed automatically can be manually performed, and conversely, the processes described as being performed manually can be performed. Can be automatically or completely performed by a known method. In addition, the processing procedure, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown.

  Each component of each device illustrated is a functional concept and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  Further, the respective embodiments described above can be appropriately combined within a range that does not contradict processing contents.

[5. program〕
The information providing apparatus 10 according to the above-described embodiment is realized by, for example, a computer 1000 having a configuration as illustrated in FIG. FIG. 38 is a diagram illustrating an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and a form in which a computing device 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected by a bus 1090. Having.

  The arithmetic device 1030 operates based on a program stored in the primary storage device 1040 or the secondary storage device 1050, a program read from the input device 1020, or the like, and executes various processes. The primary storage device 1040 is a memory device such as a RAM that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, such as a ROM (Read Only Memory), a HDD (Hard Disk Drive), and a flash memory. Is realized by:

  The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information such as a monitor and a printer. For example, a USB (Universal Serial Bus), a DVI (Digital Visual Interface), This is realized by a connector of a standard such as HDMI (registered trademark) (High Definition Multimedia Interface). The input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, and a scanner, and is realized by, for example, a USB.

  The input device 1020 includes, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), and a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), and a tape. A device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like may be used. The input device 1020 may be an external storage medium such as a USB memory.

  The network IF 1080 receives data from another device via the network N and sends the data to the arithmetic device 1030, and transmits the data generated by the arithmetic device 1030 to the other device via the network N.

  The arithmetic device 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic device 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

  For example, when the computer 1000 functions as the information providing apparatus 10, the arithmetic unit 1030 of the computer 1000 implements the function of the control unit 40 by executing a program loaded on the primary storage device 1040. For example, when the computer 1000 functions as the terminal device 100, the arithmetic device 1030 of the computer 1000 implements the function of the control unit 40 by executing a program loaded on the primary storage device 1040.

[6. Experimental results)
Next, an example of an experimental result of the above-described information providing apparatus 10 will be described with reference to FIGS.

[6-1. About data)
First, an example of the session data used for the experiment of the information providing apparatus 10 will be described. In the following experiment, labels were assigned to 4000 sessions including at least two or more utterances and responses by crowdsourcing. Specifically, 2,000 training sessions (Training), 1,000 adjustment sessions (Validation), and 1,000 test sessions (Test) were prepared. Also, each session is presented to the seven judges, and when it is determined that the user is frustrated (“Frustrated”), “Yes”, and when it is determined that the user is not frustrated, , "No" was assigned to each session.

  For example, FIG. 39 is a diagram illustrating an example of a result of labeling session data. For example, in the example shown in FIG. 39, a session in which at least one judge has attached a label of “Yes” is FL (Frustration Level) 1, and a session in which at least two judge are given a label of “Yes” is FL2, a session in which at least three judges have labeled "Yes" is described as FL3, and a session in which at least four judges have labeled "Yes" is described as FL4. FIG. 39 shows the number of sessions classified into FL1 to FL4 among the training session, the adjustment session, and the test session.

[6-2. About the first experiment]
Subsequently, the contents of the first experiment using the above-described session data will be described. In the first experiment, a method indicated by Prop_D and a method indicated by Prop_U were set as experimental objects, and methods indicated by BL_S1, BL_S2, and BL_SS were set as baselines to be compared.

  Here, Prop_D is a determination method using a repetition evaluation model M1, an apology evaluation model M2, an unfavorable index utterance evaluation model M3, and a favorable index utterance evaluation model M4, as shown in FIG. For example, in Prop_D, a test session was input to each of the models M1 to M4 to calculate a dissatisfaction score. In Prop_D, 500,000 sessions are determined to be low-satisfaction sessions (that is, the label is “Yes”) in descending order of the calculated value of the dissatisfaction score. 500,000 sessions in turn were determined to be high satisfaction sessions (ie, the label was "No").

  Here, in Prop_D, when learning the repetition evaluation model M1, learning data is generated by using the method shown in FIG. 17, and as shown in FIG. 18, a distribution image of the generated learning data has The feature was repeatedly learned by the evaluation model M1. In Prop_D, when learning the apology evaluation model M2, learning data was generated using the method shown in FIG. 19, and the features of the learning data were learned by the apology evaluation model M2. In the case of Prop_D, learning of the favorable index utterance evaluation model M3 and the non-favorable index utterance evaluation model M4 was performed using the method shown in FIG. In Prop_D, a model for performing image classification is adopted as the repetition evaluation model M1, and a model for classifying information input using LSTM is adopted as models M2 to M4.

  Prop_U is a determination method using only the repetition evaluation model M1 and the apology evaluation model M2 among the models M1 to M4 shown in FIG. Since such a determination method does not require index utterance for learning a model, it is not necessary to manually create data in advance, and it is an unsupervised method. For example, in Prop_U, the target session was repeatedly input to the evaluation model M1 and the apology evaluation model M2, and the sum of the satisfaction levels output by the models M1 and M2 was calculated as the dissatisfaction score. In Prop_U, 500,000 sessions are determined to be low-satisfaction sessions (that is, the label is “Yes”) in descending order of the calculated value of the dissatisfaction score, and the dissatisfaction score is determined in descending order of the value of the dissatisfaction score. 500,000 sessions were determined to be high satisfaction sessions (ie, the label was "No").

  BL_S1 used a model that was learned using the training session described above and adjusted using the adjustment session. In the case of BL_S1, a model for classifying information input using LSTM is employed, similarly to Prop_D and Prop_U described above.

  Further, in BL_S2, SVM in which learning was performed using a training session and adjustment was performed using an adjustment session was used. In BL_S2, the output of the LSTM layer of the model used in BL_S1 is input to SVM, and the output of SVM is adopted as the classification result.

  In addition, in BL_SS, in addition to the training session, a transductive SVM (TSVM) trained using one million unlabeled sessions was used. Further, the adjustment session and the test session were used for learning as sessions without labels by ignoring the labels.

  Here, in the first experiment, the sessions classified into FL1 to FL4 are classified by the various methods described above, and the accuracy (accuracy), precision (accuracy), recall (recall), The F1 value, which is the harmonic mean of the precision and the recall, was calculated. FIG. 40 is a diagram showing the results of the first experiment. In the example shown in FIG. 40, an asterisk is given to a method having a significant difference from Prop_D, and a sharp is given to a method having a significant difference from Prop_U. As shown in FIG. 40, when the F1 value is used as a comparison target, the accuracy of the classification result by Prop_D is highest in the classification results of the session data classified into FL1, FL2, and FL3.

[6-3. About the second experiment]
Subsequently, the following second experiment was performed to determine how much each model described above contributed to the accuracy. In the second experiment, w / o-Rep in which the repeated evaluation model M1 was removed in Prop_D, w / o-Apo in which the apology evaluation model M2 was removed in Prop_D, and w / o-Apo in which the unfavorable index utterance evaluation model M3 was removed in Prop_D. A method called w / o-Fav was set in which the favorable index utterance evaluation model M4 was removed from / o-Unf and Prop_D.

  In the second experiment, a method called w / o-RAUF was set without using the four models M1 to M4 shown in FIG. In w / o-RAUF, 500,000 high-satisfaction sessions and 500,000 low-satisfaction sessions are used as learning data, and the input session is a high-satisfaction session (that is, a session with a label “No”) or a low-satisfaction session. A model to be classified into a session (that is, a session whose label is “Yes”) was learned, and the session was classified using the learned model.

  In the second experiment, a method called w / o-CandRet was set. In the w / o-CandRet, the dissatisfaction score is calculated using the above-described models M1 to M4 with respect to the session history acquired in advance, and 500,000 sessions are labeled in descending order of the calculated score. The sessions were extracted as “Yes” sessions, and 500,000 sessions were extracted as sessions with the label “No” in ascending order of calculated score. In w / o-CandRet, learning of a model for classifying an input session is performed using the generated learning data, and session classification is performed using the learned model.

  In the second experiment, the sessions classified into FL1 to FL4 are classified by the above-described various methods, and the accuracy (accuracy), precision (accuracy), recall (recall), The F1 value was calculated. FIG. 41 is a diagram showing the results of the second experiment. As shown in FIG. 41, it was found that when the F1 value was used as a comparison target, the accuracy of Prop_D was the highest, and when any of the models M1 to M4 was not used, the accuracy was reduced. As a result, it was found that each of the models M1 to M4 contributed to the improvement of the session classification accuracy.

[6-4. About the third experiment]
Subsequently, in Prop_D described above, the following third experiment was performed in order to determine the contribution of the amount of learning data to the classification accuracy. In the following description, the amount of used learning data is indicated by a subscript k. For example, Prop_D_k learns each model M1 to M4 using k / 2 high satisfaction sessions and k / 2 low satisfaction sessions, and classifies test sessions using the learned models M1 to M4. Is performed. In the third experiment, the value of k was changed to 500,000 (500k), 1,000,000 (1M), 2,000,000 (2M), 3,000,000 (3M), and 4,000,000 (4M) to obtain the learning data. The contribution of the quantity to the accuracy in each approach was investigated.

  FIG. 42 is a diagram showing the result of Prop_D in the third experiment. In FIG. 42, an asterisk is added to a result having a significant difference from the case where k is set to 1M. As shown in FIG. 42, in Prop_D, it was found that the accuracy was highest when k = 2M, regardless of the classification of the test session. This may be because if the learning data is less than 2 million, there is room for improvement in accuracy, and if the learning data exceeds 2 million, over-learning occurs.

[6-5. About the fourth experiment]
Here, in the above-described Prop_D, a neural network is used as a model for performing evaluation based on repeated utterances, apologies, and index utterances. Here, the following fourth experiment was performed in order to examine the effect of using a neural network for each recognition. For example, in the fourth experiment, in the Prop_D method, instead of a neural network, a session based on a repeated utterance is evaluated using an edit distance, and a method based on character string matching is used to perform evaluation based on a repeated utterance or an index utterance by character string matching. Ran as

  FIG. 43 is a diagram showing the result of Prop_S in the fourth experiment. Note that in FIG. 43, results having significant differences from Prop_D were obtained in FL2, FL3, and FL4. Comparing the results of Prop_D shown in FIGS. 40 and 41 with the results of Prop_S shown in FIG. 43, Prop_S has a higher precision and lower recall and F1 value than Prop_D. That is, Prop_S is considered to have lower accuracy than Prop_D. For this reason, in Prop_D, accuracy can be improved by using a neural network when evaluating a session.

[6-6. About the fifth experiment]
Next, a fifth experiment was performed in which the output of each model used for evaluation in Prop_D was fused, and the session was evaluated using a model with improved generalization ability (prediction ability for unlearned data). More specifically, in the fifth experiment, a technique in which the accuracy of Prop_D was further improved using the ensemble technique was used as Prop_E.

  FIG. 44 is a diagram showing the result of Prop_E in the fifth experiment. In FIG. 44, the result of Prop_E and the result of Prop_D_2M are described side by side. As shown in FIG. 44, the result of Prop_E has a higher F1 value than the result of Prop_D_2M. For this reason, by using an ensemble method like Prop_E, the accuracy of evaluating a session can be further improved.

[7. Effect of Embodiment]
Hereinafter, an example of the effect of the information providing apparatus 10 that executes the above-described processes will be described.

[7-1. Extraction of learning data based on usage mode)
As described above, the information providing apparatus 10 specifies the usage mode of the interactive service by the user. Then, the information providing apparatus 10 uses the session including the utterance of the user whose specified use mode satisfies the predetermined condition as the learning data for evaluating the session including the utterance and the response in the interactive service. Extract from

  As described above, the information providing apparatus 10 extracts learning data for learning a model for evaluating a session based on the engagement of each user. Therefore, the information providing apparatus 10 can automatically extract, for example, learning data for learning a model for determining whether or not the user is satisfied with the session without human intervention. As a result of such processing, the information providing apparatus 10 can easily prepare a large amount of learning data. As a result, the accuracy of the model can be improved, and as a result, the evaluation accuracy of the session can be improved.

  In addition, the information providing apparatus 10 specifies a use period of the interactive service by the user as a use mode, and extracts a session including an utterance of the user whose use period satisfies a predetermined condition as learning data. In addition, the information providing apparatus 10 specifies the use frequency of the interactive service by the user as a use mode, and extracts a session including the utterance of the user whose use frequency satisfies a predetermined condition as learning data. For this reason, for example, when extracting learning data for the purpose of extracting a session with a low degree of user satisfaction, the information providing apparatus 10 determines whether a user with a high engagement, that is, a user with a high evaluation for a dialogue service, Sessions are extracted as examples and sessions of users with low engagement, that is, users with low evaluation of dialogue services are extracted as positive examples, so that data suitable for learning can be easily prepared. Can improve the evaluation accuracy.

  Further, the information providing apparatus 10 causes the model to learn features of the extracted session. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  In addition, the information providing apparatus 10 extracts a session including an utterance of a user whose use mode satisfies a predetermined condition as learning data having a first attribute, and extracts an utterance of a user whose use mode does not satisfy a predetermined condition. The extracted session is extracted as learning data having the second attribute. Then, the information providing apparatus 10 learns a model that classifies the input session into either data having the first attribute or data having the second attribute using the extracted learning data.

  For example, the information providing apparatus 10 extracts a session including an utterance of a user whose use mode satisfies a predetermined condition as learning data favorably evaluated by the user, and the use mode does not satisfy the predetermined condition. The session including the utterance of the user is extracted as learning data whose evaluation by the user is unfavorable. Then, the information providing apparatus 10 learns a model that determines whether the user's evaluation of the input session is favorable or unfavorable using the extracted learning data. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  In addition, the information providing apparatus 10 evaluates the session including the utterance of the user based on the specified usage mode. Then, the information providing apparatus 10 extracts, as learning data, a session whose evaluation satisfies a predetermined condition. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  Further, the information providing apparatus 10 extracts, from the history of the interactive service, an utterance similar to an utterance included in a session whose evaluation satisfies a predetermined condition, or a response similar to a response included in the session, and extracts the extracted utterance or A new session using the response is generated as a session whose evaluation satisfies a predetermined condition. Then, the information providing apparatus 10 causes the model to learn features of the generated new session. As described above, the information providing apparatus 10 generates a similar session as learning data from a session whose use mode satisfies a predetermined condition, and performs model learning using the generated learning data. For this reason, the information providing apparatus 10 can improve the session evaluation accuracy as a result of improving the model determination accuracy.

  In addition, the information providing apparatus 10 extracts a session whose evaluation satisfies a predetermined condition from the history of the interactive service, and among the utterances included in the extracted session, the utterance whose appearance frequency satisfies the predetermined condition includes the utterance. It is extracted as an index utterance that is an index of whether or not the user's evaluation of the session is favorable. For example, the information providing apparatus 10 extracts, as an index utterance, an utterance in which a ratio of an appearance frequency in a session whose evaluation satisfies a predetermined condition and an appearance frequency in a session whose evaluation does not satisfy a predetermined condition satisfies a predetermined condition. For this reason, the information providing apparatus 10 can extract the index utterance based on the usage mode of the user. Note that such an index utterance can be used for evaluating a session. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session based on the index utterance.

  Further, the information providing apparatus 10 evaluates the session based on whether or not a predetermined response indicating an apology is included in the session, in addition to the usage mode of the user. In addition, the information providing apparatus 10 determines whether or not the session includes an index utterance that is an index of whether the evaluation of the user is favorable or not, in addition to the usage mode of the user. To evaluate. For this reason, the information providing apparatus 10 can further improve the evaluation accuracy of the session.

  In addition, the information providing apparatus 10 specifies a usage mode of the interactive service by the user, and evaluates a session including the utterance of the user based on the specified usage mode. For this reason, the information providing apparatus 10 can appropriately evaluate the session without manual intervention.

[7-2. Extraction of index utterance based on usage pattern]
In addition, the information providing apparatus 10 acquires a session including an utterance of a user whose use mode of the interactive service satisfies a predetermined condition and a response to the utterance. Then, the information providing apparatus 10 extracts an index utterance that is an index of the user's evaluation of the interactive service based on the acquired appearance frequency in the session. Such an index utterance can be used for evaluating a session. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  In addition, the information providing apparatus 10 acquires a session including an utterance of a user whose usage period of the interactive service satisfies a predetermined condition. In addition, the information providing apparatus 10 acquires a session including the utterance of the user whose use frequency of the interactive service satisfies a predetermined condition. Then, the information providing apparatus 10 determines the ratio of the appearance frequency in the session including the utterance of the user whose use mode satisfies the predetermined condition to the appearance frequency in the session including the utterance of the user whose use mode does not satisfy the predetermined condition. Extracts an utterance satisfying a predetermined condition as an index utterance.

  Further, the information providing apparatus 10 indicates that the user favorably evaluates the interactive service for an utterance whose appearance frequency in a session including the utterance of the user whose usage mode satisfies the predetermined condition exceeds a predetermined threshold. It is extracted as a favorable index utterance shown. In the information providing apparatus 10, the user unfavorably evaluates the dialogue service for an utterance whose appearance frequency in a session including an utterance of a user whose use mode does not satisfy the predetermined condition exceeds a predetermined threshold. It is extracted as an unfavorable index utterance indicating the effect. As a result of these processes, the information providing apparatus 10 can appropriately extract the index utterance from the utterance of the user, so that the evaluation accuracy of the session can be improved.

  Further, the information providing apparatus 10 evaluates the session based on the appearance frequency of the extracted index utterance. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  In addition, the information providing apparatus 10 determines that, for a session to be evaluated, an utterance whose appearance frequency in a session including an utterance of a user whose usage mode satisfies a predetermined condition exceeds a predetermined threshold value, and the usage mode satisfies a predetermined condition. The session to be evaluated is evaluated based on the utterance whose appearance frequency in the session including the utterance of the user who does not exceed the predetermined threshold. For this reason, the information providing apparatus 10 can appropriately evaluate the session.

  Further, the information providing apparatus 10 causes the model to learn features of a session whose evaluation satisfies a predetermined condition. For this reason, the information providing apparatus 10 can realize a model for evaluating a session.

  Further, the information providing apparatus 10 specifies the index utterance extracted from the history in the dialog service, and extracts a series of dialogs corresponding to the specified index utterance as learning data. Then, the information providing apparatus 10 causes the model to learn the features of the extracted learning data. For example, the information providing apparatus 10 causes the model to learn characteristics of the unfavorable dialogue example UD and the favorable dialogue example FD. For this reason, the information providing apparatus 10 can realize a model for evaluating a session.

[7-3. Evaluation of Session Based on Index Utterance]
Further, the information providing device 10 acquires a session including a user's utterance and a response in the interactive service. Then, the information providing apparatus 10 evaluates the acquired session based on an index utterance which is an index of the user's evaluation on the interactive service. Therefore, the information providing apparatus 10 can automatically and accurately learn the session.

  The information providing apparatus 10 also displays a favorable index utterance indicating that the evaluation of the user for the interactive service is favorable, and an unfavorable index utterance indicating that the evaluation of the user for the interactive service is unfavorable. And evaluate the session based on Then, for example, the information providing apparatus 10 evaluates the session based on the ratio between the appearance frequency of the favorable index utterance and the appearance frequency of the unfavorable index utterance. In addition, for example, the information providing apparatus 10 may include, in a session including a favorable index utterance, a favorable score indicating how favorable the evaluation of the user for the interactive service is, and a session including a non-favorable index utterance. Evaluate the session based on the ratio of the unfavorable score indicating how unfavorable the user's rating of the interactive service is. Therefore, the information providing apparatus 10 can automatically and accurately learn the session.

  In addition, the information providing apparatus 10 evaluates whether the evaluation by the user is a favorable high satisfaction session. Then, the information providing apparatus 10 causes the model to learn the features of the high satisfaction session. Further, the information providing apparatus 10 evaluates whether or not the evaluation by the user is an unfavorable low satisfaction session. Then, the information providing apparatus 10 causes the model to learn the features of the low satisfaction session.

  In addition, the information providing apparatus 10 specifies an index utterance from the history of the interactive service, and extracts a predetermined number of responses and utterances preceding the specified index utterance as learning data. Then, the information providing apparatus 10 causes the model to learn the features of the extracted learning data. For example, the information providing apparatus 10 extracts, as a favorable dialogue example, responses and utterances a predetermined number of times earlier than a favorable index utterance indicating that the user's evaluation of the dialogue service is favorable, and uses the dialogue service. The response and the utterance a predetermined number of times ago are extracted as unfavorable dialogue examples from the unfavorable index utterance indicating that the evaluation of the user is unfavorable, and the user's evaluation of the dialogue service is favorable or unfavorable A predetermined number of previous responses and utterances from the neutral utterance indicating that it is not a target are extracted as a neutral dialogue example. Then, the information providing apparatus 10 causes the model to learn the features of the extracted favorable example, the unfavorable example, and the neutral example.

  Further, the information providing apparatus 10 evaluates the session based on whether the session includes an index utterance and a predetermined response indicating an apology. As a result of the various processes described above, the information providing apparatus 10 can learn a model that appropriately evaluates a session from various viewpoints. As a result, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  In addition, the information providing apparatus 10 estimates a future use mode of the interactive service by the user based on the evaluation result of the session including the utterance of the user. For example, the information providing apparatus 10 calculates, as the evaluation, a favorable score indicating how favorable the evaluation by the user is, or an unfavorable score indicating how unfavorable the evaluation by the user is. Then, based on a ratio between the favorable score and the non-favorable score, a future use mode of the interactive service by the user is estimated. For this reason, the information providing apparatus 10 can appropriately estimate the usage mode of the user.

  Further, the information providing apparatus 10 extracts, as an index utterance, an utterance whose appearance frequency satisfies a predetermined condition from a session whose evaluation satisfies a predetermined condition. For example, the information providing apparatus 10 evaluates whether the evaluation of the session by the user is favorable or unfavorable, and determines the frequency of appearance in the session in which the evaluation by the user is favorable, An utterance that satisfies a predetermined condition with respect to a frequency of appearance in a session evaluated as unfavorable by the user is extracted as an index utterance. For this reason, the information providing apparatus 10 can appropriately extract a new index utterance.

[7-4. Session evaluation based on distribution image)
In addition, the information providing apparatus 10 acquires a session including the utterance of the user in the interactive service and a response to the utterance. In addition, the information providing apparatus 10 generates an image indicating whether the characters of the utterance included in the session are the same as each other. Then, the information providing device 10 evaluates the acquired session based on the generated image.

  Here, when the same or similar utterance is repeatedly included in the session, the above-described image includes a predetermined pattern. Then, when it is estimated that a predetermined pattern is included in the image, it can be estimated that the same or similar utterance is repeatedly included in the session. In this way, a session in which the same or similar utterance is repeatedly included is estimated to have a low evaluation by the user. For this reason, the information providing apparatus 10 can appropriately evaluate the acquired session based on the generated image.

  Further, the information providing apparatus 10 generates, as an image, an image indicating the distribution of the same character in the session. For example, when the information providing apparatus 10 associates a character string of an utterance included in a session with each of a vertical region and a horizontal region of an image, the same character corresponds to the vertical direction and the horizontal direction. An image in which a predetermined color is applied to the attached area is generated. Then, the information providing apparatus 10 determines whether the same or similar character string repeatedly appears in the session based on the image, and evaluates the session based on the determination result. For example, when the same or similar character string repeatedly appears in a session, the information providing apparatus 10 gives an evaluation that the evaluation by the user is unfavorable to the session. For this reason, the information providing apparatus 10 can appropriately evaluate the session.

  In addition, the information providing apparatus 10 causes the model to learn features of an image generated from a session in which the same or similar character strings repeatedly appear. Then, the information providing apparatus 10 evaluates the session from which the image is based on the image using the model on which the learning has been performed. Here, it is known that a model that has learned images is easier to improve determination accuracy than a model that has learned characters. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  Further, the information providing apparatus 10 selects one of the utterances from the history in the interactive service, and generates an utterance group including a plurality of utterances that are the same as or similar to the selected utterance. Then, the information providing apparatus 10 generates an image from the generated utterance group, and causes the model to learn features of the generated image. For this reason, the information providing apparatus 10 can improve the accuracy of the model as a result of increasing the learning data of the model.

[7-5. Session evaluation based on repeated speech)
In addition, the information providing apparatus 10 acquires a session including the utterance of the user in the interactive service and a response to the utterance. Then, the information providing apparatus 10 evaluates the acquired session based on the repetition of the character string of the utterance included in the session. For this reason, the information providing apparatus 10 can accurately evaluate the session without manual intervention.

  For example, the information providing apparatus 10 extracts a pair of utterances in which the similarity of the character string exceeds a predetermined threshold from the session, and evaluates the session based on the number of the extracted pairs. The information providing apparatus 10 extracts a pair of utterances whose semantic similarity exceeds a predetermined threshold from the session, and evaluates the session based on the number of the extracted pairs. If the number of pairs extracted from the session exceeds a predetermined threshold, the information providing apparatus 10 gives the session an evaluation indicating that the evaluation by the user is unfavorable. For this reason, the information providing apparatus 10 can appropriately evaluate the session.

  In addition, the information providing apparatus 10 evaluates the session using a model that has learned features of the session in which the same or similar character strings repeatedly appear. For example, the information providing apparatus 10 selects one of the utterances from the history of the interactive service, and generates an utterance group including a plurality of utterances that are the same as or similar to the selected utterance. Further, the information providing apparatus 10 causes the model to learn the features of the generated utterance group. Then, the information providing apparatus 10 evaluates the session by using the learned model.

  In addition, the information providing apparatus 10 generates a first utterance group including a plurality of utterances that are the same as or similar to the selected utterance, selects a plurality of different utterances from the history in the interactive service, and generates a second utterance including the selected utterance. An utterance group is generated, and the model learns features of the first utterance group and features of the second utterance group. In addition, the information providing apparatus 10 learns a feature of the group of utterances by learning the model to determine whether or not the evaluation of the input session by the user is unfavorable. Therefore, the information providing apparatus 10 can accurately evaluate the session.

  In addition, the information providing apparatus 10 extracts, from a session whose evaluation satisfies a predetermined condition, an utterance whose appearance frequency satisfies a predetermined condition, as an index utterance serving as an index that is a user's evaluation of the interactive service. For this reason, the information providing apparatus 10 can automatically extract a new index utterance.

[7-6. Reinforcement learning with speech as reward)
In addition, the information providing apparatus 10 obtains an index utterance that is an index for evaluation of the interactive service among utterances for an interactive model that realizes an interactive service by generating a response to the input utterance, and obtains the obtained index. By setting a reward based on the utterance, reinforcement learning of the dialog model is performed. For this reason, the information providing apparatus 10 can realize the learning of the dialog model for realizing the dialog service without any manual operation, and can improve the accuracy of the dialog model.

  In addition, the information providing apparatus 10 outputs a favorable index utterance indicating that the evaluation of the user for the interactive service is favorable, or an unfavorable index indicating that the evaluation of the user for the interactive service is unfavorable. When a utterance is acquired and a favorable index utterance is acquired, a positive reward is set for the dialogue model, and when an unfavorable index utterance is acquired, a negative reward is set for the dialogue model I do. For this reason, the information providing apparatus 10 can appropriately realize the reinforcement learning of the conversation model.

  Further, the information providing apparatus 10 acquires the index utterance output by the utterance model that outputs the index utterance in response to the response generated by the interaction model. For example, the information providing apparatus 10 acquires, as the utterance model, the index utterance output by the model that has learned the features of the history of the interaction between the user and the interaction model in the interaction service. For this reason, the information providing apparatus 10 can realize the reinforcement learning of the dialogue model without the history of the session, for example.

[7-7. Extraction of index utterance based on co-occurrence]
Further, the information providing apparatus 10 specifies, from the history of the utterance of the user in the interactive service and the response to the utterance, an utterance serving as an index of the evaluation of the user for the interactive service, which is a preset index utterance. . Then, the information providing apparatus 10 extracts a new index utterance from the utterances included in the history based on the co-occurrence with the specified index utterance. As described above, such an index utterance can be used for evaluating a session. As a result, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  Further, the information providing apparatus 10 specifies a favorable index utterance indicating that the evaluation of the user for the interactive service is favorable, and generates a new utterance whose co-occurrence with the favorable index utterance exceeds a predetermined threshold. It is extracted as a favorable index utterance. The information providing apparatus 10 also specifies an unfavorable index utterance indicating that the user's evaluation of the dialogue service is unfavorable, and the utterance whose co-occurrence with the unfavorable index utterance exceeds a predetermined threshold value. Is extracted as a new unfavorable index utterance.

  In addition, the information providing apparatus 10 determines that the number of times that the favorable index utterance indicating that the evaluation of the user for the interactive service is favorable is included in the same session, and that the evaluation of the user for the interactive service is unfavorable. The number of times that the unfavorable index utterance indicating the presence is included in the same session is counted for each user's utterance, and the utterance in which the ratio of the counted times satisfies a predetermined condition is extracted as a new index utterance. .

  Also, the information providing apparatus 10 generates a co-occurrence network connecting utterances or responses having co-occurrence from the history of the utterance and the response, specifies an index utterance from the co-occurrence network, An utterance whose distance from the utterance satisfies a predetermined condition is extracted as a new index utterance. For example, the information providing apparatus 10 specifies, from the co-occurrence network, a favorable index utterance indicating that the evaluation of the user for the interactive service is favorable, and the distance from the favorable index utterance on the co-occurrence network is a predetermined distance. An utterance that is equal to or less than the threshold is extracted as a new favorable index utterance. In addition, the information providing apparatus 10 may include a co-occurrence network with a non-favorable index utterance indicating that the distance from the favorable index utterance is equal to or less than a predetermined threshold value and that the evaluation of the user for the interactive service is unfavorable. An utterance in which the distance above is equal to or greater than a predetermined threshold is extracted as a new favorable index utterance.

  Further, the information providing apparatus 10 specifies an unfavorable index utterance indicating that the evaluation of the user for the interactive service is unfavorable from the co-occurrence network, and determines a distance from the unfavorable index utterance on the co-occurrence network. Are extracted as new unfavorable index utterances. For example, the information providing apparatus 10 may be configured such that the distance from the unfavorable index utterance is equal to or less than a predetermined threshold value, and the co-occurrence network with the favorable index utterance indicating that the evaluation of the user for the interactive service is favorable. Is extracted as a new unfavorable index utterance. As a result of these processes, the information providing device 10 can accurately extract the index utterance indicating the impression of the user.

  Further, the information providing apparatus 10 evaluates a session including an utterance and a response in the interactive service based on whether or not the extracted index utterance is included. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

[7-8. Session evaluation using dialogue examples)
In addition, the information providing apparatus 10 extracts a series of dialogues corresponding to an index utterance that is an index of the user's evaluation of the interactive service from the history of the interactive service. That is, the information providing apparatus 10 extracts various examples of dialogue. For example, the information providing apparatus 10 extracts a predetermined number of utterances and responses preceding the index utterance as a dialog example. Then, the information providing apparatus 10 learns a model for estimating a user's evaluation of the input dialogue by using the extracted dialogue.

  As described above, the information providing apparatus 10 extracts various dialogue examples, not sessions, and causes the model to learn features of the extracted dialogue examples. Extraction of such a dialog example can be realized more easily than extraction of a session if there are various index utterances. As a result, the information providing apparatus 10 can easily realize the session evaluation.

  In addition, the information providing apparatus 10 extracts a predetermined number of utterances and responses from the favorable index utterance indicating that the evaluation of the user for the interactive service is favorable, and uses the extracted utterances and responses. Model learning is performed as learning data for dialogues that are favorably evaluated by the user. Further, the information providing apparatus 10 extracts a predetermined number of utterances and responses before the unfavorable index utterance indicating that the evaluation of the user for the interactive service is unfavorable, and extracts the extracted utterances and responses. In addition, the learning of the model is performed as learning data of the dialogue that the evaluation by the user is unfavorable. In addition, the information providing apparatus 10 extracts a predetermined number of utterances and responses prior to the neutral index utterance indicating that the evaluation of the user for the interactive service is neutral, and outputs the extracted utterances and responses to the user. The model is learned as learning data for dialogues in which the evaluation is neutral. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session based on the model.

  Further, the information providing device 10 acquires a session including a user's utterance and a response in the interactive service. Then, the information providing apparatus 10 evaluates the acquired session by using the learned model. For this reason, the information providing apparatus 10 can implement session evaluation.

  The information providing apparatus 10 extracts, as an index utterance, an utterance whose appearance frequency satisfies a predetermined condition from a session whose evaluation satisfies a predetermined condition. For example, the information providing apparatus 10 evaluates whether the evaluation of the session by the user is favorable or unfavorable, and determines the frequency of appearance in the session in which the evaluation by the user is favorable, An utterance that satisfies a predetermined condition with respect to a frequency of appearance in a session evaluated as unfavorable by the user is extracted as an index utterance. For this reason, the information providing apparatus 10 can automatically extract a new index utterance.

[7-9. Session evaluation using multiple models]
Further, the information providing device 10 acquires a session including a user's utterance and a response in the interactive service. Then, the information providing apparatus 10 evaluates the session using evaluation results of a plurality of models that evaluate the session based on different types of characteristics among a plurality of characteristics that are indexes of the user's evaluation of the session. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  Further, the information providing apparatus 10 uses at least the evaluation result of the model for evaluating the session based on whether the same or similar utterance is repeatedly included in the utterance included in the session. Further, the information providing apparatus 10 uses at least the evaluation result of the model for evaluating the session based on whether or not the response included in the session includes a predetermined response indicating an apology. Further, the information providing apparatus 10 determines the session based on whether or not the utterance included in the session includes an unfavorable index utterance indicating that the user is unfavorably evaluating the interactive service. At least the evaluation result of the model to be evaluated is used. Further, the information providing apparatus 10 evaluates the session based on whether or not the utterance included in the session includes a favorable index utterance indicating that the user is favorably evaluating the interactive service. Use at least the evaluation results of the model. For this reason, the information providing apparatus 10 can improve the evaluation accuracy of the session.

  In addition, the information providing apparatus 10 apologizes to the evaluation value output by the model for evaluating the session based on whether the same or similar utterance is repeatedly included in the utterance included in the session and the response included in the session. The session is evaluated based on the value of the sum with the evaluation value output by the model that evaluates the session based on whether or not the specified response is included. For this reason, the information providing apparatus 10 can evaluate a session using a plurality of models that can evaluate the session without manual intervention.

  In addition, the information providing apparatus 10 apologizes to the evaluation value output by the model for evaluating the session based on whether the same or similar utterance is repeatedly included in the utterance included in the session and the response included in the session. The evaluation value output by the model for evaluating the session based on whether or not the predetermined response is included is included in the utterance included in the session. From the sum of the evaluation values output by the model that evaluates the session based on whether or not the utterances included in the session are included, it is determined that the user is favorably evaluating the dialogue service for the utterance included in the session. The session is evaluated based on a value obtained by subtracting the evaluation value output from the model for evaluating the session based on whether or not the indicated favorable index utterance is included. For example, the information providing apparatus 10 evaluates the session according to whether the value is a positive value or a negative value. Therefore, the information providing apparatus 10 can accurately evaluate the session.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are exemplifications, and various modifications based on the knowledge of those skilled in the art, including the aspects described in the section of disclosure of the invention, The invention can be implemented in other modified forms.

  Further, the above-mentioned “section (section, module, unit)” can be read as “means”, “circuit”, or the like. For example, the extraction unit can be replaced with an extraction unit or an extraction circuit.

DESCRIPTION OF SYMBOLS 10 Information provision apparatus 20 Communication part 30 Storage part 31 Session log database 32 Session evaluation database 33 Index utterance database 34 Engagement database 35 Model database 40 Control part 41 Session acquisition part 42 Usage mode identification part 50 Index utterance extraction processing part 51 Index utterance identification Unit 52 utterance extraction unit 53 network generation unit 60 session evaluation processing unit 61 learning data extraction unit 62 learning unit 63 image generation unit 64 utterance group generation unit 65 evaluation unit 70 engagement prediction processing unit 71 estimation unit 80 reinforcement learning processing unit 81 index utterance Acquisition unit 82 Reward setting unit 83 Reinforcement learning unit 100 Terminal device 200 Interactive device

Claims (6)

An acquisition unit that acquires an index utterance that is an index of evaluation of the dialog service, among utterances of a dialog model that realizes a dialog service by generating a response to the input utterance;
A learning unit configured to perform a reinforcement learning of the interactive model by setting a reward based on the index utterance acquired by the acquiring unit. The acquisition unit is a favorable index utterance indicating that the evaluation of the user for the interactive service is favorable, or an unfavorable index utterance indicating that the evaluation of the user for the interactive service is unfavorable. And get
The learning unit sets a positive reward for the dialogue model when the favorable index utterance is obtained, and sets a negative reward for the dialogue model when the non-favorable index utterance is obtained. The learning device according to claim 1, wherein a reward is set. The learning device according to claim 1, wherein the obtaining unit obtains an index utterance output by an utterance model that outputs the index utterance in response to a response generated by the interaction model. The said acquisition part acquires the index utterance which the model which learned the characteristic which the history of the conversation between the user and the said conversation model in the said conversation service has in the said conversation service was output. The said utterance model, The said utterance model. Learning device. A learning method performed by the learning device,
Acquisition step of acquiring an index utterance that is an index of evaluation for the dialog service, among utterances for a dialog model that realizes a dialog service by generating a response to the input utterance,
A learning step of performing a reinforcement learning of the dialog model by setting a reward based on the index utterance acquired in the acquiring step. Acquisition procedure of acquiring an index utterance that is an index of evaluation for the dialog service, among utterances for a dialog model that realizes a dialog service by generating a response to the input utterance,
A learning program for causing a computer to execute a learning procedure of performing reinforcement learning of the dialog model by setting a reward based on the index utterance acquired by the acquisition procedure.

Images