JP5545489B2 - Learning system, simulation apparatus, and learning method - Google Patents

Description

  The present invention relates to a learning system that learns information to be used when determining a sentence to be output by an interactive device that supports a user's decision making.

  A consultation-type spoken dialogue system is considered to be a kind of decision support system. There are many cases of decision support in the field of operations research, and a representative method is the hierarchical analysis method (AHP method) (Non-patent Document 1). In the AHP method, the problem elements are divided into three layers, “Final Goal”, “Evaluation Criteria”, and “Alternative”, and the optimal weight is determined by estimating the local weight (importance) of each user's evaluation criteria. Do.

  It is conceivable to apply the AHP method as it is to a dialogue apparatus.

  Conventionally, there has been an interactive device that retrieves and presents information corresponding to a key input by a user.

Saaty, T., The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation, Mcgraw-Hill (1980)

  However, in the conventional spoken dialogue system, a great amount of labor is required to construct information necessary for the dialogue system that performs dialogue with the user to output a sentence.

In addition, when the above AHP method is applied to a dialogue apparatus as it is, a dialogue apparatus that can achieve the purpose of supporting the user's decision making cannot be configured. That is, in order to make an optimal decision for the user, first, the weight P _user = (p ₁ , p ₂ ,..., P _M ) for the evaluation criterion, and the local from the viewpoint of each evaluation criterion for each alternative The weights V _user = (v ₁₁ , v ₁₂ ,..., V _1M ,..., V _nm ) are determined. The determination of the optimal candidate is realized by selecting an alternative k that maximizes the priority “Σ _{m = 1} ^M p _m v _km ”. In the general AHP method, the above weights are determined by paired comparisons with evaluation criteria and alternatives.

  However, candidates and domain knowledge that can be presented by the device to the user are often information that can be known for the first time through the dialogue, and are rarely known at the beginning of the dialogue. In many cases, the dialogue apparatus handles a large number of candidates (alternatives) and evaluation criteria. Under such circumstances, it is not realistic to perform a paired comparison because a large number of exchanges are required.

  In addition, an interactive device that only presents information corresponding to a key input by the user cannot achieve the purpose of supporting the user's decision making.

  Therefore, an object of the present invention is to automatically learn information to be used when determining a sentence to be output by an interactive device.

  In addition, the present invention provides an interactive apparatus that can appropriately support the user's decision making by performing a conversation while dynamically changing information related to the user's knowledge and preferences according to the progress of the conversation with the user. The purpose is to do.

  The learning system according to the first aspect of the present invention is a learning system that includes an interactive device that simulates an interaction with respect to a spot and a simulation device, and learns a weight vector that is used when determining a sentence that the interactive device outputs. In the learning system, the dialogue apparatus includes spot information including a spot, one or more determinants that are factors for determining the spot, and an evaluation value indicating an evaluation of each of the one or more determinants of the spot. Two or more information recommendation methods having two or more stored knowledge bases, a method identifier for identifying an information recommendation method, evaluation information of the information recommendation method, and a weight vector indicating the weight of each element constituting the evaluation information Is an information recommendation method storage unit storing information, information indicating a user's state, and information indicating a user's preference for one or more determination factors A user state information storage unit storing user state information having a good vector and a knowledge vector indicating the user's knowledge for each of one or more determinants, and a user who is a sentence pattern input by the user from the simulation device A user sentence type identifier for identifying a sentence type, or a user sentence information identifier and a user input information receiving unit for receiving user input information having one or more determinants or one or more pieces of information of one or more spots; and information recommendation A score calculation unit that calculates two or more scores for each of the two or more information recommendation methods using the evaluation information and the weight vector of each of the two or more information recommendation methods stored in the method storage unit, and the user state information And a method identifier for identifying one information recommendation method using two or more scores calculated by the score calculation unit, or a method identifier A dialogue sentence information constituting unit comprising dialogue sentence information having a child, one or more determinants or one or more information of one or more spots, and dialogue sentence information constituted by the dialogue sentence information constituting unit are simulated. At least one spot or one or more from one or more pieces of information out of the dialogue sentence output unit sent to the device and the user input information accepted by the user input information accepting unit or the dialogue sentence information outputted by the dialogue sentence output unit And a user state information update unit that updates user state information in the user state information storage unit using one or more spots or one or more determinants, and the score calculation unit includes information recommendation Using the evaluation information and weight vector of each of the two or more information recommendation methods stored in the method storage unit and the user state information updated by the user state information update unit, two or less Two or more scores are calculated for each of the above information recommendation methods, and the simulation apparatus is information on the probability of selection of conversation probability information, which is information on the probability between each information recommendation method and each user sentence type, and information on the probability that a determinant is selected. Dialog information storage unit that can store determinant probability information and spot probability information that is information regarding the probability that a spot is selected, and a user preference vector storage unit that can store a user preference vector that is a vector indicating user preference And a dialogue sentence information receiving unit that accepts dialogue sentence information from the dialogue device, a user sentence type that determines a user sentence type and acquires a user sentence type identifier using a method identifier and dialogue probability information that the dialogue sentence information has Determining unit and one or more pieces of determinant probability information or spot probability information, or determinant probability information or spot One or more determinants or one or more spots are acquired using one or more of the rate information and one or more determinants included in the dialogue information or one or more information of one or more spots. User input information for sending user input information having a determination factor acquisition unit, user sentence type identifier, or user sentence type identifier and one or more information of one or more determination factors or one or more spots to the interactive device The sending unit, the user preference vector, and one or more evaluation values indicating the evaluation of one or more determinants of the spot included in the user input information are acquired, and the degree of coincidence between the user preference vector and the one or more evaluation values And a reward calculation unit that calculates a reward for selecting a user sentence type identified by the user sentence type identifier using the degree of match, and corresponds to the technique identifier of the dialogue apparatus using the reward A Ruomomi vector, a learning system comprising a learning unit for updating the weight vector information recommendation method storage unit interactive device.

  With this configuration, it is possible to automatically construct a weight vector necessary for an interactive apparatus that interacts with the user to output a sentence.

  Further, the learning system of the second aspect of the present invention is directed to the first aspect, wherein the reward calculation unit uses one or more evaluation values and a user preference vector when a spot is randomly determined using spot probability information. The degree of coincidence between the random selection coincidence calculating means for calculating the expected value of the degree of coincidence, the user preference vector, and one or more evaluation values indicating the evaluation of one or more determinants of the spot included in the user input information. The spot included in the user input information is calculated using the expected value of the degree of match calculated by the selected spot match degree calculating means, the random selection match value calculating means, and the match degree calculated by the selected spot match degree calculating means. It is a learning system provided with the reward calculation means which calculates the reward of having been selected.

  With this configuration, it is possible to automatically construct a weight vector necessary for an interactive apparatus that interacts with the user to output a sentence.

  The interactive apparatus according to the third aspect of the present invention is an interactive apparatus that performs a dialog with a user regarding a spot, and includes a spot, one or more determination factors that are factors for determining the spot, and one or more determinations of the spot. A knowledge base that stores two or more spot information having an evaluation value indicating the evaluation of the factor, sentence pattern information that is a sentence output by the interactive device or information indicating a sentence pattern output by the interactive device, and a sentence An information recommendation method storage unit storing two or more information recommendation methods having evaluation information of sentence pattern information used when selecting pattern information, and information indicating a user state, each of one or more A user state information having a preference vector which is information indicating a user's preference for a determinant and a knowledge vector indicating a user's knowledge for one or more determinants is stored. The state information storage unit, a reception unit that receives a sentence input by the user, and the user state information are applied to evaluation information included in each of two or more information recommendation methods stored in the information recommendation method storage unit. Using the score calculation unit that calculates two or more scores for each of the information recommendation methods and the two or more scores calculated by the score calculation unit, the sentence pattern information possessed by one information recommendation method is acquired, and the sentence pattern information is At least one of at least one of a sentence composing part that constitutes a sentence, a sentence output part that outputs a sentence constituted by the sentence composing part, a sentence accepted by the accepting part, or a sentence output by the sentence output part A user status information update unit that acquires the above spot or one or more determinants and updates the user status information in the user status information storage unit using the one or more spots or the one or more determinants, The core calculation unit applies the user state information updated by the user state information update unit to evaluation information included in each of the two or more information recommendation methods stored in the information recommendation method storage unit, and each of the two or more information recommendation methods Is an interactive device for calculating a score of 2 or more for

  With this configuration, it is possible to appropriately support the user's decision making by performing the dialogue while dynamically changing the information about the user's knowledge and preferences according to the progress of the dialogue with the user.

  Further, in the dialog device according to the fourth aspect of the invention, in contrast to the third aspect, the sentence composition unit has one information recommendation method corresponding to the largest score among two or more scores calculated by the score calculation unit. A sentence pattern information acquisition unit that acquires sentence pattern information, one or more variables included in the sentence pattern information acquired by the sentence pattern information acquisition unit, and a sentence output unit immediately preceding a spot or determination factor corresponding to the variable Variable value acquisition means for acquiring from one or more of the sentences output to the sentence or the sentence received by the reception unit immediately before, and the variable value acquisition at the location of the variable in the sentence pattern information acquired by the sentence pattern information acquisition means And a sentence constructing means for constructing a sentence by inserting the term obtained by the means.

  With this configuration, it is possible to appropriately support the user's decision making by performing the dialogue while dynamically changing the information about the user's knowledge and preferences according to the progress of the dialogue with the user.

  Further, in the dialog device according to the fifth invention, in contrast to the fourth invention, the variable value acquisition means acquires one or more variables included in the sentence pattern information acquired by the sentence pattern information acquisition means, One or more spots or one or more determinants to be the corresponding candidates are acquired from one or more sentences of the sentence output immediately before by the sentence output unit or the sentence received immediately before by the reception unit and become candidates. An interactive device that selects a spot or determinant corresponding to a variable from one or more spots or one or more determinants using a candidate evaluation value of one or more spots or one or more determinants. It is.

  With this configuration, it is possible to appropriately support the user's decision making by performing the dialogue while dynamically changing the information about the user's knowledge and preferences according to the progress of the dialogue with the user.

  Further, according to the sixth aspect of the present invention, in the third to fifth aspects of the invention, the user status information update unit obtains at least one or more determinants from the sentence received by the reception unit. A term acquisition unit, a device presentation term acquisition unit that acquires at least one or more determinants from one or more of the sentences output by the sentence output unit, and one or more determinants acquired by the user presentation term acquisition unit Preference vector update means for updating so as to increase the value of the element of the preference vector with respect to, and knowledge vector update for updating so as to increase the value of the element of the knowledge vector for one or more determinants acquired by the device presentation term acquisition means An interactive device.

  With this configuration, it is possible to appropriately support the user's decision making by performing the dialogue while dynamically changing the information about the user's knowledge and preferences according to the progress of the dialogue with the user.

  Further, according to the seventh aspect of the present invention, in any one of the third to sixth aspects, the accepting unit recognizes the speech inputted by the user and recognizes the speech and converts it into a character string. And a sentence output unit is an interactive device that outputs a sentence constituted by the sentence constructing unit.

  With this configuration, it is possible to appropriately support the user's decision making by performing the voice conversation while dynamically changing the information related to the user's knowledge and preferences according to the progress of the voice conversation with the user.

  According to the learning system of the present invention, it is possible to automatically construct a weight vector necessary for a dialog device that interacts with a user to output a sentence.

Conceptual diagram of the learning system in the first embodiment Block diagram of the dialogue device that constitutes the learning system Block diagram of a simulation device that constitutes the learning system A flowchart for explaining the operation of the dialogue apparatus A flowchart for explaining the operation of the score calculation process A flowchart for explaining the operation of the configuration processing A flowchart for explaining the operation of the user status information update process A flowchart for explaining the operation of the user status information update process Flow chart explaining operation of the simulation apparatus A flowchart for explaining the operation of the reward calculation process The figure which shows an example of the spot information management table The figure which shows an example of the information recommendation method management table The figure which shows the dialogue probability information management table A diagram showing the relationship between the number of simulated dialogues and the reward after each turn Figure showing the comparison results of rewards with the baseline method Figure showing the comparison results of rewards with the baseline method Block diagram of interactive apparatus in embodiment 2 A flowchart for explaining the operation of the dialogue apparatus A flowchart for explaining the operation of the score calculation process Flowchart explaining operation of same sentence composition processing A flowchart for explaining the operation of the recommended sentence acquisition process A flowchart for explaining the operation of the user status information update process Conceptual diagram of the dialogue device Diagram explaining the flow of the dialogue Overview of the computer system Block diagram of the computer system

Hereinafter, embodiments of a learning system and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.
(Embodiment 1)

  A learning system that automatically generates a dialog between a simulation device and a dialog device, determines a reward, and updates a weight vector in the present embodiment will be described. The weight vector is a set of weights for each element of the user state information.

  The interactive device is a device in which a consultation-type interactive device in which a user selects a candidate while receiving information presentation or information recommendation from the device is compatible with a simulation device.

  In the present embodiment, the dialogue device is a dialogue device that implements a consultation-type dialogue model for selecting a candidate suitable for the user from among a plurality of candidates (here, mainly referred to as spots).

  FIG. 1 is a conceptual diagram of a learning system in the present embodiment. The learning system includes an interactive device 1 and a simulation device 2. The interactive device 1 and the simulation device 2 update the weight vector of the interactive device 1 while exchanging interactive data (interactive sentence information and user input information described later) that is data indicating the interactive operation. Such update is referred to as learning as appropriate. In FIG. 1, the interactive device 1 and the simulation device 2 are two devices, but may be a single device.

  FIG. 2 is a block diagram showing an internal structure of the interactive apparatus 1 that constitutes the learning system according to the present embodiment.

  The dialogue apparatus 1 includes a knowledge base 11, an information recommendation method storage unit 12, a user state information storage unit 13, a user input information reception unit 14, a score calculation unit 15, a dialogue sentence information configuration unit 16, a dialogue sentence output unit 17, a user state. An information update unit 18 is provided.

  The dialogue sentence information configuration unit 16 includes a technique identifier acquisition unit 161, a variable value acquisition unit 162, and a dialogue statement information configuration unit 163.

  The user status information update unit 18 includes a user presentation term acquisition unit 181, a device presentation term acquisition unit 182, a preference vector update unit 183, and a knowledge vector update unit 184.

  FIG. 3 is a block diagram showing an internal structure of the simulation apparatus 2 constituting the learning system in the present embodiment.

  The simulation apparatus 2 includes a knowledge base 11, a dialogue information storage unit 21, a user preference vector storage unit 22, a dialogue sentence information reception unit 23, a user sentence type determination unit 24, a determination factor acquisition unit 25, a user input information transmission unit 26, A reward calculation unit 27 and a learning unit 28 are provided.

  The reward calculation unit 27 includes a random selection match value calculation means 271, a selected spot match degree calculation means 272, and a reward calculation means 273.

  Needless to say, the dialogue apparatus 1 and the simulation apparatus 2 may share the knowledge base 11.

  The knowledge base 11 stores two or more pieces of spot information. The spot information is information having a spot, one or more determinants, and an evaluation value. A spot has the same meaning as information for identifying a spot, and is, for example, a spot name. A spot is a place where a user usually visits, such as a sightseeing spot, a restaurant, or a store. However, anything can be used as long as it is a subject of dialogue. For example, a spot may be a company, a group of people, a concept, etc., and is widely understood. The determinant is a factor for determining a spot. The determinant can also be said to be an attribute of a spot. The determinant can be said to be the viewpoint of spot evaluation. When the spot is a sightseeing spot, the determining factors are, for example, “famous for gardens”, “not crowded”, “world heritage”, “scenic scenery”, and the like. The evaluation value is information indicating evaluation of one or more determinants of the spot. The evaluation value may be binary (two stages) such as “1 (◯), 0 (×)”, or may be multistage such as an integer of 1 to 5. The evaluation value is an evaluation value indicating whether or not a spot is a determining factor. The spot information may have a description corresponding to the spot and the determining factor. When spot information has an explanatory note, it usually has an explanatory note for each spot and one determining factor.

  The information recommendation method storage unit 12 stores two or more information recommendation methods. Here, the information recommendation method has a method identifier, sentence pattern information, evaluation information, and a weight vector. The method identifier is information for identifying the information recommendation method. The sentence pattern information is information indicating a sentence output from the interactive device 1 or a sentence pattern output from the interactive device 1. The evaluation information is information for evaluating the sentence pattern information used when selecting the sentence pattern information. It can be said that the evaluation information is information for evaluating the information recommendation method. The evaluation information is, for example, a vector having 29 elements as will be described later. The weight vector is a vector indicating the weight of each element constituting the evaluation information. In the information recommendation method, sentence pattern information is not essential. The information recommendation method may include a method identifier, evaluation information, and a weight vector.

  In addition, the information recommendation method includes the following six methods, for example. (1) Information recommendation regarding current topic spot (method 1), (2) Information recommendation regarding current topic determinants (method 2), (3) Open prompt (method 3), (4) Presentation of determinants 1 ( Method 4), (5) Presentation of decision factor 2 (Method 5), (6) Spot recommendation (Method 6) that the user is estimated to be interested in. The technique 1 is a current topic spot information recommendation technique having a technique identifier “1”, sentence pattern information for recommending a detailed explanation, evaluation information of sentence pattern information, and a weight vector for the spot just described. . Method 2 is current topic determinant information having method identifier “2”, sentence pattern information that recommends another tourist spot related to the determinant just described, sentence pattern information evaluation information, and a weight vector. This is a recommendation method. Method 3 is an open prompt information recommendation method that includes method identifier “3”, sentence pattern information indicating an open prompt, evaluation information of sentence pattern information, and a weight vector without particularly recommending information. Method 4 is a method for presenting determinants that can be explained by the dialogue apparatus 1. When the method 4 is selected, the user selects from the determinants having a low knowledge of the user estimated by the dialogue apparatus 1. Method 5 is a method for presenting determinants that can be explained by the dialogue apparatus 1, as in Method 4. When the method 5 is selected, the user selects from the determination factors having high knowledge of the user estimated by the dialogue apparatus 1. Method 6 is a method of selecting a spot k that the user is most interested in based on the user's interest estimated by the interactive apparatus 1 and presenting the spot k to the user.

  The user state information storage unit 13 is information indicating a user's state, a preference vector that is information indicating a user's preference with respect to one or more determinants, and a knowledge vector indicating user's knowledge with respect to one or more determinants The user status information including The user state information may include an attribute vector that is information indicating one or more attribute values of the user. User attribute values include, for example, sex (male or female), age group (10's, 20's, 30's, baby boom junior, etc.), occupation, hometown, supporting political party, and the like.

It is preferable that the user state information includes local weight information that is information on one or more local weights indicating the importance of the spot from the viewpoint of the user's determinant. The local weight v _nm for the spot n from the viewpoint of the user's determinant m is, for example, when the interactive apparatus 1 informs the user of the spot evaluation using the information recommendation method 1, the information recommendation method 2, or the information recommendation method 6. “1” shall be taken. This is based on the assumption that the user determines only from the information presented from the interactive device 1. Further, the user status information in the user status information storage unit 13 is dynamically changed as the dialogue progresses.

  Further, it is preferable that the user state information includes dialogue amount information that is information indicating the amount of dialogue between the dialogue device 1 and the simulation device 2 up to the present. Further, the user status information is one or more of information regarding a determinant corresponding to the dialog sentence information sent by the dialog device 1 immediately before or information regarding a determinant corresponding to the user input information received from the simulation device 2 just before. It is preferable that this information is included.

  The user input information receiving unit 14 receives user input information from the simulation apparatus 2. The user input information is information having a user sentence type identifier, or information having a user sentence type identifier and one or more information of one or more determinants or one or more spots. The user sentence type identifier is information for identifying a sentence pattern input by the user.

  The score calculation unit 15 uses the evaluation information and the weight vector included in each of the two or more information recommendation methods stored in the information recommendation method storage unit 12 and the user status information, and outputs 2 for each of the two or more information recommendation methods. The above score is calculated. Further, the score calculation unit 15 uses the evaluation information and the weight vector of each of the two or more information recommendation methods stored in the information recommendation method storage unit 12 and the user state information updated by the user state information update unit 18. Thus, a score of 2 or more for each information recommendation method of 2 or more is calculated.

  The score calculation unit 15 normally calculates the score before the dialog text output unit 17 outputs the dialog text (not necessarily immediately before). In addition, it is suitable for the score calculation part 15 to calculate a score whenever the user input information reception part 14 receives user input information. Here, the score calculation unit 15 is to calculate a score by, for example, an arithmetic expression “score = f (user state information, weight vector)”. For example, f is “score = user state information × weight vector”. That is, the score calculation unit 15 uses the evaluation information managed in association with the sentence pattern information and the user state information that dynamically changes in order to determine the sentence pattern information of the sentence to be output next by the dialogue apparatus 1. Are used to calculate a score for each information recommendation method.

  The dialog text information configuration unit 16 configures the dialog text information using two or more scores calculated by the score calculation unit 15. Normally, the dialog text information configuration unit 16 configures dialog text information having a method identifier for identifying an information recommendation method corresponding to the maximum score calculated by the score calculation unit 15. The dialog sentence information has a technique identifier. It is preferable that the dialog sentence information includes one or more information of one or more determinants or one or more spots. The method identifier is information for identifying one information recommendation method.

  The method identifier acquisition unit 161 acquires, from the information recommendation method storage unit 12, a method identifier that identifies one information recommendation method corresponding to the largest score among the two or more scores calculated by the score calculation unit 15.

  The variable value acquisition unit 162 is configured to obtain one or more spots or one of the dialogue text information output immediately before by the dialog text output unit 17 or the user input information received immediately before by the user input information reception unit 14. Obtain one or more information of one or more determinants.

  The dialogue sentence information configuring unit 163 obtains dialogue sentence information from one or more of the method identifier acquired by the method identifier acquisition unit 161, one or more spots acquired by the variable value acquisition unit 162, or one or more determinants. Configure.

Note that the system action a _sys (ca _sys ) is selected by the score calculation unit 15 and the dialog sentence information configuration unit 16 based on the softmax policy expressed by the following Equation 1. The system action is a technique identifier included in the dialog text information.

In Equation 1, S is user status information. k is a technique identifier. Θ is a set of parameters, and (θ ₁₁ , θ ₁₂ ,..., Θ _1I ,..., Θ _JI ) is calculated from J (number of methods) × I (number of features) parameters. Become. The parameter θ _ji is a weight for the i-th feature amount of the action j, and determines how easily the method j is selected. This θ is an object of learning in the learning system. Note that. For learning, reinforcement learning is preferably used.

  The dialog text output unit 17 sends the dialog text information configured by the dialog text information configuration unit 16 to the simulation apparatus 2.

The user status information update unit 18 is configured to obtain at least one spot or one or more information from one or more pieces of information of the user input information received by the user input information receiving unit 14 or the dialogue sentence output by the dialogue sentence output unit 17. The determination factor is acquired, and the user status information in the user status information storage unit 13 is updated using the one or more spots or the one or more determination factors. Then, the user state information update unit 18 usually sets the user state information so that the values of elements (elements such as preference vectors, knowledge vectors, and attribute vectors) constituting the user state information about the acquired spot or determination factor are increased. Update. The user status information updating unit 18 does not ask what arithmetic expression or algorithm is used to raise the value of the element constituting the user status information regarding the acquired spot or determination factor. The user status information update unit 18 may increase the value of the element constituting the user status information regarding the acquired spot or determination factor by adding a constant, or increase the value by multiplying the constant. It may be good or may be raised by other increasing functions.
Moreover, the user status information update unit 18 is normally updated every time the user input information reception unit 14 receives user input information. However, the user status information update unit 18 may update the dialog sentence output unit 17 every time the dialog sentence is sent.

  The user presentation term acquisition unit 181 acquires at least one or more determination factors from the user input information received by the user input information reception unit 14. The user-presented term acquisition unit 181 may acquire only a positive determinant, or may detect affirmation / negative and acquire a determinant for each category (positive / negative). Further, the user presented term acquisition unit 181 may acquire a focus determination factor.

  The device presentation term acquisition unit 182 acquires at least one or more determinants from the dialogue sentence output by the dialogue sentence output unit 17. The dialogue sentence output by the dialogue sentence output unit 17 has the same meaning as the dialogue sentence formed by the dialogue sentence information configuration unit 16. Further, the device presentation term acquisition unit 182 may acquire only a positive determination factor, or may detect affirmation / negative and acquire a determination factor for each category (positive / negative). In addition, the device presentation term acquisition unit 182 may acquire a focus determination factor.

  The preference vector update unit 183 updates the user state information so as to increase the value of the preference vector element for one or more determination factors acquired by the user presented term acquisition unit 181. Also, the preference vector update unit 183 is a value of an element of the preference vector for one or more determinants that cannot be acquired by the user-presented term acquisition unit 181 among one or more determinants acquired by the device presentation term acquisition unit 182. The user status information is updated so as to lower the value. This is to lower the value of the determinant that is output from the interactive device 1 but not selected by the simulation device 2.

The knowledge vector update unit 184 updates the user state information so as to increase the value of the element of the knowledge vector for one or more determination factors acquired by the device presentation term acquisition unit 182.
The user status information update unit 18 includes attribute vector update means 185 that changes the value of the element of the attribute vector for one or more determination factors acquired by the user presented term acquisition means 181 and updates the user status information. Also good.

  The dialogue information storage unit 21 constituting the simulation apparatus 2 can store dialogue probability information, determinant probability information, and spot probability information. The conversation probability information is information regarding the probability between each information recommendation method and each user sentence type. As the conversation probability information, for example, (method identifier, user sentence type identifier, user sentence type, probability) information has the number of combinations of the technique identifier and the user sentence type identifier. In such a case, the probability is that the simulation apparatus 2 generates a sentence of the type identified by the user sentence type identifier when the sentence corresponding to the technique identified by the technique identifier is sent from the dialogue apparatus (to the conversation apparatus 1). Send) probability. The determinant probability information is information regarding the probability that a determinant is selected. The determination factor probability information is, for example, a set of (determination factor, probability). The determinant probability information is managed for each group having two or more related determinant probability information. The spot probability information is information regarding the probability that a spot is selected. In addition, when the probability that all the spots are selected is the same, the spot probability information is unnecessary. The user sentence type can be said to be a type of user's answer or a type of sentence input by the user. The user's answer and the sentence entered by the user can be said to be information that the simulation apparatus 2 sends to the dialogue apparatus 1 here.

  The user preference vector storage unit 22 can store a user preference vector that is a vector indicating the user's preference. The user preference vector is a set of values indicating the preference for each of two or more determining factors. The value for each determinant may be either “0” or “1”, or may be multi-stage (for example, any integer from 1 to 5).

  The dialog text information receiving unit 23 receives dialog text information from the dialog device 1. The acceptance here is usually information delivery. The dialog sentence information has a technique identifier. It is preferable that the dialog sentence information includes one or more information of one or more determinants or one or more spots.

  The user sentence type determination unit 24 determines the user sentence type using the technique identifier and the conversation probability information included in the dialog sentence information, and acquires the user sentence type identifier. The user sentence type determining unit 24 acquires the probability of pairing with the technique identifier included in the dialog sentence information and the user sentence type identifier from the dialog probability information, and acquires the user sentence type identifier according to the probability.

  The determination factor acquisition unit 25 acquires one or more determination factors or one or more spots. The determining factor acquisition unit 25 acquires one or more determining factors or one or more spots using one or more pieces of information of the determining factor probability information or spot probability information. Further, the determination factor acquisition unit 25 obtains one or more pieces of information of the decision factor probability information or spot probability information and one or more pieces of information of one or more decision factors or one or more spots included in the dialogue sentence information. Used to obtain one or more determinants or one or more spots. For example, when the dialog sentence information includes three spots, the determination factor acquisition unit 25 acquires one spot according to the probabilities of the three spots included in the spot probability information. In addition, when the spot information is not included in the dialogue sentence information, the determining factor acquisition unit 25 preferably acquires one spot according to the probability of the spot probability information.

The user sentence type determination unit 24 and the determining factor parameter acquisition unit 25, speech acts of the user for action ^a _{t sys} system ^ca _{t user,} meaning ^sc _{t user} is acquired. The user sentence type determination unit 24 and the determination factor acquisition unit 25 acquire the user's utterance action and meaning content by using the following Equation 2. Here, the user's speech act is a user sentence type identifier. The semantic content is one or more information of one or more determinants or one or more spots.

That is, Equation 2, speech acts ^ca _{t user} of the user, a conditional probability ^{Pr |} based on _{(ca t user ca t sys)} , are obtained. The conditional probability is the above-described dialog probability information. The meaning content sc ^t _user of the user utterance is determined based on the user's preference under the user's knowledge. The sc is acquired based on whether or not the user is interested (based on corpus statistics) from the determinants known by the user (km = 1).

  The user input information sending unit 26 sends user input information to the dialogue apparatus 1. The user input information has a user sentence type identifier. Further, the user input information may include a user sentence type identifier and one or more determinants or one or more information of one or more spots. The user sentence type identifier here is a user sentence type identifier acquired by the user sentence type determination unit 24. Further, one or more determining factors or one or more spots are information acquired by the determining factor acquisition unit 25.

The reward calculation unit 27 calculates a reward when the spot acquired by the determining factor acquisition unit 25 is selected. The reward calculation unit 27 acquires from the knowledge base 11 one or more evaluation values indicating the evaluation of one or more determination factors of the spot acquired by the determination factor etc. acquisition unit 25. Then, the reward calculation unit 27 reads the user preference vector from the user preference vector storage unit 22. Then, the reward calculation unit 27 calculates the degree of coincidence between the read user preference vector and one or more evaluation values (appropriately referred to as “spot evaluation vectors”) indicating the evaluation of each of the one or more determining factors of the acquired spot. The reward for selecting the user sentence type identified by the user sentence type identifier is calculated using the degree of match. For example, when the user preference vector is (1, 0, 1, 1, 1) and the acquired spot evaluation vector is (1, 1, 0, 1, 1), the matching degree is “3”. The degree of match is, for example, the number of elements that match between the user preference vector and the spot evaluation vector. The reward calculation unit 27 uses a random selection match value calculation unit 271, a selected spot match degree calculation unit 272, and a reward calculation unit 273, which will be described later, and a determination factor acquisition unit 25. It is preferable to compare the case where the spot acquired by is selected and calculate the reward.
The reward calculation unit 27 calculates the reward using, for example, a reward function of the following formula 3.

The reward function is a function for calculating a reward based on a match rate between an attribute (one or more evaluation values) of a spot selected by the user and a user preference (user preference vector). It is assumed that the user selects a spot k having the highest priority (Σ _m k _k , p _k , v _km ) under the knowledge K _user and the local weight V _user in the current dialog state. The reward R is given based on how good the spot k determined by the user is compared to the case where the spot is randomly determined. In Equation 3, M is the number of one or more evaluation values, the number of elements of the user preference vector, and is 29, for example. In Equation 3, N is the number of spots. Further, in Equation 3, _{p m} is a user preference vector. Further, e _{k, m} are each element of the evaluation information of the spot k.

  The random selection match value calculation means 271 uses the spot probability information to calculate an expected value of the degree of match between one or more evaluation values and a user preference vector when a spot is randomly determined. Note that the spot probability information may be the same for all spots. In such a case, “when spots are randomly determined” means “when spots are determined at an equal ratio”.

  The selected spot coincidence degree calculating means 272 calculates the degree of coincidence between the user preference vector and one or more evaluation values indicating the evaluation of one or more determinants of spots included in the user input information. In addition, the selected spot match degree calculation unit 272 calculates a match degree between the user preference vector and one or more evaluation values indicating the evaluation of one or more determination factors of the spot determined by the determination factor acquisition unit 25. Also good. A spot included in the user input information or a spot determined by the determination factor etc. acquiring unit 25 is referred to as a spot of interest here.

  The reward calculation unit 273 selects a spot included in the user input information using the expected value of the degree of match calculated by the random selection match value calculation unit 271 and the match degree calculated by the selected spot match level calculation unit 272. To calculate the reward.

  The learning unit 28 uses the reward to update the weight vector corresponding to the method identifier of the dialog device 1 and the information recommendation method storage unit 12 of the dialog device 1. For example, when the reward is a positive number, the learning unit 28 updates the weight vector corresponding to the method identifier of the dialog device 1 so that the information recommendation method included in the dialog text information is more easily selected. This weight vector is a weight vector of the information recommendation method storage unit 12. Here, updating means that the learning unit 28 may directly rewrite the weight vector in the information recommendation method storage unit 12 or may instruct the dialog device 1 to update. When the interactive device 1 receives an update instruction, the interactive device 1 rewrites the weight vector. The method and degree by which the learning unit 28 updates the weight vector does not matter. Usually, the larger the reward is, the learning unit 28 updates the weight vector corresponding to the technique identifier of the dialogue apparatus 1 according to the magnitude of the reward so that the information recommendation technique included in the dialog sentence information is more easily selected. To do. For example, when the reward is a negative number, the learning unit 28 updates the weight vector corresponding to the technique identifier of the dialogue apparatus 1 so that the information recommendation technique included in the dialogue sentence information is more difficult to be selected. For example, the learning unit 28 updates the weight vector according to a later-described Natural Actor Critic (NAC) algorithm, which is one of natural policy gradient methods. NAC is described in “Otake Hachiya, Masaru Sugiyama: How to make a strong robotic game player, Mainichi Communications (2008).” Since it is a well-known technique, detailed description is omitted. NAC is a method for optimizing policies and is one of natural policy gradient methods. In the policy gradient method, instead of directly estimating the value function for the state S or estimating the action value function Q (S, A), the reward of the dialogue episode obtained by the policy before the update is used. Update policy π directly by natural gradient method to increase.

  The knowledge base 11, the information recommendation method storage unit 12, the user state information storage unit 13, the dialogue information storage unit 21, and the user preference vector storage unit 22 are preferably non-volatile recording media, but are also realized by volatile recording media. Is possible. The process of storing spot information or the like in the knowledge base 11 or the like is not limited. For example, spot information or the like may be stored in the knowledge base 11 or the like via a recording medium, and spot information or the like transmitted via a communication line or the like is stored in the knowledge base 11 or the like. Alternatively, spot information or the like input via the input device may be stored in the knowledge base 11 or the like.

  The user input information receiving unit 14 and the dialog text information receiving unit 23 are realized by, for example, wireless or wired communication means. The user input information receiving unit 14 or the like may be realized by an MPU, a memory, or the like. The reception here may be reception or reception of information by a function or the like.

  The score calculation unit 15, the dialog sentence information configuration unit 16, the user state information update unit 18, the user sentence type determination unit 24, the determination factor acquisition unit 25, the reward calculation unit 27, and the learning unit 28 are usually from an MPU, a memory, or the like. Can be realized. The processing procedure of the score calculation unit 15 or the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The user input information sending unit 26 and the dialogue sentence output unit 17 are realized by, for example, wireless or wired communication means. The user input information sending unit 26 and the like may be realized by an MPU, a memory, or the like. The sending here may be sending or delivery of information to other processes.

  Next, the operation of the learning system will be described. First, the operation of the interactive apparatus 1 will be described using the flowchart of FIG.

  (Step S <b> 401) The dialogue sentence output unit 17 sends the initial dialogue sentence information stored in advance to the simulation apparatus 2.

  (Step S402) The user input information receiving unit 14 determines whether user input information has been received from the simulation apparatus 2. If user input information is accepted, the process goes to step S403, and if user input information is not accepted, the process returns to step S402.

  (Step S403) The user input information receiving unit 14 or a unit (not shown) determines whether or not the user input information received by the user input information receiving unit 14 satisfies an end condition. If the end condition is satisfied, the process ends. If the end condition is not satisfied, the process goes to step S405. The end condition is, for example, a case where the user input information includes a user sentence type identifier corresponding to a predetermined sentence pattern. Examples of the sentence pattern determined in advance include “go to <spot>” and “determine <spot>”.

  (Step S404) The score calculation unit 15 calculates two or more scores for two or more information recommendation methods. Details of the score calculation processing will be described with reference to the flowchart of FIG.

  (Step S405) The dialog text information configuration unit 16 configures dialog text information to be sent. Details of the processing for constructing dialogue text information will be described with reference to the flowchart of FIG.

  (Step S <b> 406) The dialog text output unit 17 sends the dialog text information configured by the dialog text information configuration unit 16 to the simulation apparatus 2.

  (Step S407) The user state information update unit 18 performs a user state information update process, and returns to step S402. Details of the user status information update processing will be described with reference to the flowchart of FIG.

  In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

  Next, details of the score calculation processing in step S404 will be described with reference to the flowchart of FIG.

  (Step S501) The score calculation unit 15 reads the user state information from the user state information storage unit 13.

  (Step S502) The score calculation unit 15 substitutes 1 for a counter i.

  (Step S503) The score calculation unit 15 determines whether or not the i-th information recommendation method exists in the information recommendation method storage unit 12. If the i-th information recommendation method exists, the process goes to step S504, and if it does not exist, the process returns to the upper process.

  (Step S504) The score calculation part 15 reads the weight vector which the i-th information recommendation method has.

  (Step S505) The score calculation unit 15 calculates the score of the i-th information recommendation method using the user state information read in step S501, the evaluation information read in step S504, and the weight vector, and calculates the score. Temporary storage is performed in association with the i-th information recommendation method. The score calculation unit 15 calculates a score using, for example, “user state information × weight vector”. Note that user state information and evaluation information are also vectors.

  (Step S506) The score calculation unit 15 increments the counter i by one. The process returns to step S503.

  In the flowchart of FIG. 5, the score calculation method of the score calculation unit 15 does not matter.

  Next, details of the dialogue text information configuration processing in step S405 will be described with reference to the flowchart of FIG.

  (Step S601) The dialogue sentence information configuration unit 16 acquires a spot included in the immediately preceding user input information and / or the dialogue sentence information of the immediately preceding dialogue apparatus 1. However, here, there may be a case where the spot cannot be acquired.

  (Step S602) The dialog statement information configuration unit 16 acquires a determination factor included in the immediately preceding user input information and / or the dialog statement information of the immediately preceding dialog device 1. However, here, there may be a case where the determination factor cannot be acquired.

  (Step S603) The dialog text information configuration unit 16 determines whether or not a spot has been acquired in Step S601. If it can be acquired, the process goes to step S604, and if it cannot be acquired, the process goes to step S605.

  (Step S604) The dialogue sentence information configuration unit 16 substitutes the spot acquired in Step S601 for the variable “target spot”. Note that the value of the variable “spot of interest” is a spot that is currently focused on in the dialogue. The value of the variable “spot of interest” is usually one spot.

  (Step S605) The dialog statement information configuration unit 16 determines whether or not the determination factor has been acquired in Step S602. If it can be acquired, the process goes to step S606, and if it cannot be acquired, the process goes to step S607.

  (Step S606) The dialog statement information configuration unit 16 substitutes the determination factor acquired in Step S602 for the variable “focus determination factor”. Note that the value of the variable “focused determinant” is a determinant currently focused on in the dialogue. In addition, the value of the variable “target decision factor” may be two or more decision factors.

  (Step S <b> 607) The dialogue sentence information configuration unit 16 searches the knowledge base 11 using the value of the variable “focus spot” and the value of the variable “focus determination factor”, and explains corresponding to the focus spot and the focus determination factor. Sentence information is read from the knowledge base 11. The explanatory text information is response text information for an input text from the user. Normally, the dialogue sentence information configuration unit 16 reads from the knowledge base 11 the value of the variable “target spot” and the explanatory text corresponding to the variable “target decision factor”. The explanatory note information may be the explanatory note itself or information for identifying the explanatory note.

  (Step S608) The dialogue text information configuration unit 16 performs a recommendation text information acquisition process. The recommended sentence information acquisition process will be described with reference to the flowchart of FIG. The recommendation sentence information has a technique identifier. Further, it is preferable that the recommendation sentence information has one or more spots or one or more determinants.

  (Step S609) The dialog text information configuring unit 163 configures dialog text information from the explanatory text information acquired in Step S607 and the recommended text information acquired in Step S608.

  In the flowchart of FIG. 4, answer sentence information and recommendation sentence information are acquired. However, in the flowchart of FIG. 4, it is conceivable to perform various sentence information acquisition processing, such as acquiring only recommended sentence information, acquiring answer sentence information, recommended sentence information, and other sentence information.

  Next, the recommended sentence information acquisition process in step S608 (FIG. 6) will be described with reference to the flowchart of FIG.

  (Step S701) The method identifier acquisition unit 161 of the dialogue sentence information configuration unit 16 sets a method identifier for identifying one information recommendation method corresponding to the largest score among the two or more scores calculated by the score calculation unit 15 as information Obtained from the recommendation method storage unit 12.

  (Step S702) The variable value acquisition means 162 substitutes 1 for the counter i.

  (Step S703) The variable value acquisition unit 162 determines whether or not the i-th variable exists in the sentence pattern information corresponding to the technique identifier acquired in step S701. If it exists, the process goes to step S704, and if it does not exist, the process returns to the upper process.

  (Step S704) The variable value acquisition unit 162 acquires the i-th variable in the sentence pattern information acquired in step S701. This variable also holds information on where to obtain the variable value from.

  (Step S705) The variable value acquisition unit 162 acquires one or more terms to be substituted into the i-th variable. This term is usually a spot or a determinant (may be a word specifying the determinant).

  (Step S706) The variable value acquisition unit 162 increments the counter i by one.

  Next, details of the user status information update processing in step S407 will be described with reference to the flowchart of FIG.

  (Step S801) The user presentation term acquisition means 181 of the user status information update unit 18 acquires one or more determination factors from the latest (immediately preceding) user input information received by the user input information reception unit 14. When one or more determination factors cannot be acquired from the sentence received immediately before, the user-presented term acquisition unit 181 acquires the focus determination factor. The user presented term acquisition unit 181 acquires one or more spots from the latest (immediately preceding) user input information received by the user input information receiving unit 14. Then, the user-presented term acquisition unit 181 temporarily stores the acquired determination factor or / and the acquired spot in the buffer.

  (Step S802) The preference vector update unit 183 reads the preference vector included in the user status information in the user status information storage unit 13. Then, the preference vector update unit 183 updates the preference vector included in the user state information so that the value of the element corresponding to the determination factor acquired in step S801 is increased. Here, the attribute vector update unit 185 is configured so that the value of the element corresponding to the determination factor acquired in step S801 is increased, or the value of the element corresponding to the determination factor is set. The attribute vector included in the user state information may be updated so as to approach the value of the element corresponding to the factor. For example, the attribute vector element is gender, which is defined by male “1” and female “0”, and the value of the element corresponding to the determination factor acquired in step S801 is “0” (female). If there is, the attribute vector update unit 185 changes the attribute value so that the value of the attribute “gender” is closer to “0”. Note that the attribute vector update unit 185 does not matter how the attribute value is changed. For example, the attribute vector update unit 185 sets the average value of the element values (1 or 0) corresponding to the acquired determination factor as the current attribute value.

  (Step S803) The device presentation term acquisition means 182 acquires one or more determinants from the latest (immediately previous) dialog sentence information sent by the dialog sentence output unit 17. Further, the device presentation term acquisition means 182 acquires one or more spots from the latest (immediately previous) dialog sentence information sent by the dialog sentence output unit 17. Then, the device presentation term acquisition unit 182 temporarily stores the acquired determination factor or / and the acquired spot in the buffer.

  (Step S804) The knowledge vector update means 184 reads the knowledge vector possessed by the user state information in the user state information storage unit 13. The knowledge vector update unit 184 updates the knowledge vector included in the user state information so that the value of the element corresponding to the determination factor acquired in step S803 is increased.

  (Step S805) The user state information update unit 18 reads the number of conversation turns included in the user state information in the user state information storage unit 13. The number of dialogue turns is a logarithm of dialogue repeated. Then, the user status information update unit 18 updates the user status information in the user status information storage unit 13 with a value obtained by adding 1 to the read number of turns as a new number of turns.

  (Step S806) The user state information update unit 18 updates the immediately preceding user utterance action information. The immediately preceding user utterance action information is information regarding the user input information received by the user input information receiving unit 14 immediately before, and corresponds to the type of information requested by the user (spot only, determinant name only, or both). Information.

  (Step S807) The user state information update unit 18 updates the immediately preceding system utterance action information. The immediately preceding system utterance action information is information related to the dialog text information sent by the dialog text output unit 17 immediately before, and is information (method identifier) that identifies the selected information recommendation method.

  (Step S808) The user state information update unit 18 updates the system presentation history information. The system presentation history information is the number of spots and determination factors sent from the interactive apparatus 1 (system) to the simulation apparatus 2. The user status information update unit 18 performs a unique process on the determination factor and spot written in the buffer in step S803, and acquires the number of determination factors and the number of spots in the buffer. Then, the user status information update unit 18 acquires the number of determining factors and the number of spots in the buffer as system presentation history information.

  In the flowchart of FIG. 8, the information updated in steps S805 to S808 is an example of information constituting the user status information, and other information may constitute the user status information.

  Next, the operation of the simulation apparatus 2 will be described with reference to the flowchart of FIG.

  (Step S901) The dialog text information receiving unit 23 determines whether dialog text information has been received. If dialog text information is accepted, the process goes to step S902. If dialog text information is not accepted, the process returns to step S901.

  (Step S902) The user sentence type determination unit 24 acquires a technique identifier included in the dialog sentence information received in Step S901.

  (Step S903) The user sentence type determination unit 24 determines the user sentence type using the method identifier acquired in step S902 and the conversation probability information in the conversation information storage unit 21, and acquires the user sentence type identifier.

  (Step S904) The determination factor acquisition unit 25 acquires one or more determination factors or / and one or more spots included in the dialog sentence information received in Step S901. Here, there are cases where the determination factor and one or more spots cannot be acquired.

  (Step S905) The determination factor etc. acquiring unit 25 acquires one or more determination factors or one or more spots by using one or more determination factors or / and one or more spots acquired in step S904. Here, the determination factor and the spot may not be determined. In addition, when the determination factor acquisition unit 25 cannot acquire a spot using one or more determination factors or / and one or more spots acquired in step S904, one spot is determined according to the probability of the spot probability information. You may get

  (Step S906) The user input information sending unit 26 configures user input information using the user sentence type identifier acquired in step S903 and zero or more determinants or / and zero or more spots acquired in step S905. . Here, the user input information is to configure a user sentence type identifier and user input information having zero or more determinants or / and zero or more spots in a data structure to be transmitted.

  (Step S907) The user input information sending unit 26 sends the user input information configured in step S906 to the dialogue apparatus 1.

  (Step S908) The reward calculation unit 27 calculates a reward when the spot acquired by the determination factor acquisition unit 25 is selected. The reward calculation process will be described with reference to the flowchart of FIG.

  (Step S909) The learning unit 28 uses the reward calculated in step S908 to update the weight vector of the information recommendation method storage unit 12 of the dialog device 1 that is a weight vector corresponding to the method identifier of the dialog device 1. Return to step S901.

  In the flowchart of FIG. 9, the process is terminated by power-off or a process termination interrupt.

  Next, the reward calculation process in step S908 will be described with reference to the flowchart of FIG.

  (Step S1001) The reward calculation unit 27 reads a user preference vector from the user preference vector storage unit 22.

  (Step S1002) The random selection match value calculation means 271 substitutes 1 for the counter i.

  (Step S1003) The random selection match value calculation means 271 determines whether or not the i-th spot exists in the knowledge base 11. If the i-th spot exists, the process goes to step S1004, and if not, the process goes to step S1007.

  (Step S1004) The random selection match value calculation means 271 reads one or more evaluation values of the i-th spot from the knowledge base 11.

  (Step S1005) The random selection match value calculation means 271 calculates the user preference vector and the i-th spot from the user preference vector read in step S1001 and one or more evaluation values of the i-th spot read in step S1004. The degree of coincidence with one or more evaluation values is calculated, and the degree of coincidence is temporarily stored in a buffer (not shown). The random selection match value calculation means 271 sets, for example, the number of matches between each element of the user preference vector and one or more evaluation values of the i-th spot as the match level.

  (Step S1006) The random selection match value calculation means 271 increments the counter i by 1, and returns to step S1003.

  (Step S1007) The random selection match value calculation means 271 calculates the expected value of the match degree of all spots from the match degree of all spots temporarily accumulated in step S1005. The random selection match value calculation means 271 may use an average value of the match degrees of all spots as an expected value. In addition, the random selection match value calculation unit 271 may use the sum of “the degree of match of each spot × the probability of each spot indicated by the spot probability information” as an expected value.

  (Step S1008) The selected spot coincidence degree calculation means 272 reads from the knowledge base 11 one or more evaluation values indicating the evaluation of one or more determinants of the spot of interest.

  (Step S1009) The selected spot coincidence degree calculation means 272 calculates the user preference vector and one or more evaluation values of the spot of interest from the user preference vector read in Step S1001 and one or more evaluation values read in Step S1008. The degree of match is calculated. For example, the selected spot coincidence degree calculating means 272 sets the number of coincidence between each element of the user preference vector and one or more evaluation values of the spot of interest as the coincidence degree.

  (Step S1010) The reward calculation means 273 is included in the user input information using the expected value of the degree of match calculated by the random selection match value calculation means 271 and the degree of match calculated by the selected spot match degree calculation means 272. A reward for selecting a spot is calculated. The reward calculation means 273 calculates a reward by, for example, “reward = the degree of match calculated in step S1009−the expected value calculated in step S1007”. It is assumed that the calculation formula is stored in advance by the reward calculation unit 273.

  Hereinafter, an evaluation experiment of the learning system in the present embodiment will be described. Note that the dialogue device 1 in the evaluation experiment is a device that assists in the decision making of the visit destination of the simulation device 2 while performing a dialogue with the simulation device 2 that simulates a user, and provides sightseeing guidance in Kyoto.

  FIG. 11 is an example of a spot information management table held by the knowledge base 11. The spot information management table stores two or more pieces of spot information, which is information regarding spots. The spot here is a sightseeing spot in Kyoto. In this specification, a spot may be information that identifies a spot (which may be referred to as a spot identifier) or a concept of a spot. The spot information includes “spot”, “determining factor information”, “evaluation value”, and “description”. The “determining factor information” includes a “determining factor identifier” that identifies a determinant and a “determining factor” that is a phrase that accurately indicates the determinant (including a positive or negative term). Also, in this specific example, the determinants are “Garden is famous”, “Not crowded”, “World heritage”, “Scenic view”, “Good access”, “Colorful leaves”, “Sakura is famous”, “History in history” Suppose that there are 10 factors (in no particular order): “Walking” and “There is an event”. The “evaluation value” is information indicating an evaluation with respect to the determinant of each spot. Here, each spot has “1” when it has the characteristic of the determinant indicated by the determinant information. In this case, a value of “0” is taken. The “evaluation value” may be a multi-level evaluation (for example, “1” to “5”) instead of the binary values “1” and “0”.

  FIG. 12 is an example of an information recommendation method management table in the information recommendation method storage unit 12. The information recommendation method management table stores six information recommendation methods. The information recommendation method includes “ID”, “method identifier”, “sentence pattern information”, “evaluation information”, and “weight vector”. “ID” is a numerical value for identifying an information recommendation method. The “method identifier” is a character string that identifies the information recommendation method and indicates its significance. The “sentence pattern information” is information that constitutes a recommended sentence of information, or a recommended sentence of information (ID = 3 only). In the recommendation sentence of information, the tag (information of the beginning “<”, end “>”) is a variable. Also, information enclosed by “{” and “}” in the sentence pattern information is information (hereinafter referred to as an acquisition operation description) indicating a method for acquiring the value of the immediately preceding variable (acquisition operation). The current target spot is substituted for <target spot>. The current focus determination factor is substituted into <focus determination factor>. For <one or more undecided determinants>, a determinant that has not appeared in the previous dialogue for the spot of interest and whose evaluation value of the spot of interest is “1” is substituted. For <1 or more unspotted spots>, a spot whose attention determining factor has not appeared in the previous dialogue and whose evaluation value of the attention determining factor is “1” is substituted. <Highest favorite spot> is a spot that becomes the highest point when the rank (preference value) of each spot is calculated based on the preference vector of the user status information. The acquisition operation description “{decision factor equal to or less than 3 select where in the knowledge vector has the lowest value}” indicates an operation of acquiring a decision factor of 3 or less in the descending order of the value of the decision element in the knowledge vector. Since this acquisition operation description is attached to <one or more undecided determinants>, it indicates an operation of acquiring three or less undecided determinants in descending order of the value of the determinant element in the knowledge vector. It will be. The acquisition operation description “{decision factor equal to or smaller than select 3 where preference value in order of descending value}” indicates an operation of acquiring a determinant of 3 or less in descending order of value in the preference vector. Since this acquisition operation description is attached to <one or more undecided determinants>, it indicates an operation for acquiring three or less undecided determinants in descending order of the value in the preference vector. . The acquisition operation description “{select highest preference spot where the preference value is the maximum}” indicates that a spot estimated by the system to be the maximum preference value when viewed from the user is obtained from the current user status information. Show. For example, this acquisition operation description substitutes the preference vector and the evaluation value of each determinant for each spot into the arithmetic expression for calculating the preference value, calculates the preference value, and acquires the spot having the maximum preference value. Show. “Evaluation information” is information held for each information recommendation method. Then, using the current user status information and the evaluation information of the information recommendation method, the score of each information recommendation method is calculated, and using the sentence pattern information corresponding to the information recommendation method with the maximum score, the recommended sentence Is configured. For example, the score of the information recommendation method is calculated by multiplying the current user state information (vector) and the evaluation information (vector) of the information recommendation method.

In addition, it is assumed that the user status information storage unit 13 stores user status information described below. For example, it is assumed that the user state information has three elements: a knowledge vector “K _user ”, a preference vector “P _user ”, and a local weight matrix “V _user ”. Here, for the sake of simplicity, the elements of the _user preference vector “P _user = (p ₁ , p ₂ ,..., P _M )” are parameters consisting of binary values of “1” or “0”. And That is, the user is interested in is determinant m (or potentially interested), p _m is assumed to take the "1" when emphasized in determining spot. In addition, in order to express a state in which the user has a potential preference (the user himself / herself is unaware), the user's knowledge vector “K _user = (k ₁ , k ₂ ,..., K _M ) Is introduced. When the user knows that the system (interactive device 1) can handle the determining factor m, or when the system recommends the determining factor m, the vector element k _M takes “1”. By using these vectors, for example, determinants m is is a factor that the user has potentially interesting, the state that the user is not aware of it _{(k m = 0, p m} = 1) Can be expressed as Further, assuming that the user determines only the local weight v _nm for the spot n from the viewpoint of the user's determinant m from the information presented from the system, the system uses “ID = “1” is assumed when the user is informed of the evaluation of the spot using any one of the recommended methods “1, 2, 6”. Note that the user status information may have an attribute vector “A _user ”.

The user state information here corresponds to a feature vector of the dialog state. More specifically, here, the user status information includes the following six types of information. The first is the number of turns. Here, the number of turns is expressed by five parameters using a sawtooth function. The second is immediately preceding user utterance action information. The immediately preceding user utterance action information is, for example, a vector (x _i , x _{i + 1} , x _{i + 2} , x _{i + 3} , x _{i + 4} ), and (1 if a _user ^t−1 = x _i , otherwise 0). Here, “a _user ^t−1 ” is the utterance of the previous user. In addition, when the utterance of the previous user includes only a spot (which may be referred to as a spot name) recommended by the system (dialogue device 1) or a determinant, “x _i = 1” and other vector elements Is “0”. When a spot not recommended by the system is included, “x _{i + 1} = 1” and the elements of other vectors are “0”. When only the determinants not recommended by the system are included, “x _{i + 2} = 1” and the elements of the other vectors are “0”. Further, when both spots and determinants not recommended by the system are included, “x _{i + 3} = 1” and the elements of other vectors are “0”. Further, when none of these is included, “x _{i + 4} = 1” and the elements of the other vectors are “0”. The third is the immediately preceding system utterance action information. The immediately preceding system utterance action information is, for example, a vector (y _i , y _{i + 1} , y _{i + 2} , y _{i + 3} , y _{i + 4} , y _{i + 5} , y _{i + 6} ), and (1 if a _sys ^t-1 = y _i , otherwise 0) Here, “a _sys ^t-1 ” is the utterance of the immediately preceding system. Further, when the information recommendation method 1 is used as the utterance of the immediately preceding system, “y _{i + 1} = 1” and the elements of the other vectors are “0”. When the information recommendation method 2 is used, “y _{i + 2} = 1”, and the elements of the other vectors are “0”. That is, when the information recommendation method n is used, “y _{i + n} = 1”, and the elements of other vectors are “0”. The fourth is a knowledge vector for a user's determinant. Note that the knowledge vector may be calculated by “Σ _{n = 1} ^N Pr (k _n = 1)”, which is the knowledge of the user's decision factor. Here, k _n is the element values of the knowledge vector for the n th determinant. Pr (k = 1) is a posterior probability (certainty factor) estimated by the system that “k is 1”. Here, it is assumed that the element value of the preference vector is “1” or “0”. The fifth is the number of spots and determinants presented by the system. That is, the fifth item is system presentation history information, for example, “Σ _{n = 1} ^N Σ _{m = 1} ^M v _nm ”. Here, v _nm is an evaluation value for the determinant n of the spot m that the interactive apparatus 1 has. The sixth is a preference vector. The preference vector is information indicating a user's preference for each determining factor. Note that the preference vector may be replaced with an expected value of the probability that the user attaches importance to the determination factor. This expected value is indicated by “Pr (k _n = 1) × Pr (p _n = 1))”, and has 10 parameters in total for each decision factor. Here, “p _n ” is an element value of the preference vector. Here, it is assumed that the element value of the preference vector is “1” or “0”.

  The user state information stored in the user state information storage unit 13 is a vector having the element values described above, and is a vector (s1, s2,..., S29) having 29 element values. , And. The initial value of the user state information is (s0001, s0002,..., S0029).

  Further, it is assumed that the dialog text output unit 17 stores initial dialog text information (method identifier “3”) that is dialog text information output when the dialog device 1 is activated. The sentence corresponding to the method identifier “3” is, for example, “Kyoto sightseeing system. Recommend your favorite sightseeing spot.”

  Further, the user input information receiving unit 14 holds a user sentence type identifier which is a conversation end condition. The user sentence type identifier here is a user sentence type identifier corresponding to the sentence pattern “go to <spot>” or “determined to <spot>”.

  Further, the dialogue probability information in the dialogue information storage unit 21 is, for example, information indicated by the dialogue probability information management table shown in FIG. The dialogue probability information management table has user sentence types and the probabilities of Method 1 to Method 3. That is, FIG. 13 shows the user sentence identified by the user sentence type identifier when the information recommendation technique identifier “method 1”, “method 2”, or “method 3” is included in the dialog sentence information. The probability that the type is selected is shown. In FIG. 12, it is assumed that the probability based on the information recommendation method identifier “method 1” is used for user action selection (selection of user sentence type identifier) for the recommendation methods 4 to 6.

  In this state, an evaluation experiment was performed. In the evaluation experiment, a simulation speaker (Puser, Kuser, Vuser) is sampled for each simulation dialogue. The pseudo-speaker is assumed to have four preferences (= four elements of the preference vector User are “1” and the remaining elements are “0”). For preference selection, the distribution of user preferences (see Fig. 12) examined by questionnaires conducted after subject experiments were used. Similarly, the user's knowledge Kuser was set based on the ratio of the user's utterance before the system recommendation in the preliminary experiment. The user's local weight Vuser is initialized to “0”, assuming that the user has no prior knowledge. Similarly, the user preference Psys and knowledge Ksys estimated by the system based on the result of the preliminary experiment were initialized for the parameters on the interactive apparatus 1 side (system side). The following assumptions were made when performing the simulation. The system determines the recommended content based on the policy π at that time without performing speech recognition and understanding error of the user's utterance. The user is assumed to continue the dialogue for 20 turns, and the simulation apparatus 2 generates a response (user input information). The interactive device 1 is rewarded based on the reward function. Under the above conditions, the dialogue was simulated, and the policy (parameter θ) was updated by NAC every 2000 dialogues.

  The experimental results will be described below. First of all, we investigated the improvement of remuneration by repeating the policy. Since the method in this experiment includes random elements, all the experimental results are the average of five trials. FIG. 14 shows the relationship between the number of simulation dialogues performed (2,000 dialogues are 1 batch) and rewards after 2, 5, 10, 15, and 20 turns. In FIG. 14, the case where the determination is made when the user has all the knowledge about the domain is also shown as (Oracle). The system policy converged with 30,000 dialogues.

  In addition, the dialogue strategy was analyzed by comparing and analyzing the value of the learned parameter θ (weight vector). In the methods 4 and 5, the weight for the parameter indicating that the number of turns of the dialogue from the start is small is large, and in the methods 2 and 6, the weight of the parameter indicating that the number of turns is large is large. This result indicates that in the dialogue strategy after learning, first, the user is given knowledge of the determinants, and the user's preference is estimated, and then the dialogue strategy for presenting specific candidates is performed.

Next, the learned dialogue strategy was compared with the following two baseline approaches.
(1) No recommendation (B1)
In the baseline method (B1), the system only presents requested information and does not recommend it. This is equivalent to the case where Method 3 is always selected.
(2) Recommendation at random (B2)
In the baseline method (B2), the system randomly selects a recommended method from six selectable methods. This is equivalent to the strategy at the initial value (all 0) of the parameter θ.

  FIG. 15 shows the results of comparison with these baseline methods. The dialogue strategy optimized by NAC was able to obtain significantly larger rewards compared to the baseline approach (n = 500, p <0.01).

  Furthermore, consider the trade-off problem between the fitness of the spot to be determined and the length of dialogue. It is troublesome for the user to recommend information when the item to be asked next is clearly determined, or to repeatedly recommend known content. Therefore, we consider the optimal dialogue strategy after taking the penalty into the recommendation act. The dialogue strategy was learned and evaluated by an evaluation function that gave a penalty of 0.05 to recommended methods other than Method 3. The result is shown in FIG. Compared with the method of selecting a recommendation method at random, the dialogue strategy learned by the proposed method has a small amount of decrease in reward even when the dialogue is prolonged. This is thought to avoid unnecessary recommendations in the learned dialogue strategy. The reward obtained by the proposed method was statistically significant compared to the baseline method (p <.01).

  As described above, according to the present embodiment, it is possible to automatically construct a weight vector necessary for a dialog device that performs a dialog with a user to output a sentence.

  In addition, according to the present embodiment, a dialogue state model that considers both the user's knowledge and preferences is proposed, optimization is performed by reinforcement learning, and the user can make better decisions as compared to the baseline method. I confirmed that I can do it.

  In the present embodiment, the learning algorithm of the weight vector does not matter.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. Note that the software that implements the interactive apparatus 1 in the present embodiment is the following program. That is, this program has a spot having a spot, one or more determinants that are factors for determining the spot, and an evaluation value indicating evaluation of each of the one or more determinants of the spot on the storage medium. Two or more information having at least two pieces of information and having a method identifier for identifying the information recommendation method, evaluation information of the information recommendation method, and a weight vector indicating the weight of each element constituting the evaluation information An information recommendation method is stored, and information indicating a user's state, a preference vector that is information indicating a user's preference for one or more determinants, and a user's knowledge for one or more determinants User state information having a knowledge vector to be stored, and from a simulation device, a user sentence type which is a pattern of a sentence input by the user is stored. User input information receiving unit for receiving user input information having another user sentence type identifier or user sentence type identifier and one or more determinants or one or more information of one or more spots, and stored in the storage medium A score calculation unit that calculates two or more scores for each of the two or more information recommendation methods using the evaluation information and weight vector of each of the two or more information recommendation methods and the user status information; Using the two or more scores calculated by the score calculation unit, a method identifier for identifying one information recommendation method, or a method identifier, and one or more determinants or one or more information of one or more spots Dialogue sentence information constituting the dialogue sentence information, and the dialogue sentence output for sending the dialogue sentence information constituted by the dialogue sentence information constituting unit to the simulation apparatus. And at least one spot or one or more determinants from the user input information received by the user input information received by the user input information or the dialog sentence information output by the dialog sentence output unit And a user state information update unit that updates user state information of the storage medium using the one or more spots or one or more determinants, wherein the score calculation unit is stored in the storage medium. Two or more scores for each of the two or more information recommendation methods are calculated using the evaluation information and the weight vector of each of the two or more information recommendation methods and the user status information updated by the user status information update unit. It is a program for making it function as a thing.

  Moreover, the software which implement | achieves the simulation apparatus 2 in this Embodiment is the following programs. In other words, this program selects dialogue probability information that is information about the probability of each information recommendation method and each user sentence type, determinant probability information that is information about the probability that a determinant is selected, and a spot on a storage medium. Dialogue probability information, which is information relating to the probability of being played, and a user preference vector, which is a vector indicating the user's preference, is received, and dialogue text information for accepting dialogue text information from a dialogue device Using the reception unit, the method identifier included in the dialogue sentence information and the dialogue probability information, determine a user sentence type, obtain a user sentence type identifier, and the determination factor probability information or the One or more pieces of information in the spot probability information, or one or more pieces of information in the determinant probability information or the spot probability information And one or more determinants or one or more information of one or more spots included in the dialogue sentence information, a determination factor etc. acquisition unit for acquiring one or more determinants or one or more spots, A user sentence information identifier, or a user sentence information identifier and one or more determinants or user input information having one or more information of one or more spots; The user preference vector and one or more evaluation values indicating the evaluation of each of the one or more determining factors of the spot included in the user input information are acquired, and the degree of coincidence between the user preference vector and the one or more evaluation values A reward calculation unit for calculating a reward for selecting a user sentence type identified by the user sentence type identifier using the degree of match, and using the reward, A weight vector corresponding to the method identifier of the device, a program, to function as a learning unit for updating the weight vectors of the storage medium of the interactive device.

In the above program, the reward calculation unit uses the spot probability information to calculate an expected value of the degree of coincidence between one or more evaluation values when the spot is randomly determined and the user preference vector. Selected spot match degree calculating means for calculating a match degree between the match value calculating means, the user preference vector, and one or more evaluation values indicating the evaluation of the one or more determining factors of the spot included in the user input information The spot included in the user input information is selected using the expected value of the degree of match calculated by the random selection match value calculation unit and the match level calculated by the selected spot match level calculation unit. It is preferable that the program includes a reward calculation unit that calculates a reward, and is a program for causing a computer to function.
(Embodiment 2)

  In the present embodiment, an interactive device using the weight vector learned in the first embodiment will be described. This dialogue apparatus is a consultation type dialogue apparatus in which a user selects a candidate while receiving information presentation or information recommendation from the apparatus. In the dialogue apparatus according to the present embodiment, interaction with the user is performed by voice, but voice input / output is not essential.

  When selecting a candidate that suits the taste, it is necessary to consider many factors (same meaning as a determinant described later). Since the user who uses the interactive device does not necessarily grasp all such factors, the interactive device recommends information to the user, and the gap between the knowledge held by the interactive device and the user's knowledge is obtained. Need to fill.

  In this embodiment, an interactive apparatus that implements a consultation-type conversation model for selecting a candidate suitable for the user from among a plurality of candidates (here, mainly referred to as spots) will be described.

  FIG. 17 is a block diagram showing the internal structure of the interactive apparatus 3 in the present embodiment. The dialogue apparatus 3 includes a knowledge base 11, an information recommendation method storage unit 12, a user state information storage unit 13, a reception unit 34, a score calculation unit 15, a sentence composition unit 36, a sentence output unit 37, and a user state information update unit 18. .

  The reception unit 34 includes a voice reception unit 341 and a voice recognition unit 342.

The sentence composition unit 36 includes sentence pattern information acquisition means 361, variable value acquisition means 362, and sentence composition means 363.
Here, the user status information update unit 18 includes a user presentation term acquisition unit 181, a device presentation term acquisition unit 182, a preference vector update unit 183, a knowledge vector update unit 184, and an attribute vector update unit 185.

  The accepting unit 34 accepts a sentence input by the user. Here, the reception is usually reception of voice. However, reception means reception of information input from an input device such as a keyboard, mouse, touch panel, reception of information transmitted via a wired or wireless communication line, a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory This is a concept including reception of information read out from. In other words, the input includes voice input, character string input, and the like, and any input means may be used.

  The sentence input means may be anything such as a microphone, keyboard, mouse or menu screen. The receiving unit 34 can be realized by a device driver for input means such as a microphone and a keyboard, control software for a menu screen, and the like.

  The voice receiving unit 341 receives voice input by the user from a microphone.

  The voice recognition unit 342 recognizes the voice received by the voice reception unit 341 and converts it into a character string. The voice recognition method in the voice recognition means 342 does not matter. Since the voice recognition unit 342 is a known technique, a detailed description thereof is omitted. The voice recognition unit 342 can be usually realized by an MPU, a memory, or the like. The processing procedure of the voice recognition means 342 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The score calculation unit 15 applies the user state information stored in the user state information storage unit 13 to evaluation information included in each of the two or more information recommendation methods stored in the information recommendation method storage unit 12, and each of the two or more information recommendations A score of 2 or more for the method is calculated. The score calculation unit 15 applies the user state information updated by the user state information update unit 18 to evaluation information included in each of the two or more information recommendation methods stored in the information recommendation method storage unit 12. A score of 2 or more for each information recommendation method is calculated.

  The score calculation unit 15 usually calculates a score before the sentence output unit 37 outputs a sentence (not necessarily immediately before). In addition, it is suitable for the score calculation part 15 to calculate a score whenever the reception part 34 receives a sentence. Here, the application is, for example, calculating a score by an arithmetic expression “score = f (user state information, weight vector)”. For example, f is “score = user state information × weight vector”. That is, the score calculation unit 15 uses the evaluation information managed in association with the sentence pattern information and the user state information that dynamically changes in order to determine the sentence pattern information of the sentence to be output next by the dialogue apparatus 1. Are used to calculate a score for each information recommendation method.

  The sentence composing unit 36 uses the two or more scores calculated by the score calculating unit 15 to acquire sentence pattern information included in one information recommendation method, and composes a sentence from the sentence pattern information. Normally, the sentence composing unit 36 acquires sentence pattern information included in the information recommendation method having the maximum score calculated by the score calculating unit 15 and composes a sentence from the sentence pattern information. Here, when the sentence pattern information is a sentence, the structure of the sentence is NOP (no processing is performed). In addition, if the sentence pattern information is pattern information of a sentence including variables, the sentence structure is determined from the immediately preceding output sentence, the immediately preceding accepted sentence, the determinant of interest, the spot of interest, etc. This is a process of acquiring the value of a variable and substituting the acquired value of the variable into sentence pattern information. Note that the determinant of interest is the determinant from which one or more spots are output. The spot of interest is a spot from which one or more determinants are output. In addition, the sentence structure part 36 also performs the process which acquires an attention spot and an attention determination factor from the user's input sentence immediately before and / or the output sentence of the dialog apparatus 1 immediately before. Details of such processing will be described later.

  The sentence constructing unit 36 can be usually realized by an MPU, a memory, or the like. The processing procedure of the sentence composing unit 36 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The sentence pattern information acquisition unit 361 acquires, from the information recommendation method storage unit 12, sentence pattern information included in one information recommendation method corresponding to the largest score among the two or more scores calculated by the score calculation unit 15.

  The variable value acquisition unit 362 acquires one or more variables included in the sentence pattern information acquired by the sentence pattern information acquisition unit 361, and the sentence output unit 37 outputs the spot or determination factor corresponding to the variable immediately before. Or from one or more of the sentences received by the receiving unit 34 immediately before. Further, the variable value acquisition unit 362 acquires one or more variables included in the sentence pattern information acquired by the sentence pattern information acquisition unit 361, and sets one or more spots or one or more determinants as candidates corresponding to the variables. From one or more of the sentences output immediately before by the sentence output unit 37 or the sentence received immediately before by the receiving unit 34, and from one or more spots or one or more determinants as candidates, Using the evaluation values of the knowledge base 11 corresponding to one or more spots or one or more determinants, a spot or determinant corresponding to the variable is selected. The variable value acquisition unit 362 may acquire a spot or a determinant corresponding to the variable from a target spot that is a spot that is currently focused or a focus determination factor that is a decisive factor that is currently focused.

  The sentence constructing unit 363 composes a sentence by inserting the term (usually a spot or a determinant) acquired by the variable value acquiring unit 362 into the variable portion of the sentence pattern information acquired by the sentence pattern information acquiring unit 361. The sentence constructing unit 363 may perform a transformation process so that the output sentence becomes a natural and fluent sentence. Since this processing is a known technique, detailed description thereof is omitted.

  The sentence output unit 37 outputs the sentence constructed by the sentence constructing unit 36. Here, the output is usually an audio output. That is, it is preferable that the sentence output unit 37 outputs the sentence formed by the sentence structure unit 36 by voice. However, output refers to display on a display, projection using a projector, printing on a printer, transmission to an external device (such as an audio output device or display device), storage in a recording medium, other processing devices, etc. This is a concept that includes delivery of processing results to other programs.

  The sentence output unit 37 may be considered as including or not including an output device such as a display or a speaker. The sentence output unit 37 may be implemented by output device driver software, or output device driver software and an output device.

  The user status information update unit 18 acquires at least one or more spots or one or more determinants from one or more sentences out of the sentences received by the reception unit 34 or the sentences output by the sentence output unit 37. The user status information in the user status information storage unit 13 is updated using one or more spots or one or more determining factors. The user status information update unit 18 performs morphological analysis on one or more sentences of the sentence received by the reception part 34 or the sentence output by the sentence output part 37, and stores a specific part of speech (noun, adjective, adjective verb, etc.). The term may be acquired, the knowledge base 11 may be searched using the term as a key, and the term stored in the knowledge base 11 may be acquired as a spot or a determinant. In addition, the user state information update unit 18 acquires a kanji string from one or more sentences out of the sentences received by the receiving unit 34 or the sentences output by the sentence output unit 37, and uses the kanji string as a key for the knowledge base. 11 may be retrieved, and a Chinese character string stored in the knowledge base 11 may be acquired as a spot or a determining factor. Then, the user state information update unit 18 usually updates the user state information so that the values of elements (elements such as a preference vector and a knowledge vector) constituting the user state information regarding the acquired spot or determination factor are increased. . In addition, the user status information update unit 18 normally updates every time the reception unit 34 receives a sentence. However, the user state information update unit 18 may update the sentence output unit 37 each time a sentence is output.

  The user presented term acquisition unit 181 acquires at least one or more determinants from the sentence received by the receiving unit 34. The user-presented term acquisition unit 181 may acquire only a positive determinant, or may detect affirmation / negative and acquire a determinant for each category (positive / negative). For example, the user-presented term acquisition unit 181 acquires an independent word from a sentence received by the reception unit 34, and acquires the independent word as a determining factor when the independent word is a determining factor of the knowledge base 11. Further, the user presented term acquisition unit 181 may acquire a focus determination factor.

  The device presentation term acquisition means 182 acquires at least one or more determinants from one or more sentences out of the sentences output by the sentence output unit 37. For example, the device presentation term acquisition unit 182 acquires an independent word from a sentence output by the sentence output unit 37, and acquires the independent word as a determining factor when the independent word is a determining factor of the knowledge base 11. Note that the sentence output by the sentence output unit 37 has the same meaning as the sentence configured by the sentence configuration unit 36. Further, the device presentation term acquisition unit 182 may acquire only a positive determination factor, or may detect affirmation / negative and acquire a determination factor for each category (positive / negative). In addition, the device presentation term acquisition unit 182 may acquire a focus determination factor.

  The preference vector update unit 183 updates the user state information so as to increase the value of the preference vector element for one or more determination factors acquired by the user presented term acquisition unit 181. Also, the preference vector update unit 183 is a value of an element of the preference vector for one or more determinants that cannot be acquired by the user-presented term acquisition unit 181 among one or more determinants acquired by the device presentation term acquisition unit 182. The user status information is updated so as to lower the value. This is to lower the value of the determinant that is output from the interactive device 1 but not selected by the user.

The knowledge vector update unit 184 updates the user state information so as to increase the value of the element of the knowledge vector for one or more determination factors acquired by the device presentation term acquisition unit 182.
The attribute vector update unit 185 changes the value of the element of the attribute vector for one or more determination factors acquired by the user presented term acquisition unit 181 and updates the user state information. That is, the attribute vector update unit 185 increases the value of the element corresponding to the determination factor acquired by the user presented term acquisition unit 181 or the value of the element corresponding to the determination factor. The value of the element of the attribute vector included in the user state information is updated so as to approach the value of the element corresponding to the factor.

  Next, the operation of the interactive apparatus 1 will be described using the flowchart of FIG.

  (Step S1801) The sentence output unit 37 outputs an initial sentence held in advance. The initial sentence is a sentence that is output to the user when the interactive apparatus 1 starts operation (for example, at the time of activation). The initial sentence is, for example, the sentence “Kyoto Tourist Information System. Recommend your favorite sightseeing spot”.

  (Step S1802) The reception unit 34 determines whether a sentence has been received from the user. If a sentence is accepted, the process goes to step S1803, and if no sentence is accepted, the process returns to step S1802.

  (Step S1803) The voice recognition means 342 of the reception unit 34 recognizes the sentence received in step S1802, and acquires a sentence of a character string (a sentence of a character code string).

  (Step S1804) The receiving unit 34 or a means (not shown) determines whether or not the sentence received by the receiving unit 34 satisfies the end condition. If the end condition is satisfied, the process ends. If the end condition is not satisfied, the process proceeds to step S1805. The termination condition is, for example, that the user input sentence includes a sentence that matches a predetermined sentence pattern. Examples of the sentence pattern determined in advance include “go to <spot>” and “determine <spot>”.

  (Step S1805) The score calculation unit 15 calculates two or more scores for two or more information recommendation methods. Details of the score calculation processing will be described with reference to the flowchart of FIG.

  (Step S1806) The sentence composition unit 36 composes one or more sentences to be output. Details of the sentence composition process will be described with reference to the flowchart of FIG.

  (Step S1807) The sentence output unit 37 outputs one or more sentences configured by the sentence constructing unit 36.

  (Step S1808) The user status information update unit 18 performs user status information update processing, and returns to step S1802. Details of the user status information update processing will be described with reference to the flowchart of FIG.

  In the flowchart of FIG. 18, the sentence output unit 37 may output a predetermined sentence or a sentence composed of a predetermined sentence pattern before the end of the process.

  Further, in the flowchart of FIG. 4, the process is terminated by powering off or a process termination interrupt.

  Next, details of the score calculation processing in step S1805 will be described with reference to the flowchart of FIG.

  (Step S1901) The score calculation unit 15 reads the user status information from the user status information storage unit 13.

  (Step S1902) The score calculation part 15 substitutes 1 to the counter i.

  (Step S1903) The score calculation unit 15 determines whether or not the i-th information recommendation method exists in the information recommendation method storage unit 12. If the i-th information recommendation method exists, the process goes to step S1904, and if it does not exist, the process returns to the upper process.

  (Step S1904) The score calculation part 15 reads the evaluation information which the i-th information recommendation method has.

  (Step S1905) The score calculation unit 15 calculates the score of the i-th information recommendation method using the user state information read in step S1901 and the evaluation information read in step S1904, and calculates the score as the i-th Temporary storage in association with the information recommendation method. For example, the score calculation unit 15 multiplies the user state information read in step S1901 and the evaluation information read in step S1904 to calculate a score.

  (Step S1906) The score calculation unit 15 increments the counter i by 1, and returns to step S1903.

  In the flowchart of FIG. 19, the score calculation method of the score calculation unit 15 does not matter.

  Next, details of the sentence composition processing in step S1806 will be described with reference to the flowchart of FIG.

  (Step S2001) The sentence composing unit 36 performs natural language processing on the immediately preceding user input sentence or / and the immediately preceding output sentence of the interactive device 1 to obtain a spot. The sentence constructing unit 36, for example, performs morphological analysis on the immediately preceding user input sentence or / and the immediately preceding output sentence of the dialogue apparatus 1, acquires an independent word, searches the knowledge base 11 using the independent word as a key, A spot existing in the base 11 is acquired. However, here, there may be a case where the spot cannot be acquired.

  (Step S2002) The sentence composing unit 36 performs natural language processing on the immediately preceding user input sentence or / and the immediately preceding output sentence of the interactive device 1, and acquires a determination factor. The sentence constructing unit 36, for example, performs morphological analysis on the immediately preceding user input sentence or / and the immediately preceding output sentence of the dialogue apparatus 1, acquires an independent word, searches the knowledge base 11 using the independent word as a key, The determination factor existing in the base 11 is acquired. However, here, there may be a case where the determination factor cannot be acquired.

  (Step S2003) The sentence structure part 36 judges whether the spot was able to be acquired by step S2001. If it can be acquired, the process goes to step S2004, and if it cannot be acquired, the process goes to step S2005.

  (Step S2004) The sentence structure part 36 substitutes the spot acquired by step S2001 to the variable "focus spot". Note that the value of the variable “spot of interest” is a spot that is currently focused on in the dialogue. The value of the variable “spot of interest” is usually one spot.

  (Step S2005) The sentence structure part 36 judges whether the decision factor was able to be acquired by step S2002. If it can be acquired, go to step S2006, and if it cannot be acquired, go to step S2007.

  (Step S2006) The sentence structure part 36 substitutes the determination factor acquired by step S2002 to variable "focus determination factor". Note that the value of the variable “focused determinant” is a determinant currently focused on in the dialogue. In addition, the value of the variable “target decision factor” may be two or more decision factors.

  (Step S2007) The sentence composing unit 36 searches the knowledge base 11 using the value of the variable “focused spot” and the value of the variable “focused determination factor”, and provides an explanatory sentence corresponding to the focused spot and the focused determination factor. Read from the knowledge base 11. This explanatory text is an answer text to the input text from the user. Normally, the sentence composing unit 36 reads out the value of the variable “focus spot” and the explanatory text corresponding to the variable “focus determination factor” from the knowledge base 11.

  (Step S2008) The sentence structure part 36 performs the recommendation sentence acquisition process, and returns to the higher-level process. The recommended sentence acquisition process will be described with reference to the flowchart of FIG.

  In the flowchart of FIG. 20, an answer sentence and a recommendation sentence are acquired. However, in the flowchart of FIG. 20, various sentence acquisition processes are conceivable, such as obtaining only a recommended sentence, obtaining an answer sentence, a recommended sentence, and other sentences.

  Next, the recommendation sentence acquisition process of step S2008 is demonstrated using the flowchart of FIG.

  (Step S2101) The sentence pattern information acquisition unit 361 of the sentence composition unit 36 recommends the sentence pattern information included in one information recommendation method corresponding to the largest score among the two or more scores calculated by the score calculation unit 15 as information recommendation. Obtained from the technique storage unit 12.

  (Step S2102) The variable value acquisition unit 362 substitutes 1 for the counter i.

  (Step S2103) The variable value acquisition unit 362 determines whether or not the i-th variable exists in the sentence pattern information acquired in step S2101. If it exists, go to Step S2104, and if not, go to Step S2108.

  (Step S2104) The variable value acquisition unit 362 acquires the i-th variable in the sentence pattern information acquired in step S2101. This variable also holds information on where to obtain the variable value from.

  (Step S2105) The variable value acquisition unit 362 acquires one or more terms to be substituted into the i-th variable. This term is usually a spot or a determinant (may be a word specifying the determinant).

  (Step S2106) The sentence composing means 363 substitutes one or more terms acquired in step S2104 for the i-th variable in the sentence pattern information.

  (Step S2107) The variable value acquisition unit 362 increments the counter i by 1, and returns to step S2103.

  (Step S2108) The sentence composing means 363 changes the acquired sentence to a natural sentence, and returns to the upper process. If it is not necessary to change to a natural sentence, nothing is processed in step S2108. Moreover, since the technique for changing a sentence to a natural sentence is a known technique, detailed description thereof is omitted. As a technique for changing a sentence to a natural sentence, for example, a statistical-based method is used.

  Next, details of the user status information update processing in step S1808 will be described with reference to the flowchart of FIG.

  (Step S2201) The user presentation term acquisition unit 181 of the user state information update unit 18 acquires one or more determinants from the latest (immediately preceding) sentence received by the reception unit 34. When one or more determination factors cannot be acquired from the sentence received immediately before, the user-presented term acquisition unit 181 acquires the focus determination factor. Further, the user-presented term acquisition unit 181 acquires one or more spots from the latest (immediately preceding) sentence received by the receiving unit 34. Then, the user-presented term acquisition unit 181 temporarily stores the acquired determination factor or / and the acquired spot in the buffer.

  (Step S2202) The preference vector update unit 183 reads the preference vector included in the user status information in the user status information storage unit 13. Then, the preference vector update unit 183 updates the preference vector included in the user state information so that the value of the element corresponding to the determination factor acquired in step S2201 is increased. Further, the attribute vector update unit 185 corresponds to the determination factor so that the value of the element corresponding to the determination factor acquired in step S801 is increased, or to the value of the element corresponding to the determination factor. The attribute vector included in the user state information is updated so as to approach the value of the element to be performed.

  (Step S2203) The device presentation term acquisition means 182 acquires one or more determinants from the latest (immediately preceding) sentence output by the sentence output unit 37. Further, the device presentation term acquisition unit 182 acquires one or more spots from the latest (immediately preceding) sentence output by the sentence output unit 37. Then, the device presentation term acquisition unit 182 temporarily stores the acquired determination factor or / and the acquired spot in the buffer.

  (Step S2204) The knowledge vector update unit 184 reads the knowledge vector included in the user state information in the user state information storage unit 13. The knowledge vector update unit 184 updates the knowledge vector included in the user state information so that the value of the element corresponding to the determination factor acquired in step S2203 is increased.

  (Step S2205) The user state information update unit 18 reads the number of conversation turns included in the user state information in the user state information storage unit 13. The number of dialogue turns is a logarithm of dialogue repeated. Then, the user status information update unit 18 updates the user status information in the user status information storage unit 13 with a value obtained by adding 1 to the read number of turns as a new number of turns.

  (Step S2206) The user state information update unit 18 updates the immediately preceding user utterance action information. The immediately preceding user utterance action information is information related to the sentence received by the receiving unit 34 immediately before, and is information corresponding to the type of information requested by the user (spot only, determinant name only, or both).

  (Step S2207) The user state information update unit 18 updates the immediately preceding system utterance action information. The immediately preceding system utterance action information is information related to the sentence output by the sentence output unit 37 immediately before, and is information for specifying the selected information recommendation method.

  (Step S2208) The user status information update unit 18 updates the system presentation history information, and returns to the upper process. The system presentation history information is the number of spots and determination factors output by the interactive device 1 (system). The user status information update unit 18 uniquely processes the determination factor and the spot written in the buffer in step S2203, respectively, and acquires the number of determination factors and the number of spots in the buffer. Then, the user status information update unit 18 acquires the number of determining factors and the number of spots in the buffer as system presentation history information.

  In the flowchart of FIG. 22, the information updated in steps S2205 to S2208 is an example of information constituting the user status information, and other information may constitute the user status information.

  Hereinafter, a specific operation of the interactive apparatus 3 in the present embodiment will be described. FIG. 23 is a conceptual diagram of the dialogue apparatus 1. In this specific example, the dialogue device 3 is a system that assists the decision making of the user's visit destination while promoting dialogue with the user, and provides sightseeing guidance in Kyoto.

  An example of the spot information management table held by the knowledge base 11 is the above-described FIG. An example of the information recommendation method management table in the information recommendation method storage unit 12 is the above-described FIG.

In addition, it is assumed that the user status information storage unit 13 stores user status information described below. For example, it is assumed that the user state information has three elements: a knowledge vector “K _user ”, a preference vector “P _user ”, and a local weight matrix “V _user ”. Here, for the sake of simplicity, the elements of the _user preference vector “P _user = (p ₁ , p ₂ ,..., P _M )” are parameters consisting of binary values of “1” or “0”. And That is, the user is interested in is determinant m (or potentially interested), p _m is assumed to take the "1" when emphasized in determining spot. In addition, in order to express a state in which the user has a potential preference (the user himself / herself is unaware), the user's knowledge vector “K _user = (k ₁ , k ₂ ,..., K _M ) Is introduced. When the user knows that the system (dialogue device 3) can handle the determination factor m, or when the system recommends the determination factor m, the vector element k _M takes “1”. By using these vectors, for example, determinants m is is a factor that the user has potentially interesting, the state that the user is not aware of it _{(k m = 0, p m} = 1) Can be expressed as Further, assuming that the user determines only the local weight v _nm for the spot n from the viewpoint of the user's determinant m from the information presented from the system, the system uses “ID = “1” is assumed when the user is informed of the evaluation of the spot using any one of the recommended methods “1, 2, 6”.

The user state information here corresponds to a feature vector of the dialog state. More specifically, here, the user status information includes the following six types of information. The first is the number of turns. Here, the number of turns is expressed by five parameters using a sawtooth function. The second is immediately preceding user utterance action information. The immediately preceding user utterance action information is, for example, a vector (x _i , x _{i + 1} , x _{i + 2} , x _{i + 3} , x _{i + 4} ), and (1 if a _user ^t−1 = x _i , otherwise 0). Here, “a _user ^t−1 ” is the utterance of the previous user. Also, if the utterance of the previous user includes only a spot (which may be referred to as a spot name) recommended by the system (dialogue device 3) or a determinant, “x _i = 1” and other vector elements Is “0”. When a spot not recommended by the system is included, “x _{i + 1} = 1” and the elements of other vectors are “0”. When only the determinants not recommended by the system are included, “x _{i + 2} = 1” and the elements of the other vectors are “0”. Further, when both spots and determinants not recommended by the system are included, “x _{i + 3} = 1” and the elements of other vectors are “0”. Further, when none of these is included, “x _{i + 4} = 1” and the elements of the other vectors are “0”. The third is the immediately preceding system utterance action information. The immediately preceding system utterance action information is, for example, a vector (y _i , y _{i + 1} , y _{i + 2} , y _{i + 3} , y _{i + 4} , y _{i + 5} , y _{i + 6} ), and (1 if a _sys ^t-1 = y _i , otherwise 0) Here, “a _sys ^t-1 ” is the utterance of the immediately preceding system. Further, when the information recommendation method 1 is used as the utterance of the immediately preceding system, “y _{i + 1} = 1” and the elements of the other vectors are “0”. When the information recommendation method 2 is used, “y _{i + 2} = 1”, and the elements of the other vectors are “0”. That is, when the information recommendation method n is used, “y _{i + n} = 1”, and the elements of other vectors are “0”. The fourth is a knowledge vector for a user's determinant. Note that the knowledge vector may be calculated by “Σ _{n = 1} ^N Pr (k _n = 1)”, which is the knowledge of the user's decision factor. Here, k _n is the element values of the knowledge vector for the n th determinant. Pr (k = 1) is a posterior probability (certainty factor) estimated by the system that “k is 1”. Here, it is assumed that the element value of the preference vector is “1” or “0”. The fifth is the number of spots and determinants presented by the system. That is, the fifth item is system presentation history information, for example, “Σ _{n = 1} ^N Σ _{m = 1} ^M v _nm ”. Here, v _nm is an evaluation value for the determinant n of the spot m that the interactive apparatus 3 has. The sixth is a preference vector. The preference vector is information indicating a user's preference for each determining factor. Note that the preference vector may be replaced with an expected value of the probability that the user attaches importance to the determination factor. This expected value is indicated by “Pr (k _n = 1) × Pr (p _n = 1))”, and has 10 parameters in total for each decision factor. Here, “p _n ” is an element value of the preference vector. Here, it is assumed that the element value of the preference vector is “1” or “0”.

  The user state information stored in the user state information storage unit 13 is a vector having the element values described above, and is a vector (s1, s2,..., S29) having 29 element values. , And. The initial value of the user state information is (s0001, s0002,..., S0029).

  Further, it is assumed that the sentence output unit 37 stores an initial sentence “Kyoto sightseeing system. Recommend your favorite sightseeing spot”, which is a sentence output when the interactive device 3 is activated.

  Further, it is assumed that the reception unit 34 holds the sentence patterns “go to <spot>” and “determined <spot>”, which are conditions for terminating the conversation. Here, <spot> is a variable to which a spot can be substituted.

  In this state, it is assumed that the user has activated the dialogue apparatus 3. Next, the sentence output unit 37 reads the initial sentence and outputs the initial sentence “Kyoto sightseeing system. Recommend your favorite sightseeing spot” by voice.

  Next, it is assumed that the user inputs a voice message “Tell me about cherry blossoms at Ninna-ji Temple”. Then, the voice reception means 341 of the reception unit 34 receives the voice “Tell me about cherry blossoms at Ninna-ji.” Next, the voice recognition means 342 recognizes this voice and acquires the sentence “Tell me about cherry blossoms at Ninna-ji.”

  Next, the reception unit 34 changes to the sentence pattern “I will go to <spot>” or “I decided to <spot>” with the input sentence “Tell me about cherry blossoms at Ninna-ji.” Judge that they do not match.

  Next, the score calculation unit 15 calculates six scores for each information recommendation method. That is, first, the score calculation unit 15 reads the user state information (s0001, s0002,..., S0029) from the user state information storage unit 13.

  Next, the score calculation unit 15 stores in the information recommendation method storage unit 12 the evaluation information (vector), the weight vector, and the user status information (s0001, s0002, , S0029) for each information recommendation method. And the score calculation part 15 calculates the score of six information recommendation methods.

  Next, the sentence composing unit 36 performs natural language processing on the immediately preceding user input sentence, and acquires the spot “Ninna-ji”. In addition, the sentence composing unit 36 performs natural language processing on the immediately preceding user input sentence, and acquires the determination factor “sakura”. For example, the sentence composing unit 36 performs morphological analysis on the user input sentence, and acquires the independent words “Ninnaji”, “Sakura”, and “Teach me”. Then, the sentence composition unit 36 searches the knowledge base 11 using the three independent words as keys, detects that “Ninna-ji” is a spot and “Sakura” is a determining factor, and determines the spot “Ninna-ji” and the determination Get the factor “Sakura”.

  Next, the sentence composing unit 36 substitutes the acquired spot “Ninna-ji” for the variable “spot of interest”. In addition, the sentence composition unit 36 substitutes the acquired determinant “sakura” for the variable “focus determinant”.

  Next, the sentence composition unit 36 searches the knowledge base 11 using the value “Ninna-ji” of the variable “focused spot” and the value “cherry blossom” of the variable “focused determination factor”, and searches for the focused spot and the focused determination factor. Read from the knowledge base 11 "Omuro Sakura is a rare cherry tree with a low tree height and a single fragrant white flower that blooms from the base. The flowering time is late and decorates the end of spring in Kyoto." This explanatory text becomes an answer text.

  Next, the sentence composing unit 36 performs a recommendation sentence acquisition process. That is, first, the sentence pattern information acquisition unit 361 of the sentence composing unit 36 has one information recommendation method corresponding to the largest score among the six scores calculated by the score calculation unit 15 (in this case, the method of ID = 1). Is acquired from the information recommendation method management table of FIG. 12. The sentence pattern information “<the spot of interest> is <one or more undecided factors>.

  Next, the variable value acquisition unit 362 acquires the first variable <target spot> from the sentence pattern information. Next, the variable value acquisition unit 362 acquires one or more terms (the value “Ninnaji” of the variable “target spot”) that is substituted into the variable <target spot>. Next, the sentence composing means 363 substitutes the acquired one or more terms “Ninna-ji” into the location of the first variable in the sentence pattern information. > Would you explain something?

  Next, the variable value acquisition unit 362 acquires the second variable <one or more undecided determination factors> from the sentence pattern information. Then, the variable value acquisition unit 362 acquires a variable <one or more undecided determination factors>. That is, the variable value acquisition unit 362 has the evaluation value of the value “Ninna-ji” of the variable “spot of interest” as “1”, and the “scenery is good” Is famous, "Colorful leaves are famous", "World Heritage", "Event is". Then, the sentence composing means 363 substitutes the acquired one or more terms into the second variable location in the sentence pattern information, and says, “Ninna-ji Temple has good scenery, famous gardens, and autumn leaves. There are famous, world heritage, and events. Next, the sentence composition means 363 is a natural sentence "Ninna-ji Temple is" Ninna-ji Temple has a good view, garden is famous, autumn leaves are famous, there are world heritage, and an event. The view is good, the garden is famous for its autumn leaves, it is a World Heritage site, and there are events.

In other words, the above-described processing is executed as “Method 1 ((Spot _Ninnaji ), (Det _scenery , Det _garden , Det _{autumn leaves} , Det _{world heritage} , Det _event ))”. Method1 () means that the information recommendation method 1 is applied. “Spot _Ninna-ji ” is the spot “Ninna-ji”, “Det _Landscape ” is the determinant “Scenery”, “Det _Garden ” is the determinant “Garden”, “Det Colored _Leaves ” is the determinant “Autumn Leaves”, “Det _{World Heritage} ” Means determinant “world heritage”, “Det _event ” means determinant “event”.

  Next, the sentence output unit 37 outputs an answer sentence “Omuro Sakura is a rare cherry tree with a low tree height and a single fragrant white flower that blooms from the root. The flowering time is late and decorates the end of spring in Kyoto.” . Subsequently, the sentence output unit 37 outputs a recommendation sentence “Ninna-ji Temple has a good view, is famous for its garden and autumn leaves, is a World Heritage site, and has an event.

  Next, the user status information update unit 18 performs user status information update processing as follows. That is, first, the user presentation term acquisition unit 181 of the user status information update unit 18 acquires one or more determinants “sakura” from the latest sentence “Tell me about cherry blossoms at Ninna-ji Temple” received by the reception unit 34. To do.

  And the preference vector update means 183 reads the preference vector which the user state information of the user state information storage part 13 has. Then, the preference vector update unit 183 updates the preference vector included in the user state information so that the value of the element corresponding to the acquired determination factor “Sakura” becomes large. It does not matter how large the value of the element corresponding to the determinant factor “sakura” is. The value to be increased may be a fixed value, a fixed ratio, or dynamically change.

  Next, the device presentation term acquisition means 182 uses the latest recommended sentence output by the sentence output unit 37: “Ninnaji Temple has a good view, is famous for its garden, autumn leaves, is a World Heritage site, and has an event. Get one or more determinants “scenery” “garden” “autumn leaves” “world heritage” “event” from

  Next, the knowledge vector update unit 184 reads the knowledge vector included in the user state information in the user state information storage unit 13. The knowledge vector update means 184 updates the knowledge vector included in the user state information so that the value of the element corresponding to the acquired determinants “scenery”, “garden”, “autumn leaves”, “world heritage”, “event” becomes large. . It does not matter how much the value of the element corresponding to the determinant “scenery” or the like is increased. The value to be increased may be a fixed value, a fixed ratio, or dynamically change.

  Next, the user state information update unit 18 reads the number of conversation turns “0” included in the user state information in the user state information storage unit 13. Then, the user state information update unit 18 updates the user state information in the user state information storage unit 13 with the value “1” obtained by adding 1 to the read turn number “0” as the new turn number.

  Next, the user state information update unit 18 updates the immediately preceding user utterance action information from “0” to “1”.

  Next, the user status information update unit 18 updates the immediately preceding system utterance action information from “0” to “1”.

  Next, the user status information update unit 18 updates the number of spots constituting the system presentation history information from “0” to “1” (since “Ninna-ji” was output). In addition, the user status information update unit 18 changes the number of determinants constituting the system presentation history information from “0” to “5” (because “landscape”, “garden”, “autumn leaves”, “world heritage”, and “event” are output). Update to

  As described above, the user status information is updated to the latest value (s0101, s0102,..., S0129).

  Next, it is assumed that the user has interacted with the dialog device 3 several times, and the fifth utterance “How is the scenery here?” Is voice input by the user. The current spot of interest is “Ninna-ji”.

  Next, the voice reception means 341 of the reception unit 34 receives a voice sentence “How is the scenery here?” From the user.

  Next, the voice recognition unit 342 of the reception unit 34 recognizes the received sentence “How is the scenery here?” And acquires a sentence of a character string.

  Next, the reception unit 34 does not match the sentence pattern “go to <spot>” or “decided to <spot>” whose input condition is “How is the scenery here?” Judge that.

  Next, the score calculation unit 15 calculates six scores for each of the six information recommendation methods in the same manner as described above.

  That is, first, the score calculation unit 15 reads the current user state information (s0501, s0502,..., S0529) from the user state information storage unit 13.

  Next, the score calculation unit 15 stores in the information recommendation method storage unit 12 the evaluation information (vector), the weight vector, and the user state information (s0501, s0502, each of the first to sixth information recommendation methods. .., S0529) for each information recommendation method. And the score calculation part 15 calculates the score of six information recommendation methods.

  Next, the sentence composing unit 36 performs natural language processing on the immediately preceding user input sentence, and acquires the determination factor “scenery”. In addition, the sentence structure part 36 was not able to acquire a spot from a user input sentence here.

  Next, the sentence composition unit 36 substitutes the acquired determination factor “scenery” for the variable “focused determination factor”.

  Next, the sentence composition unit 36 searches the knowledge base 11 using the value “Ninna-ji” of the variable “focused spot” and the value “scenery” of the variable “focused determination factor” to find the focused spot and the focused determination factor. Read from the knowledge base 11 the explanation corresponding to “From the top of Sanmon. This explanatory text becomes an answer text.

  Next, the sentence composing unit 36 performs a recommendation sentence acquisition process. That is, first, the sentence pattern information acquisition unit 361 of the sentence composing unit 36 is one information recommendation method corresponding to the largest score among the six scores calculated by the score calculation unit 15 (in this case, the method of ID = 2). Is acquired from the information recommendation method management table of FIG. 12. <Focus determining factor> Then, <1 or more unseen spots> etc. can be introduced].

  Next, the variable value acquisition unit 362 acquires the first variable <focus determination factor> from the sentence pattern information. Next, the variable value acquisition unit 362 acquires one or more terms (the value “scenery” of the variable “focus determination factor”) that is substituted into the variable <focus determination factor>. Next, the sentence composing means 363 substitutes the acquired one or more terms “good scenery” into the position of the first variable in the sentence pattern information, and “if the scenery is good, <1 or more Unspotted spot> etc. can be introduced. Note that, when assigning a value to a variable related to “determination factor”, the variable value acquisition unit 362 acquires the attribute value “determination factor” in FIG. 11.

  Next, the variable value acquisition unit 362 acquires the second variable <one or more unspotted spots> from the sentence pattern information. Then, the variable value acquisition unit 362 acquires a variable <one or more unspotted spots>. In other words, the variable value acquisition means 362 has a value “1” for the value “scenery” of the variable “focused determinant”, and spots “Kiyomizu-dera” and “Fushimi Inari Taisha” excluding the existing determinant “Ninna-ji”. “Kuramaji” is acquired from the knowledge base 11 (FIG. 11). Next, the sentence composing means 363 substitutes the acquired terms “Kiyomizu-dera”, “Fushimi Inari-taisha” and “Kurama-ji” for the second variable in the sentence pattern information. Kiyomizu Temple, Fushimi Inari Taisha Shrine, Kurama Temple, etc. ”can be introduced. Next, the sentence composing means 363 tries to convert “If the scenery is good, you can introduce Kiyomizu Temple, Fushimi Inari Taisha, Kurama Temple, etc.” to natural sentences, but there is no need to change and recommend Acquired the sentence “If you have a good view, you can introduce Kiyomizu Temple, Fushimi Inari Taisha Shrine, Kurama Temple, etc.”

In other words, the above-described processing is executed “Method 2 ((Spot _{Kiyomizu Temple} , Spot _{Fushimi Inari Taisha} , Spot _{Kurama Temple} ), (Det _Landscape ))”. Method2 () means that the information recommendation method 2 is applied. “Spot _{Kiyomizu-dera} ” etc. means the spot “Kiyomizu-dera” etc.

  Next, the sentence output unit 37 outputs an answer sentence “From the top of Sanmon. Subsequently, the sentence output unit 37 outputs a recommendation sentence “If the scenery is good, you can introduce Kiyomizu Temple, Fushimi Inari Taisha, Kurama Temple, etc.”.

  Next, the user status information update unit 18 performs user status information update processing as follows. That is, first, the user presentation term acquisition unit 181 of the user state information update unit 18 acquires one or more determinants “scenery” from the latest sentence “how is the scenery here” received by the reception unit 34.

  And the preference vector update means 183 reads the preference vector which the user state information of the user state information storage part 13 has. Then, the preference vector updating unit 183 updates the preference vector included in the user state information so that the value of the element corresponding to the acquired determination factor “scenery” becomes large. It does not matter how large the value of the element corresponding to the determining factor “scenery” is.

  Next, the device presentation term acquisition means 182 has one or more of the following recommended sentences output by the sentence output unit 37: “If the scenery is good, Kiyomizu Temple, Fushimi Inari Taisha, Kurama Temple, etc. can be introduced.” Get the determinant "scenery".

  Next, the knowledge vector update unit 184 reads the knowledge vector included in the user state information in the user state information storage unit 13. The knowledge vector update unit 184 updates the knowledge vector included in the user state information so that the value of the element corresponding to the acquired determination factor “scenery” becomes large. For example, when the value of the element corresponding to the determination factor “scenery” is already the maximum value, the element value does not change.

  Next, the user state information update unit 18 reads the number of conversation turns “5” included in the user state information in the user state information storage unit 13. Then, the user state information update unit 18 updates the user state information in the user state information storage unit 13 with the value “6” obtained by adding 1 to the read turn number “5” as the new turn number.

  Next, the user status information update unit 18 keeps the previous user utterance action information as “1”. Further, the user status information update unit 18 keeps the previous system utterance action information as “1”.

  Next, the user status information update unit 18 updates the number of spots constituting the system presentation history information from “1” to “4” (since “Kiyomizu Temple”, “Fushimi Inari Taisha” and “Kurama Temple” are output). Since “scenery” has already appeared, the user state information update unit 18 keeps the number of determining factors constituting the system presentation history information as “5”.

  As described above, the user status information is updated to the latest value (s0601, s0602,..., S0629).

  Next, it is assumed that the user hears the answer sentence and the recommendation sentence output by the dialogue apparatus 3 and answers “Kiyomizu Temple” (sixth utterance).

  Next, the voice reception unit 341 of the reception unit 34 receives a voice sentence “Kiyomizu Temple” from the user.

  Next, the voice recognition means 342 of the reception unit 34 recognizes the received sentence “Kiyomizu-dera” and acquires a sentence of a character string.

  Next, the reception unit 34 determines that the input sentence “Kiyomizu-dera” does not match the sentence pattern “go to <spot>” or “determined <spot>” as the end condition.

  Next, the score calculation unit 15 calculates six scores for each of the six information recommendation methods in the same manner as described above.

  That is, first, the score calculation unit 15 reads the current user status information (s0601, s0602,..., S0629) from the user status information storage unit 13.

  Next, the score calculation unit 15 stores in the information recommendation method storage unit 12 the evaluation information (vector), the weight vector, and the user state information (s0601, s0602, each of the first to sixth information recommendation methods. , S0629) is multiplied for each information recommendation method. And the score calculation part 15 calculates the score of six information recommendation methods.

  Next, the sentence composing unit 36 performs natural language processing on the immediately preceding user input sentence and acquires the spot “Kiyomizu Temple”. In addition, the sentence structure part 36 was not able to acquire a decision factor from a user input sentence here.

  Next, the sentence composing unit 36 substitutes the acquired spot “Kiyomizu-dera” for the variable “target spot”. Note that the current variable “Focus Determination Factor” is “Scenery”.

  Next, the sentence composition unit 36 searches the knowledge base 11 using the value “Kiyomizu-dera” of the variable “focused spot” and the value “scenery” of the variable “focused determination factor”, and sets the focused spot and the focused determination factor. The corresponding explanation “The stage of Shimizu is built on the slope and the cityscape you want from here is wonderful” is read from the knowledge base 11. This explanatory text becomes an answer text.

  Next, the sentence composing unit 36 performs a recommendation sentence acquisition process. That is, first, the sentence pattern information acquisition unit 361 of the sentence composition unit 36 has one information recommendation method corresponding to the largest score among the six scores calculated by the score calculation unit 15 (here, the method of ID = 4). The sentence pattern information possessed by “Other than 1 or more undecided determinants> {Determinants of select 3 or less where where the value in the knowledge vector is low} can be explained.” Obtain from the recommended method management table.

  Next, the variable value acquisition unit 362 acquires the first variable <one or more undecided determination factors> from the sentence pattern information. Next, the variable value acquisition means 362 has one or more terms assigned to the variable <one or more undecided determinants> in accordance with the acquisition operation description {decision factor of select 3 or less where value in the knowledge vector is low}. (The value “scenery” of the variable “focus determination factor”) is acquired. That is, the variable value acquisition means 362 has three of the determinants “congested”, “world heritage”, “walk” other than the previously determined determinants “cherry blossom” “landscape”, etc. (Here, “World Heritage” “Walk” “History”). Next, the sentence composing means 363 substitutes the acquired three determinants “world heritage”, “walk”, and “history” into the first variable part in the sentence pattern information, and “other world heritage, Can explain the famous places that can be walked in history. "

In other words, the above-mentioned process, so that you run the "Method4 ((NULL), (Det _{World Heritage Site,} Det _walk, Det _history))". Method4 () means that the information recommendation method 4 is applied. “Det _{World Heritage} ” etc. means determinant “World Heritage” etc.

  Next, the sentence output unit 37 outputs an answer sentence “The stage of Shimizu is built on a slope and the scenery of the city you want from here is wonderful”. Subsequently, the sentence output unit 37 outputs a recommendation sentence “I can explain other world heritage sites, walkable places, famous places in history, etc.”.

  Next, it is assumed that the user state information update unit 18 performs the user state information update process in the same manner as described above, and the user state information is updated to the latest value (s0701, s0702,..., S0729).

  After that, it is assumed that the dialogue between the user and the dialogue device 3 proceeds, and the user voice-inputs “I will go to Nanzenji” in the 16th utterance.

  Next, the voice reception means 341 of the reception unit 34 receives a voice sentence “I will go to Nanzenji” from the user.

  Next, the speech recognition means 342 of the reception unit 34 recognizes the received sentence “I will go to Nanzenji” and acquires a sentence of a character string.

  Next, the reception unit 34 determines that the input sentence “I will go to Nanzenji” matches the sentence pattern “I will go to <spot>”, which is the end condition. This is because the sentence “Let's go to Nanzenji” includes a sentence that matches the sentence pattern “Go to <spot>.”

  Then, the process ends. At the end of the process, the sentence output unit 37 determines a predetermined sentence “Thank you for using the Kyoto Tourist Information System” and a predetermined sentence pattern “<Spot>”. Do you have anything else you want to know? " Here, the sentence output unit 37 has decided to Nanzenji from the sentence pattern “I decided to <spot>. Do you have anything else to know?” Is there anything else you want to know? ? "Is output.

  The flow of the above dialogue is shown in FIG. In FIG. 24, S [numerical value] indicates an utterance from the dialogue apparatus 3 (system), and U [numeric value] indicates an utterance from the user. Ans indicates an answer sentence, and Rec indicates a recommendation sentence.

  As described above, according to the present embodiment, it is possible to appropriately support the user's decision making by performing the conversation while dynamically changing the information related to the user's knowledge and preferences according to the progress of the conversation with the user. .

  In addition, according to this Embodiment, the content of user state information does not ask | require. Further, it does not matter how the user status information is changed according to the progress of the dialogue with the user. However, a value related to a preference (such as an element value constituting a preference vector) that is a spot or determinant included in the user's utterance and is not a negative content is selected (selected by the dialogue apparatus 3). To make it easier). In addition, the value (such as element values constituting the knowledge vector) related to the knowledge about the spots and determination factors included in the utterance output from the dialog device 3 is increased (to be easily selected by the dialog device 3).

Further, an example of changing the user state information by dialogue in the present embodiment will be described in detail below. The knowledge vector of the prior user state information is “Ksys = (0.22, 0.01, 0.02, 0.18,...)”, And the prior preference vector is “Psys = (0.37). , 0.19, 0.48, 0.38, ...) ". Then, it is assumed that the dialogue apparatus 3 has selected the information recommendation method 1 in response to the user's utterance. In other words, it is assumed that the dialogue apparatus 3 executes “a _sys = Method1 {(Spot ₅ ), (Det ₁ , Det ₃ , Det ₄ )}”. Then, it is assumed that the dialogue apparatus 3 has obtained the recommendation sentence “Regarding Ninnaji (Spot ₅ ), it can explain the garden (Det ₁ ), the world heritage (Det ₃ ), and the autumn leaves (Det ₄ )”. Then, it is assumed that the user inputs “Tell me about World Heritage (Det ₃ )”.

Then, the user state information update unit 18 uses the subsequent knowledge vector “Ksys = ((1.00, 0.01, 1.00, 1.00,...)” And the subsequent preference vector “Psys = (0 .26, 0.19, 0.65, 0.22, ...) ", and the vector is (garden, crowded, world heritage, autumn leaves, scenery, access, cherry blossom, history, walk, event) As the determinants “garden”, “world heritage”, and “colored leaves” are output from the dialogue device 3 (system), the subsequent knowledge vector “garden” “world heritage” The element value of “Autumn leaves” has improved, and the element value of “World Heritage” in the preference vector has increased due to the input of the determinant “World Heritage” by the user. "By not selecting" Autumn leaves " Element values of the "garden", "autumn leaves" of good vector is reduced.
In the present embodiment, the interactive apparatus using the weight vector learned in the first embodiment has been described. However, the weight vector used by the dialogue apparatus 3 in the present embodiment may not be the weight vector learned in the first embodiment. That is, the dialogue apparatus 3 and the simulation apparatus 2 described in the first embodiment may not be linked.

  Further, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. Note that the software that realizes the interactive device 3 in the present embodiment is the following program. That is, this program has a spot having a spot, one or more determinants that are factors for determining the spot, and an evaluation value indicating evaluation of each of the one or more determinants of the spot on the storage medium. Used when selecting a knowledge base that stores two or more pieces of information, sentence pattern information that indicates a sentence pattern output by the interactive device or a sentence pattern output by the interactive device, and the sentence pattern information Two or more information recommendation methods having evaluation information of sentence pattern information to be performed; information indicating user status; preference vector which is information indicating user preference for one or more determination factors; and one or more information User status information having knowledge vectors indicating the user's knowledge for each determinant is stored, and the computer accepts a sentence input by the user And a score calculation unit that applies the user status information to evaluation information of each of the two or more information recommendation methods stored in the storage medium and calculates two or more scores for the two or more information recommendation methods And using the two or more scores calculated by the score calculation section, the sentence pattern information possessed by one information recommendation method is acquired, and a sentence composition section that composes a sentence from the sentence pattern information; and the sentence composition section Acquire at least one spot or one or more determinants from one or more sentences out of a sentence output part that outputs the composed sentence and a sentence received by the reception part or a sentence output by the sentence output part. The score calculation is a program that functions as a user status information update unit that updates user status information of the storage medium using the one or more spots or one or more determining factors. Applies the user status information updated by the user status information update unit to the evaluation information of each of the two or more information recommendation methods stored in the information recommendation method storage unit, and the two or more information recommendation methods This is a program that functions as one that calculates a score of 2 or more for.

  In the above program, the sentence composing unit includes sentence pattern information obtaining means for obtaining sentence pattern information included in one information recommendation method corresponding to the largest score among two or more scores calculated by the score calculating unit; , One or more variables included in the sentence pattern information acquired by the sentence pattern information acquisition unit, and a spot or determination factor corresponding to the variable, the sentence output by the sentence output unit immediately before, or the reception unit The variable value acquisition unit that acquires from one or more of the sentences received immediately before, and the term acquired by the variable value acquisition unit in the variable part of the sentence pattern information acquired by the sentence pattern information acquisition unit It is preferable to make a computer function as what comprises the sentence structure means which inserts and comprises a sentence.

  In the above program, the variable value acquisition unit acquires one or more variables included in the sentence pattern information acquired by the sentence pattern information acquisition unit, and is one or more spots or 1 that are candidates corresponding to the variable. The above determinants are obtained from one or more sentences of the sentence output by the sentence output unit or the sentence received by the reception unit immediately before, and are one or more spots or one or more candidates as the candidates. Let the computer function as one that selects a spot or a determinant corresponding to the variable using one or more spots as candidates or an evaluation value of the knowledge base corresponding to one or more determinants from the determinants That is preferred.

  In the above program, the user status information update unit includes a user presentation term acquisition unit that acquires at least one or more determinants from the sentence received by the reception unit, and one of the sentences output by the sentence output unit. From the above sentence, the device presentation term acquisition means for acquiring at least one or more determination factors, and the preference for updating the preference vector element value for the one or more determination factors acquired by the user presentation term acquisition means. It is possible to cause a computer to function as comprising a vector updating unit and a knowledge vector updating unit that updates the value of an element of a knowledge vector for one or more determinants acquired by the device presentation term acquisition unit. Is preferred.

  In the above program, the reception unit includes a voice reception unit that receives a voice input by a user, and a voice recognition unit that recognizes the voice and converts the voice into a character string. It is preferable to make a computer function as a voice output of a sentence formed by the component.

  FIG. 25 shows the external appearance of a computer that executes the program described in this specification to realize the interactive apparatus or the like of the above-described embodiment. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 25 is an overview diagram of the computer system 340, and FIG. 26 is a block diagram of the computer system 340.

  25, the computer system 340 includes a computer 341 including an FD drive and a CD-ROM drive, a keyboard 342, a mouse 343, a monitor 344, and a microphone 345. Note that the dialogue apparatuses 1 and 3 and the simulation apparatus 2 do not have to include the microphone 345.

  26, in addition to the FD drive 3411 and the CD-ROM drive 3412, the computer 341 stores an MPU 3413, a bus 3414 connected to the CD-ROM drive 3412 and the FD drive 3411, and a program such as a bootup program. A RAM 3416 for temporarily storing application program instructions and providing a temporary storage space; and a hard disk 3417 for storing application programs, system programs, and data. Although not shown here, the computer 341 may further include a network card that provides connection to the LAN.

  A program for causing the computer system 340 to execute the functions of the interactive apparatus or the like of the above-described embodiment is stored in the CD-ROM 3501 or FD 3502, inserted into the CD-ROM drive 3412 or FD drive 3411, and further stored in the hard disk 3417. May be forwarded. Alternatively, the program may be transmitted to the computer 341 via a network (not shown) and stored in the hard disk 3417. The program is loaded into the RAM 3416 at the time of execution. The program may be loaded directly from the CD-ROM 3501, the FD 3502, or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 341 to execute the functions of the interactive device according to the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 340 operates is well known and will not be described in detail.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that two or more communication units existing in one apparatus may be physically realized by one medium.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the learning system according to the present invention has an effect that an interactive device that interacts with a user can automatically construct a weight vector necessary for outputting a sentence, and is useful as a learning system. is there.

DESCRIPTION OF SYMBOLS 1, 3 Dialogue apparatus 2 Simulation apparatus 11 Knowledge base 12 Information recommendation method storage part 13 User state information storage part 14 User input information reception part 15 Score calculation part 16 Dialogue sentence information structure part 16 Dialogue sentence sentence information structure part 17 Dialogue sentence output Unit 18 User status information update unit 21 Dialog information storage unit 22 User preference vector storage unit 23 Dialogue sentence information reception unit 24 User sentence type determination unit 25 Determination factor acquisition unit 26 User input information transmission unit 27 Reward calculation unit 28 Learning unit 34 Reception unit 36 Sentence configuration unit 37 Sentence output unit 161 Method identifier acquisition unit 162, 362 Variable value acquisition unit 163 Dialogue sentence information configuration unit 181 User presentation term acquisition unit 182 Device presentation term acquisition unit 183 Preference vector update unit 184 Knowledge vector update unit 271 Random selection match value calculation means 2 2 Select spot matching degree calculating unit 273 reward calculation means 341 voice accepting unit 342 speech recognition means 361 sentence pattern information acquiring unit 363 sentences constituting unit

Claims (5)

A learning system comprising a dialog device and a simulation device for simulating a dialog about a spot, and a learning system for learning a weight vector used when determining a sentence output by the dialog device,
The interactive device is:
Knowledge storing two or more pieces of spot information including a spot, one or more determinants that are factors for determining the spot, and an evaluation value indicating the evaluation of each of the one or more determinants of the spot Base and
Information recommendation method storage that stores two or more information recommendation methods having a method identifier for identifying an information recommendation method, evaluation information of the information recommendation method, and a weight vector indicating the weight of each element constituting the evaluation information And
Stores user state information including information indicating a user state and a preference vector which is information indicating a user's preference for one or more determinants and a knowledge vector indicating a user's knowledge for one or more determinants A user status information storage unit,
From the simulation device, a user sentence type identifier that identifies a user sentence type that is a pattern of a sentence input by a user, or a user sentence type identifier and one or more information of one or more determinants or one or more spots A user input information receiving unit for receiving user input information,
Using the evaluation information and weight vector of each of the two or more information recommendation methods stored in the information recommendation method storage unit, and the user status information, two or more scores for the two or more information recommendation methods are obtained. A score calculation unit to calculate,
Using two or more scores calculated by the score calculation unit, a method identifier for identifying one information recommendation method, or a method identifier, and one or more determinants or one or more information of one or more spots A dialogue sentence information constituting unit constituting dialogue sentence information having;
Dialog message output unit configured to send dialog message information configured by the dialog message information configuration unit to the simulation device;
Acquiring at least one spot or one or more determinants from one or more pieces of information of user input information received by the user input information receiving unit or dialogue sentence output by the dialogue sentence output unit; Using one or more spots or one or more determinants, and a user state information update unit for updating user state information in the user state information storage unit,
The score calculation unit
Each of the two or more pieces of information using the evaluation information and the weight vector of each of the two or more pieces of information recommendation methods stored in the information recommendation method storage unit and the user state information updated by the user state information update unit. Calculate a score of 2 or higher for the recommendation method,
The simulation apparatus includes:
Dialogue probability information that is information about the probability of each information recommendation method and each user sentence type, determinant probability information that is information about the probability that a determinant is selected, and spot probability information that is information about the probability that a spot is selected A dialogue information storage unit capable of storing
A user preference vector storage unit that can store a user preference vector that is a vector indicating the user's preference;
A dialog text information receiving unit that receives dialog text information from the dialog device;
A user sentence type determining unit that determines a user sentence type using the technique identifier and the conversation probability information included in the dialog sentence information, and acquires a user sentence type identifier;
One or more information of the determinant probability information or the spot probability information, or one or more determinants of the one or more information of the determinant probability information or the spot probability information and the dialogue sentence information or 1 Using one or more pieces of information among the above spots, one or more determining factors or a determining factor acquiring unit for acquiring one or more spots,
A user input information sending unit for sending user input information having the user sentence type identifier or the user sentence type identifier and one or more determinants or one or more information of one or more spots to the interactive device;
The user preference vector and one or more evaluation values indicating the evaluation of each of the one or more determinants of the spot included in the user input information are acquired, and the user preference vector matches the one or more evaluation values A reward calculating unit for calculating a reward for selecting a user sentence type identified by the user sentence type identifier using the degree of match;
A learning system comprising: a learning unit that uses the reward to update a weight vector corresponding to the method identifier of the dialog device and the information recommendation method storage unit of the dialog device. The reward calculation unit
Random selection match value calculation means for calculating an expected value of the degree of match between one or more evaluation values when the spot is randomly determined using the spot probability information and the user preference vector;
A selected spot coincidence degree calculating means for calculating a degree of coincidence between the user preference vector and one or more evaluation values indicating evaluations of the one or more determination factors of the spot included in the user input information;
A reward for selecting a spot included in the user input information using the expected value of the degree of match calculated by the random selection match value calculation unit and the match level calculated by the selected spot match level calculation unit. The learning system according to claim 1, further comprising reward calculating means for calculating. The user status information storage unit
A preference vector that is information indicating a user's state and indicating user's preference for one or more determinants, a knowledge vector indicating user knowledge for one or more determinants, and one or more determinants The learning system according to claim 1, further comprising user state information having attribute vectors that are information indicating user attributes with respect to. The simulation apparatus which comprises the learning system in any one of Claims 1-3. On the recording medium,
Dialogue probability information that is information about the probability of each information recommendation method and each user sentence type, determinant probability information that is information about the probability that a determinant is selected, and spot probability information that is information about the probability that a spot is selected When,
Stores a user preference vector, which is a vector indicating the user's preference,
It is a learning method that can be realized by a dialogue sentence information reception unit, a user sentence type determination unit, a determination factor acquisition unit, a user input information transmission unit, a reward calculation unit, and a learning unit,
The dialog text information receiving unit receives dialog text information from a dialog device, and receives a dialog text information receiving step;
The user sentence type determination unit determines a user sentence type using the technique identifier and the conversation probability information included in the dialog sentence information, and acquires a user sentence type identifier; and
The determination factor etc. acquiring unit may include one or more pieces of information of the decision factor probability information or the spot probability information, or one or more pieces of information of the decision factor probability information or the spot probability information and the dialogue sentence information. Using one or more determinants or one or more pieces of information of one or more spots having one or more determinants or a determinant acquisition step for acquiring one or more spots;
The user input information sending unit sends user input information having the user sentence type identifier or the user sentence type identifier and one or more determinants or one or more information of one or more spots to the dialogue apparatus. A user input information sending step,
The reward calculation unit acquires the user preference vector and one or more evaluation values indicating the evaluation of each of the one or more determinants of spots included in the user input information, and the user preference vector and the one or more A degree of coincidence with the evaluation value, and using the degree of coincidence, a reward calculating step of calculating a reward for selecting the user sentence type identified by the user sentence type identifier;
A learning method comprising: a learning step in which the learning unit updates a weight vector of the information recommendation method storage unit of the dialog device, the weight vector corresponding to the method identifier of the dialog device using the reward .