JP2018045413A - Information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device, information processing method, and program that improve the satisfaction of a user with a message which is transmitted to a terminal device of the user.SOLUTION: An information processing device includes a reception unit, a generation unit, and a transmission unit. The reception unit receives a user message from a terminal device of a user. The generation unit generates a candidate message including candidates narrowed down using a candidate selection model learned by reinforcement learning. The transmission unit transmits the candidate message generated by the generation unit, to the terminal device.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  Conventionally, in transmitting a question message to a user terminal device in response to a message transmitted from a user terminal device, before transmitting the question message, content is selected from a plurality of content sets and transmitted to the user terminal device. An information processing apparatus that transmits content is known (see Patent Document 1).

Japanese Patent Laying-Open No. 2015-69608

  However, in the information processing apparatus, content selected from a plurality of content sets is selected on average, and content desired by the user may not be transmitted to the user's terminal device, which reduces user satisfaction. There is a case.

  The present application has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and a program that improve a user's satisfaction with a message transmitted to the user's terminal device.

  An information processing apparatus according to the present application includes a reception unit, a generation unit, and a transmission unit. The receiving unit receives a user message from the user terminal device. The generation unit generates a candidate message including candidates narrowed down using the candidate selection model learned by reinforcement learning for the user message. The transmission unit transmits the candidate message generated by the generation unit to the terminal device.

  According to one embodiment of the present invention, it is possible to provide an information processing device, an information processing method, and a program that improve user satisfaction with a message transmitted to a user terminal device.

FIG. 1 is an explanatory diagram of information processing according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the information processing system. FIG. 3 is a diagram illustrating a configuration example of the information processing apparatus. FIG. 4 is a diagram illustrating an example in which menus are narrowed down by the information processing apparatus. FIG. 5 is a diagram illustrating an example of a display screen of the terminal device. FIG. 6 is a flowchart illustrating an example of menu selection processing according to the embodiment. FIG. 7 is a diagram illustrating an example of a hardware configuration of a computer that executes a program.

  Hereinafter, an information processing apparatus, an information processing method, and a form for implementing a program (hereinafter referred to as “embodiment”) according to the present application will be described in detail with reference to the drawings. Note that the information processing apparatus, the information processing method, and the program according to the present application are not limited by this embodiment.

[1. Information processing]
An example of information processing according to the embodiment will be described. FIG. 1 is an explanatory diagram of information processing according to the embodiment. Here, although the case where the information processing apparatus 1 performs the process with respect to the menu search received from the user's terminal device 2 is demonstrated, it is not limited to this.

  The information processing apparatus 1 receives menu search data, which is a user message based on the user's utterance (step S1). For example, when the user speaks “I want to use cabbage today”, the information processing apparatus 1 receives menu search data corresponding to “I want to use cabbage today”.

  The information processing apparatus 1 uses a candidate selection model (hereinafter referred to as reinforcement learning model) learned by reinforcement learning based on the ingredient data extracted from the menu search data to select and select a menu that uses the ingredient. Text data corresponding to the response sentence including the menu (hereinafter may be referred to as “response sentence”) is generated (step S2). The response sentence is also a candidate message including candidates narrowed down with respect to the user message.

  Although the reinforcement learning model will be described in detail later, the reinforcement learning model according to the embodiment is a model in which a menu having a high reward integrated value (hereinafter referred to as a score) is selected for the food material data.

  For example, the information processing apparatus 1 selects “roll cabbage” having a high score from a menu using “cabbage” as a food material. Then, the information processing apparatus 1 generates a response sentence “How about the roll cabbage?”.

  The information processing apparatus 1 transmits data relating to the generated response sentence to the user terminal device 2 (step S3). For example, the information processing apparatus 1 transmits a response sentence “How about a roll cabbage?”.

  The information processing apparatus 1 receives the menu selection result based on the operation of the terminal device 2 from the user terminal device 2 (step S4). For example, the information processing apparatus 1 receives information in which “roll cabbage” is selected as a menu based on the operation of the terminal device 2.

  The information processing apparatus 1 updates the reinforcement learning model based on the menu selection result (step S5). For example, the information processing apparatus 1 updates the reinforcement learning model so that the score of “roll cabbage” is high as a menu.

  As described above, the information processing apparatus 1 receives the menu search data including the food material information from the user terminal device 2 and selects a menu using the reinforcement learning model based on the received menu search data. Then, the information processing apparatus 1 generates a response sentence including the selected menu, and transmits the response sentence to the terminal device 2. By selecting a menu using the reinforcement learning model, the information processing apparatus 1 can quickly provide a menu that the user likes, and can improve user satisfaction with respect to menu search.

[2. Configuration of information processing system 5]
FIG. 2 is a diagram illustrating a configuration example of the information processing system 5. As illustrated in FIG. 2, the information processing system 5 according to the embodiment includes an information processing device 1, a terminal device 2, a speech recognition server 3, and a speech synthesis server 4.

  The terminal device 2, the speech recognition server 3, the speech synthesis server 4, and the information processing device 1 are connected to be communicable with each other wirelessly or via a network N. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

  The terminal device 2 is realized by a smartphone, a tablet terminal, a desktop PC (Personal Computer), a notebook PC, a PDA (Personal Digital Assistant), or the like.

  The speech recognition server 3 is a device that performs natural language processing on speech information and converts speech data into text data. When the speech recognition server 3 receives speech speech data from the terminal device 2, the speech recognition server 3 converts the speech data into text data. The voice recognition server 3 transmits text data obtained by converting the voice data to the information processing apparatus 1.

  The voice synthesis server 4 converts the text data of the response sentence generated by the information processing apparatus 1 into voice data. The voice synthesis server 4 transmits the voice data obtained by converting the text data to the terminal device 2.

  The information processing device 1 generates text data of a response sentence based on the text data transmitted from the terminal device 2 or the text data obtained by converting the speech data via the speech recognition server 3. The information processing apparatus 1 transmits the generated text data of the response sentence to the speech synthesis server 4 and the terminal device 2.

  Note that the speech recognition server 3 and the speech synthesis server 4 may be configured integrally with the information processing apparatus 1. When the terminal device 2 has a voice recognition function or a voice synthesis function, the voice data and the text data may be converted using these functions.

[3. Configuration of information processing apparatus 1]
Next, the information processing apparatus 1 according to the embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of the information processing apparatus 1.

  Here, an example in which voice data based on a user's speech is transmitted from the terminal device 2 will be described as an example, but text data may be transmitted.

Further, an example in which the following response is performed between the terminal device 2 (user) and the information processing device 1 will be described as an example.
(User utterance 1): “I want to use cabbage today”
(Response 1): "What about roll cabbage if you have ground meat?"
(User utterance 2): “It's better to be different because there is no meat”
(Response 2): “If you have bean sprouts, how about fried vegetables?”
(User utterance 3): “There are eggs and carrots”
(Response 3): “Displays a menu that can be made with cabbage, eggs and carrots”

  When the menu search is performed by the user, the information processing device 1 generates a response sentence to the menu search data transmitted from the terminal device 2 (see FIG. 2) via the voice recognition server 3 (see FIG. 2). It is a response generation device. The information processing apparatus 1 includes a reception unit 10, a transmission unit 20, a storage unit 30, and a processing unit 40.

  The receiving unit 10 receives a user message from the terminal device 2 via the network N. The user message includes text data such as menu search data based on utterances and data related to operation information of the terminal device 2. The operation information of the terminal device 2 is information that does not include text data, such as selection information (click or touch) in the terminal device 2.

  The storage unit 30 includes a dialogue model storage unit 31, a reinforcement learning model storage unit 32, a menu storage unit 33, and a food storage unit 34. The storage unit 30 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

  The dialogue model storage unit 31 stores a dialogue model for generating a response sentence set in advance for the content of the user's utterance.

  The dialogue model is a model for generating text data to be paired with text data corresponding to the utterance content of the user. For example, in the dialogue model, when the text data from the user's utterance is “I want to use *** today”, a response sentence such as “What is XXX with XXX?” This is a model for generating a response sentence such as "What about XX?"

  The reinforcement learning model storage unit 32 stores a reinforcement learning model for selecting a menu based on the food material data. The reinforcement learning model is learned by reinforcement learning, for example, Q-learning.

  Reinforcement learning is a learning method that preferentially selects an action that maximizes the reward obtained by the given action when the action is given to a certain state. Various actions are tried for the state, and the reinforcement learning model is updated, that is, learned by receiving a reward corresponding to the tried action.

  In the reinforcement learning according to the present embodiment, the food data obtained from the menu search data is set to “state”, the menu selected based on the food data is set to “action”, and the score based on the response of the user who provided the menu is “ Reward ”is done. In the reinforcement learning model, a score is assigned to each menu. When the reinforcement learning model is used when selecting a menu, the menu having the highest score is selected.

  For example, the reward of the menu selected by the user is “+1”. Further, for example, the reward of the menu not selected by the user with respect to the menu provided to the user is “−0.1”.

  If the user has written a review about the menu, the reward of the menu that wrote the review may be “+2”. When the review of the menu is written, it is known that the user has actually created the dish, so the reward is increased.

  Note that the initial value of the score in the reinforcement learning model is set based on, for example, a general menu ranking. The initial value of the score is set according to the degree of popularity. For example, the initial value of the score for a menu that is popular as a menu increases.

  The menu storage unit 33 stores a menu and a recipe including necessary ingredients in association with each other. For example, in the “roll cabbage” menu, “cabbage”, “ground meat”, and “onion” are stored in association with each other as ingredients.

  The food storage unit 34 temporarily stores the food data extracted by the analysis unit 41 described later. The stored food data is erased when a predetermined condition, for example, when a menu is selected by the user or when a preset predetermined time has elapsed. The predetermined time is, for example, a time when it can be determined that the menu selection is completed.

  The food storage unit 34 represents the state of the food data that the user has as a vector. For example, the food data that the user has or is unknown is “0”, and the food data that is known to be “1”. , The food data that is known not to be stored is stored as “−1”.

  The processing unit 40 includes an analysis unit 41, a menu selection unit 42, a response generation unit 43, and a learning unit 44.

  The analysis unit 41 determines whether the user message received by the reception unit 10 is text data or a message related to operation information of the terminal device 2. Moreover, the analysis part 41 determines whether it is the operation information regarding a menu selection result, for example, when a user message is a message regarding the operation information of the terminal device 2. FIG.

  If the message includes text data, the analysis unit 41 analyzes the user message using morphological analysis or the like, and extracts a word group included in the text data. Specifically, the analysis unit 41 extracts food material data and menu data included in the text data. Moreover, the analysis part 41 specifies whether it is the positive content or negative content with respect to foodstuff data or menu data.

  That is, the analysis unit 41 specifies whether the content of the text data is content related to menu search, and whether the content is positive content or negative content with respect to the food material data. The food data extracted by the analysis unit 41 is temporarily stored in the food storage unit 34.

  Moreover, the analysis part 41 specifies whether the content of text data is the content regarding a menu selection result, and the content is a positive content or a negative content with respect to a menu.

  For example, when the receiving unit 10 receives (user utterance 1): “I want to use cabbage today” as the menu search data, the analysis unit 41 extracts “cabbage” as food data, To be identified. As a result, the food storage unit 34 stores “Cabbage” as the food data.

  Further, the analysis unit 41 extracts “meat” as the food data when the receiving unit 10 receives (user utterance 2): “May be different because there is no meat” as the menu search data, for example. , To identify negative content. Thereby, it is further memorize | stored in the foodstuff memory | storage part 34 that it does not have "meat" as foodstuff data.

  Further, for example, when the receiving unit 10 receives (user utterance 3): “There is an egg and carrot” as the menu search data, the analysis unit 41 extracts “egg” and “carrot” as food data. And identify it as positive. Thereby, it is further memorize | stored in the foodstuff storage part 34 that it has "egg" and "carrot" as foodstuff data.

  The menu selection unit 42 selects a menu having a high score using the reinforcement learning model based on the food data stored in the food storage unit 34.

  Specifically, the menu selection unit 42 ranks the menus in descending order of the score among the menus using the ingredients of the ingredient data stored in the ingredient storage unit 34.

  For example, based on (user utterance 1): “I want to use cabbage today” and the food data stored in the food storage unit 34 is only “cabbage”, “cabbage” is used as the food. Ranking the menu with the highest score in the menu.

  Thereby, for example, as shown in FIG. 4, in the order of “roll cabbage”, “boiled pot meat”, “fried vegetables”, “vegetable soup”, “okonomiyaki”, “salad”, “fried cabbage” The menu selection unit 42 selects “roll cabbage” having the highest score in the ranking as a menu. FIG. 4 is a diagram illustrating an example in which menus are narrowed down by the information processing apparatus 1.

  In addition, for example, based on (user utterance 2): “There is no meat and it is good to be different”, the food data stored in the food storage unit 34 is “cabbage” and “meat”, and “meat” is If it is found that the menu selection unit 42 does not have the menu, the menu selection unit 42 ranks the menu having a high score among the menus in which “cabbage” is used as the food and “meat” is not used.

  As a result, for example, as shown in FIG. 4, ranking is performed in the order of “fried vegetables”, “vegetable soup”, “okonomiyaki”, “salad”, and “fried cabbage”. In this ranking, the menu is narrowed down by excluding “roll cabbage”, which uses “meat” as a food, and “roasted meat” compared to the ranking when the food data is only “cabbage”. In this case, the menu selection unit 42 selects “stir-fried vegetables” having the highest score in the ranking as a menu.

  For example, based on (user utterance 3): “There are eggs and carrots”, the food data stored in the food storage unit 34 is “cabbage”, “meat”, “eggs”, and “carrots”. If it turns out that it does not have "meat", the menu selection part 42 uses "cabbage", "egg", and "carrot" as a foodstuff, and a score is high in the menu in which "meat" is not used. Ranking menus.

  Thereby, for example, as shown in FIG. 4, ranking is performed in the order of “vegetable soup” and “fried cabbage”. In this ranking, the ingredients data is “cabbage” and “meat”, and “ranked egg” or “carrot” is not used as an ingredient compared to the ranking when “meat” is not included. , “Okonomiyaki”, and “salad” are excluded, and the menu is narrowed down.

  The menu selection unit 42 selects all menus when the number of menus having a score equal to or higher than a predetermined score is a predetermined number, for example, three or less. The predetermined score is a preset value. For example, when “vegetable soup” and “fried cabbage” remain in the ranking as a menu having a score equal to or higher than a predetermined score, the menu selection unit 42 adds to the “vegetable soup” having the highest score in the menu. Select “Cabbage fried rice” as the menu.

  Note that the menu selection unit 42 may select a predetermined number of higher-order menus in the ranking when a predetermined number of responses are made after starting the menu search. The predetermined number of times is a preset number of times, for example, three times.

  The response generation unit 43 generates a response sentence including the selected menu based on the menu selected by the menu selection unit 42 and the dialogue model.

  The response generation unit 43 generates a response sentence asking whether there is an important food among the foods necessary for the menu selected by the menu selection unit 42 and not stored as food data in the food storage unit 34. . That is, the response generation unit 43 generates a response sentence including a hint for extracting food data from the user.

  For example, the response generation unit 43 generates a response sentence asking whether there is a food with a high priority among the ingredients without information based on the priority of the food with respect to the menu. In addition, the priority order of foodstuffs is memorize | stored in the menu memory | storage part 33 with the menu.

  For example, priority is given to “roll cabbage” in the order of “cabbage”, “meat”, “onion”, and “bread crumbs” as priority of ingredients, and “cabbage” is stored in the ingredient storage unit 34 as ingredient data. "Is stored, the response generation unit 43 generates a response sentence asking whether or not" meat "exists.

  For example, in response to (user utterance 1): “I want to use cabbage today”, the response generation unit 43 generates (response 1): “If there is minced meat, what about roll cabbage?”.

  In addition, when (user utterance 2): “There is no meat and it is good to be different”, “stir-fried vegetables” is selected as the menu by the menu selection unit 42, and “sprouts” is high as the priority of the ingredients. The response generation unit 43 generates (Response sentence 2): “If there is bean sprouts, how about fried vegetables?”.

  Also, for (user utterance 3): “There are eggs and carrots”, the menu selection unit 42 selects “vegetable soup” and “cabbage fried rice”, and the menu is narrowed down, so the response generation unit 43 (Response sentence 3): “Display menu that can be made with cabbage, eggs and carrots” is generated.

  Note that the response generation unit 43 may generate a response sentence including only the menu selected by the menu selection unit 42.

  The transmission unit 20 transmits the response sentence generated by the response generation unit 43 to the terminal device 2 and the voice synthesis server 4 (see FIG. 2) via the network N. The transmission unit 20 narrows down the menu. For example, when “vegetable soup” and “cabbage fried rice” are selected by the menu selection unit 42, “vegetable soup” and “vegetable soup” and “ The recipe data of “fried cabbage” is transmitted to the terminal device 2.

  The learning unit 44 updates the reinforcement learning model based on the menu selection result by the user. The learning unit 44 gives a reward for the menu selected by the menu selection unit 42 based on the menu selection result by the user.

  For example, (Response 3): “Display menu that can be made with cabbage, eggs and carrots” is sent to the terminal device 2, and recipes for “vegetable soup” and “fried cabbage” are as shown in FIG. Is displayed on the terminal device 2. FIG. 5 is a diagram illustrating an example of a display screen of the terminal device 2.

  When “vegetable soup” is selected by the user, the learning unit 44 gives a reward “+1” to the “vegetable soup” menu and updates the reinforcement learning model.

  Further, the learning unit 44 rewards “Roll cabbage” included in (Response sentence 1), “Fried vegetables” included in (Response sentence 2), and “Cabbage fried rice” that was not finally selected. -0.1 "is given, and the reinforcement learning model is updated.

  As a result, the menu selection unit 42 has a higher probability that “vegetable soup” will be selected in the next and subsequent times than when this menu is selected, and “roll cabbage”, “fried vegetables”, and “fried cabbage” The probability of being selected is reduced. That is, the menu selection reflecting the user's preference is performed, and the information processing apparatus 1 can quickly provide the menu that the user likes.

  Note that the learning unit 44, for example, responds to (Response 1): “If there is minced meat, what about roll cabbage?”, The user utters “Roll cabbage is good”, and the menu is selected and determined. If it is, the reward “+1” is given to the “roll cabbage” menu included in (response sentence 1), and the reinforcement learning model is updated.

  In addition, the learning unit 44 may, for example, (response sentence 1): “How about roll cabbage if there is minced meat?” When there is no user utterance for a predetermined time after the text data of the response generation sentence is transmitted. On the other hand, if there is no next utterance of the user even after the predetermined time has passed, the reward “−0.1” is given to the “roll cabbage” menu included in (response sentence 1), and the reinforcement learning model is Update.

[4. Menu selection process]
Next, menu selection processing according to the embodiment will be described with reference to FIG. FIG. 6 is a flowchart illustrating an example of menu selection processing according to the embodiment.

  When the user message is received by the receiving unit 10 (step S10), the processing unit 40 determines whether the user message is a message including text data (step S11).

  When the user message is a message including text data (step S11: Yes), the processing unit 40 analyzes the text data, extracts food material data and menu data included in the text data, and extracts these data. It is specified whether the content of the text data is a positive content or a negative content (step S12).

  When the content of the text data includes food data and is related to menu search (step S13: Yes), the processing unit 40 stores the extracted food data in the food storage unit 34 (step S14).

  The processing unit 40 selects a menu with the highest score using the reinforcement learning model based on the food data stored in the food storage unit 34 (step S15).

  The processing unit 40 generates a response sentence including the selected menu based on the selected menu and the interaction model (step S16).

  The processing unit 40 transmits the generated response sentence from the transmission unit 20 to the user terminal device 2 (step S17).

  In addition, when the user message is a message that does not include text data, that is, a message regarding operation information of the terminal device 2 (step S11: No), or the content of the text data is not related to menu search. In the case (step S13: No), it is determined whether or not the user message is content related to the menu selection result (step S18).

  When the user message is content related to the menu selection result (step S18: Yes), the processing unit 40 updates the reinforcement learning model based on the menu selection result (step S19).

  When the content of the text data is not the content related to the menu selection result (step S18: No), the processing unit 40 ends the current process.

[5. Modified example]
In the above-described embodiment, the menu selection unit 42 selects the menu having the highest score with respect to the ranking. However, the menu selection unit 42 may select the menu with a predetermined probability from the ranking. For example, for the menu with the highest score, the probability of selection may be 50%, and for the menu with the next highest score, the probability of selection may be 20%.

  Thereby, it can suppress that the same menu is selected by the menu selection part 42, and can diversify the menu provided to a user.

  Moreover, the menu selection part 42 may select a menu based on the foodstuff data represented by the real number vector showing the characteristic with respect to a foodstuff. For example, a real vector expression corresponding to each food is learned from a large amount of recipe text using a Skip-gram model or the like, and the menu selection unit 42 selects a menu based on the real vector expression.

  By representing the food data as a real vector, the menu selection unit 42 selects a menu having a high score from a menu that uses preset alternative foods according to the similarity of the real vector representing the food. Is possible.

  In addition, when food material data is represented by a real vector expression, the reinforcement learning model and the dialogue model may be learned as one model by a neural network to generate a response sentence.

  The menu selection unit 42 may select, for example, a menu that the user has not created or a menu with a low ranking score at a predetermined timing. The predetermined timing is set in advance, for example, once every several months. Further, the learning unit 44 may weight the score based on the recommendation result of collaborative filtering based on the history on the website.

  Thereby, it can suppress that the same menu is selected by the menu selection part 42, and can provide the menu which a user does not anticipate.

  In addition, a predetermined time for storing the food material data may be used as a food consumption time. The food storage unit 34 deletes the food data after the consumption time of the food has elapsed. The consumption time of the ingredients is, for example, the average consumption time of the ingredients, or the average consumption time reduced according to the probability of existence of the ingredients, for example, the probability associated with the amount used in the selected menu, i.e. It is a time when it can be estimated that all ingredients remain without being consumed.

  In addition to the food data obtained in the current menu search, the menu selection unit 42 has not consumed the food consumption time, and the food data stored in the food storage unit 34, for example, obtained during the previous menu search. The menu is selected based on the food data that is estimated to be not consumed.

  Moreover, the response production | generation part 43 may produce | generate the response sentence of the content which asks a user about the residual amount of a foodstuff, and the foodstuff memory | storage part 34 may memorize | store the residual quantity of a foodstuff.

  Thereby, the menu which is easy to be selected by the user can be proposed at an early stage, and the satisfaction of the user can be improved.

  For example, the learning unit 44 may temporarily reduce the score of a similar menu for a predetermined period with respect to a menu selected by the user. The similar menu is a menu that is the same as or similar to the menu selected by the user. A similar menu is a menu that can be created using a substitute ingredient for a certain ingredient. The predetermined period is set in advance, for example, one month. For example, the learning unit 44 weights the score so that the score for the similar menu becomes small.

  Thereby, it can suppress that a similar menu is selected by the menu selection part 42 in a short period, and can diversify the menu provided to a user.

  Moreover, the menu selection part 42 may select a menu based on a user's health information. The health information is, for example, information on weight, blood pressure, exercise amount, and daily meals. For example, the health information is extracted and acquired by the analysis unit 41 from text data obtained by dialogue with the user. Moreover, health information may be acquired through an external service. For example, the learning unit 44 sets the health state as the “state” of reinforcement learning, and weights the score based on the health information so that the score of the menu with high health becomes large.

  Thereby, a healthy menu can be provided to the user.

  In addition, the menu selection unit 42 may select a menu based on information such as user preferences and user utterance history acquired by interaction with the user. For example, the learning unit 44 weights the score according to the user's preference. Further, the response generation unit 43 may generate a response sentence based on user preferences and the like. For example, the response generation unit 43 may generate a response sentence such as “How about fried vegetables if you don't like roll cabbage?”.

  Thereby, natural utterance can be realized and a menu preferred by the user can be provided early.

  Moreover, the menu selection part 42 may select a menu based on the user's preference information of foodstuffs. For example, when the user utters “I don't like bell peppers”, the food storage unit 34 stores that there is no “green pepper” as long-term information that is not erased in a predetermined time. In addition, the learning unit 44 may reduce the score of a menu that uses “green pepper” as a food material.

  This makes it difficult to select a menu that uses “green pepper” as a food material, and provides a menu according to the user's preference.

  Further, the menu selection unit 42 can be configured not to display a predetermined menu. For example, a menu including allergic ingredients may not be selected for a user who has allergies. For example, the learning unit 44 performs weighting on the score so that the score of the menu including the allergic food is small. Moreover, the menu selection part 42 may exclude the menu containing an allergic foodstuff from ranking.

  Thereby, it can suppress providing a menu which a user does not like.

  In the above embodiment, the response generation unit 43 generates a response sentence including the menu selected by the menu selection unit 42, but may generate a response sentence that does not include the menu. For example, the response generation unit 43 may generate a response sentence such as “What foods do you have?” That does not include a menu. Thereby, the ingredient data which a user has can be withdraw | derived from a user, and the information processing apparatus 1 can provide the menu which a user likes at an early stage based on much ingredient data.

  The response generation unit 43 may generate a response sentence including a plurality of menus. For example, a response sentence such as “If you have bean sprouts, you can make fried vegetables, but what about salads?” May be generated for the above (Response sentence 2).

  Thereby, the menu which a user likes can be provided at an early stage.

  In the above embodiment, the learning unit 44 gives a reward based on the menu selection result by the user and updates the reinforcement learning model. However, the number of responses with the user (number of interactions) may be taken into account. For example, when the user selects a menu and there are a plurality of responses so far, “+ 1 / number of responses” may be given to the selected menu as a reward according to the number of responses.

  The learning unit 44 may change the reward when the menu is selected by the user according to the amount of food data in the menu selection process.

  For example, when the user performs a menu search and selects a menu of one item after the user performs a menu search of two items and selects a menu of two items, a reward for the menu of two items is calculated. Make it smaller than the reward for one menu item.

  The two-item menu search is performed in a state in which information on food ingredients is obtained from the user when the one-item menu is selected. Therefore, for example, even when the user selects a menu of one item and a menu of two items with the same number of responses, the menu selection of one item in the information processing apparatus 1 is more effective than the menu selection of two items. It is based on a small amount of food data.

  Therefore, the learning unit 44 makes the reward for the menu of the two items smaller than the reward for the menu of the one item, and increases the reward for the menu selected by the user in a state where the food data is small, and the reinforcement learning. Train the model.

  For example, the reward for the menu of one item is “+1”, and the reward for the menu of two items is weighted to “+0.1”.

  Thereby, when the menu which a user likes can be provided based on the data amount of few foodstuffs, a reward becomes large and the menu which a user likes can be provided at an early stage.

[6. effect]
The information processing apparatus 1 according to the embodiment includes a reception unit 10, a response generation unit 43, and a transmission unit 20. The receiving unit 10 receives a user message from the user terminal device 2. The response generation unit 43 generates a response sentence including the narrowed menu using a reinforcement learning model (candidate selection model) in response to the user message. The transmission unit 20 transmits the generated response sentence to the terminal device 2.

  As a result, a response sentence including a menu selected using the reinforcement learning model can be provided for a menu search of the user, a menu that the user likes can be provided at an early stage, and user satisfaction is improved. be able to.

  The response generation unit 43 generates a response sentence further narrowing down the menu for a new user message for the response sentence.

  Thereby, since the response sentence which narrowed down a menu can be provided, the menu which a user likes can be provided at an early stage, and a user's satisfaction can be improved.

  The information processing apparatus 1 includes a learning unit 44 that learns a reinforcement learning model based on a reward obtained according to a user message for a response sentence. The response generation unit 43 generates a user message using the learned reinforcement learning model.

  Thereby, the menu according to a user's liking can be provided at an early stage using the reinforcement learning model learned based on the reward obtained in the past.

  The learning unit 44 increases the reward as the number of responses until the menu is selected by the user is smaller.

  Thereby, when the menu that the user likes can be quickly provided, the reward increases, so the menu that the user likes can be provided early.

  The learning unit 44 sets the initial value of the score according to the popularity, and increases the initial value of the score as the popularity increases.

  Thereby, it can suppress that a menu with low popularity is offered to a user in the initial stage of reinforcement learning, and can suppress that a user's satisfaction falls.

  The learning unit 44 temporarily reduces the score for a similar menu that is the same as or similar to the menu selected by the user for a predetermined period.

  Thereby, it can suppress that a similar menu is selected by the menu selection part 42 in a short period, and can diversify the menu provided to a user.

  The response generation unit 43 generates a response sentence using the reinforcement learning model and the user's health state.

  Thereby, a healthy menu can be provided to the user.

[7. Hardware configuration]
The information processing apparatus 1 according to the embodiment described above is realized by a computer 1000 configured as shown in FIG. 7, for example. FIG. 7 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing apparatus 1. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 receives data from other devices via the network N and sends the data to the CPU 1100, and transmits data determined by the CPU 1100 to other devices via the network N.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the determined data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the information processing apparatus 1 according to the embodiment, the CPU 1100 of the computer 1000 implements the function of the processing unit 40 by executing a program loaded on the RAM 1200. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them. However, as another example, these programs may be acquired from other devices via the network N.

  As described above, some of the embodiments and modifications of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various aspects can be made based on the knowledge of those skilled in the art including the aspects described in the disclosure line of the invention. It is possible to carry out the present invention in other forms that have been modified and improved.

[8. Other]
The above-described reinforcement learning model can be used for other than the menu search described above. For example, it can be used in a dialog system with a user. In a dialogue system, “state” is the user's utterance content, “behavior” is the proposal (response content) to the user, “reward” is the user's response to the proposal, and reinforcement learning is used to satisfy the user. It is possible to make a proposal with a high degree.

  For example, in response to an inquiry (product search) from a user, a proposal with high user satisfaction (a user is satisfied) using a reinforcement learning model in which user preferences are learned from past proposal contents based on user utterances and the like. Product proposal).

  In addition, among the processes described in the above-described embodiments and modifications, all or a part of the processes described as being automatically performed can be manually performed, or are described as being performed manually. All or part of the processing can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each drawing is not limited to the illustrated information.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  In addition, the above-described embodiments and modifications can be combined as appropriate within a range that does not contradict processing contents.

  In addition, the “section (module, unit)” described above can be read as “means” or “circuit”. For example, the receiving unit 10 can be read as receiving means or a receiving circuit.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 2 Terminal device 10 Reception part 20 Transmission part 30 Storage part 40 Processing part 41 Analysis part 42 Menu selection part 43 Response generation part (generation part)
44 Learning Department

Claims (9)

A receiving unit for receiving a user message from a user terminal device;
For the user message, a generation unit that generates a candidate message including candidates narrowed down using a candidate selection model learned by reinforcement learning;
A transmission unit that transmits the candidate message generated by the generation unit to the terminal device;
An information processing apparatus comprising: The generator is
The information processing apparatus according to claim 1, wherein a new candidate message further narrowing down the candidates is generated for a new user message corresponding to the candidate message. A learning unit that learns the candidate selection model based on a reward obtained according to the new user message for the candidate message;
The generator is
The information processing apparatus according to claim 2, wherein the candidate message is generated using the candidate selection model learned by the learning unit in the past. The learning unit
The information processing apparatus according to claim 3, wherein the reward for the candidate selected by the user is increased as the number of responses before the candidate is selected by the user is reduced. The learning unit
5. The information processing apparatus according to claim 3, wherein an initial value of a score in the candidate selection model is set according to popularity, and the initial value is increased as the popularity is higher. The learning unit
6. The score in the candidate selection model for a similar candidate that is the same as or similar to the candidate selected by the user is temporarily reduced for a predetermined period. 6. Information processing device. An acquisition unit for acquiring the health information of the user;
The user message is:
Message about cooking recipes,
The generator is
The information processing apparatus according to any one of claims 1 to 6, wherein the candidate message is generated for the user message using the candidate selection model and the health information. An information processing method executed by an information processing apparatus,
A receiving step of receiving a user message from the user terminal device;
Generating a candidate message including candidates narrowed down using a candidate selection model learned by reinforcement learning for the user message;
A transmission step of transmitting the candidate message generated by the generation step to the terminal device;
An information processing method comprising: A receiving procedure for receiving a user message from a user terminal device;
For the user message, a generation procedure for generating a candidate message including candidates narrowed down using a candidate selection model learned by reinforcement learning;
A transmission procedure for transmitting the candidate message generated by the generation procedure to the terminal device;
A program that causes a computer to execute.

Images