JP7418106B2 - Information processing device, information processing method and program - Google Patents

Description

The present invention relates to a technique for realizing a response imitating a specific person.

In recent years, as the processing performance of personal computers and smartphones has improved, technology has been developed in which chatbots and avatars generated on personal computers and smartphones automatically respond to users' questions and inquiries. As a technology related to chatbots, Patent Document 1 discloses dividing a long response message into appropriate lengths in a chatbot system that automatically responds to questions sent from multiple users' terminals. . When communicating with chatbots and avatars, users are not only able to quickly obtain the information they need, but are also able to enjoy the communication itself. Therefore, in order to realize more human-like communication using artificial intelligence (AI), computers are being trained to learn conversations in various cases. Machine-learned avatars can respond automatically as if a real, specific person was responding.

Japanese Patent Application Publication No. 2009-3533

However, communication with chatbot systems and avatars in the prior art may not be able to provide a sufficient sense of satisfaction to users. For example, when communicating with an avatar that imitates a celebrity such as an actor, it is desirable to communicate with an avatar that reproduces the celebrity when he or she was most active in the past, rather than an avatar that imitates the current celebrity. There is. Furthermore, when communicating by reproducing a deceased person as an avatar, it may be desirable to communicate with a deceased person of a desired age rather than with the deceased person who was about to die.

The present invention has been made in view of the above problems, and an object of the present invention is to generate a learning model for realizing a response imitating a specific person of a predetermined age.

The present invention provides an acquisition means for acquiring information on a specific person's response together with age information of the specific person at the time of the response, a setting device for setting a predetermined age, and a set of the predetermined age and the age information. an extraction means for extracting response information corresponding to the predetermined age from the response information; and a generation means for generating a learning model of the specific person's response from the extracted response information. It is characterized by

According to the present invention, a response imitating a specific person of a predetermined age can be realized.

Device and system configuration diagram in the first embodiment Functional configuration diagram in the first embodiment Diagram showing Q&A data and age information Conceptual diagram of the response model in the first embodiment Diagram showing the concept of neural networks Flowchart of response model generation in the first embodiment Flowchart of communication using response model in first embodiment Conceptual diagram of response model in second embodiment Example of avatar display using age-specific response model Conceptual diagram of a model that selects response models by age

(First embodiment)
The first embodiment will be described below with reference to the drawings. FIG. 1 is an apparatus and system configuration diagram in this embodiment. The information processing device 1100 is, for example, a general computer, and generates a response model (learning model) for realizing a response imitating a specific person.

The information processing device 1100 includes a CPU 1101, a DRAM 1102, a secondary storage device 1103, a network IF 1104, and an I/O controller 1105. The CPU 1101 is a central processing unit and performs various processes according to instructions from a computer program. DRAM 1102 is a volatile memory that temporarily stores computer programs and various data. The secondary storage device 1103 is a nonvolatile storage device such as a hard disk drive (HDD) or a solid state disk (SSD), and stores computer programs and various data. The network 1400 is a wired cable or a wireless network, and various data are sent and received to and from external devices via the network 1400. Computer programs and various data are acquired from the secondary storage device 1103 or the network 1400 and temporarily stored in the DRAM 1102. The CPU 1101 performs information processing based on computer programs and various data stored in the DRAM 1102. The I/O controller 1105 is a controller that controls input/output between an externally connected input device 1106, monitor 1107, and speaker 1108. The input device 1106 is, for example, a keyboard, mouse, or microphone device, and accepts user input. The user can use the input device 1106 to ask questions to the virtual specific person reproduced on the information processing device 1100 through text input or voice input. The monitor 1107 is, for example, a general liquid crystal display, and displays the processing results of the information processing apparatus 1100. Computer graphics of a specific person reproduced on the information processing device 1100 may be displayed on the monitor 1107. Further, the response of the specific person may be displayed in text on the monitor 1107. The speaker 1108 may output a voice of the specific person's response reproduced on the information processing device 1100.

The user can ask a question or the like using the input device 1106 while viewing the computer graphics of the specific person displayed on the monitor 1107, and can listen to the response outputted as a voice from the speaker 1108. Note that communication with the specific person reproduced on the information processing device 1100 may be performed without using the input device 1106, monitor 1107, or speaker 1108. For example, it is possible to communicate on Saturdays using a client PC or smartphone 1300 that is connected to the information processing device 1100 via the network 1400.

FIG. 2 is a functional configuration diagram in this embodiment. Each function shown in FIG. 2 is realized by the information processing apparatus 1100 performing processing based on a computer program and various data stored in the DRAM 1102. Storage means 2001 is realized by DRAM 1102 and secondary storage device 1103, and holds Q&A data and age information. Q&A data is data that collects responses to questions asked to a specific person. The Q&A data may be in text format or audio format. Alternatively, the data may include a response to silence as a spontaneous utterance by a specific person. Q&A data may be collected from conversation logs on mobile phones or smartphones, or from chat logs recorded on personal computers. The age information is information indicating how old a specific person was when making each response included in the Q&A data. Age information is stored in association with each response included in the Q&A data. FIG. 3 is a diagram showing Q&A data and age information in this embodiment. As shown in the figure, Q&A data 3001 includes answers to many questions. Furthermore, age information 3002 indicating the age at which each Q&A was made is associated. The storage unit 2001 may store not only Q&A data 3001 and age information 3002 of one specific person, but also Q&A data 3001 and age information 3002 of a plurality of specific persons. Furthermore, the age information 3002 may be set in an age range such as 25 to 30 years old.

The acquisition means 2002 is means for acquiring the Q&A data and age information stored in the storage means 2001. The setting means 2003 is means for setting a predetermined age based on a user's instruction or the like. The predetermined age may be determined by the user. For example, if you want to communicate by reproducing a virtual person when the specific person is 25 years old, set the age of 25. Furthermore, an age range may be set, such as 25 to 30 years old. The extraction means 2004 is means for extracting corresponding response information from the Q&A data based on the predetermined age set by the setting means 2003. For example, if 25 years old is set in the setting means 2003, the Q&A data 3001 shown in FIG. , extracts response information only for 25-year-olds. The generation unit 2005 generates a response model (learning model) of the specific person using the response information extracted by the extraction unit 2004. The simplest response model may be the Q&A data itself extracted by the extraction means 2004. In that case, the model becomes such that only the questions included in the Q&A data stored in the storage means 2001 can be answered. The determining means 2006 is a determining means that determines whether or not to perform communication using the generated response model, and whether or not to stop communication. The determining means 2006 stops communication using the response model generated by learning when there is no question from the user for a predetermined period of time or when an instruction to stop communication is received from the user. Input means 2007 is means for accepting questions from the user and inputting question data into the response model. Note that if there are multiple generated response models, a desired response model is selected based on the input from the input means 2007. The output means 2008 is a means for outputting a response of the response model to a question from a user.

FIG. 4 is a conceptual diagram of an example of a response model in this embodiment. As described above, if the Q&A data itself extracted by the extraction means 2004 is used as a response model, only the questions included in the Q&A data stored in the storage means 2001 can be answered. Therefore, in order to respond to questions other than those included in the Q&A data stored in the storage means 2001, a semantic analysis model 4102 is introduced. The semantic analysis model 4102 is created in advance using machine learning such as a neural network or a support vector machine. The semantic analysis model 4102 inputs question data in text format or audio format, and outputs a feature amount 4103 of the question that can identify the question. By performing machine learning using a large amount of question data in which similar questions are given the same label as training data, it is possible to create a semantic analysis model 4102 that outputs the same feature amount for similar questions. If there is relatively little Q&A data, the feature amount 4103 may be a scalar value, but if there is a lot of Q&A data, the feature amount 4103 may be a vector value. Here, we will explain the concept of neural networks, which is a type of machine learning.

Note that since the principle of the neural network itself is well known, it will be briefly explained. FIG. 5 is a diagram illustrating a neural network. In FIG. 5, the intermediate layer is one layer, but it is desirable to configure the intermediate layer with two or more layers. In the neural network shown in Figure 5, the input layer has Mi nodes (n11, n12,..., n1Mi), the middle layer has Mh nodes (n21, n22,..., n2Mh), and the output layer (Final layer) has Mo nodes (n31, n32, . . . , n3Mo). The nodes of each layer are connected to all the nodes of adjacent layers, forming a three-layer hierarchical neural network that transmits information between layers.

When inputting an image to the input layer, nodes for the number of pixels are provided in the input layer so that there is a one-to-one relationship between pixels and nodes. Further, nodes for the number of pixels to be output are set in the output layer as well. That is, in this embodiment, a block image of 16 pixels x 16 pixels is input, and pixel values of 16 pixels x 16 pixels are output, so there are 256 nodes in the input layer and the output layer. Data is passed from left to right in FIG. 5, that is, in the order of input layer, intermediate layer, and output layer. Each node in the input layer is connected to all nodes in the hidden layer, and each connection between nodes has a weight. The output value transmitted from one node to another node through a connection is enhanced or attenuated by the weight of the connection. A set of weighting coefficients and bias values determined for such connections are parameters of the learning model. Note that the activation function is not particularly limited, but a logistic sigmoid function, a rectified linear unit (ReLU) function, or the like may be used. As a learning method, various neural network learning methods that have been proposed may be applied. For example, when student data is input to the input layer and a neural network is operated, the difference between the output obtained from the output layer and the teacher data that is previously associated with the student data is calculated, and the difference is calculated. The weighting coefficient and bias value are adjusted to minimize them.

As shown in FIG. 4, a response model 4100 is created by serially connecting the semantic analysis model 4102 created as described above and the Q&A model 4101 created from the response information extracted by the extraction means 2004. I can do it. The Q&A model 4101 is a collection of Q&A data, but in order to connect it to the semantic analysis model 4102, the input question is changed to the feature amount of the semantic analysis model 4102. As for the processing of the response model 4100, first, when a question is input, the semantic analysis model 4102 outputs a feature amount 4103 that can identify the question. The Q&A model 4101 inputs the feature amount 4103 output by the semantic analysis model 4102 and outputs a response corresponding to the feature amount 4103. Through the above processing, it is possible to realize a response model 4100 that can respond to questions other than those stored in advance. In the above embodiment, the semantic analysis model 4102 is generated separately from the Q&A model 4101 by machine learning, but the response model 4100 may be created including the Q&A model 4101 at the machine learning stage.

FIG. 6 is a flowchart showing generation of a response model in this embodiment. Each step will be explained below. Note that each of the following steps is performed by at least one of the information processing device 1100, the client PC 1200, and the smartphone 1300.

S601 is a step in which the acquisition unit 2002 acquires the specific person's Q&A data and corresponding age information from the storage unit 2001. Note that the specific person may be set in advance or may be set based on the user's instructions.

S602 is a step in which the setting unit 2003 sets the user's desired age. The age set here is not a specific age, but may be an age range such as 25 to 30 years old. Alternatively, the age may be calculated based on the specific person's date of birth by specifying the Western calendar or era name.

S603 is a process in which the extraction unit 2004 extracts corresponding response information from the Q&A data of the specific person based on the predetermined age set in S602. For example, if 25 years old is set in S602, only the response information of 25 years old is extracted from the Q&A data of the specific person by referring to the age information acquired in S601.

S604 is a step of generating a response model using the response information extracted in S603. Details of response model generation have been described above, so they will be omitted here.

Through the above processing, it is possible to generate a response model that can reproduce the response of a specific person of the age desired by the user.

Next, a method of communicating using the generated response model using at least one of the information processing device 1100, the client PC 1200, and the smartphone 1300 will be described.

FIG. 7 is a flowchart of communication using the response model in the first embodiment. Each step will be explained below.

S701 is a step in which the input means 2007 selects a response model of a specific person of the age with whom communication is desired based on an input instruction from the user. Note that selection is performed here on the premise that response models for multiple specific persons or multiple ages have already been generated. However, if only one desired response model has been generated, the selection in this step may be omitted.

S702 is a step in which the determining means 2006 determines whether to stop communication with the response model selected in S701. The determination is made based on a user's input instruction, a predetermined period of time, etc. If it is determined that communication is to be stopped, the processing in this flow is ended. If it is determined that the communication is not to be stopped, that is, if it is determined that the communication is to be continued, the process advances to S703.

S703 is a step in which the determining means 2006 determines whether there is a question from the user. Questions from the user are input using the input device 1106, the client PC 1200, and the smartphone 1300 in the form of text or audio data. Furthermore, if a spontaneous statement is desired from a specific person, by setting the spontaneous mode in advance, the process proceeds as if a question had been input even if there was no question from the user. When a question is input, it is determined that there is a question, and the process advances to S704. If it is determined that there are no questions, the process returns to S702 to determine whether to stop communication.

S704 is a step in which the output means 2008 outputs a response to the question input in S703. For the response, the question input in S703 is input into the response model, and the response is output from the response model. The response is output from the speaker 1108, the monitor 1107, the client PC 1200 via the network 1400, and the smartphone 1300. After the output, the process returns to S703 in order to receive a question from the next user.

As described above, according to the present embodiment, it is possible to create a response model of a specific person of the user's desired age and to communicate using the generated response model.

(Second embodiment)
In the first embodiment, a response model 4100 is generated by connecting a semantic analysis model 4102 and a Q&A model 4101 in series. However, in order to appropriately respond to a variety of user questions in a way that is appropriate for a specific person, it may be better to create a response model using a different method.

In general, it is thought that whether or not a response seems to be a specific person is determined based on the content of the response and the tone, such as the habit of ending words and the presence or absence of dialect. Therefore, in this embodiment, a response model incorporating a tone model is generated. Note that the second embodiment is the same as the first embodiment except for the response model generation method, so only the response model generation method in this embodiment will be described below.

FIG. 8 is a conceptual diagram of a response model 8100 in this embodiment. In FIG. 8, a semantic analysis model 8102 is the same as the semantic analysis model 4102 in the first embodiment, and a feature amount 8103 is the same as the feature amount 4103 in the first embodiment. The response model 8100 in this embodiment is realized by connecting a semantic analysis model 8102, a retrained standard response model 8101, and a tone model 8104 in series.

The retrained standard response model 8101 is generated by relearning the standard response model using the Q&A data 3001 and age information 3002. The standard response model is a model that can perform general standard responses, and can be generated by using a standard conversation log or the like as training data. Although it is not possible to learn responses unique to a specific person from standard conversation logs, it is possible to collect a huge amount of Q&A data. Therefore, response models capable of responding to various questions can be generated. In this embodiment, a standard response model capable of responding to various questions is retrained using Q&A data 3001 and age information 3002. That is, relearning is performed using the Q&A data 3001 corresponding to the age set by the user, and responses included in the Q&A data 3001 are given to questions included in the Q&A data 3001. The computer learns to give standard responses to questions that are not included in the Q&A data 3001.

The tone model 8104 is a model that outputs the input text or voice as is, but is made to learn the tone using Q&A data 3001 corresponding to the age set by the user. In other words, the system is trained to output input text or audio that has the same content, but adds the tone of a specific person, such as the habit of the ending or the dialect.

In the response model 8100 in this embodiment, when a question is input, as in the first embodiment, the semantic analysis model 8102 outputs a feature amount 8103 that can identify the question. The retrained standard response model 8101 receives the feature amount 8103 output by the semantic analysis model 8102 as input, and outputs a response corresponding to the feature amount 8103. The response here will be a response typical of a specific person to a question included in the Q&A data 3001, but will be a standard response to a question not included in the Q&A data 3001. The output of the retrained standard response model 8101 is input to the tone model 8104, and is output as a response with a tone similar to that of a specific person. Through the above processing, even if the output from the retrained standard response model 8101 is a standard response, the tone model 8104 outputs a response with a tone that is typical of a specific person, so the user does not feel that the response is similar to a specific person. I can do it. If the output from the retrained standard response model 8101 is a question included in the Q&A data 3001, it becomes possible to make the content and tone of the response similar to that of a specific person.

Note that model learning in this embodiment can be realized using various machine learning techniques such as the neural network and support vector machine described above. As described above, according to the present embodiment, it is possible to generate a response model that appropriately responds to a variety of questions asked by a user in a way that is typical of a specific person.

FIG. 9 shows an example of how avatars are displayed using age-specific response models. By displaying the images of avatars according to age shown in FIG. 9 on the monitor 1107, client PC 1200, or smartphone 1300, the user can feel as if they are facing avatars of different ages. . By setting pre-created age-specific response models to avatars of each age, it is possible to respond to user questions and inquiries in a way that is unique to each age group. You can feel as if you are having a conversation with an avatar.

FIG. 10 is an example of a response model for realizing the response by the avatar in FIG. 9, and is a conceptual diagram of a model for selecting response models by age. As shown in the figure, the response model 10100 includes a standard response model 10101 that has been retrained at the age of 25, a standard response model 10201 that has been retrained at the age of 50, and a standard response model 10301 that has been retrained at the age of 70. ing. Furthermore, a tone model 10104 for a 25-year-old is connected to a standard response model 10101 that has been retrained at a 25-year-old. A 50-year-old tone model 10204 is connected to the standard response model 10201 that has been retrained at the age of 50, and a 70-year-old tone model 10304 is connected to the standard response model 10301 that has been retrained at the age of 70. The functionality of the model is similar to the response 8100 shown in FIG. The major difference is that in the semantic analysis model and age selection 10102 for input (question), in addition to semantic analysis of the input (question), age selection is performed, and features are applied to the model corresponding to the selected age. The amount 10103 is output. The age selection may be set in advance or may be selected according to a setting instruction from the user. Moreover, you may select not only a specific age but also a range of ages. With this configuration, a response model corresponding to each age-specific avatar shown in FIG. 9 is selected, and an appropriate output (response) is made.

(Other examples)
The present invention provides a system or device with a program that implements one or more of the functions of the embodiments described above via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

2002 Acquisition means 2003 Setting means 2004 Extraction means 2005 Generation means

Claims (11)

acquisition means for acquiring information on a specific person's response together with age information of the specific person at the time of the response;
a setting means for setting a predetermined age;
Extracting means for extracting response information corresponding to the predetermined age from the response information based on the predetermined age and the age information;
An information processing apparatus comprising: a generation means for generating a learning model of the specific person's response from the extracted response information. The information processing apparatus according to claim 1, wherein the setting means sets the predetermined age in an age range. 3. The information processing apparatus according to claim 1, wherein each response included in the response information is associated with age information of the specific person. The information processing device according to any one of claims 1 to 3, wherein the response learning model is a model that outputs a response corresponding to the predetermined age in response to an input from a user. . 5. The information processing apparatus according to claim 1, wherein the generating means generates a response learning model including an analysis model for analyzing the meaning of input from a user. The information processing apparatus according to any one of claims 1 to 5, wherein the generating means generates a response learning model including a tone model that reproduces the tone of the specific person of the predetermined age. . 7. The generating means generates a learning model of the response based on the extracted response information and a standard response model that performs a standard response. The information processing device described in . 8. The information processing apparatus according to claim 1, wherein the generating means generates a learning model of the response using a neural network. 9. The information processing apparatus according to claim 1, further comprising a display means for displaying an avatar associated with a learned model of the response of the specific person. An information processing method executed by an information processing device, the method comprising:
an acquisition step in which the acquisition means acquires information on the specific person's response together with age information of the specific person at the time of the response;
a setting step in which the setting means sets a predetermined age;
an extraction step in which the extraction means extracts response information corresponding to the predetermined age from the response information based on the predetermined age and the age information;
An information processing method , characterized in that the generating means has a generating step of generating a learning model of the specific person's response from the extracted response information. computer,
acquisition means for acquiring information on a specific person's response together with age information of the specific person at the time of the response;
a setting means for setting a predetermined age;
Extracting means for extracting response information corresponding to the predetermined age from the response information based on the predetermined age and the age information;
A program that functions as an information processing apparatus, comprising: a generation unit that generates a learning model of the specific person's response from the extracted response information.

Images