JP2020166359A - Program and information processing device - Google Patents

Abstract

To provide a program and information processing device capable of relieving loneliness or anxiety of a user.SOLUTION: A computer is configured to: collect user-specific information based on what a user has said; make utterance based on the collected user-specific information via an avatar set for the user; and output an execution result of the utterance based on the user-specific information made via the avatar representing the user.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to programs and information processing devices.

In recent years, the number of elderly people who live alone and those who move into facilities by themselves is increasing, and such elderly people have few opportunities to talk in their daily lives and feel isolated. Therefore, for example, Patent Document 1 proposes a stuffed animal that can have a pseudo conversation by outputting a voice stored in advance when a voice emitted by a user is acquired. In the technique disclosed in Patent Document 1, the user can give healing to the user and alleviate the feeling of isolation by having a pseudo conversation with the stuffed animal while holding the stuffed animal.

JP-A-2018-117821

People who live alone, not limited to the elderly, may feel lonely or anxious because they have no one to talk to when they are alone at home, for example. Also, for example, even a person who lives with a family member or a roommate may feel lonely or anxious when he / she is alone in his / her room. However, the technique disclosed in Patent Document 1 is intended for the elderly, and it is difficult to provide healing to all those who feel lonely or anxious.

The present disclosure has been made in view of such circumstances, and an object of the present disclosure is to provide a program or the like capable of alleviating a user's feeling of loneliness or anxiety.

The program according to the present disclosure collects user-specific information based on the user's words and actions, executes the collected words and actions based on the user-specific information via an avatar set corresponding to the user, and describes the above. Have the computer execute the process of outputting the result of the avatar executing the words and actions based on the user-specific information.

In the present disclosure, the avatar corresponding to the user performs the behavior based on the user-specific information collected from the behavior of the user. Therefore, for example, the user can have a conversation with his / her own avatar, and can alleviate the user's feeling of loneliness or anxiety. In addition, since the user can know the result of the words and actions performed by his / her avatar based on his / her own information, he / she can experience the words and actions performed by the avatar, that is, the words and actions not performed by the user himself / herself. .. Therefore, users will be able to gain new awareness and lead a mentally richer life through simulated experiences through avatars and simulated experiences by themselves, and will be able to expand the points of contact with reality. By causing actual behavioral changes, it is possible to feel a connection with society and alleviate the feeling of loneliness or anxiety of the user.

It is a schematic diagram which shows the configuration example of an information processing system. It is a block diagram which shows the configuration example of own terminal. It is a schematic diagram which shows the structural example of the DB stored in the own terminal. It is a schematic diagram which shows the structural example of the DB stored in the own terminal. It is a flowchart which shows the procedure example of the collection process of the user-specific information by a local terminal. It is a schematic diagram which shows the screen example. It is a flowchart which shows the procedure example of the word-action processing by own terminal and other terminal. It is a flowchart which shows the procedure example of the word-action processing by own terminal and other terminal. It is a flowchart which shows the procedure example of the word-action processing by own terminal and other terminal. It is a flowchart which shows the procedure example of the word-action processing by own terminal and other terminal. It is a schematic diagram which shows the screen example. It is a flowchart which shows the procedure example of the word-action processing by own terminal. It is a flowchart which shows the procedure example of the reproduction processing of the word and action history by own terminal. It is a schematic diagram which shows the screen example. It is a schematic diagram which shows the structural example of the information processing system of Embodiment 2. It is a flowchart which shows the procedure example of the collection process of the user-specific information by a local terminal. It is a block diagram which shows the configuration example of the own terminal of Embodiment 3. It is a flowchart which shows the procedure example of the word-action processing by own terminal and other terminal of Embodiment 3. It is a block diagram which shows the configuration example of a server. It is a flowchart which shows the procedure example of the word-action processing by a server.

Hereinafter, the program and the information processing apparatus of the present disclosure will be described in detail with reference to the drawings showing the embodiments thereof.

(Embodiment 1)
FIG. 1 is a schematic diagram showing a configuration example of an information processing system. The information processing system of the present embodiment is a system in which a large number of users communicate with other users' terminals using their own terminals. In the information processing system of the present embodiment, an avatar corresponding to each user is set in the terminal of each user, and each user not only communicates with another user via the terminal but also an avatar of another user. You can also interact with. Therefore, each user's avatar set in each user's terminal can not only communicate with other users via the terminal, but also communicate with other users' avatars. The exchange includes, for example, transmission / reception of information related to dialogue processing, collection of information published on a network N such as the Internet, and the like. The information processing system of the present embodiment includes a large number of user terminals 10 and 20, and the user terminals 10 and 20 mutually transmit and receive information via the network N. In the following, one user will be referred to as oneself, another user will be referred to as another person, one's own user terminal will be referred to as own terminal 10, and the other's user terminal will be referred to as another terminal 20. Similar user terminals can be used for the own terminal 10 and the other terminal 20, and for example, a smartphone, a tablet terminal, a personal computer, a portable game machine, or the like can be used. Further, the own terminal 10 and the other terminal 20 may be configured by a dedicated terminal used in the information processing system. In the information processing system of the present embodiment, each of the user terminals 10 and 20 is a process of collecting user-specific information (user-specific information) of each user of the terminals 10 and 20, and another user terminal 10 via network N. Performs various information processing and communication processing such as processing for transmitting and receiving information to and from 20 or other devices.

FIG. 2 is a block diagram showing a configuration example of the own terminal 10. The own terminal 10 includes a control unit 11, a storage unit 12, a processing unit 13, a communication unit 14, an input unit 15, a display unit 16, a speaker 17, a microphone 18, a reading unit 19, and the like, and each of these units is via a bus. They are interconnected. The control unit 11 includes one or a plurality of processors such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit). By appropriately executing the control program 12P stored in the storage unit 12, the control unit 11 causes the own terminal 10 to execute various information processing, control processing, and the like to be performed by the information processing apparatus of the present disclosure.

The storage unit 12 includes a RAM (Random Access Memory), a flash memory, a hard disk, an SSD (Solid State Drive), and the like. The storage unit 12 stores in advance various data and the like necessary for executing the control program 12P and the control program 12P executed by the control unit 11. Further, the storage unit 12 temporarily stores data or the like generated when the control unit 11 executes the control program 12P. Further, the storage unit 12 stores the unique information DB (database) 12a, the destination information DB 12b, and the behavior history DB 12c, which will be described later. The unique information DB 12a, the destination information DB 12b, and the behavior history DB 12c may be stored in an external storage device connected to the own terminal 10, or may be stored in an external storage device capable of communicating with the own terminal 10. Further, the storage unit 12 stores information about the avatar set in association with the user of the own terminal 10. The information about the avatar includes image data for displaying the avatar, voice data for making the avatar speak, motion data for adding motion to the displayed image of the avatar, and the like.

The processing unit 13 includes an interactive processing unit 13a, a unique information collection unit 13b, a word / action execution unit 13c, a history output unit 13d, and the like. Each unit of the processing unit 13 may be realized, for example, by the control unit 11 executing the control program 12P stored in the storage unit 12, or a part of these may be realized by a dedicated hardware circuit. Good. The dialogue processing unit 13a has a dialogue with the user of the own terminal 10. For example, the dialogue processing unit 13a acquires dialogue information from the user by voice input via the microphone 18 or input via the input unit 15 (for example, text input), and the response information to the acquired dialogue information is transmitted via the speaker 17. It is configured to respond to the user by outputting voice or displaying it on the display unit 16 and to have a dialogue with the user of the own terminal 10. The dialogue processing unit 13a stores a plurality of response information to be performed for the dialogue information from the user as a template in advance, and selects and outputs the response information according to the dialogue information from the user from the template. You may have a dialogue. Further, the dialogue processing unit 13a may perform a dialogue by generating and outputting response information corresponding to the dialogue information from the user based on the user-specific information stored in the unique information DB 12a described later. Further, the dialogue processing unit 13a speaks to the user by displaying dialogue information such as a greeting message and a question message on the voice output or the display unit 16 via the speaker 17, and the voice input or the input unit 15 via the microphone 18 is input. By acquiring the response information from the user by inputting through, the dialogue with the user of the own terminal 10 is performed. In this case, the dialogue processing unit 13a may store a plurality of dialogue information such as greeting messages and question messages as templates in advance, select an arbitrary template, and output it to talk to the user.

The dialogue processing unit 13a also performs a dialogue with the user of the other terminal 20 or the avatar of the user of the other terminal 20 (the avatar of the other terminal 20). For example, the dialogue processing unit 13a acquires the dialogue information from the user of the other terminal 20 or the avatar of the other terminal 20 transmitted from the other terminal 20, and returns the response information to the acquired dialogue information to the other terminal 20. It responds to the user or avatar of the other terminal 20 and has a dialogue with the user or avatar of the other terminal 20. In this case as well, the dialogue processing unit 13a may perform the dialogue by selecting and outputting the response information corresponding to the dialogue information from the user or the avatar of the other terminal 20 from the template, or the unique information DB 12a described later. The dialogue may be performed by generating and outputting the response information based on the user-specific information accumulated in the template. Further, the dialogue processing unit 13a speaks to the user or avatar of the other terminal 20 by transmitting dialogue information such as a greeting message and a question message to the user or avatar of the other terminal 20, and the user or the other terminal 20 of the other terminal 20. By acquiring the response information from the avatar of the other terminal 20, a dialogue with the user of the other terminal 20 or the avatar is performed. In this case as well, the dialogue processing unit 13a may talk to the user or avatar of the other terminal 20 by selecting the dialogue information stored in advance and transmitting it to the user or avatar of the other terminal 20.

The unique information collecting unit (collecting unit) 13b collects user-specific information of the user of the own terminal 10 based on the words and actions of the user of the own terminal 10. For example, the unique information collecting unit 13b collects user-specific information from the contents of the dialogue conducted by the dialogue processing unit 13a with the user of the own terminal 10. Further, the unique information collecting unit 13b asks the user of the own terminal 10 a question about the user-specific information by, for example, the dialogue processing unit 13a, and when the answer to the question is obtained, collects the user-specific information from the acquired answer contents. You may. Further, the unique information collecting unit 13b may collect user-specific information from information related to processing that can be executed by various application programs installed in the own terminal 10. For example, the unique information collecting unit 13b allows the user of the own terminal 10 to send and receive e-mails, LINE (registered trademark) messages and LINE stamps, and SNS (Social Network Service) sent and received by the own terminal 10 to and from the other terminal 20. User-specific information may be collected from posted messages, viewed messages, data transmitted and received by various services provided via the network N, and the like. Further, the unique information collecting unit 13b may collect user-specific information based on the actions taken by the user of the own terminal 10. For example, when the user of the own terminal 10 returns a predetermined LINE stamp or LINE message when receiving the LINE message, the unique information collecting unit 13b returns the predetermined LINE stamp or LINE message in response to the reception of the LINE message. Acquire what to do as user-specific information. User-specific information includes, for example, user attributes, behavior history, hobbies, preferences, behavioral tendencies, thinking tendencies, and the like.

The word / action execution unit (execution unit) 13c has a dialogue with the user of the other terminal 20 or the avatar of the other terminal 20 according to an instruction or an action tendency from the user of the own terminal 10, for example, and is open to the public on the network N. Collect various kinds of information. Specifically, the word / action execution unit 13c transmits / receives information to / from another user via the other terminal 20, and the avatar set in the other terminal 20 via the other terminal 20 (of the other terminal 20). By transmitting and receiving information with the word-and-action execution unit 23c), a dialogue with a user or an avatar of another terminal 20 is performed. The word / action execution unit 13c uses the dialogue processing unit 13a to perform a dialogue with a user or an avatar of another terminal 20. In addition, the word / action execution unit 13c searches for various information via the network N and acquires necessary information. In addition, the photographed image taken by the camera outdoors or indoors is published on the network N, and by acquiring such a photographed image via the network N, the behavior execution unit 13c goes out to the shooting place in a pseudo manner. it can. Therefore, the word / action execution unit 13c can execute a pseudo outing process via the avatar of the own terminal 10.

The history output unit (output unit) 13d outputs the history information related to the result of the words and actions executed by the words and actions execution unit 13c, for example, on the display unit 16 or the speaker 17. For example, the history output unit 13d outputs information (dialogue information) transmitted / received to / from another user or another user's avatar via the other terminal 20, and various information collected via the network N. The history output unit 13d displays a screen based on the image data or text data on the display unit 16 when the information to be output includes image data or text data, and when the information includes voice data or text data, the history output unit 13d displays the screen based on the image data or text data. Audio is output from the speaker 17.

The communication unit 14 is an interface for connecting to the network N by wired communication or wireless communication, and transmits / receives information to / from an external device via the network N. The input unit 15 receives an operation input by the user and sends a control signal corresponding to the operation content to the control unit 11. The display unit 16 is a liquid crystal display, an organic EL display, or the like, and displays various information according to instructions from the control unit 11. The input unit 15 and the display unit 16 may be a touch panel integrally configured.

The speaker 17 is an audio output device having an amplifier or the like, and after amplifying audio data according to an instruction from the control unit 11 by the amplifier, the speaker 17 outputs audio. The microphone 18 is an audio input device having an amplifier, an A / D (analog / digital) converter, and the like, collects ambient audio, acquires analog audio data, and amplifies the acquired audio data with an amplifier. , Convert to digital audio data with an A / D converter to acquire audio data. The microphone 18 collects sound according to an instruction from the control unit 11, and sequentially sends the acquired voice data to the control unit 11. The speaker 17 and the microphone 18 may be built in the own terminal 10, may be externally attached to the own terminal 10 via a cable or the like, or may be an integrated headset.

The reading unit 19 reads information stored in a portable storage medium 1a including a CD (Compact Disc) -ROM, a DVD (Digital Versatile Disc) -ROM, or a USB (Universal Serial Bus) memory. The control program 12P and various data stored in the storage unit 12 may be read by the control unit 11 from the portable storage medium 1a via the reading unit 19 and stored in the storage unit 12. Further, the control program 12P and various data stored in the storage unit 12 may be downloaded by the control unit 11 from an external device via the communication unit 14 and stored in the storage unit 12. Further, the control program 12P and various data may be stored in the semiconductor memory 1b, and the control unit 11 may read the data from the semiconductor memory 1b and store the data in the storage unit 12.

3 and 4 are schematic views showing a configuration example of DB 12a to 12c stored in the own terminal 10. FIG. 3 shows the unique information DB 12a, FIG. 4A shows the destination information DB 12b, and FIG. 4B shows the behavior history DB 12c. The unique information DB 12a stores user-specific information about the user of the own terminal 10. The user-specific information is collected by the unique information collecting unit 13b from the user of the own terminal 10 and stored in the unique information DB 12a. The unique information DB 12a shown in FIG. 3 includes an information ID string, an information content string, a unique information string, and the like. The information ID column stores identification information assigned in advance to each of the user-specific information. The information content column stores information indicating the content and type of user-specific information. If the content of the user-specific information corresponding to each information ID is clear, it is not necessary to provide the information content string in the unique information DB 12a. The unique information string stores user-specific information collected by the unique information collecting unit 13b. The information ID and information content stored in the unique information DB 12a are stored in advance, and the unique information is stored or updated by the control unit 11 when the unique information collecting unit 13b acquires various user-specific information.

In the unique information DB 12a shown in FIG. 3, as user-specific information, personal information including name, age, gender, etc., address, work company name, school name, family structure, family information, etc., personal information, personality, Information about tastes including hobbies, favorite things, favorite things, favorite foods, favorite drinks, favorite fruits, dislikes, dislikes, dislike foods, dislike drinks, dislike fruits, etc., I was happy. Information about emotions, including things, things that are angry, things that are of concern, places of concern, etc. are registered. The registered contents of the unique information DB 12a are not limited to these, and are configured so that various information about the user can be registered. For example, annual income, commuting route, job title at the company, work information including colleague information and boss information, information about pets, behavior history and behavior tendency including the place of visit, purchase history including purchased products and Information about the user's inner side such as purchasing tendency, personality, and way of thinking may also be registered in the unique information DB 12a. Further, the unique information DB 12a may be divided into a plurality of DBs according to the type of user-specific information to be registered and the like.

The destination information DB 12b is an address book that stores the destination information of another terminal 20 used when the own terminal 10 transmits / receives information via the network N. The destination information DB 12b shown in FIG. 4A includes a terminal ID string, an owner name string, a destination information string, and the like. The terminal ID column stores the identification information assigned in advance to each other terminal 20, and the owner name column stores the name of the user (other person) of each other terminal 20 in association with the terminal ID. The destination information string stores the destination information assigned to each other terminal 20 in association with the terminal ID. As the destination information, for example, a telephone number, an e-mail address, an IP (Internet Protocol Address) address, a LINE-ID, or the like assigned to each other terminal 20 can be used. The terminal ID stored in the destination information DB 12b is issued and stored by the control unit 11 when the control unit 11 acquires new information of the other terminal 20 via the input unit 15 or the communication unit 14. Each information other than the terminal ID stored in the destination information DB 12b is added or changed by the control unit 11 each time the control unit 11 acquires an addition or change instruction via the input unit 15 or the communication unit 14. The stored contents of the destination information DB 12b are not limited to the example shown in FIG. 4A, and various information related to the other terminal 20 can be stored.

The word / action history DB 12c stores history information related to the result of the word / action executed by the word / action execution unit 13c. The word / action history DB 12c shown in FIG. 4B includes a word / action ID string, a word / action content string, a partner information string, a place information string, a word / action data string, and the like. The word / action ID string stores identification information assigned in advance to each word / action (word / action processing) executed by the word / action execution unit 13c, and the word / action content string is associated with the word / action ID to perform the word / action (word / action execution unit 13c). Stores information indicating the content of (word and action processing). The contents of words and actions include dialogues with other users or avatars of other users, and going out simulated via network N. The other party information string stores the information of the other party in the words and actions executed by the words and actions execution unit 13c in association with the words and actions ID. The other party in the words and actions is the other party of the dialogue executed by the words and actions execution unit 13c, the companion who goes out executed by the words and actions execution unit 13c, and the terminal ID, owner name, or destination information of the other terminal 20 of the other party is the other party information string. Is remembered in. The place information string stores information on the place in the word / action executed by the word / action execution unit 13c in association with the word / action ID. The place in the words and actions is, for example, a place where the words and actions execution unit 13c has performed in a simulated manner, and information on the shooting place of the photographed image acquired via the network N is stored in the place information string. The word / action data string stores the word / action data in the word / action executed by the word / action execution unit 13c in association with the word / action ID. The behavioral data is, for example, dialogue data (voice data or text data) related to a dialogue performed with another user or an avatar of another user, a photographed image (image data) acquired via the network N, or the like. The word / action data is not limited to the configuration stored in the word / action history DB 12c, and may be stored in a predetermined area of the storage unit 12 or an external storage device connected to the own terminal 10. In this case, information for reading out the behavioral data (dialogue data, captured image, etc.) (for example, a file name indicating a storage location of the data) may be stored in the behavioral data string. The stored content of the behavior history DB 12c is not limited to the example shown in FIG. 4B, and various information related to the behavior history can be stored.

The other terminal 20 has the same configuration as the above-described configuration of the own terminal 10, and each of the constituent units 11 to 19 of the own terminal 10 will be described as the respective constituent units 21 to 29 in the other terminal 20. ..

The processes performed by the respective user terminals 10 and 20 will be described below. Since the own terminal 10 and the other terminal 20 perform the same processing, the processing performed by the own terminal 10 will be described below as an example. FIG. 5 is a flowchart showing an example of a procedure for collecting and processing user-specific information by the own terminal 10, and FIG. 6 is a schematic diagram showing a screen example. The following processing is executed by the control unit 11 according to the control program 12P stored in the storage unit 12 of the own terminal 10. The control unit 11 of the own terminal 10 collects user-specific information about the user of the own terminal 10 by the unique information collecting unit 13b at an appropriate timing. The appropriate timing may be, for example, a regular timing such as once a week or once a month, or a continuous timing until a certain amount of user-specific information is accumulated in the unique information DB 12a. You may.

For example, the unique information collecting unit 13b has a dialogue with the user of the own terminal 10 by the dialogue processing unit 13a, and generates a dialogue message (dialogue information) to be talked with the user of the own terminal 10 (S11). Here the dialogue message, "Hello. Genki?" May be a greeting message such as, may be a response message to the dialogue message from the user. In addition, the unique information collection unit 13b may generate a question message as an interactive message so that user-specific information can be collected. The unique information collecting unit 13b outputs the generated dialogue message to the speaker 17 or the display unit 16 (S12). The unique information collecting unit 13b outputs voice from the speaker 17 when the dialogue message includes voice data, and displays the dialogue message on the display unit 16 when the dialogue message includes display data such as image data or text data. At this time, the unique information collecting unit 13b (or the dialogue processing unit 13a) has, for example, based on the image data of the avatar stored in the storage unit 12, as shown in FIG. 6, the image of the avatar of the user of the own terminal 10. May be displayed on the display unit 16. Further, the unique information collecting unit 13b (or the dialogue processing unit 13a) may output a dialogue message from the speaker 17 by voice based on the voice data related to the avatar stored in the storage unit 12. In this case, the user can feel as if he / she is interacting face-to-face with the avatar displayed on the screen shown in FIG. As the display image of the avatar, a live-action image of the user of the own terminal 10, an image of a character similar to a human being, an image of an animal or the like, an image of a character such as an animal, or the like can be used.

The unique information collecting unit 13b acquires the information answered by the user in response to the output dialogue message by the microphone 18 or the input unit 15 (S13). The unique information collecting unit 13b acquires the answer information by voice input via the microphone 18 when the user answers by utterance, and when the user answers via the input unit 15, the answer information is input to the input unit 15. Get through. Then, the unique information collecting unit 13b extracts the user-specific information from the acquired response information (S14). For example, when the unique information collecting unit 13b outputs a question message (dialogue message) by voice saying "Please tell me your name" and the user utters "It is ○○○○", the unique information collecting unit 13b is the user. The user's name (user-specific information) "○○○○" is extracted from the answer information "○○○○". In addition, when the user utters "I had a quarrel with my father today" in the dialogue with the dialogue processing unit 13a, the unique information collecting unit 13b uses the content of the user's utterance as information on recent events (user-specific information). I quarreled with my dad today. " Further, for example, when the user utters "I watched a movie of XX today", the unique information collecting unit 13b uses the content of the user's utterance as information on the movie that the user has recently watched (user-specific information). ○ ”is extracted.

The unique information collecting unit 13b stores the acquired user-specific information in the unique information DB 12a in association with the information ID and the information content (S15). For example, a question message corresponding to the user-specific information to be registered in the unique information DB 12a is stored in the storage unit 12 in advance, and the unique information collecting unit 13b asks the user of the own terminal 10 a question about each question message. , Unique information User-specific information to be registered in the DB 12a can be collected. Further, each time the dialogue processing unit 13a interacts with the user of the own terminal 10, the unique information collecting unit 13b can collect user-specific information from the contents of the dialogue by the user and store the user-specific information in the unique information DB 12a.

Since the user-specific information is sequentially extracted and registered from the contents of the dialogue with the user by the above-described processing, the user-specific information can be accumulated without the user being aware of it. If the user-specific information stored in the unique information DB 12a is already stored in the unique information DB 12a, it may be overwritten and stored, or it may be stored (stored) as time-series data together with the acquisition date and time. .. By accumulating various information (user-specific information) about the user in this way, the avatar (user's avatar) set in the own terminal 10 can be made to perform words and actions based on the user-specific information. That is, it is possible to make the avatar of the own terminal 10 perform words and actions that are likely to be performed by the user of the own terminal 10. When collecting user-specific information, the user's heart rate, electroencephalogram, electromyogram, etc. are measured to determine whether the user's mental and physical condition is relaxed and calm, and collected when the user is calm. Only the user-specific information obtained may be stored in the unique information DB 12a. In this case, by discarding the user-specific information collected when the mental and physical state is not calm and excited, only the user-specific information collected in the normal state of the user can be accumulated.

7 and 8 are flowcharts showing an example of a procedure for word-and-action processing by the own terminal 10 and the other terminal 20. The following processing is executed by the control units 11 and 21 according to the control programs 12P and 22P stored in the storage units 12 and 22 of the terminals 10 and 20. In FIGS. 7 and 8, the processing performed by the own terminal 10 is shown on the left side, and the processing performed by the other terminal 20 is shown on the right side. The processing shown in FIGS. 7 and 8 is performed by the control unit 11 of the own terminal 10 based on the user-specific information stored in the unique information DB 12a in the avatar set in the own terminal 10 by the word / action execution unit 13c. This is a process for performing a process (dialogue process) and performing a dialogue with a user or an avatar of another terminal 20.

Specifically, whether or not the word / action execution unit 13c has received a request for executing a dialogue with any user of the other terminal 20 or an avatar set in the other terminal 20 via the microphone 18 or the input unit 15. Is determined (S21). For example, the user of the own terminal 10 inputs the information of the other party to interact with (information indicating the user or avatar of the other terminal 20, destination information of the other terminal 20, etc.) and the content to be interacted with via the microphone 18 or the input unit 15. Then, instruct the avatar of the own terminal 10 to have a dialogue with the dialogue partner. Specifically, when the user of the own terminal 10 utters "Please confirm △△ with Mr. XX.", The word / action execution unit 13c announces "Mr. XX" as the dialogue partner and "Mr. XX" as the dialogue content. "About △△" is extracted and accepted as an instruction to have a dialogue with the dialogue partner. When the other party to interact with is the avatar of the other terminal 20, the word / action execution unit 13c extracts "Mr. XX's avatar" as the dialogue partner. Further, the word / action execution unit 13c extracts the other party and the dialogue content that the user of the own terminal 10 wants to have a dialogue with based on the user-specific information stored in the unique information DB 12a, and if the extraction can be performed, an instruction to perform a dialogue with the dialogue partner. Accept as (dialogue execution request).

When it is determined that the dialogue request is not accepted (S21: NO), the word / action execution unit 13c waits until the dialogue request is accepted. When it is determined that the dialogue request has been accepted (S21: YES), the word / action execution unit 13c reads the requested destination information of the dialogue partner from the destination information DB 12b (S22). In step S22, the word / action execution unit 13c reads out the destination information stored in the destination information DB 12b in association with the requested information of the dialogue partner (for example, the name and the owner name of the other terminal 20). Then, the word / action execution unit 13c requests the other terminal 20 of the requested dialogue partner to execute the dialogue based on the read destination information (S23). In step S23, the word / action execution unit 13c executes a dialogue with the user or avatar of the other terminal 20 depending on whether the requested dialogue partner is the user of the other terminal 20 or the avatar set in the other terminal 20. To request. Further, the word / action execution unit 13c performs a dialogue (transmission / reception of a message) with another terminal 20 by using, for example, a chat application or a message application. The chat application or the message application is stored in the storage unit 12 in advance.

When the control unit 21 (word / action execution unit 23c) of the other terminal 20 is requested to execute the dialogue from the own terminal 10, whether or not the requested dialogue partner (user or avatar of the other terminal 20) can interact with the other terminal 20. Is determined (S24). In step S24, when the requested dialogue partner is a user of another terminal 20, the speech / action execution unit 23c notifies (outputs) that the dialogue request has been accepted, for example, via the speaker 27 or the display unit 26, and the microphone 28. Alternatively, it accepts whether or not the requested dialogue can be executed via the input unit 25. For example, the word / action execution unit 23c has a message such as "A dialogue request has come from Mr. XX" on the display unit 26, an OK button for instructing the start of the dialogue execution, and a cancel button for refusing the execution of the dialogue. And are displayed. Then, the word / action execution unit 23c accepts an operation for either the OK button or the cancel button via the input unit 25, and determines whether or not the requested dialogue can be executed according to the accepted operation. In addition, the word / action execution unit 23c outputs a message such as "A dialogue request has been received from Mr. XX. What should I do?" On the speaker 27, and receives a response from the user by voice input via the microphone 28. Depending on the response received, it is determined whether the requested dialogue can be performed. Further, when the requested dialogue partner is an avatar of another terminal 20, the word / action execution unit 23c depends on whether or not it is in a dialogue with a terminal other than the own terminal 10 (another other terminal 20), for example. To determine if the requested dialogue can be performed. For example, when the word / action execution unit 23c is in a dialogue with a terminal other than the own terminal 10, it determines that the requested dialogue cannot be executed, and is not in a dialogue with a terminal other than the own terminal 10. , Judge that the requested dialogue can be performed.

When it is determined that the requested dialogue partner is not capable of dialogue (S24: NO), the word / action execution unit 23c notifies the own terminal 10 that the requested dialogue partner is not capable of dialogue (dialogue is not possible) (S25). Upon receiving the notification that dialogue is not possible, the word / action execution unit 13c of the own terminal 10 waits for a predetermined time (S26), returns to the process of step S23, and again, based on the destination information read in step S22, the other terminal of the dialogue partner. Request 20 to execute a dialogue (S23).

When it is determined that the dialogue requested by the own terminal 10 is possible (S24: YES), the word / action execution unit 23c of the other terminal 20 notifies the own terminal 10 that the dialogue is possible (S27). When the other terminal 20 notifies that dialogue is possible, the word / action execution unit 13c of the own terminal 10 interacts based on the dialogue content requested via the microphone 18 or the input unit 15 or the dialogue content extracted from the user-specific information. Generate information (S28). In step S28, the word / action execution unit 13c generates dialogue information based on the information extracted as the dialogue content, and generates a dialogue message such as "What about △△?". The word / action execution unit 13c may generate a dialogue message to the dialogue partner (user or avatar of another terminal 20) by the dialogue processing unit 13a. Then, the word / action execution unit 13c transmits the generated dialogue information to the other terminal 20 of the dialogue partner (S29).

When the dialogue information is received from the own terminal 10, the word / action execution unit 23c of the other terminal 20 determines whether or not the requested dialogue partner is a user of the other terminal 20 (S30). When it is determined that the dialogue partner is a user of the other terminal 20 (S30: YES), the word / action execution unit 23c outputs the received dialogue information (S31). In step S31, the word / action execution unit 23c outputs the received dialogue information (dialogue message) to the speaker 27 or the display unit 26. The word / action execution unit 23c receives the response information (response message) from the user of the other terminal 20 to the output dialogue information via the microphone 28 or the input unit 25 (S32), and transmits the accepted response information to the own terminal 10. (S33).

When it is determined that the dialogue partner is not a user of the other terminal 20 (S30: NO), that is, when the dialogue partner is an avatar of the other terminal 20, the word / action execution unit 23c of the other terminal 20 stores it in the unique information DB 22a. Based on a certain user-specific information, response information (response message) for the dialogue information received from the own terminal 10 is generated (S34). In step S34, when the dialogue information received from the own terminal 10 is "What about △△", the word / action execution unit 23c searches the stored contents of the unique information DB 22a for information related to "△△". Use the search results to generate a response message such as "□□". The word / action execution unit 23c transmits the generated response information to the own terminal 10 (S33).

The word / action execution unit 13c of the own terminal 10 stores the response information received from the other terminal 20 in the word / action history DB 12c as history information (S35). The word / action execution unit 13c also stores the transmitted dialogue information as history information in the word / action history DB 12c even when the dialogue information is transmitted to the other terminal 20 in step S29. As a result, the message (dialogue content) transmitted / received to / from the other terminal 20 can be stored in the behavior history DB 12c. The word / action execution unit 13c determines whether or not the dialogue with the other terminal 20 has ended (S36), and if it determines that the dialogue has not ended (S36: NO), returns to the process of step S28 and returns to the process of the other terminal 20. Further generation of dialogue information to be transmitted to 20 (S28). The words and actions execution units 13c and 23c of the own terminal 10 and the other terminal 20 engage in dialogue by repeating the processes of steps S28 to S35, and the result of the dialogue (dialogue information transmitted and received) is accumulated in the words and actions history DB 12c. When the word / action execution unit 13c of the own terminal 10 determines that the dialogue with the other terminal 20 has ended (S36: YES), the dialogue with the other terminal 20 ends.

By the above-mentioned processing, the user of the own terminal 10 can interact with the user or the avatar of the other terminal 20 by the own avatar set in the own terminal 10. Therefore, when it is difficult for the user of the own terminal 10 to directly interact with the user of the other terminal 20, he / she can interact with the user of the other terminal 20 via his / her own avatar. For example, when the user of the own terminal 10 quarrels with the user of the other terminal 20, the user of the own terminal 10 can apologize to the user of the other terminal 20 and make up with his / her own avatar. Further, even when the time for the user of the own terminal 10 and the user of the other terminal 20 to interact cannot be secured, the dialogue time can be secured by, for example, having a dialogue between the avatars. Since the word / action execution units 13c and 23c cause the avatars of the terminals 10 and 20 to perform word / action processing (dialogue processing) based on the user-specific information stored in the unique information DBs 12a and 22a, the words / actions execution units 13c and 23c perform the word / action processing (dialogue processing) on the terminals 10 and 20. It is possible to realize the interactive processing that is likely to be performed by the user.

The user of the other terminal 20 may be an expert such as a doctor, a nurse, an occupational therapist, or a dietitian. In this case, the user of the own terminal 10 specializes in physical condition, illness, nutritional management, etc. by his / her avatar. You can talk to your house. Further, the avatar of the other terminal 20 may be an avatar of these experts (expert avatar). For example, the word / action execution unit 13c of the own terminal 10 receives a consultation (dialogue) execution request for a predetermined expert or an expert's avatar from the user of the own terminal 10 via the microphone 18 or the input unit 15. Then, the word / action execution unit 13c may consult with the expert or the expert's avatar by conducting a dialogue with the received expert or the expert's avatar based on the user-specific information accumulated in the unique information DB 12a. it can. In this way, the content (dialogue content) consulted with the expert or the expert's avatar is also stored in the behavior history DB 12c as the dialogue history. Then, in response to the request from the user of the own terminal 10, the history output unit 13d outputs the dialogue content related to the dialogue history, so that the user of the own terminal 10 can know the consultation result. With such a configuration, even if the user of the own terminal 10 consults with a content that is difficult to hear by himself / herself, the avatar consults with an expert instead of himself / herself and obtains an answer from the expert. Ten users can feel free to consult. Also, when consulting with an expert avatar, the expert avatar will answer the consultation instead of the busy expert.

9 and 10 are flowcharts showing a procedure example of speech / action processing by the own terminal 10 and another terminal 20, and FIG. 11 is a schematic diagram showing a screen example. In FIGS. 9 and 10, the processing performed by the own terminal 10 is shown on the left side, and the processing performed by the other terminal 20 is shown on the right side. The processes shown in FIGS. 9 and 10 are processes when the user of the own terminal 10 has a dialogue with the user or the avatar of the other terminal 20 using the own terminal 10. In the processes shown in FIGS. 9 and 10, detailed description of the same processes as those shown in FIGS. 7 and 8 will be omitted.

The word / action execution unit 13c of the own terminal 10 determines whether or not the execution request of the dialogue with any user or avatar of the other terminal 20 has been received via the microphone 18 or the input unit 15 (S41). For example, the user of the own terminal 10 inputs information of the other party to talk to (information indicating the user or avatar of the other terminal 20, destination information of the other terminal 20, etc.) through the microphone 18 or the input unit 15, and the user of the own terminal 10 talks with the other party. Instruct to perform a dialogue with. Specifically, when the user of the own terminal 10 utters "I want to talk to Mr. XX", the word / action execution unit 13c extracts "Mr. XX" as the dialogue partner and executes the dialogue with the dialogue partner. Accept as an instruction. When the other party to interact with is the avatar of the other terminal 20, the word / action execution unit 13c extracts "Mr. XX's avatar" as the dialogue partner.

When it is determined that the dialogue request is not accepted (S41: NO), the word / action execution unit 13c waits until the dialogue request is accepted. When it is determined that the dialogue request has been accepted (S41: YES), the word / action execution unit 13c reads the requested destination information of the dialogue partner from the destination information DB 12b (S42). Then, the word / action execution unit 13c requests the other terminal 20 of the requested dialogue partner to execute the dialogue based on the read destination information (S43). The word / action execution unit 13c requests execution of a dialogue with the user or avatar of the other terminal 20 depending on whether the requested dialogue partner is a user or an avatar of the other terminal 20.

When the own terminal 10 requests the execution of a dialogue, the word / action execution unit 23c of the other terminal 20 determines whether or not the requested dialogue partner (user or avatar of the other terminal 20) can interact (S44). ). When it is determined that the requested dialogue partner is not capable of dialogue (S44: NO), the word / action execution unit 23c notifies the own terminal 10 that the requested dialogue partner is not capable of dialogue (cannot be dialogue) (S45). When the other terminal 20 notifies that the dialogue is not possible, the word / action execution unit 13c of the own terminal 10 displays a message indicating that the user or the avatar of the other terminal 20 cannot interact with the display unit 16 or the speaker 17 By outputting voice from, the user of the own terminal 10 is notified (S46), and the process is terminated.

When the word / action execution unit 23c of the other terminal 20 determines that the dialogue requested by the own terminal 10 is possible (S44: YES), the own terminal 10 notifies that the dialogue is possible (S47). When the other terminal 20 notifies that dialogue is possible, the word / action execution unit 13c of the own terminal 10 displays a message indicating that the user or avatar of the other terminal 20 is capable of dialogue on the display unit 16 or the speaker 17 Notifies the user of the own terminal 10 by outputting voice from (S48). When the user terminals 10 and 20 have a camera and are configured to be able to make a videophone call to send and receive a photographed image of the user taken by the camera between the user terminals 10 and 20, the behavior of the own terminal 10 is executed. The unit 13c may display a screen as shown in FIG. 11A on the display unit 16 and start a videophone call with a user of another terminal 20. Note that FIG. 11A shows an example in which a photographed image of a user of another terminal 20 is displayed, and the user of the own terminal 10 can interact with the user of the other terminal 20 via a screen as shown in FIG. 11A. .. When the dialogue partner is an avatar of another terminal 20, the word / action execution unit 23c of the other terminal 20 transmits an image of the avatar set in the other terminal 20 to the own terminal 10 and the word / action execution unit of the own terminal 10. The 13c displays a screen as shown in FIG. 11B on the display unit 16. FIG. 11B shows an example in which an image of the avatar of the other terminal 20 is displayed, and the user of the own terminal 10 can interact with the avatar of the other terminal 20 via the screen as shown in FIG. 11B.

The speech / action execution unit 13c receives dialogue information from the user of the own terminal 10 to the dialogue partner via the microphone 18 or the input unit 15 (S49), and transmits the received dialogue information to the other terminal 20 of the dialogue partner (S50). .. When the dialogue information is received from the own terminal 10, the word / action execution unit 23c of the other terminal 20 determines whether or not the requested dialogue partner is a user of the other terminal 20 (S51). When it is determined that the dialogue partner is a user of the other terminal 20 (S51: YES), the word / action execution unit 23c outputs the received dialogue information (dialogue message) to the speaker 27 or the display unit 26 (S52). Then, the word / action execution unit 23c receives the response information (response message) from the user of the other terminal 20 to the output dialogue information via the microphone 28 or the input unit 25 (S53), and receives the received response information to the own terminal 10. Transmit (S54).

When it is determined that the dialogue partner is not a user of the other terminal 20 (S51: NO), that is, when the dialogue partner is an avatar of the other terminal 20, the word / action execution unit 23c of the other terminal 20 stores it in the unique information DB 22a. Based on a certain user-specific information, response information (response message) for the dialogue information received from the own terminal 10 is generated (S55). Then, the word / action execution unit 23c transmits the generated response information to the own terminal 10 (S54). The word / action execution unit 13c of the own terminal 10 outputs the reply information (reply message) received from the other terminal 20 to the speaker 17 or the display unit 16 (S56), and further stores it in the word / action history DB 12c as history information (S57). .. The word / action execution unit 13c also stores the transmitted dialogue information as history information in the word / action history DB 12c even when the dialogue information is transmitted to the other terminal 20 in step S50. As a result, the message (dialogue content) transmitted / received to / from the other terminal 20 can be stored in the behavior history DB 12c.

The word / action execution unit 13c of the own terminal 10 determines whether or not the dialogue with the other terminal 20 has ended (S58), and if it determines that the dialogue has not ended (S58: NO), the process of step S49 is performed. Returning, the dialogue information to be transmitted to the other terminal 20 is received from the user of the own terminal 10 via the microphone 18 or the input unit 15 (S49). The words and actions execution units 13c and 23c of the own terminal 10 and the other terminal 20 engage in dialogue by repeating the processes of steps S49 to S57, and the result of the dialogue (dialogue information transmitted and received) is accumulated in the words and actions history DB 12c. When the word / action execution unit 13c of the own terminal 10 determines that the dialogue with the other terminal 20 has ended (S58: YES), the dialogue with the other terminal 20 ends.

By the process described above, the user of the own terminal 10 can interact with the user or the avatar of the other terminal 20 by using the own terminal 10. Therefore, the user of the own terminal 10 can interact with the user of the other terminal 20 via the terminals 10 and 20, and when the captured images are transmitted and received from each other, the videophone can be made. Further, when the user of the own terminal 10 has difficulty in directly interacting with the user of the other terminal 20, or when the user of the other terminal 20 is busy, the dialogue with the user of the other terminal 20 is simulated by interacting with the avatar of the other terminal 20. Can be realized. The user of the other terminal 20 may be an expert such as a doctor, a nurse, an occupational therapist, or a dietitian. In this case, the user of the own terminal 10 becomes an expert on physical condition, illness, nutritional management, and the like. You can consult. Further, the avatar of the other terminal 20 may be an avatar of these experts (expert avatar). In this case, the user of the own terminal 10 can feel free to consult because the answer to the consultation content can be obtained by the expert's avatar instead of the busy expert.

FIG. 12 is a flowchart showing an example of a procedure for word-and-action processing by the own terminal 10. In the process shown in FIG. 12, the control unit 11 of the own terminal 10 causes the avatar of the own terminal 10 to perform a pseudo outing action based on the user-specific information stored in the unique information DB 12a by the word / action execution unit 13c. It is a process. The word / action execution unit 13c determines whether or not there is a place where the user of the own terminal 10 wants to go, based on the information stored in the unique information DB 12a (S61). Further, the word / action execution unit 13c may determine the place where the user of the own terminal 10 wants to go, depending on whether or not the information of the place where the user of the own terminal 10 wants to go is received through the microphone 18 or the input unit 15. You may decide whether or not there is.

When the user of the own terminal 10 determines that there is a place he / she wants to go to (S61: YES), the word / action execution unit 13c searches for information about the place he / she wants to go from various information published on the network N (S62). ). Then, the word / action execution unit 13c collects the captured images taken by the camera at the place where the user of the own terminal 10 wants to go and published on the network N from the search results via the network N (S63). For example, when the user wants to climb Mt. Fuji, the behavior execution unit 13c collects images of Mt. Fuji taken by a camera installed near Mt. Fuji, images taken by climbers of Mt. Fuji, and the like. As a result, the word / action execution unit 13c can perform a pseudo-going-out act of going to a place where the user of the own terminal 10 wants to go and acquiring a photographed image via the avatar of the own terminal 10. The behavior execution unit 13c stores the collected captured image as history information in the behavior history DB 12c (S64). The information to be collected is not limited to the captured image, and may be any information as long as it is information about the location. Further, the word / action execution unit 13c may store the collected information in the word / action history DB 12c after editing the collected information into information suitable for the user's preference based on, for example, user-specific information.

When it is determined that there is no place where the user of the own terminal 10 wants to go (S61: NO), the word / action execution unit 13c wants to do it based on the information stored in the unique information DB 12a. It is determined whether or not there is a case (S65). When the user of the own terminal 10 determines that there is something he / she wants to do (S65: YES), the word / action execution unit 13c searches various information published on the network N for information about what he / she wants to do (S66). Then, the word / action execution unit 13c collects captured images published on the network N regarding what the user of the own terminal 10 wants to do from the search results via the network N (S67). For example, when the user wants to skydive, the behavior execution unit 13c collects an image of a person doing skydiving, an image taken during skydiving, and the like. As a result, the word / action execution unit 13c can pseudo-execute what the user of the own terminal 10 wants to do through the avatar of the own terminal 10. The behavior execution unit 13c stores the collected captured image as history information in the behavior history DB 12c (S68). The information to be collected is not limited to the captured image, and may be any information as long as it is information about the desired thing to be done. Further, the word / action execution unit 13c may store the collected information in the word / action history DB 12c after editing the collected information into information suitable for the user's preference based on, for example, user-specific information.

When the user determines that there is nothing he / she wants to do (S65: NO), the word / action execution unit 13c returns to the process of step S61, and each time he / she determines that he / she wants to go or has something he / she wants to do, he / she outputs various information by the above-mentioned process. It can be collected and the history information of the pseudo action can be stored in the behavior history DB 12c. By the above-described processing, the own terminal 10 appropriately extracts the place or what the user wants to go to based on the user-specific information stored in the unique information DB 12a, acts on behalf of the user, and obtains the result of the action. Information can be accumulated as history information. Therefore, even if the user of the own terminal 10 does not have time to go to the place he / she wants to go or do what he / she wants to do, he / she can use the avatar of the own terminal 10 to pseudo-go to the place he / she wants to go and do what he / she wants to do. It can be done in a pseudo manner. Since the word / action execution unit 13c performs word / action processing (action, going out) based on the user-specific information stored in the unique information DB 12a via the avatar of the own terminal 10, the user of the own terminal 10 may perform the word / action processing. Can achieve high behavior.

FIG. 13 is a flowchart showing an example of a procedure for reproducing the behavior history by the own terminal 10, and FIG. 14 is a schematic diagram showing a screen example. In the own terminal 10 in which the behavior history (history information) is accumulated in the behavior history DB 12c by the dialogue processing shown in FIGS. 7 to 10 and the action processing shown in FIG. Is reproduced on the display unit 16 or the speaker 17. Specifically, the history output unit 13d determines whether or not a reproduction request for the word / action history (history information) accumulated in the word / action history DB 12c has been received via the microphone 18 or the input unit 15 (S71). When the user of the own terminal 10 wants to reproduce any of the behavior history, he / she inputs a reproduction request of the behavior history via the microphone 18 or the input unit 15.

When the history output unit 13d determines that the reproduction request for the behavior history is not accepted (S71: NO), the history output unit 13d waits until the reproduction request is accepted. When the history output unit 13d determines that the reproduction request has been accepted (S71: YES), the history output unit 13d outputs information related to each behavior history as shown in FIG. 14 based on the behavior history (history information) accumulated in the behavior history DB 12c. Generate a history screen to be displayed (S72). The history output unit 13d displays the generated history screen on the display unit 16 (S73). The history screen shown in FIG. 14 displays an image of the avatar of the own terminal 10 on the upper left, and the activity (words and actions) performed by the avatar of the own terminal 10 today is based on each of the words and actions history accumulated in the words and actions history DB 12c. A thumbnail image of history information and information indicating the content of words and actions are displayed. When the behavior data included in the history information is image data, the thumbnail image of the history information is generated based on this image data. If the word / action data included in the history information is not image data, for example, if the word / action data is only voice data, a “voice only” message may be displayed instead of the thumbnail image. The information indicating the behavioral content displayed on the history screen is generated based on the behavioral content, partner information, location information, etc. included in the history information. Further, the history screen displays a "past activity history" button for displaying the word / action history (history information) in the activity (word / action) performed by the avatar of the own terminal 10 in the past. Therefore, when the "past activity history" button is operated via the input unit 15 on the history screen, the history output unit 13d displays the information on the past behavior history accumulated in the behavior history DB 12c on the history screen.

The history screen is configured so that any one of the behavior history can be selected as a playback target, and a play button for requesting playback of the selected behavior history and a cancel button for ending the playback process are displayed. are doing. The history output unit 13d makes a reproduction request (output request) for the selected history information according to whether or not any of the words and actions history is selected and the play button is operated via the input unit 15 on the history screen. It is determined whether or not it has been accepted (S74). When it is determined that the reproduction request is not accepted (S74: NO), the history output unit 13d waits until the reproduction request is accepted. When it is determined that the reproduction request has been accepted (S74: YES), the history output unit 13d reads the selected history information from the behavior history DB 12c (S75), and outputs the read history information on the display unit 16 or the speaker 17. (S76). For example, when the history information of the dialogue processing includes the dialogue data (dialogue voice) of the voice data, the voice data is output from the speaker 17, and when the dialogue data of the text data is included, the text data is displayed on the display unit 16. .. As a result, it is possible to reproduce the interactive processing performed by the avatar of the own terminal 10 with the user or the avatar of the other terminal 20 in a pseudo manner. Further, for example, when the history information of the outing process includes image data (captured image), the image data is displayed on the display unit 16, and the outing process simulated by the avatar of the own terminal 10 can be reproduced.

By the above-described processing, in the present embodiment, the avatar of the own terminal 10 can perform a dialogue with the user of the other terminal 20 or the avatar on behalf of the user of the own terminal 10. In addition, the user of the own terminal 10 pseudo-goes to the place he / she wants to go, and the user of the own terminal 10 pseudo-executes what he / she wants to do, and provides the user of the own terminal 10 with history information indicating the execution result. be able to. Therefore, the user of the own terminal 10 can confirm the content of the avatar interacting with the user of the other terminal 20 or the avatar by playing back the history information provided by the avatar of the own terminal 10. In addition, the user of the own terminal 10 can trace the behavior simulated by the avatar by playing back the history information provided by the avatar of the own terminal 10. Therefore, the user of the own terminal 10 can select the place he / she wants to actually go to or what he / she wants to do based on the history information provided by the avatar, and can move to the actual action.

In the present embodiment, the dialogue processing units 13a and 23a of the user terminals 10 and 20 can interact with the user of the user terminals 10 and 20, and the user can interact with the user of the user terminals 10 and 20 (user terminals 10 and 20). You can relieve loneliness and anxiety by interacting with (avatar). Further, the unique information collecting units 13b and 23b of the user terminals 10 and 20 collect user-specific information of each user from the contents of dialogue with the users of the user terminals 10 and 20. Therefore, it is possible to accumulate unique information such as user attributes, behavior history, hobbies, preferences, and thinking tendencies. Then, by causing the user's avatar to perform words and actions based on such unique information, the avatar can be made to perform words and actions similar to those of the user instead of the user's words and actions. Therefore, the user's avatar can interact with other users on behalf of the user. Also, the avatar can go where the user wants to go instead, and the avatar can do what the user wants to do instead. By accumulating history information about the words and actions performed by the avatar on behalf of the user and reproducing the accumulated history information, the user can experience the words and actions by the avatar in a simulated manner. Through such simulated experiences through avatars and simulated experiences by oneself, users can gain new awareness and lead a mentally richer life, and contact points in the real world. It can be expected that the expansion of will cause actual behavioral changes.

Further, in the present embodiment, the word / action execution unit 13c makes an inquiry to, for example, an expert (expert terminal) regarding an event that may occur in the future, such as several years or several decades later, and the opinion of the expert. Alternatively, a process for acquiring advice (information related to the event) may be performed. At this time, the word / action execution unit 13c may make an inquiry via the avatar of the own terminal 10. For example, the doctor (or doctor's avatar) asks the doctor (or doctor's avatar) for information on diseases that may develop in the future based on the age, history, hospital visit history, etc. of the user of the own terminal 10. Information or advice regarding the disease may be acquired and stored in the behavior history DB 12c as history information. In addition, by inquiring about information on entrance examinations, employment examinations, etc. that will come in the future from the age of the user of the own terminal 10 to a predetermined terminal (for example, an expert terminal), or by searching for information published on the network N. , These information may be acquired and stored in the behavior history DB 12c as history information. Furthermore, from the age of the child of the user of the own terminal 10, inquire information about events such as 753, admission, graduation, admission, graduation, etc. of the child coming in the future to a predetermined terminal (for example, a specialist terminal), or network N. By searching the information published above, these information may be acquired and stored in the behavior history DB 12c as history information. With such a configuration, the user of the own terminal 10 can obtain various information by browsing the history information collected by the avatar without making an inquiry to an expert or searching for information. it can.

Also, when the avatar of the own terminal 10 interacts with another user (the user of the other terminal 20) based on the user-specific information instead of the user of the own terminal 10, whether or not the feeling matches with the other user is determined. It can be determined by the avatar of the own terminal 10. Therefore, by recommending to the user of the own terminal 10 another user whose feeling is judged to be suitable by the avatar of the own terminal 10, the user of the own terminal 10 can select another user who has a high possibility of matching the feeling. I can grasp it. It is also possible for the avatar of the own terminal 10 to do what the user of the own terminal 10 has forgotten to do based on the user-specific information instead of the user of the own terminal 10. For example, an avatar of the own terminal 10 can process an order based on user-specific information for an item that the user of the own terminal 10 has forgotten to shop. Specifically, the word / action execution unit 13c of the own terminal 10 identifies the product that the user was trying to purchase based on the user-specific information, and sets the order information of the specified product to a predetermined terminal (terminal of the product sales site). ) May be executed. Further, when the user of the own terminal 10 searches for a product to be purchased based on the user-specific information, the avatar of the own terminal 10 outputs information about this product to the speaker 17 or the display unit 16 and outputs the information about the product to the own terminal 10. You may propose to the user of. In this case, the word / action execution unit 13c of the own terminal 10 extracts the product that the user was trying to purchase based on the user-specific information, and obtains the information about the extracted product via, for example, the speaker 17 or the display unit 16. The avatar of the terminal 10 proposes to the user of the own terminal 10. After that, when the word / action execution unit 13c of the own terminal 10 receives a purchase permission instruction from the user via, for example, the microphone 18 or the input unit 15, the order information of this product is set to a predetermined terminal (terminal of the product sales site). Order processing can be performed by outputting to. Further, when the word / action execution unit 13c of the own terminal 10 searches for a product that the user of the own terminal 10 is likely to like based on the user-specific information, the word / action execution unit 13c of the own terminal 10 uses the avatar of the own terminal 10 to search for the product. It can also be recommended to users of.

In the present embodiment, it is possible to reproduce the words and actions that the user of the own terminal 10 is likely to perform based on the user-specific information stored in the unique information DB 12a. Therefore, for example, even when the user of the own terminal 10 cannot interact due to illness or the like, it is possible to predict what the user wants to say or what he / she wants to do according to the words and actions based on the user-specific information.

(Embodiment 2)
An information processing system that considers the user's state based on the user's biometric information when registering user-specific information will be described. Since the information processing system of the present embodiment is realized by the same user terminals 10 and 20 as the information processing system of the first embodiment, the description of the same configuration will be omitted. The information processing system of the present embodiment further includes a biometric information measuring instrument 30 that measures biometric information of each user of the user terminals 10 and 20.

FIG. 15 is a schematic diagram showing a configuration example of the information processing system of the second embodiment. FIG. 15 shows only the biometric information measuring device 30 that measures the biometric information of the own terminal 10 and the user of the own terminal 10, but the information processing system of the present embodiment measures the biometric information of the user of each of the other terminals 20. Includes biometric measuring instruments. In the user terminals 10 and 20 of the present embodiment, the communication units 14 and 24 are not only an interface for connecting to the network N but also an interface for communicating with the corresponding biometric information measuring instrument 30 by wired communication or wireless communication. Information is transmitted and received to and from the biometric information measuring instrument 30. The biological information measuring instrument 30 is a wristwatch-type wearable device such as a smart watch, and measures biological information such as a user's heart rate, pulse rate, and blood pressure. The biometric information measuring instrument 30 sequentially transmits the measured biometric information to the user terminals 10 and 20, and the communication units 14 and 24 of the user terminals 10 and 20 each transmit the biometric information transmitted from the communicable biometric information measuring instrument 30. Get information in sequence.

Also in the information processing system of the present embodiment, each of the user terminals 10 and 20 performs a process of collecting user-specific information of the user of each terminal. The user terminals 10 and 20 of the present embodiment consider the state of the user based on the biometric information of each user when collecting (registering) the user-specific information in the unique information DB 12a. FIG. 16 is a flowchart showing an example of a procedure for collecting and processing user-specific information by the own terminal 10. Note that FIG. 16 shows the processing performed by the own terminal 10, but the other terminal 20 also performs the same processing.

The unique information collecting unit 13b of the own terminal 10 has a dialogue with the user of the own terminal 10 by the dialogue processing unit 13a. Specifically, the unique information collecting unit 13b generates dialogue information (dialogue message) indicating the content of the dialogue with the user by the dialogue processing unit 13a, and outputs it to the speaker 17 or the display unit 16 (S81). The dialogue information may be a dialogue message stored in the storage unit 12 in advance, or may be a response message according to the dialogue content from the user acquired via the microphone 18 or the input unit 15. The unique information collecting unit 13b acquires the response information (response message) that the user replied to the output dialogue information with the microphone 18 or the input unit 15 (S82).

The control unit 11 of the own terminal 10 acquires the user's biometric information transmitted from the biometric information measuring instrument 30 while interacting with the user by the unique information collecting unit 13b (S83). The control unit 11 detects the psychological state of the user based on the biometric information acquired from the biometric information measuring instrument 30 (S84). For example, the control unit 11 detects the state of the user's autonomic nerve from the heart rate and pulse rate measured by the biological information measuring device 30. As is generally known, autonomic nerves include sympathetic nerves that become active during activity and parasympathetic nerves that become active at night or at rest, and the user can adjust the balance between the sympathetic nerve and the parasympathetic nerve. Detects whether you are in a relaxed state or a tense / anxious state.

Specifically, the control unit 11 performs spectral analysis on the measurement results (to be exact, electrocardiographic waveform and pulse wave) of the user's heart rate and pulse rate, and obtains a low frequency band component (LF; Low Frequency). The high frequency band component (HF; High Frequency) is extracted, and the LF / HF value indicating the ratio between the two is calculated. The low-frequency band component mainly contains a sympathetic nerve component, the high-frequency band component mainly contains a parasympathetic nerve component, and the control unit 11 calculates the LF / HF value to balance the sympathetic nerve and the parasympathetic nerve. Can be obtained. The control unit 11 determines the psychological state of the user based on the calculated LF / HF value. For example, the control unit 11 determines that the user is in a tense / anxious state if the LF / HF value is equal to or higher than a predetermined value, and determines that the user is in a relaxed state if the LF / HF value is less than the predetermined value. You may. The control unit 11 may calculate a numerical value other than the LF / HF value and detect the user's state based on the calculation result.

The control unit 11 determines whether or not the detected user's psychological state is in the relaxed state (S85), and if it is determined to be in the relaxed state (S85: YES), the unique information collecting unit 13b determines in step S82. User-specific information is extracted from the acquired response information (S86). Then, the unique information collecting unit 13b stores the extracted user-specific information in the unique information DB 12a in association with the information ID and the information content (S87). As a result, the user-specific information extracted from the content of the dialogue with the user when the user is in the relaxed state is registered in the unique information DB 12a. On the other hand, when it is determined that the detected user's state is a tense / anxious state (S85: NO), the control unit 11 skips steps S86 to S87. As a result, the user-specific information extracted from the content of the dialogue with the user when the user is in a tense / anxious state is not registered in the unique information DB 12a. Therefore, the dialogue content issued by the user who is not in the relaxed state (tension / anxiety state) may not be the user's intention. Only user-specific information in the user's original state can be registered.

When the control unit 11 determines whether or not the unique information collection unit 13b (dialogue processing unit 13a) has terminated the dialogue with the user (S88) and has determined that the dialogue has not been terminated (S88: NO), The process returns to the process of step S81. Then, the unique information collecting unit 13b generates dialogue information (dialogue message) to be next interacted with the user, outputs it to the speaker 17 or the display unit 16 (S81), and repeats the processes of steps S82 to S87. When the control unit 11 determines that the unique information collection unit 13b (dialogue processing unit 13a) has finished the dialogue with the user (S88: YES), the control unit 11 ends the process. By the above-mentioned processing, when the user is in a relaxed state, the user-specific information is sequentially extracted and registered from the contents of the dialogue with the user, so that the user-specific information in the original user can be accumulated without the user being aware of it. It is possible. When the user is in a tense / anxious state, the user-specific information is not accumulated, but the user interacts with the user. Therefore, the user interacts with the own terminal 10 regardless of his / her own state. Can be done.

In the information processing system of the present embodiment, the user terminals 10 and 20 perform the same processing as the processing shown in FIGS. 7 to 8. As a result, also in this embodiment, the avatar (user's avatar) set in the own terminal 10 has a dialogue with the user or avatar of the other terminal 20 based on the user-specific information stored in the unique information DB 12a. Can be made. Further, the user terminals 10 and 20 of the present embodiment perform the same processing as the processing shown in FIGS. 9 to 10. Thereby, also in this embodiment, the user of the own terminal 10 can have a dialogue with the user or the avatar of the other terminal 20 by using the own terminal 10. Further, the user terminals 10 and 20 of the present embodiment perform the same processing as the processing shown in FIG. As a result, also in the present embodiment, the avatar (user's avatar) set in the own terminal 10 is made to perform a pseudo outing act (action) based on the user-specific information stored in the unique information DB 12a. Can be done. Further, the user terminals 10 and 20 of the present embodiment perform the same processing as the processing shown in FIG. As a result, the history of dialogue with the user or avatar of the other terminal 20 performed by the avatar of the own terminal 10, the history of going out related to the pseudo outing act performed by the avatar of the own terminal 10, and the like can be reproduced on the own terminal 10. Therefore, the user of the own terminal 10 simulates the dialogue with the user or the avatar of the other terminal 20 performed by the avatar of the own terminal 10, going out to the place where he / she wanted to go, the experience of what he / she wanted to do, and the like. Can be done.

Also in this embodiment, the same effect as that of the first embodiment can be obtained. Further, in the present embodiment, when registering user-specific information, the user's heart rate, pulse rate, etc. are measured to determine whether or not the user's mental and physical condition is relaxed and calm, and when the user is calm. Only the collected user-specific information can be stored in the unique information DB 12a. In this case, by not registering (discarding) the user-specific information collected when the mental and physical state is not calm and excited, only the user-specific information collected in the user's normal state can be accumulated.

(Embodiment 3)
An information processing system in which each of the user terminals 10 and 20 executes a word / action process using a user model constructed based on the word / action of each user will be described. The information processing system of the present embodiment is realized by the same user terminals 10 and 20 as the information processing system of the first embodiment. FIG. 17 is a block diagram showing a configuration example of the own terminal 10 of the third embodiment. Since the local terminal 10 of the present embodiment has the same configuration as the local terminal 10 of the first embodiment, the description of the same configuration will be omitted. In the own terminal 10 of the present embodiment, the user model 12M is stored in the storage unit 12 instead of the unique information DB 12a, and the processing unit 13 does not have the unique information collecting unit 13b.

The user model 12M is, for example, a deep learning model trained by using a deep learning algorithm, and is constructed by, for example, a Seq2Seq (Sequence to Sequence) model using an RNN (Recurrent Neural Network). The user model 12M (first user model) uses teacher data including presentation data presented to the user (first user) of the own terminal 10 and response data given by the user to the presentation data. This is a trained model trained to output the answer data corresponding to the input presentation data when the presentation data is input. The presented data is, for example, dialogue information for the user of the own terminal 10, and the response data is the response information that the user replied in response to the dialogue information. The presented data includes, for example, dialogue information presented to the user via the speaker 17 or the display unit 16, and the response data includes the response information input by the user to the dialogue information via the microphone 18 or the input unit 15. Further, the presented data may include dialogue information received from the other terminal 20 by the user of the own terminal 10 via an e-mail, a LINE message, a LINE stamp, an SNS, etc. The reply information replied to 20 may be included. Further, the presented data may include, for example, information on an action performed on the user of the own terminal 10, and the response data may include information on an action performed by the user on this action. For example, when the user of the own terminal 10 receives the LINE message, it is assumed that a predetermined LINE stamp is returned. In this case, the reception of the LINE message can be used as the presentation data, and the reply of the predetermined LINE stamp can be used as the reply data. The trained model performs a predetermined operation on the input value and outputs the operation result, and data such as coefficients and thresholds of the function that defines this operation is stored in the storage unit 12 as the user model 12M. To.

The user model 12M may be trained in the own terminal 10 or may be stored in the storage unit 12 of the own terminal 10 after being learned in another learning device. The user model 12M learns so that when the presentation data included in the teacher data is input, the answer data included in the teacher data is output. In the learning process, the user model 12M optimizes data such as coefficients and thresholds of functions that define predetermined operations performed on input data. As a result, the user model 12M learned to output the response data (response information) that is likely to be performed by the user of the own terminal 10 is obtained based on the input dialogue information (presentation data).

The dialogue processing unit 13a of the present embodiment uses the user model 12M to perform a dialogue with the user of the own terminal 10 and a dialogue with the user or the avatar of the other terminal 20. Specifically, when the dialogue processing unit 13a acquires the dialogue information (presentation data) for the avatar (user model 12M) of the own terminal 10, the acquired dialogue information is input to the user model 12M and output from the user model 12M. The response information (response data) to the dialogue information to be performed is acquired. The dialogue information is acquired from the user of the own terminal 10 via, for example, the microphone 18 or the input unit 15 in the dialogue with the user of the own terminal 10, and is communicated in the dialogue with the user or the avatar of the other terminal 20. Obtained from another terminal 20 (user or avatar of another terminal 20) via unit 14.

By specifying the response information for the dialogue information using the user model 12M, the dialogue processing unit 13a can identify the response information (response data) that the user of the own terminal 10 is likely to respond to. Then, when the dialogue processing unit 13a interacts with the user of the own terminal 10, the dialogue processing unit 13a responds to the user of the own terminal 10 by outputting the specified reply information on the speaker 17 or the display unit 16. Further, when the dialogue processing unit 13a interacts with the user or avatar of the other terminal 20, the dialogue processing unit 13a replies to the user or avatar of the other terminal 20 by returning the specified reply information from the communication unit 14 to the other terminal 20. As described above, when the dialogue processing unit 13a of the present embodiment acquires the dialogue information for the avatar of the own terminal 10, the response information for the dialogue information is specified by using the user model 12M, so that the user of the own terminal 10 Can make a reply that is likely to be made by. The user model 12M is not limited to the configuration constructed by using the Seq2Seq model.

Also in this embodiment, the other terminal 20 has the same configuration as the own terminal 10. Therefore, in the other terminal 20, the teacher data including the presentation data presented to the user (second user) of the other terminal 20 and the response data given by the user of the other terminal 20 to the presentation data is used. The user model 22M (second user model), which is a trained model trained to output the answer data corresponding to the input presentation data when the presentation data is input, is stored in the storage unit 22. Has been done.

FIG. 18 is a flowchart showing an example of a procedure for word-and-action processing by the own terminal 10 and the other terminal 20 of the third embodiment. The process shown in FIG. 18 is a process in which step S91 is added before step S27 and step S92 is added before step S28 in the processes shown in FIGS. 7 and 8. In FIG. 18, illustrations of steps S21 to S26 shown in FIG. 7 are omitted, and description of the same steps as in FIGS. 7 and 8 will be omitted. It is assumed that the user terminals 10 and 20 store the user models 12M and 22M learned by using the teacher data based on the respective users in the storage units 12 and 22.

In the present embodiment, in step S21, the word / action execution unit 13c of the own terminal 10 receives a dialogue execution command (user model 12M) from the user of the own terminal 10 via the microphone 18 or the input unit 15 (user model 12M). Execution request) is accepted. The dialogue execution command here is a dialogue execution command with the user or avatar of any other terminal 20. Further, in the present embodiment, when the word / action execution unit (setting unit) 23c of the other terminal 20 determines that the dialogue requested by the own terminal 10 is possible (S24: YES), the connection with the own terminal 10 is established. Set (S91). As a result, the word / action execution unit 23c of the other terminal 20 sets a connection between the user or avatar (user model 12M) of the own terminal 10 that has requested the dialogue and the user or avatar (user model 22M) of the other terminal 20. .. Then, the word / action execution unit 23c of the other terminal 20 notifies the own terminal 10 that the user or avatar of the other terminal 20 requested to have a dialogue can have a dialogue (S27). On the other hand, the word / action execution unit (setting unit) 13c of the own terminal 10 sets a connection with the other terminal 20 after being notified by the other terminal 20 that dialogue is possible (S92). As a result, the word / action execution unit 13c of the own terminal 10 sets a connection between the user or avatar of the own terminal 10 (user model 12M) who requested the dialogue and the user or avatar of the other terminal 20 (user model 22M). Then, the word / action execution unit 13c of the own terminal 10 generates dialogue information based on the dialogue content requested via the microphone 18 or the input unit 15 (S28). The word / action execution unit 13c (dialogue processing unit 13a) inputs a message such as "what do you want to talk about" into the user model 12M as presentation data, and the response information (answer) output from the user model 12M. The data) may be used as dialogue information to be transmitted to the user or avatar of the other terminal 20.

After that, the words and actions execution units 13c and 23c of the own terminal 10 and the other terminal 20 perform the processing after step S29. The word / action execution unit 23c of the other terminal 20 operates as an acquisition unit for acquiring dialogue information (presentation data presented from the own terminal 10) transmitted from the own terminal 10, and the word / action execution unit 13c of the own terminal 10 operates. , Operates as an acquisition unit that acquires dialogue information (presentation data presented by the other terminal 20) transmitted from the other terminal 20. Further, in the present embodiment, in step S34, the word / action execution unit 23c (dialogue processing unit 23a) of the other terminal 20 uses the user model 22M to provide response information (response message) corresponding to the dialogue information received from the own terminal 10. To generate. Specifically, the dialogue processing unit 23a inputs the dialogue information received from the own terminal 10 into the user model 22M as presentation data for the avatar of the other terminal 20, and the response information (answer data) output from the user model 22M. Is acquired, and the acquired response information is used as the response information to be returned to the own terminal 10. Further, in step S28, the word / action execution unit 13c (dialogue processing unit 13a) of the own terminal 10 uses the user model 12M to generate response information corresponding to the dialogue information (response information) received from the other terminal 20. Here, too, the dialogue processing unit 13a of the own terminal 10 inputs the dialogue information received from the other terminal 20 into the user model 12M of the own terminal 10 as presentation data for the avatar of the own terminal 10, and outputs the dialogue information from the user model 12M. The response information is acquired, and the acquired response information is used as the response information to be returned to the other terminal 20.

By the above-described processing, also in the present embodiment, the user of the own terminal 10 can interact with the user or the avatar of the other terminal 20 by the own avatar set in the own terminal 10. Further, in the present embodiment, using the user models 12M and 22M learned for each of the user terminals 10 and 20, the dialogue information and the dialogue information that the avatar of the own terminal 10 exchanges with the user or the avatar of the other terminal 20 Response information, or dialogue information and response information that the avatar of the other terminal 20 exchanges with the avatar of the own terminal 10 is generated. Therefore, the dialogue that is likely to be performed by the users of the user terminals 10 and 20 can be reproduced by the avatars of the user terminals 10 and 20.

The user terminals 10 and 20 of the present embodiment perform the same processing as the behavior processing shown in FIGS. 9 and 10. In the present embodiment, when the word / action execution unit 23c of the other terminal 20 determines in step S47 that the dialogue requested by the own terminal 10 is possible (S44: YES), the connection with the own terminal 10 is set. After that, the user or avatar of the other terminal 20 requested to have a dialogue is notified to the own terminal 10 that the dialogue is possible. Further, in step S48, the word / action execution unit 13c of the own terminal 10 sets a connection with the other terminal 20 after being notified by the other terminal 20 that dialogue is possible, and then receives dialogue information to the other terminal 20. Further, in the present embodiment, in step S55, the word / action execution unit 23c (dialogue processing unit 23a) of the other terminal 20 uses the user model 22M to provide response information (response message) corresponding to the dialogue information received from the own terminal 10. Generate. Specifically, the dialogue processing unit 23a inputs the dialogue information received from the own terminal 10 into the user model 22M as presentation data for the avatar of the other terminal 20, and the response information (answer data) output from the user model 22M. Is acquired, and the acquired response information is used as the response information to be returned to the own terminal 10. By such processing, even in this embodiment, the user of the own terminal 10 can interact with the user of the other terminal 20 or the avatar. In the present embodiment, the user model 22M learned for each of the other terminals 20 is used to generate dialogue information and response information that the avatar of the other terminal 20 exchanges with the user of the own terminal 10. Therefore, the dialogue that is likely to be performed by the user of the other terminal 20 can be reproduced on the avatar of the other terminal 20, and the user of the own terminal 10 simulates the dialogue with the user of the other terminal 20 via the avatar of the other terminal 20. Can be done

The user terminals 10 and 20 of the present embodiment perform the same processing as the behavior processing shown in FIG. In the present embodiment, the control unit 11 of the own terminal 10 executes a pseudo outing act based on the user model 12M by the word / action execution unit 13c via the avatar of the own terminal 10. Specifically, in step S61, the word / action execution unit 13c of the own terminal 10 uses the user model 22M to determine whether or not there is a place where the user of the own terminal 10 wants to go. For example, the word / action execution unit 13c inputs the “desired place” as presentation data into the user model 12M, and acquires the information (answer data) regarding the “desired place” output from the user model 12M. The word / action execution unit 13c determines whether or not there is a place where the user of the own terminal 10 wants to go, depending on whether or not the information about the "place to go" is output from the user model 12M. .. Further, in step S65, the word / action execution unit 13c of the own terminal 10 uses the user model 22M to determine whether or not the user of the own terminal 10 has something he / she wants to do. For example, the word / action execution unit 13c inputs "what you want to do" into the user model 12M as presentation data, and acquires information (answer data) about "what you want to do" output from the user model 12M. The word / action execution unit 13c determines whether or not the user of the own terminal 10 has something he / she wants to do, depending on whether or not the information about “what he / she wants to do” is output from the user model 12M. By the above-described processing, the own terminal 10 of the present embodiment uses the user model 12M to extract the place or what the user of the own terminal 10 wants to go to, and acts on behalf of the user, and is obtained as a result of the action. Information can be stored as history information.

The user terminals 10 and 20 of the present embodiment perform the same processing as the reproduction processing of the behavior history shown in FIG. As a result, the user terminals 10 and 20 display a history screen as shown in FIG. 14, and the dialogue processing and the outing processing performed by the word / action execution units 13c and 23c of the user terminals 10 and 20 via the history screen. Accepts playback instructions. Therefore, it is possible to reproduce the dialogue processing performed by the avatar of the own terminal 10 with the user or the avatar of the other terminal 20 on behalf of the user of the own terminal 10, and the user of the own terminal 10 is the user of the other terminal 20 or You can experience the interaction with the avatar in a simulated manner. In addition, it is possible to reproduce the outing process simulated by the avatar of the own terminal 10, and the user of the own terminal 10 can trace the action executed by the avatar.

Also in this embodiment, the same effect as that of the first embodiment can be obtained. Further, in the present embodiment, it is possible to specify the dialogue content and the action content that each user is likely to perform based on the user models 12M and 22M learned for each of the user terminals 10 and 20. Therefore, by realizing the dialogue of the dialogue content and the action of the action content that are likely to be performed by each user, it is possible to realize the pseudo dialogue and action by the avatar of each user.

In the present embodiment, the own terminal 10 stores the user model 12M of the user of the own terminal 10, the other terminal 20 stores the user model 22M of the user of the other terminal 20, and the word / action execution unit 13c of each of the terminals 10 and 20. , 23c is not limited to the configuration in which the word / action processing based on the respective user models 12M and 22M is performed via the avatars of the respective terminals 10 and 20. For example, the own terminal 10 may store not only the user model 12M of the own terminal 10 but also the user model 22M of the other terminal 20 by acquiring the user model 22M of the user of the other terminal 20 from the other terminal 20. .. According to such a configuration, the own terminal 10 performs the word / action processing based on the user model 12M of the own terminal 10 via the avatar of the own terminal 10, and performs the word / action processing based on the user model 22M of the other terminal 20. It can be done via 20 avatars. Therefore, in the own terminal 10, the dialogue process between the avatar of the own terminal 10 and the avatar of the other terminal 20 can be executed. In this case, the user of the other terminal 20 can confirm the dialogue content at a later date by transmitting the history information of the dialogue processing executed between the avatar of the own terminal 10 and the avatar of the other terminal 20 to the other terminal 20. .. On the contrary, the other terminal 20 acquires the user model 12M of the own terminal 10 from the own terminal 10, and executes the dialogue processing between the avatar of the own terminal 10 and the avatar of the other terminal 20 on the other terminal 20. May be.

In the present embodiment, the user models 12M and 22M include presentation data presented to each user of the user terminals 10 and 20, response data given by the user to the presentation data, and execution data of the executed application program. It may be a trained model trained using teacher data including and. In this case, when the presentation data is input, the user model outputs the response data corresponding to the input presentation data and the execution data of the application program. The presented data here is, for example, action information related to an action performed by the user of the own terminal 10 using the own terminal 10, and the response data is process information related to the process (action) performed by the user in response to this action information. , Execution data can be information on the application program executed (started) by the user in response to this action information.

For example, when a user of the own terminal 10 acquires information on a conference schedule via e-mail, a LINE message, an SNS, or the like, a reservation application for reserving the use of the conference room using the own terminal 10 thereafter. (Application program) is executed to reserve a conference room. In this case, the action information indicating that the information on the conference schedule has been acquired is used as the presentation data, and the processing information indicating that the conference room is reserved using the reservation application is used as the response data to reserve the use of the conference room. The reservation application of can be used as execution data. Further, when the user of the own terminal 10 makes a travel reservation through the travel reservation site on the network N, then the restaurant reservation or the optional through the restaurant reservation site is made using the own terminal 10. Suppose you have applied for an optional tour through the tour reservation site. In this case, the action information indicating that the travel reservation has been made through the travel reservation site is used as the presentation data, and the processing information indicating that the restaurant is reserved or the optional tour is applied for through the reservation site is used as the response data. , The browser for accessing the restaurant reservation site or the optional tour reservation site can be used as the execution data. By training the user model 12M using teacher data including such presentation data, response data, and execution data, a user who predicts the next action (process) that is likely to be performed after the user has performed a certain action. Model 12M can be generated.

As described above, information on actions performed by the user of the own terminal 10 in the past using the own terminal 10 is used as presentation data, and information on actions (processes) further performed by the user of the own terminal 10 in response to this action. Is used as the answer data, and the user model 12M is trained using the teacher data using the information of the application program executed by the user as the execution data. The user model 12M learns so that when the presentation data included in the teacher data is input, the answer data and the execution data included in the teacher data are output. As a result, it was learned to output processing information (answer data) and application program information (execution data) that are likely to be performed by the user of the own terminal 10 based on the input action information (presentation data). A user model 12M is obtained. By using such a user model 12M, the word / action execution unit 13c of the own terminal 10 predicts an action (process) that is likely to be performed next when the user of the own terminal 10 performs a certain action. By executing the corresponding application program, the operational burden on the user of the own terminal 10 can be reduced. The word / action execution unit 13c stores the execution result of the application program executed based on the user model 12M in the word / action history DB, and displays the accumulated execution result on the speaker 17 or in response to a request from the user of the own terminal 10. It can be reproduced via the part 16. As a result, the user of the own terminal 10 can later confirm the action (process) performed by the avatar (word / action execution unit 13c) of the own terminal 10 based on the user model 12M.

(Embodiment 4)
An information processing system in which a user model learned for each user of each of the user terminals 10 and 20 is stored in a server and word-action processing based on each user model is executed in the server will be described. The information processing system of the present embodiment includes a server 40 in addition to the user terminals 10 and 20 in the information processing system of the third embodiment. FIG. 19 is a block diagram showing a configuration example of the server 40. The server 40 includes a control unit 41, a storage unit 42, a processing unit 43, a communication unit 44, an input unit 45, a display unit 46, a reading unit 47, and the like, and each of these units is connected to each other via a bus. Since each of the constituent units 41 to 47 of the server 40 has the same configuration as the respective constituent units 11 to 16 and 19 of the own terminal 10, the details thereof will be omitted.

The storage unit 42 of the server 40 has a plurality of user models 42M (three first user models 42M1, second user model 42M2, and third user model 42M3 in FIG. 19) provided corresponding to the user terminals 10 and 20 respectively. ) And the behavior history DB42a are stored. The behavior history DB 42a may be provided corresponding to each of the user terminals 10 and 20, for example, and in this case, has the same configuration as that shown in FIG. 4B. Further, the behavior history DB 42a may have a configuration having a user ID string in addition to the configuration shown in FIG. 4B. In this case, the behavior history DB 42a corresponds to each user in association with the user ID assigned to each of the user terminals 10 and 20. History information is accumulated.

Further, the processing unit 43 of the server 40 has an dialogue processing unit 43a and a word / action execution unit 43b. The dialogue processing unit 43a and the word / action execution unit 43b perform the same processing as the dialogue processing unit 13a and the word / action execution unit 13c in the own terminal 10 of the third embodiment. The word / action execution unit 43b of the server 40 uses the dialogue processing unit 43a to perform a dialogue between a plurality of avatars (avatars of the plurality of user terminals 10 and 20) based on the plurality of user models 42M. FIG. 20 is a flowchart showing an example of a procedure for processing words and actions by the server 40. The following processing is executed by the control unit 41 according to the control program 42P stored in the storage unit 42 of the server 40.

The word / action execution unit 43b of the server 40 determines whether or not the avatar of each corresponding user requests a dialogue with another avatar based on each user model 42M (S101). For example, the word / action execution unit 43b inputs a message such as "Is there a person you want to talk to?" As presentation data for each user model 42M, and the user model 42M provides information (answer data) about the person you want to talk to. Whether or not one avatar requests a dialogue with another avatar is determined depending on whether or not it is output. For example, when answer data such as "I have a partner I want to talk to" or "I want to talk to Mr. XX" is output from the user model 42M, the word / action execution unit 43b is from the avatar corresponding to this user model 42M. Accepts dialogue requests with other avatars.

When the word / action execution unit 43b determines that the dialogue request is not accepted (S101: NO), it waits until the dialogue request is accepted. When the word / action execution unit 43b determines that the dialogue request has been accepted (S101: YES), the word / action execution unit 43b identifies the user model 42M (first dialogue partner) who has requested the dialogue (S102), and the first dialogue partner wants to have a dialogue with the second dialogue. Identify the other party (S103). Here, the word / action execution unit 43b specifies the user model 42M that outputs the response data regarding the partner to be interacted with as the first dialogue partner, and the dialogue partner designated by the response data output from the user model 42M of the first dialogue partner. Is specified as the second dialogue partner.

Next, the word / action execution unit 43b generates dialogue information from the specified first dialogue partner to the second dialogue partner (S104). The word / action execution unit 43b inputs a message such as "what do you want to talk about" as presentation data into the user model 42M of the first dialogue partner, and the response information (answer data) output from the user model 12M. Based on, the dialogue information from the first dialogue partner is generated. The word / action execution unit 43b generates response information from the second dialogue partner to the generated dialogue information (S105). Here, the word / action execution unit 43b inputs the generated dialogue information from the first dialogue partner as presentation data into the user model 42M of the second dialogue partner, and uses the response information (answer data) output from the user model 42M. Based on this, the response information from the second dialogue partner to the first dialogue partner is generated. The word-and-action execution unit 43b uses the first dialogue partner and the first dialogue partner to provide the dialogue information from the first dialogue partner to the second dialogue partner and the response information from the second dialogue partner to the first dialogue partner generated in steps S104 and S105. 2. As history information in the dialogue processing with the dialogue partner, for example, it is stored in the behavior history DB 42a in association with the user (user ID) of the first dialogue partner and the second dialogue partner (S106).

The word / action execution unit 43b determines whether or not the dialogue between the first dialogue partner and the second dialogue partner has ended (S107), and if it determines that the dialogue has not ended (S107: NO), the step. Returning to the process of S104, the response information from the first dialogue partner to the dialogue information (response information) from the second dialogue partner generated in step S105 is generated (S104). Here, the word / action execution unit 43b inputs the dialogue information (response information) from the second dialogue partner into the user model 42M of the first dialogue partner as presentation data, and the response information (response data) output from the user model 42M. ), The response information from the first dialogue partner to the second dialogue partner is generated. The word / action execution unit 43b repeats the processes of steps S104 to S106 until the dialogue between the first dialogue partner and the second dialogue partner is completed. That is, the dialogue information output from the user model 42M of the first dialogue partner is input to the user model 42M of the second dialogue partner, and the dialogue information output from the user model 42M of the second dialogue partner is input to the user of the first dialogue partner. Input to model 42M. As a result, the user model 42M of the first dialogue partner and the user model 42M of the second dialogue partner can have a dialogue, and the result of the dialogue can be stored in the behavior history DB 42a.

When it is determined that the dialogue between the first dialogue partner and the second dialogue partner has been completed (S107: YES), the word / action execution unit 43b ends the above-described processing. The word / action execution unit 43b determines the end of the dialogue based on the response data (response information) output from the user model 42M of the first dialogue partner or the user model 42M of the second dialogue partner. The dialogue may be terminated when a round trip of the dialogue is made.

In the present embodiment, the word / action execution unit 43b of the server 40 performs the same processing as the word / action processing shown in FIG. 12 based on the respective user models 42M, similarly to the word / action execution unit 13c of the own terminal 10 of the third embodiment. It can be carried out. That is, the word / action execution unit 43b can specify the place where the user corresponding to each user model 42M wants to go or what he / she wants to do, and make each avatar perform a pseudo outing action. Specifically, in step S61, the word / action execution unit 43b of the server 40 uses each user model 42M to determine whether or not there is a place where each user wants to go. For example, the word / action execution unit 43b inputs the “desired place” as presentation data into the user model 42M, and acquires the information (answer data) regarding the “desired place” output from the user model 42M, thereby this user model. Acquire the place where the user corresponding to 42M wants to go. Further, in step S65, the word / action execution unit 43b of the server 40 uses each user model 42M to determine whether or not each user has something he / she wants to do. For example, the word / action execution unit 43b inputs "what he wants to do" into the user model 42M as presentation data, and acquires information (answer data) about "what he wants to do" output from the user model 42M to the user model 42M. Get what the corresponding user wants to do. By the above-mentioned processing, the server 40 can extract the place or what each user wants to do based on each user model 42M. Further, the word / action execution unit 43b of the server 40 searches and collects various information about the place or what each user wants to go to via the network N. Then, the word / action execution unit 43b stores the collected information in the word / action history DB 42a in association with each user (user ID) corresponding to the user model 42M. As a result, the avatar of each user can accumulate the result of the pseudo action performed on behalf of the user as history information.

According to the above-described configuration, in the present embodiment, the server 40 performs interactive processing between avatars on the respective user terminals 10 and 20, performs various actions by each avatar, and accumulates the results of the actions as history information. .. Then, the server 40 provides the user terminals 10 and 20 with the history information accumulated in association with each user in response to the request from the user terminals 10 and 20. Therefore, each user of the user terminals 10 and 20 can grasp the dialogue processing and various actions performed by his / her avatar on the server 40. Therefore, also in the present embodiment, the users of the respective user terminals 10 and 20 can experience the dialogue that their avatar has performed with other avatars on their behalf, and also themselves. You can follow the outing process that the avatar of.

In the present embodiment, instead of storing the user model 42M of each user in the storage unit 42 of the server 40, the unique information DB 12a of each user in the first embodiment is stored, and the words and actions based on the unique information DB 12a of each user are stored. It may be configured to perform processing.

In the above-described embodiment 1-2, when the user or avatar of the own terminal 10 and the user or avatar of the other terminal 20 interact with each other, the dialogue information is transmitted and received between the own terminal 10 and the other terminal 20. However, it is not limited to such a configuration. For example, a server that relays communication between the own terminal 10 and the other terminal 20 may be provided, and dialogue information may be transmitted / received between the own terminal 10 and the other terminal 20 via the server.

The embodiments disclosed this time should be considered as exemplary in all respects and not restrictive. The scope of the present invention is indicated by the scope of claims, not the above-mentioned meaning, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

10,20 User terminal (information processing device)
11,21 Control unit 12,22 Storage unit 13,23 Processing unit 15,25 Input unit 16,26 Display unit 17,27 Speaker 18,28 Microphone 40 Server 41 Control unit 42 Storage unit 43 Processing unit 12a, 22a Unique information DB
12c, 22c Word and action history DB
13a, 23a Dialogue processing unit 13b, 23b Unique information collection unit 13c, 23c Word / action execution unit 13d, 23d History output unit 12M, 42M User model 42a Word / behavior history DB
43a Dialogue processing unit 43b Word / action execution unit

Claims (12)

Collects user-specific information based on what the user says and does,
The words and actions based on the collected user-specific information are executed via the avatar set corresponding to the user.
A program that causes a computer to execute a process that outputs the result of the avatar executing words and actions based on the user-specific information. The execution result of the words and actions based on the user-specific information executed via the avatar corresponding to the user is stored in the storage unit.
The program according to claim 1, wherein the computer executes a process of outputting a stored execution result of the words and actions in response to a request from the user. Images related to other avatars set for other users are displayed on the display.
Accepting the content of dialogue with the other avatars,
The received dialogue content is output to the terminal related to the other avatar, and
Obtain the dialogue content from the other avatar from the terminal related to the other avatar, and
The program according to claim 1 or 2, wherein the computer executes a process of outputting the acquired dialogue content from the other avatar. Accepts requests to execute dialogues with other users or other avatars corresponding to other users,
A dialogue with the other user or the other avatar that has received the execution request is executed via the avatar corresponding to the user based on the user-specific information.
The program according to any one of claims 1 to 3, which causes the computer to execute a process of outputting an execution result. Accepting requests to perform dialogues with a given expert or an expert avatar corresponding to a given expert,
A dialogue with the predetermined expert or the expert avatar that has received the execution request is executed via the avatar corresponding to the user based on the user-specific information.
The program according to any one of claims 1 to 4, which causes the computer to execute a process of outputting an execution result. Acquisition of information related to an event that may occur in the future is executed via the avatar corresponding to the user based on the user-specific information.
The program according to any one of claims 1 to 5, which causes the computer to execute a process of outputting information related to the event acquired via an avatar corresponding to the user. Acquire the biometric information of the user and
One of claims 1 to 6 for causing the computer to execute a process of collecting the user-specific information when the user's psychological state determined based on the acquired biometric information is in a relaxed state. The program described. The program according to any one of claims 1 to 7, which collects user-specific information based on the utterance content of the user. Any of claims 1 to 8 for causing the computer to execute a process of outputting product order information to a predetermined terminal based on the collected user-specific information via an avatar corresponding to the user. The program described in one. Based on the collected user-specific information, the product to be purchased is extracted via the avatar corresponding to the user.
Information about the product extracted via the avatar corresponding to the user is output.
Any of claims 1 to 9 for causing the computer to execute a process of outputting order information of the product to a predetermined terminal after receiving a purchase permission for the product via an avatar corresponding to the user. The program described in one. Information related to the execution result of words and actions executed based on the user-specific information is displayed on the display unit.
Based on the displayed information, accept the output request for one of the execution results,
The program according to any one of claims 1 to 10, which causes the computer to execute a process of outputting the received execution result. An information processing device that executes the program according to any one of claims 1 to 11.

Images