JP7360855B2 - Information processing method, program and information processing device - Google Patents

Description

The present disclosure relates to an information processing method, a program, and an information processing device.

In recent years, the number of elderly people living alone and living alone in facilities has been increasing, and some of these elderly people feel isolated because they have few opportunities for conversation in their daily lives. Therefore, for example, Patent Document 1 proposes a stuffed toy that can have a simulated conversation by outputting a pre-stored voice when the voice uttered by the user is acquired. In the technology disclosed in Patent Document 1, the user can have a simulated conversation with the stuffed animal while hugging the stuffed animal, which can provide comfort to the user and alleviate the feeling of isolation.

Japanese Patent Application Publication No. 2018-117821

People who live alone, not just the elderly, sometimes feel lonely or anxious because they have no one to talk to when they are alone at home, for example. Furthermore, even if a person lives with family or cohabitants, they may feel lonely or anxious if they are alone in their room. However, the technique disclosed in Patent Document 1 is targeted at elderly people, and it is difficult to provide comfort to all people who feel lonely or anxious.

The present disclosure has been made in view of such circumstances, and its purpose is to provide an information processing method and the like that can alleviate a user's sense of loneliness or anxiety.

An information processing method according to the present disclosure stores a first learning model learned based on interaction history information of a first user in a storage unit, and specifies an information processing device capable of deploying the first learning model. and deploys the first learning model to the specified information processing device, and the information processing device to which the first learning model is deployed is configured to perform operations based on the deployed first learning model. If a second learning model learned based on interaction history information of a second user different from the first user is deployed to the information processing device, the first learning model by the information processing device Operations based on the learning model are prohibited .

In the present disclosure, a first learning model learned based on interaction history information of a first user is deployed to a designated information processing device. This enables the information processing device to execute processing based on the first learning model. For example, the information processing device can interact with the user of the information processing device based on the first learning model. Specifically, interaction information from a user of the information processing device is input into a first learning model, response information is generated based on output information from the first learning model, and response information is output to the user of the information processing device. . Therefore, the user of the information processing device can interact with the information processing device, and the user's feelings of loneliness or anxiety can be alleviated. Furthermore, if the first learning model is deployed to an information processing device of a user different from the first user, the user of this information processing device (a user different from the first user) in the deployed information processing device You can have a dialogue between them. Note that when the results of a dialogue performed on an information processing device of a user different from the first user are fed back to the first user, the first user can experience a simulated dialogue that the first user did not perform. Therefore, through simulated experiences through their own learning models, users will be able to gain new awareness and lead a mentally richer life, and will be able to make changes in their behavior by expanding their contact with reality. By causing the user to feel connected to society, it is possible to alleviate the user's feelings of loneliness or anxiety.

1 is a schematic diagram showing a configuration example of an information processing system. FIG. 2 is a block diagram showing a configuration example of a user terminal. FIG. 7 is a block diagram showing a modification of the user model. FIG. 2 is a schematic diagram showing a configuration example of a DB stored in a user terminal. 3 is a flowchart illustrating an example of an interaction processing procedure by a user terminal. It is a schematic diagram which shows the example of a screen in a user terminal. It is a flowchart which shows an example of the action processing procedure by a user terminal. 3 is a flowchart illustrating an example of an interaction processing procedure by a user terminal. 3 is a flowchart illustrating an example of an interaction processing procedure by a user terminal. It is a schematic diagram which shows the example of a screen in a user terminal. 3 is a flowchart illustrating an example of an interaction processing procedure by a user terminal. It is a flowchart which shows an example of the action processing procedure by a user terminal. It is a flowchart which shows an example of the action processing procedure by a user terminal. 3 is a flowchart illustrating an example of a history information reproduction processing procedure. It is a schematic diagram which shows the example of a screen in a user terminal. FIG. 2 is a block diagram illustrating a configuration example of an information processing system according to a second embodiment. FIG. 2 is a schematic diagram showing a configuration example of a model information DB. It is a flowchart which shows an example of the lending process procedure of a lending model. It is a schematic diagram which shows the example of a screen in a user terminal. FIG. 3 is a block diagram illustrating a configuration example of an information processing system according to a third embodiment. It is a block diagram showing an example of a composition of a rental terminal. FIG. 2 is a schematic diagram showing a configuration example of a terminal management server. FIG. 2 is a schematic diagram showing a configuration example of a DB stored in a terminal management server. 12 is a flowchart illustrating an example of a reservation processing procedure for renting out a rental terminal. It is a schematic diagram which shows the example of a screen in a user terminal. It is a flowchart which shows an example of the lending process procedure of a lending terminal. It is a flowchart which shows an example of the lending process procedure of a lending terminal. It is a schematic diagram which shows the modification of the structure of a user terminal and a rental terminal.

Below, an information processing method, an information processing device, a program, and an information processing system of the present disclosure will be described in detail based on drawings showing embodiments thereof.

(Embodiment 1)
FIG. 1 is a schematic diagram showing an example of the configuration of an information processing system. The information processing system of this embodiment includes a large number of user terminals 10 (information processing devices) owned by a large number of users, and each of the user terminals 10 mutually sends and receives information via a network N such as the Internet. conduct. For example, the first user owns the first user terminal 10 at his home, and the second user owns the second user terminal 10 at his home, but the first user terminal 10 and the second user terminal The installation location of 10 is not limited to the home. Note that the first user terminal 10 and the second user terminal 10 have similar configurations, and will be collectively described as the user terminal 10. The user terminal 10 is, for example, a dedicated terminal used in an information processing system, and is configured using, for example, a personal computer. Note that the external appearance of the user terminal 10 can be formed into various shapes, such as a housing type like the first user's user terminal 10 shown in FIG. 1, or a housing type like the second user's user terminal 10 shown in FIG. It may be formed into a robot type or the like.

The first user's user terminal 10 shown in FIG. 1 is configured such that a mirror 10a is rotatably attached to one surface (the top surface in FIG. 1) of a rectangular prism-shaped housing 10b, for example, by a pivot (not shown). There is. The pivot axis is an axis along one side of the top surface of the casing 10b, and the mirror 10a is moved between a position covering the top surface of the casing 10b (closed position) and a position rising from the top surface of the casing 10b (open position). Rotatably supported. FIG. 1 shows the user terminal 10 with the mirror 10a in the open position. The mirror 10a is attached so that its outer surface (upper surface) becomes a reflective surface when it is in the closed position, and a display section 16 is built into the mirror 10a. The user terminal 10 having such a configuration is configured so that the reflection surface of the mirror 10a and the display surface of the display section 16 can be switched. For example, when the display section 16 is not operating, the reflection surface of the mirror 10a is switched. The surface can be used as a mirror. Further, in the user terminal 10 of the first user shown in FIG. 1, for example, a camera 19 is provided at a predetermined location on the mirror 10a, and a speaker 17 and a microphone 18 are provided at appropriate locations on the housing 10b. In the example shown in FIG. 1, a camera 19 is provided at the top end of the mirror 10a when the mirror 10a is in the open position, and a speaker 17 and a microphone 18 are provided on the top surface of the housing 10b. The arrangement position is not limited to this example. For example, the camera 19 only needs to be installed at a position where it can photograph the user who is in front of the mirror 10a when the mirror 10a is in the open position, and the speaker 17 can send output audio to users near the user terminal 10. The microphone 18 may be provided at a position where it can collect sounds emitted by users near the user terminal 10. Furthermore, the first user's user terminal 10 shown in FIG. It is configured to run automatically.

The second user's user terminal 10 shown in FIG. 1 is configured such that a head 10c is connected to a cylindrical body portion 10d, for example. Further, the second user's user terminal 10 shown in FIG. 1 is provided with a camera 19, a display unit 16, and a speaker 17 at appropriate locations on the head 10c, and a microphone 18 is provided at an appropriate location on the body portion 10d. ing. In the example shown in FIG. 1, a display section 16 is provided at the center of the head 10c, a camera 19 and a speaker 17 are provided at the top and bottom of the display section 16, respectively, and a microphone 18 is provided at a position above the body section 10d. However, their arrangement positions are not limited to this example. For example, the display section 16 and the camera 19 may be provided on the same surface (for example, the front surface) of the user terminal 10, and the display section 16 may be provided at a position where the user who is in front of the user terminal 10 can see it. The camera 19 only needs to be provided at a position where it can photograph the user. Further, the speaker 17 may be provided at a position where output audio can reach users near the user terminal 10, and the microphone 18 may be provided at a location where it can collect sounds emitted by users near the user terminal 10. . Furthermore, the second user's user terminal 10 shown in FIG. 1 is provided with a plurality of tires 20a on the lower surface (bottom surface) of the body portion 10d, and by rotating the tires 20a by the moving mechanism 20 (see FIG. 2), the second user's user terminal 10 shown in FIG. It is configured to run automatically. Note that the second user's user terminal 10 shown in FIG. 1 is configured such that the head 10c rotates about the vertical direction with respect to the body portion 10d, and the head 10c swings at an appropriate angle with respect to the horizontal plane. It may have a swinging mechanism like this.

In this embodiment, the user terminal 10 will be described as a dedicated terminal having an appearance as shown in FIG. It doesn't have to be. For example, the user terminal 10 does not need to be provided with the movement mechanism 20, and the user terminal 10 can also be configured using, for example, a smartphone or a tablet terminal. In the information processing system of this embodiment, each user terminal 10 constructs a user model in which unique information of each user is accumulated based on each user's words and actions (dialogue history information and action history information), and each Executes various processes based on the user model. Note that an avatar corresponding to each user is set in each user terminal 10, and each user terminal 10 uses the avatar as necessary when executing processing based on its own user model.

FIG. 2 is a block diagram showing an example of the configuration of the user terminal 10. As shown in FIG. The user terminal 10 includes a control section 11, a storage section 12, a processing section 13, a communication section 14, an input section 15, a display section 16, a speaker 17, a microphone 18, a camera 19, a movement mechanism 20, a sensor section 21, a reading section 22, etc. These parts are interconnected via a bus. The control unit 11 includes one or more processors such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), or a GPU (Graphics Processing Unit). The control unit 11 executes various information processing, control processing, etc. that should be performed by the user terminal 10 by appropriately executing the control program 12P stored in the storage unit 12.

The storage unit 12 includes a RAM (Random Access Memory), a flash memory, a hard disk, an SSD (Solid State Drive), and the like. The storage unit 12 stores in advance a control program 12P executed by the control unit 11 and various data necessary for executing the control program 12P. The storage unit 12 also temporarily stores data and the like generated when the control unit 11 executes the control program 12P. The storage unit 12 also stores a user model 12M in which user-specific information collected based on the interaction history and action history of the user of the user terminal 10 is accumulated. Furthermore, the storage unit 12 stores a destination information DB (database) 12a, a dialogue history DB 12b, and an action history DB 12c, which will be described later. The destination information DB 12a, the dialogue history DB 12b, and the action history DB 12c may be stored in an external storage device connected to the user terminal 10, or an external storage device with which the user terminal 10 can communicate. Furthermore, the storage unit 12 stores information regarding an avatar (avatar information) that is set in association with the user of the user terminal 10. The information regarding the avatar includes image data for displaying the avatar, audio data for making the avatar speak, motion data for adding movement to the displayed image of the avatar, and the like.

The user model 12M is, for example, a deep learning model learned using a deep learning algorithm, and is composed of, for example, a Seq2Seq (Sequence to Sequence) model using an RNN (Recurrent Neural Network) or a CNN (Convolution Neural Network) model. There is. Note that the user model 12M is not limited to a configuration constructed using these models, but can be constructed using various DNN (Deep Neural Network) models. The user model 12M is a trained model that is trained based on the interaction history and behavior history of the user of the user terminal 10. Specifically, the user model 12M uses dialogue history information that includes dialogue information presented to the user and reply information from the user to the presented dialogue information as teacher data, and inputs the dialogue information. It has been trained to output response information corresponding to the input dialogue information when the input dialogue information is input. Such training data includes, for example, when predetermined dialogue information is presented to the user via the speaker 17 or the display unit 16, response information of the user's response to the presented dialogue information is transmitted to the microphone 18 or the input unit. 15. By training the user model 12M using the teacher data obtained in this way, it is possible to generate a user model 12M that can output information about the user, dialogue information to which the user is likely to respond, and the like. Specifically, for example, when the dialog information "What's your name?" is presented and the user returns the response information "It's ○○○○", the user's name is "○○" based on the user's response information. By using such training data, the user model 12M can learn that the user's name (user-specific information) is "○○○○". Can be done. In addition, if the user returns the response information "I had a fight with my father today" when the dialogue information "Did something happen?" is presented, the user's recent events are determined based on the user's response information. Teacher data such as "I had a fight with my father today" is obtained, and by using such teacher data, the user model 12M can learn information about the user's recent events (user-specific information). Furthermore, when the dialog information of "Where did you go today?" is presented, and the user returns the response information of "I saw a movie about ○○," based on the user's response information, the movie that the user has watched recently is displayed. By using such training data, it is possible to make the user model 12M learn information (user-specific information) about movies that the user has watched recently. Also, for example, if the user returns the response information "Good morning" when the dialogue information "Good morning" is presented, based on the user's response information, the user responds with "Good morning" in response to the dialogue information "Good morning". By using such training data, it is possible to make the user model 12M learn that the user's morning greeting is "Good morning."

Further, interaction history information may be collected from information related to processing executed by various application programs installed on the user terminal 10. For example, when a user (user terminal 10) receives interaction information from another user (other user terminal 10) via email, LINE (registered trademark) message, LINE stamp, SNS (Social Network Service), etc. , when a user (user terminal 10) returns response information to another user (other user terminal 10) in response to this dialog information, even if this dialog information and dialog history information including the response information are used as teacher data. good. In addition, when a user (user terminal 10) views a message via SNS and posts (reply) a message to the viewed message, interaction history information including the viewed message and the posted message is stored in the teacher data. May be used for. In this way, dialogue history information may be collected from data sent and received by various services provided via the network N and used as teacher data.

In addition, the user model 12M uses behavior history information including information regarding actions performed on the user (behavior information) and behavior information performed by the user in response to this action (reaction information) as the teacher data. , has been trained to output reaction information corresponding to the input behavioral information when behavioral information is input. Such training data can be, for example, if a user replies with a predetermined LINE stamp or LINE message when receiving a LINE message, the reception of the LINE message is treated as behavior information, and the predetermined LINE stamp or reply to the LINE message is treated as a reaction. It can be obtained by converting it into information. For example, when a user receives information regarding a conference schedule via e-mail, LINE message, SNS, etc., a reservation application (application program) is then used to reserve the use of a conference room using the user terminal 10. Suppose you execute the following to reserve a conference room. In this case, teacher data may be acquired by using reception of information regarding a conference schedule as behavior information and reservation of a conference room using a reservation app as reaction information. Further, suppose that the user receives a message regarding a meal invitation via e-mail, LINE message, SNS, etc., and then uses the user terminal 10 to make a restaurant reservation through a restaurant reservation site. In this case, the teacher data may be acquired by using the reception of a message regarding a meal invitation as behavior information and the activation of a browser to access a restaurant reservation site as reaction information. Furthermore, when a user makes a travel reservation through a travel reservation site on network N, he or she then uses the user terminal 10 to make a restaurant reservation through a restaurant reservation site or an optional tour reservation site. Assume that you have applied for an optional tour through . In this case, training data may be obtained by using the travel reservation through a travel reservation site as behavioral information and the activation of a browser to access a restaurant reservation site or an optional tour reservation site as reaction information. good. Furthermore, suppose that, for example, when the user receives advertisement information regarding a book from a book sales site, the user then uses the user terminal 10 to place a purchase order for the book via the book sales site. In this case, teacher data may be acquired by using reception of book advertisement information as behavior information and activation of a browser for accessing a book sales site as reaction information.

Further, for example, if the user takes a photo when looking at a flower, the teacher data may be acquired by using the action of looking at the flower as action information and taking the photo as reaction information. Further, for example, if the user uses the user terminal 10 to play music when starting to drive a car, teacher data can be acquired by using the driving of the car as action information and the playback of music as reaction information. You can. Further, for example, if the user uses the user terminal 10 to play music when starting jogging, teacher data may be acquired by using jogging as action information and music playback as reaction information. Further, for example, if the user jogs and then uses the user terminal 10 to launch a health management app and input the running time or distance, jogging is treated as action information, and activation of the health management app is treated as reaction information. The teacher data may be obtained by doing the following. For example, when a user arrives at a predetermined location such as Tokyo Station or an airport, and uses the user terminal 10 to launch a transportation route search app to search for a route to the destination, it is possible to search for a route to the destination. Teacher data may be acquired by using arrival as action information and starting a route search application as reaction information.

The above-mentioned training data is obtained by acquiring captured images, audio data, and motion information of the user using the camera 19, microphone 18, and sensor section 21 of the user terminal 10 when the user is performing some action, for example. It is obtained by acquiring behavior information and reaction information related to the user's behavior based on the acquired captured image, audio data, and behavior information. For example, if a captured image of a user taking a photo of a flower is obtained, teacher data indicating that the user takes a photo when seeing a flower is obtained from the captured image (the behavior of looking at a flower is used as behavior information, (teaching data whose response information is photo taking) can be obtained. In addition, if captured images and audio data of a user driving a car while listening to music are obtained, a teacher indicating that music should be played when the user drives a car based on the captured images and audio data. It is possible to obtain data (teacher data that uses car driving as behavioral information and music playback as reaction information). Furthermore, if captured images, audio data, and motion information of a user jogging while listening to music are obtained, it is possible to decide to play music while the user jogs based on the captured images, audio data, and motion information. It is possible to obtain teacher data (teacher data that uses jogging as behavior information and music playback as reaction information).

By training the user model 12M using the above-mentioned training data, reaction information (behavior information) that is likely to cause a user to react when some action is performed on the user, or the possibility for the user to respond. It is possible to generate a user model 12M that can output highly interactive dialogue information and the like. Furthermore, it is possible to generate a user model 12M that can output the content of the action or dialogue content that is highly likely to be performed next when the user performs a certain action. As described above, the user model 12M collects (learns) various information about the user based on the user's interaction history information and behavior history information, and collects (learns) various information about the user based on the user's personal information, hobbies, preferences, behavior history, behavioral tendencies, and thinking tendencies. etc., accumulates various information regarding users. Note that the trained model performs a predetermined calculation on the input value and outputs the calculation result, and data such as coefficients and threshold values of the function that defines this calculation are stored in the storage unit 12 as the user model 12M. be remembered.

The user model 12M may be trained in the user terminal 10 of each user, or may be stored in the storage unit 12 of the user terminal 10 of each user after being learned in another learning device. The user model 12M learns so that when dialogue information included in the teacher data is input, response information included in the teacher data is output. Furthermore, the user model 12M learns so that when behavior information included in the teacher data is input, reaction information included in the teacher data is output. Note that in the learning process, the user model 12M optimizes data such as coefficients and threshold values of functions that define predetermined operations to be performed on input data. As a result, it is possible to generate a user model 12M that is trained to output response information or reaction information that the user of the user terminal 10 is likely to make based on the input interaction information or behavior information. As described above, the user terminal 10 of the present embodiment performs processing based on the user model 12M constructed based on the interaction history and action history of the user of the own terminal, thereby determining the interactions and actions that the user is likely to perform. is configured to accomplish this on behalf of the user.

FIG. 3 is a block diagram showing a modification of the user model 12M. The user model 12M may be configured using multiple types of learning models, as shown in FIG. 3, for example. The user model 12M shown in FIG. 3 includes a dialog user model 12M1 and a behavior user model 12M2, and the behavior user model 12M2 further includes an image discrimination model 12M3, a voice discrimination model 12M4, and a sensor output discrimination model 12M5. The dialog user model 12M1 is configured, for example, by a Seq2Seq model, and when dialog information from the user acquired via the microphone 18 or input unit 15 of the user terminal 10 is input, the dialog user model 12M1 corresponds to the dialog information. Output reply information. The image discrimination model 12M3 of the behavioral user model 12M2 is configured, for example, by a CNN model, and when a captured image of the user captured by the camera 19 of the user terminal 10 is input, a dialogue corresponding to this captured image is generated. Output information or behavior information. Here, the image discrimination model 12M3 outputs dialogue information or behavior information that the user terminal 10 should perform in response to the photographed user's behavior. The voice discrimination model 12M4 of the behavior user model 12M2 is configured, for example, by an RNN model or a DNN model, and when the user's voice data collected by the microphone 18 of the user terminal 10 is input, this voice discrimination model 12M4 Outputs dialogue information or behavior information corresponding to. Here, the voice discrimination model 12M4 outputs dialogue information or action information that the user terminal 10 should perform in response to the collected user's utterance content. Furthermore, the sensor output discrimination model 12M5 of the behavior user model 12M2 is configured, for example, by an RNN model, and receives the motion information (output signal from the sensor) related to the user's motion detected by the sensor unit 21 of the user terminal 10. When the sensor is activated, dialogue information or behavior information corresponding to the output signal from this sensor is output. Here, the sensor output discrimination model 12M5 outputs interaction information or action information that the user terminal 10 should perform in response to the detected user's action. By using the user model 12M having the above-described configuration, it is possible to determine the dialog information or behavior information that the user terminal 10 should perform in response to the dialog information from the user or the user's behavior information.

The processing unit 13 includes a dialogue processing unit 13a, a behavior processing unit 13b, and the like. Each part of the processing unit 13 may be realized, for example, by the control unit 11 executing the control program 12P stored in the storage unit 12, or some of these may be realized by a dedicated hardware circuit. good. The dialogue processing unit 13a performs dialogue processing based on, for example, instructions from the user of the user terminal 10 or the user model 12M. The dialogue processing unit 13a performs a dialogue with the user of the user terminal 10, for example, and in this case, receives dialogue information (utterance information) from the user by voice input through the microphone 18 or input (for example, text input) through the input unit 15. ), generates dialogue information (response information) for the acquired dialogue information, and responds to the user by outputting the generated reply information as audio through the speaker 17 or displaying it on the display unit 16. Thereby, the interaction processing unit 13a can interact with the user of the user terminal 10. Note that by using the user model 12M when generating response information to utterance information from the user, the dialog processing unit 13a can realize a dialog (reply) in which the user is likely to respond. Specifically, when acquiring utterance information from the user, the dialogue processing unit 13a inputs the acquired utterance information to the user model 12M, and acquires response information to the utterance information output from the user model 12M. and output the obtained response information.

Note that the dialogue processing unit 13a stores in advance a plurality of templates of response information to be made to utterance information from the user, and selects and outputs response information corresponding to the utterance information from the user from the template. A dialogue with the user may also take place. The dialogue processing unit 13a also stores dialogue information (utterance information) such as greeting messages and question messages in advance, and talks to the user by outputting the utterance information through the speaker 17 or displaying it on the display unit 16, A dialogue with the user may be performed by obtaining response information from the user through voice input via the microphone 18 or input via the input unit 15 . In this case, the dialogue processing unit 13a may talk to the user by previously storing a plurality of dialogue information such as greeting messages and question messages as templates, and selecting and outputting an arbitrary template.

Note that in the present embodiment, the user model 12M of the first user (first user model 12M, first learning model) is transmitted to the second user terminal 10 of the second user and deployed on the second user terminal 10. By doing so, the second user terminal 10 is configured to be able to execute processing based on the first user model 12M. That is, by the second user terminal 10 performing processing based on the first user model 12M, it is possible to realize an interaction that is highly likely to be performed by the first user at the second user terminal 10. In this case, the second user can interact with the pseudo first user (first user model 12M) via the second user terminal 10. Note that when the first user model 12M is deployed to the second user terminal 10, the control unit 11 of the second user terminal 10 controls the control unit 11 until the process based on the first user model 12M is completed (until the first user model 12M is uninstalled). (until the first user model 12M is deployed), each hardware part of the second user terminal 10 is configured to be occupied by processing based on the first user model 12M. That is, multiple user models 12M cannot be deployed to one user terminal 10 at the same time. Specifically, in the second user terminal 10, when the first user model 12M is deployed, the second user is prohibited from executing (operating) a process based on the second user model 12M (second learning model). Ru. Note that in the second user terminal 10, when the first user model 12M is undeployed, the control unit 11 resumes execution of the process based on the second user model 12M. Prohibition of processing based on the second user model 12M may be performed, for example, by moving the second user model 12M to a predetermined evacuation area of the storage unit 12. In this case, restarting the processing may be performed using the second user model 12M. This can be done by returning 12M to a predetermined area of the storage unit 12. Further, by prohibiting access to the second user model 12M, processing based on the second user model 12M may be prohibited; in this case, by allowing access to the second user model 12M, processing based on the second user model 12M may be prohibited. Processing based on the user model 12M is restarted. Furthermore, in this embodiment, interaction between the first user model 12M of the first user and the second user model 12M of the second user can also be realized. In this case, for example, the dialog processing unit 13a of the first user terminal 10 receives dialog information (utterance information) from the second user terminal 10, and specifies response information to the received speech information based on the first user model 12M. , transmits the specified reply information to the second user terminal 10. On the other hand, the dialog processing unit 13a of the second user terminal 10 receives dialog information (utterance information) from the second user terminal 10, specifies response information to the received speech information based on the second user model 12M, and specifies The received response information is transmitted to the first user terminal 10. Thereby, the first user terminal 10 performs a dialogue based on the first user model 12M, and the second user terminal 10 performs a dialogue based on the second user model 12M, thereby creating a pseudo-interaction between the first user and the second user. It is possible to have a dialogue.

The behavior processing unit 13b performs behavior processing based on, for example, instructions from the user of the user terminal 10 or the user model 12M. The behavior processing unit 13b acquires (photographs) video and audio using, for example, the camera 19 and the microphone 18, outputs the audio and video using the speaker 17 and the display unit 16, and uses the moving mechanism 20 to perform arbitrary recording. Move to the location. The behavior processing unit 13b determines whether to shoot at the installation location of the user terminal 10, whether to output audio and video, and whether to move based on instructions from the user or the user model 12M. For example, the behavior processing unit 13b determines whether the user of the user terminal 10 wants to take a picture at the installation location of the user terminal 10, and when determining that the user wants to take a picture, the behavior processing unit 13b determines whether the user of the user terminal 10 wants to take a picture at the installation location of the user terminal 10. It is determined that the photograph will be taken at . The behavior processing unit 13b also determines whether the user of the user terminal 10 wants to output audio and video at the installation location of the user terminal 10, and if it determines that the user wants to output audio and video, the user terminal It is determined that audio and video are to be output at the installation location No. 10. Further, the behavior processing unit 13b determines whether the user of the user terminal 10 wants to move at the installation location of the user terminal 10, and if it is determined that the user wants to move, the behavior processing unit 13b determines whether the user of the user terminal 10 wants to move at the installation location of the user terminal 10. It is determined that the object will move at . When the behavior processing unit 13b determines to take a picture, it uses the camera 19 and the microphone 18 to take the picture, and when it determines to output audio and video, it outputs it using the speaker 17 and the display unit 16, and when the action processing unit 13b moves. If it is determined, the movement is performed using the movement mechanism 20.

The behavior processing unit 13b also determines, for example, whether there are other users with whom the user of the user terminal 10 wants to interact, whether there is a place the user of the user terminal 10 would like to go to away from the installation location of the user terminal 10; Identify whether or not you have something on your mind. In addition, since the user model 12M has accumulated the user's hobbies, preferences, behavior history, behavior trends, etc., the behavior processing unit 13b can determine other users with whom the user wants to interact, places the user wants to go, etc. based on the user model 12M. You can specify what you want to do. Specifically, the behavior processing unit 13b can obtain information regarding the person with whom the user wants to interact, which is output from the user model 12M, by inputting a message such as "Who do you want to interact with?" to the user model 12M. In addition, the behavior processing unit 13b can obtain information regarding the place the user wants to go, which is output from the user model 12M, by inputting a message such as "Where do you want to go?" to the user model 12M. Further, the behavior processing unit 13b can obtain information regarding what the user wants to do, which is output from the user model 12M, by inputting a message such as "What do you want to do?" to the user model 12M. When the behavior processing unit 13b specifies the person with whom the user wants to interact, the place they want to go, the things they want to do, etc., the behavior processing unit 13b specifies another user terminal 10 (referred to as a second user terminal 10) corresponding to the identification result on the go. For example, when the behavior processing unit 13b specifies the person with whom one wishes to interact, the behavior processing unit 13b specifies the user terminal 10 (second user terminal 10) of the other party with whom one wants to interact outside the home, and when the behavior processing unit 13b specifies the place one wants to go to, the behavior processing unit 13b specifies the user terminal 10 (second user terminal 10) of the other party one wants to interact with, and when the behavior processing unit 13b specifies the place one wants to go, the action processing unit 13b determines the user terminal 10 (second user terminal 10) of the other party one wants to have a conversation with. When you specify the user terminal 10 (for example, the second user terminal 10 installed at a place you want to go) outside the home and specify what you want to do, the user terminal 10 corresponding to what you want to do (for example, installed at a place where you can do what you want to do) The second user terminal 10) that has been installed is specified at a location outside the home. The behavior processing unit 13b then transmits the first user model 12M of its own user terminal 10 (first user terminal 10) to the second user terminal 10 specified outside the home.

Note that when the second user terminal 10 receives the first user model 12M from the first user terminal 10, it can store the received first user model 12M in the storage unit 12 and perform processing based on the first user model 12M. Deploy it like this. Thereby, the second user terminal 10 can execute the process based on the first user model 12M, and the interaction processing section 13a of the second user terminal 10 performs the interaction process based on the first user model 12M. The second user terminal 10 can execute a dialogue that is likely to be performed by the first user, and the behavior processing unit 13b of the second user terminal 10 performs behavior processing based on the first user model 12M. It is possible to perform an action that is highly likely to be performed by the first user on the terminal 10. Through the above-described process, the first user model 12M (pseudo first user) goes to the second user's home (second user terminal 10) to play in a pseudo manner, and interacts with the interaction processing unit of the second user terminal 10. 13a allows interaction with the second user. In addition, the first user model 12M (pseudo first user) goes out to an arbitrary place (second user terminal 10 to be deployed), and uses the behavior processing unit 13b of the second user terminal 10 to display the camera 19. It is also possible to perform actions such as photographing using the microphone 18, outputting audio and video using the speaker 17 and display unit 16, and moving using the moving mechanism 20.

The communication unit 14 is an interface for connecting to the network N by wired communication or wireless communication, and transmits and receives information to and from an external device via the network N. The input unit 15 receives an operation input by the user and sends a control signal corresponding to the operation content to the control unit 11. The display unit 16 is a liquid crystal display, an organic EL display, or the like, and displays various information according to instructions from the control unit 11. The input section 15 and the display section 16 may be a touch panel configured as one unit.

The speaker 17 is an audio output device having an amplifier and the like, and outputs audio after amplifying audio data according to instructions from the control unit 11 using the amplifier. The microphone 18 is an audio input device having an amplifier, an A/D (analog/digital) converter, etc., and collects surrounding sounds to obtain analog audio data, and amplifies the acquired audio data with the amplifier. , the audio data is converted into digital audio data using an A/D converter, and the audio data is obtained. The microphone 18 collects sound according to instructions from the control unit 11 and sequentially sends the acquired audio data to the control unit 11. The speaker 17 and the microphone 18 may be built into the user terminal 10, or may be externally attached to the user terminal 10 via a cable or the like, or may be configured to wirelessly communicate with the user terminal 10. Good too. The speaker 17 and the microphone 18 may be a headset configured as one unit.

The camera 19 is an imaging unit having a lens, an image sensor, etc., and acquires image data by photoelectrically converting light incident through the lens using the image sensor. The camera 19 performs photography according to instructions from the control unit 11 and sequentially sends the acquired image data (photographed images) to the control unit 11. The camera 19 may be built into the user terminal 10 or may be externally attached to the user terminal 10 via a cable or the like.

The moving mechanism 20 includes a plurality of tires 20a, a motor (not shown) for rotating each tire 20a, and the like. The moving mechanism 20 rotates each tire 20a by driving each motor with a motor drive amount according to an instruction from the control unit 11, and moves or changes the direction of the user terminal 10 in a predetermined direction. The moving mechanism 20 is not limited to a two-wheel running type, a four-wheel running type, or a six-wheel running type moving mechanism using tires 20a, and may be a caterpillar type moving mechanism, and may be a bipedal walking type or a quadrupedal walking type. It may be a walking-type movement mechanism such as the above, or it may be a walking-type movement mechanism such as a snake-type walking type or an earthworm type walking-type movement mechanism. Furthermore, the moving mechanism 20 is not limited to a structure that moves on a floor surface or the ground, and may include a flight mechanism that flies indoors or outdoors.

The sensor unit 21 includes various sensors such as a GPS (Global Positioning System) sensor, an acceleration sensor, and a geomagnetic sensor, and detects the position and movement of the user terminal 10. The sensor unit 21 can detect the position and movement of the user carrying the user terminal 10 by detecting the position and movement of the user terminal 10. The sensor unit 21 may be built into the user terminal 10 or may be configured to be able to communicate wirelessly with the user terminal 10. When the sensor unit 21 is configured to be able to wirelessly communicate with the user terminal 10, the user only needs to carry the sensor unit 21, and the sensor unit 21 transmits the detected position and movement of the user to the user terminal 10. .

The reading unit 22 reads information stored in a portable storage medium 1a including a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disc)-ROM, or a USB (Universal Serial Bus) memory. The control program 12P and various data stored in the storage unit 12 may be read by the control unit 11 from the portable storage medium 1a via the reading unit 22 and stored in the storage unit 12. Further, the control program 12P and various data stored in the storage unit 12 may be downloaded by the control unit 11 from an external device via the communication unit 14 and stored in the storage unit 12. Furthermore, the control program 12P and various data may be stored in the semiconductor memory 1b, and the control unit 11 may read them from the semiconductor memory 1b and store them in the storage unit 12.

FIG. 4 is a schematic diagram showing a configuration example of the DBs 12a to 12c stored in the user terminal 10. FIG. 4A shows the destination information DB 12a, FIG. 4B shows the dialogue history DB 12b, and FIG. 4C shows the action history DB 12c. The destination information DB 12a is an address book that stores destination information of other user terminals 10 used when the user terminal 10 transmits and receives information via the network N. The destination information DB 12a shown in FIG. 4A includes a terminal ID column, an owner name column, a destination information column, and the like. The terminal ID column stores identification information assigned in advance to each of the other user terminals 10, and the owner name column stores the names of the users of the other user terminals 10 in association with the terminal IDs. The destination information string stores destination information assigned to other user terminals 10 in association with terminal IDs. As the destination information, for example, a telephone number, email address, IP (Internet Protocol Address) address, LINE-ID, etc. assigned to another user terminal 10 can be used. The terminal ID stored in the destination information DB 12a is issued and stored by the control unit 11 when the control unit 11 acquires new information about another user terminal 10 via the input unit 15 or the communication unit 14. . Each piece of information other than the terminal ID stored in the destination information DB 12a is added or changed by the control unit 11 each time the control unit 11 obtains an addition or change instruction via the input unit 15 or the communication unit 14. The storage contents of the destination information DB 12a are not limited to the example shown in FIG. 4A, and various information regarding other user terminals 10 can be stored.

The dialogue history DB 12b stores history information of dialogues executed by the dialogue processing unit 13a in the user terminal 10 based on the user model 12M. Specifically, the dialogue history DB 12b of the first user terminal 10 stores history information of the dialogue with the first user that the dialogue processing unit 13a of the first user terminal 10 has executed based on the first user model 12M. Furthermore, when the second user model 12M is deployed to the first user terminal 10, the interaction processing unit 13a of the first user terminal 10 becomes capable of interaction based on the second user model 12M. Therefore, the dialogue history DB 12b stores history information of the dialogue with the first user that the first user terminal 10 has executed based on the second user model 12M. Further, when the user model 12M (first user model 12M) is deployed to another user terminal 10 (second user terminal 10), the interaction processing unit 13a of the second user terminal 10 becomes possible. Therefore, the dialogue history DB 12b stores history information of the dialogue with the second user that the dialogue processing unit 13a of the second user terminal 10 has executed based on the first user model 12M. Note that history information of the dialogue executed at the second user terminal 10 is transmitted from the second user terminal 10 to the first user terminal 10 and stored in the dialogue history DB 12b of the first user terminal 10.

The dialogue history DB 12b shown in FIG. 4B includes a dialogue ID string, a partner information string, a dialogue information string, and the like. The dialogue ID column stores identification information assigned to each dialogue process executed by the dialogue processing unit 13a of the first user terminal 10 or the second user terminal 10 to which the first user model 12M is deployed. The partner information string stores information about the dialogue partner in the dialogue process in association with the dialogue ID. The dialogue partner is, for example, the partner (for example, the first user) with whom the dialogue processing unit 13a of the first user terminal 10 performs the dialogue via the first user terminal 10, and the dialogue processing unit 13a of the second user terminal 10 is the second user. The information includes the person with whom the conversation is performed via the terminal 10 (e.g., the second user), and the information on the person with whom the conversation occurs may include, for example, the name of the person with whom the conversation occurs, the terminal ID of the user terminal 10 of the person with whom the conversation occurs, or destination information. . The dialogue information string stores various data (dialogue data) acquired during dialogue processing in association with the dialogue ID. For example, the dialogue data is generated by the camera of the first user terminal 10 or the second user terminal 10 while the dialogue processing unit 13a of the first user terminal 10 or the second user terminal 10 on which the first user model 12M is deployed executes dialogue processing. 19 and microphone 18, text data obtained or displayed using input section 15 and display section 16, etc. Note that the dialogue data is not limited to the configuration stored in the dialogue history DB 12b, and may be stored in a predetermined area of the storage unit 12 or an external storage device connected to the user terminal 10. In this case, the dialog information string may store information for reading the dialog data (for example, a file name indicating the storage location of the data). The storage contents of the dialogue history DB 12b are not limited to the example shown in FIG. 4B, but include various kinds of dialogue processing related to the dialogue processing executed by the dialogue processing unit 13a of the second user terminal 10 to which the first user terminal 10 or the first user model 12M is deployed. Able to memorize information.

The action history DB 12c stores history information of actions performed by the action processing unit 13b in the user terminal 10 based on the user model 12M. Specifically, the behavior history DB 12c of the first user terminal 10 includes information such as photography using the camera 19 and microphone 18, photography performed by the behavior processing unit 13b of the first user terminal 10 based on the first user model 12M, History information of actions such as audio and video output using the display unit 16 and movement using the moving mechanism 20 is stored. In addition, the behavior history DB 12c records that when the user model 12M (first user model 12M) is deployed to another user terminal 10 (second user terminal 10), the behavior processing unit 13b of the second user terminal 10 Stores history information of actions executed based on the model 12M. Specifically, the behavior processing unit 13b of the second user terminal 10 performs photography using the camera 19 and microphone 18 based on the first user model 12M, and outputs audio and video using the speaker 17 and display unit 16. , stores history information of actions such as movement using the movement mechanism 20. Note that history information of actions performed at the second user terminal 10 is transmitted from the second user terminal 10 to the first user terminal 10 and stored in the action history DB 12c of the first user terminal 10.

The action history DB 12c shown in FIG. 4C includes an action ID string, a location information string, an action information string, and the like. The behavior ID column stores identification information assigned to each behavior process executed by the behavior processing unit 13b of the first user terminal 10 or the second user terminal 10 to which the first user model 12M is deployed. The location information column stores information about the location of the outing in the behavior process in association with the behavior ID. The destination is the second user terminal 10 determined by the behavior processing unit 13b of the first user terminal 10 based on the instruction from the first user or the first user model 12M, and 2 user terminals 10. For example, the terminal ID, destination information, or installation location information of the second user terminal 10 determined to be out of the home, the name of the owner (second user) of the second user terminal 10, etc. are used as the information on the location outside the home. be able to. The behavior information string stores various data (behavior data) acquired during behavior processing in association with the behavior ID. For example, the behavior data is generated by the camera of the first user terminal 10 or the second user terminal 10 while the behavior processing unit 13b of the first user terminal 10 or the second user terminal 10 to which the first user model 12M is deployed executes behavior processing. 19 and microphone 18, text data obtained and displayed using the input section 15 and display section 16, etc. Note that the behavior data is not limited to the configuration stored in the behavior history DB 12c, and may be stored in a predetermined area of the storage unit 12 or an external storage device connected to the user terminal 10. In this case, the behavior information string may store information for reading the behavior data (for example, a file name indicating the storage location of the data). The memory contents of the behavior history DB 12c are not limited to the example shown in FIG. 4C, and include various types of behavior processing related to the behavior processing executed by the behavior processing unit 13b of the second user terminal 10 to which the first user terminal 10 or the first user model 12M is deployed. Able to memorize information. For example, if there is a companion in the action (going out) performed by the action processing unit 13b, information on the companion may be stored in the action history DB 12c.

The processing performed by each user terminal 10 will be described below. The following processing is executed by the control unit 11 in each user terminal 10 according to the control program 12P stored in the storage unit 12. In this embodiment, the following processing is realized by the control unit 11 executing the control program 12P, but some of the processing may be realized by a dedicated hardware circuit. FIG. 5 is a flowchart showing an example of a dialog processing procedure by the user terminal 10, and FIG. 6 is a schematic diagram showing an example of a screen on the user terminal 10.

The control unit 11 of the user terminal 10 determines whether a message (dialogue information) from the user of the own terminal has been received via the microphone 18 or the input unit 15 (S11). When the user of the user terminal 10 wishes to interact with his/her own user terminal 10 (his or her user model 12M), the user inputs a message to be spoken to by voice input via the microphone 18 or text input via the input unit 15. When the control unit 11 of the user terminal 10 determines that a message from the user has been accepted (S11: YES), the interaction processing unit 13a generates a reply message to the accepted message (S12). The control unit 11 (dialogue processing unit 13a), for example, inputs the received message into the user model 12M, and generates a reply message based on output information from the user model 12M. Note that the user model 12M is trained to output a reply message to the input message when a message is input, and the control unit 11 directly outputs the message output from the user model 12M as a reply message. Good too. For example, if a reply message to be sent to a predetermined message is stored as a template, the control unit 11 (dialogue processing unit 13a) may select a reply message to the received message from the template.

The control unit 11 outputs the generated response message via the speaker 17 or the display unit 16 (S13). Specifically, the control unit 11 responds to the user by outputting a response message audibly through the speaker 17 or displaying it on the display unit 16. Thereby, the user terminal 10 can return a reply message to the received message from the user, and can have a dialogue with the user. Note that when outputting the reply message, the control unit 11 selects the avatar set by the user of the user terminal 10 based on the image data related to the avatar stored in the storage unit 12, as shown in FIGS. 6A and 6B. The image may be displayed on the display unit 16, and a response message may be outputted from the speaker 17 using audio based on the audio data related to the avatar stored in the storage unit 12. In this case, the user can feel as if he or she is interacting face-to-face with the avatar displayed on the screen shown in FIGS. 6A and 6B. At this time, it is also possible to add movement to the avatar displayed on the display section 16 based on the motion data of the avatar stored in the storage section 12. In this case, the user can enjoy interacting with the avatar even more. I can do it. Note that, as the display image of the avatar, a real image of the user of the user terminal 10, an image of a character resembling a human, an image of an animal, etc., an image of a character such as an animal, etc. can be used.

If it is determined that no message has been received from the user (S11: NO), the control unit 11 determines whether it is time to start a dialogue with the user of the user terminal 10 (S14). For example, if the user has set a timer to an arbitrary date and time as the timing to start dialogue, the control unit 11 determines whether it is the timing to start dialogue depending on whether the timer set date and time has arrived. do. For example, the control unit 11 inputs a message such as "Do you want to talk?" to the user model 12M, and determines whether it is time to start a dialogue based on the output information from the user model 12M. It's okay. In this case, for example, when the user model 12M outputs a message indicating that it wants to start a dialogue with the user, the control unit 11 may determine that it is time to start the dialogue. Furthermore, the control unit 11 photographs the surroundings of the user terminal 10 using, for example, the camera 19, determines whether or not there is a user around the user terminal 10 based on the photographed image, and if there is a user, at the timing of starting the dialogue. It may be determined that there is.

If it is determined that it is time to start a dialogue (S14: YES), the control unit 11 generates a dialogue message to be dialogued with the user of the user terminal 10 (S15). For example, when the control unit 11 (dialogue processing unit 13a) inputs a message such as “Is there anything you would like to talk about” to the user model 12M and outputs a message regarding the dialogue with the user, the control unit 11 (dialogue processing unit 13a) The output message may be an interactive message. Also, greeting messages such as "Hello. How are you?" and question messages that can collect user information (personal information, unique information) such as "What's your name?" and "Did something happen?" etc. are stored as a template, the control unit 11 (dialogue processing unit 13a) may select a message from the template that takes into account the current date and time, the content of past interactions with the user, etc. The control unit 11 outputs the generated dialog message via the speaker 17 or the display unit 16 (S16). Here too, the control unit 11 talks to the user by outputting a dialogue message via the speaker 17 or displaying it on the display unit 16. Thereby, the user terminal 10 can talk to the user at an appropriate timing and can start a dialogue with the user. If it is determined that it is not the timing to start a dialogue with the user (S14: NO), the control unit 11 skips the processing of steps S15 to S16.

After the processing in step S13 or step S16, the control unit 11 determines whether to end the interaction with the user of the user terminal 10 (S17). For example, when the control unit 11 receives a message from the user via the microphone 18 or the input unit 15 to end the dialogue, such as "See you soon" or "Bye-bye", the control unit 11 determines to end the dialogue. If it is determined that the dialogue is not to be ended (S17: NO), the control unit 11 returns to the process of step S11 and repeats the processes of steps S11 to S16. Thereby, the control unit 11 (dialogue processing unit 13a) can return a response message to a dialogue message from the user, output a dialogue message to be spoken to the user, and realize dialogue with the user. In addition, as the interaction processing unit 13a interacts with the user based on the user model 12M through the process described above, the user model 12M learns things about the user based on the interaction message to the user and the response message from the user, and information (user-specific information) is stored in the user model 12M. Therefore, information regarding the user is collected and stored in the user model 12M without the user being aware of it. By performing dialogue processing and behavior processing based on the user model 12M in which various types of information regarding the user are accumulated, it is possible to realize dialogues and actions that are highly likely to be performed by the user.

If it is determined that the dialogue is to be ended (S17: YES), the control unit 11 stores history information of dialogues conducted with the user in the dialogue history DB 12b (S18), and ends the process. Note that when the dialogue processing unit 13a is having a dialogue with the user, the behavior processing unit 13b uses the camera 19 and the microphone 18 to acquire (photograph) the user's video and audio data, for example. Then, the user's video and audio data acquired by the behavior processing unit 13b and the avatar image and audio data output by the interaction processing unit 13a using the speaker 17 and display unit 16 are stored as interaction information in the interaction history DB 12b. Remember. Thereby, history information of the dialogue that the dialogue processing unit 13a has had with the user is stored in the dialogue history DB 12b. Through the above-described processing, the user can interact with his or her own avatar via the user's user terminal 10, thereby alleviating feelings of loneliness and anxiety. Further, since the user's own avatar performs dialogue and actions based on the user model 12M, dialogue and actions similar to those of the user can be realized, and the user can enjoy comfortable dialogue.

In the above-described process, the dialogue processing unit 13a not only generates a dialogue message and a reply message when having a dialogue with the user, but also may analyze the user's emotions based on the captured image of the user. In this case, the dialogue processing unit 13a determines whether the user's emotion is likely to be, for example, joy, anger, sadness, surprise, disgust, relaxation, etc., based on the user's facial expression and posture in the captured image. Analyze. Then, the dialogue processing unit 13a inputs the analyzed user's emotion and the dialogue message from the user into the user model 12M, and specifies a reply message (dialogue information) based on the output information from the user model 12M. Also, at this time, the user model 12M may specify, in addition to the reply message, the emotion that the user model 12M (avatar) should have in response to the user's emotion. In this case, the user model 12M is learning using teacher data including dialogue history information and emotions, and dialogue information and behavior information (emotions) corresponding to these. As a result, when dialogue information (dialogue message from the user) and emotion (user's emotion) are input, the user model 12M can output dialogue information (response message) and emotion (response to user's emotion). (reacting emotions) can be output. Note that, for example, emotions that the user (user) is likely to feel depending on the emotions of the conversation partner are stored as templates, and the conversation processing unit 13a creates a user model 12M (avatar) based on the user's analyzed emotions. Emotions that are likely to be felt may be identified from the template. When the emotion that the user model 12M (avatar) should have is specified, the dialogue processing section 13a or the behavior processing section 13b outputs the specified emotion (behavior information). For example, the dialogue processing unit 13a may express the emotion of the avatar by adding a facial expression or movement corresponding to the specified emotion to the avatar displayed on the display unit 16. For example, the behavior processing unit 13b may express the emotion of the avatar by displaying an image of eyes corresponding to the specified emotion on the display unit 16, as shown in FIG. 6C. In the example shown in FIG. 6C, the upper eye expresses the emotion of joy, and the lower eye expresses the emotion of sadness. Furthermore, the behavior processing unit 13b may express the emotion of the avatar through the movement of the user terminal 10 by using the movement mechanism 20 to perform a movement according to the specified emotion. For example, when the identified emotion is joy, the behavior processing unit 13b may express the emotion of joy by rotating or moving the user terminal 10 using the moving mechanism 20 to cause the user terminal 10 to dance. Furthermore, when the user terminal 10 has a hand or foot portion, the behavior processing unit 13b may express the emotion of the avatar by causing the user terminal 10 to perform gestures and hand gestures corresponding to the specified emotion. With such a configuration, the user terminal 10 can not only exchange messages with the user, but also exchange emotions, and the user can have a more realistic interaction with the avatar. be able to.

In the user terminal 10 that performs the above-described processing, when the user model 12M learns information about the user, it measures the user's heart rate, brain waves, electromyogram, etc., and checks that the user's mental and physical state is relaxed and calm. It is also possible to determine whether or not the user is in a calm state, and to accumulate only information about the user collected from the contents of the dialogue that took place when the user was calm. In this case, by discarding the information about the user that was collected from the content of the dialogue that took place when the user's physical and mental state was not calm and excited, it is possible to accumulate (learn) only the information about the user that was collected during the user's usual state. .

FIG. 7 is a flowchart illustrating an example of a behavior processing procedure by the user terminal 10. The control unit 11 of the user terminal 10 determines whether the user terminal 10 is to be moved near the installation location (S21). For example, the control unit 11 determines whether or not there is a place that the user of the own terminal wants to go to (a place to which he or she wants to move) based on an instruction from the user or the user model 12M, and if there is a place that the user wants to go to, the control unit 11 determines to move. do. For example, if the user has set up a process to periodically photograph a pet kept at the location where the user terminal 10 is installed, or if the user has such a behavioral tendency, the control unit 11 can take pictures of the pet at a predetermined timing. It is determined that the action process of moving to the vicinity and photographing is to be performed. In this case, the control unit 11 determines to move near the pet at a predetermined timing. For example, the control unit 11 inputs a message such as "Is there a place you want to go?" to the user model 12M, and determines whether there is a place you want to go based on the output information from the user model 12M. You may decide to move if there is a place you want to go. Further, for example, if the user terminal 10 is set to move following the user's movement, the control unit 11 may determine that the user terminal 10 moves according to the user's movement. If it is determined to move (S21: YES), the control unit 11 (behavior processing unit 13b) uses the movement mechanism 20 to move to the desired location (S22). For example, when performing the action of photographing a pet, the behavior processing unit 13b uses the moving mechanism 20 to move the user terminal 10 to a position where the pet can be photographed. Specifically, the behavior processing unit 13b outputs a processing command including a movement processing command according to movement information (behavior information) indicating movement to the vicinity of the pet to the movement mechanism 20 (hardware), and The movement process by the mechanism 20 is executed. Note that at this time, the behavior processing unit 13b photographs the surroundings of the user terminal 10 using the camera 19, and moves the user terminal 10 to the vicinity of the pet while confirming the position of the pet based on the photographed image. In addition, when following the user's movement, the behavior processing unit 13b photographs the user using the camera 19, and while confirming the user's position based on the photographed image, the behavior processing unit 13b moves the user terminal 10 while maintaining a predetermined distance from the user. move it. If it is determined not to move (S21: NO), the control unit 11 skips the process of step S22.

The control unit 11 determines whether to perform photographing and/or recording at the installation location of the user terminal 10 or at the location after movement (S23). For example, the control unit 11 determines whether or not the user of the own terminal wants to shoot and/or record based on an instruction from the user or the user model 12M, and when the user of the own terminal wants to shoot and/or record. It is determined that the video will be photographed and/or recorded. For example, the control unit 11 determines to perform the action process of photographing the pet, and when the pet moves close to the pet, determines to photograph and record the pet. For example, the control unit 11 inputs a message such as "Is there something you want to photograph?" to the user model 12M, and determines whether there is something you want to photograph based on the output information from the user model 12M. You may decide to take a picture if there is something you want to take a picture of. When it is determined that photography and/or audio recording is to be performed (S23: YES), the control unit 11 (behavior processing unit 13b) performs photography and audio recording using the camera 19 and microphone 18 (S24). For example, when performing the action of photographing a pet, the behavior processing unit 13b uses the camera 19 and the microphone 18 to acquire a photographed image (video) and audio of the pet. Specifically, the behavior processing unit 13b outputs a processing command to the camera 19 and the microphone 18 (hardware) according to the behavior information indicating photographing and recording of the pet, and causes the camera 19 and the microphone 18 to photograph and record the pet. Execute the process. Note that at this time, the behavior processing unit 13b acquires video and audio of the pet while tracking the pet's position based on the captured images sequentially acquired by the camera 19. If it is determined that photographing and recording are not to be performed (S23: NO), the control unit 11 skips the process of step S24.

The control unit 11 determines whether to output audio and/or video at the installation location of the user terminal 10 or at the location after movement (S25). For example, the control unit 11 determines whether the user of the own terminal wants to output audio and/or video based on an instruction from the user or the user model 12M, and outputs the audio and/or video. It is determined that audio and/or video will be output when the user wants to output the audio and/or video. For example, when the user is in the room (the installation location of the user terminal 10), if a process for talking to the user is set, or if the user has such behavioral tendencies (habits), the control unit 11 If so, it is determined to output audio and/or video (talk). For example, the control unit 11 inputs messages such as "Is there any audio or video you want to output?" or "Is there anything you want to talk about?" to the user model 12M, and outputs the messages based on the output information from the user model 12M. It may be determined whether or not there is audio or video, and if there is audio or video that is desired to be output, it may be determined that the audio and/or video is to be output. If it is determined that audio and/or video should be output (S25: YES), the control unit 11 (behavior processing unit 13b) outputs the audio and/or video using the speaker 17 and display unit 16 (S26). For example, when talking to a user in a room, the behavior processing unit 13b displays an avatar image on the display unit 16 and outputs a message such as "Hello. Are you free now?" from the speaker 17, and Talk to the user in . Specifically, the behavior processing unit 13b outputs a processing command according to the behavior information indicating the output of audio to the speaker 17 (hardware), and causes the speaker 17 to perform audio output processing. If it is determined that audio and video are not to be output (S25: NO), the control unit 11 skips the process of step S26.

The control unit 11 determines whether to end the action based on the instruction from the user or the user model 12M (S27). For example, the control unit 11 determines to perform an action process of photographing a pet, and when photographing and recording of the pet are completed, determines to end the action. If it is determined that the action is not to be completed (S27: NO), the control unit 11 returns to the process of step S21 and repeats the processes of steps S21 to S26. Thereby, the control unit 11 (behavior processing unit 13b) can perform an action that is highly likely to be performed by the user of the own terminal, based on instructions from the user or the user model 12M.

If it is determined that the action is to be ended (S27: YES), the control unit 11 stores history information of the action performed by the user terminal 10 in the action history DB 12c (S28), and ends the process. Note that when the movement mechanism 20 is moving and the speaker 17 and the display unit 16 are outputting audio and video, the behavior processing unit 13b uses the camera 19 and the microphone 18 to monitor the video and audio around the user terminal 10. The video and audio data, including such video and audio data, acquired by the camera 19 and the microphone 18 are stored as behavior information in the behavior history DB 12c. Thereby, history information of actions performed by the action processing unit 13b is stored in the action history DB 12c.

In the above-described process, the user model 12M can also learn things about the user as the user's behavior history and behavior tendency regarding the behavior that the behavior processing unit 13b executes based on instructions from the user or the user model 12M. Therefore, the user's behavior history and behavior trends are collected one after another and stored in the user model 12M without the user being aware of it.

8 and 9 are flowcharts showing an example of a dialog processing procedure by the user terminal 10, and FIG. 10 is a schematic diagram showing an example of a screen on the user terminal 10. 8 and 9, the first user model 12M of the first user (avatar of the first user) interacts with the second user or the second user model 12M of the second user (avatar of the second user). This is the process when performing. In FIGS. 8 and 9, the left side shows the process performed by the first user terminal 10, and the right side shows the process performed by the second user terminal 10.

The control unit 11 (behavior processing unit 13b) of the first user terminal 10 determines whether there is a second user with whom the user wants to interact (S31). For example, the control unit 11 determines whether there is a second user with whom the first user wants to interact, based on an instruction from the first user or the first user model 12M. For example, if the first user utters "How is Mr. ___?" while interacting with the first user terminal 10 (first user model 12M), the control unit 11 determines that the first user is Mr. (the second user). For example, the control unit 11 inputs a message such as "Do you want to interact with someone?" to the first user model 12M, and based on the output information from the first user model 12M, the control unit 11 inputs a message such as "Do you want to interact with someone?" If there is a second user with whom the user wants to interact, the second user is identified. In this way, by specifying the second user with whom the first user wants to interact based on the first user model 12M, it is possible to specify the second user with whom the first user is likely to want to interact. If it is determined that there is no second user with whom the user wants to interact (S31: NO), the control unit 11 waits.

If it is determined that there is a second user with whom the user wants to interact (S31: YES), the control unit 11 reads the second user's destination information from the destination information DB 12a (S32). For example, the control unit 11 reads destination information stored in the destination information DB 12a in association with the second user's information (for example, name, name of the owner of the second user terminal 10). Then, the control unit 11 requests the second user terminal 10 of the second user to perform a dialogue based on the read destination information (S33). Note that the control unit 11 transmits a request signal to the second user terminal 10 to request execution of a dialogue with the second user.

When the control unit 11 of the second user terminal 10 receives the interaction request signal from the first user terminal 10, it outputs, for example, via the speaker 17 or the display unit 16, that the interaction request has been received from the first user. (S34). For example, the control unit 11 outputs a message from the speaker 17 such as "Mr. ○○ has requested a dialogue. What should you do?" to notify the second user that the first user has requested a dialogue. The control unit 11 also displays a message on the display unit 16 such as "Mr. ○○ has requested a dialogue", an OK button for instructing to start dialogue, and a cancel button for refusing execution of the dialogue. is displayed to notify the second user that there is a dialogue request from the first user.

The control unit 11 of the second user terminal 10 determines whether the second user can interact with the first user's avatar (the first user model 12M of the first user terminal 10) (S35). For example, the control unit 11 of the second user terminal 10 receives a response from the second user indicating whether dialogue is possible through voice input via the microphone 18, and determines whether the second user is capable of dialogue based on the received response. (whether or not interaction is permitted by the second user). The control unit 11 may also accept an operation on the OK button or the cancel button displayed on the display unit 16 through the input unit 15, and determine whether or not the second user can interact based on the accepted operation. good. If it is determined that the second user is not able to interact (S35: NO), the control unit 11 of the second user terminal 10 causes the second user's avatar (second user model 12M) to interact with the first user instead of the second user. Decide to perform an interaction with the user's avatar (hereinafter referred to as the first avatar), and notify the first user terminal 10 that the second user's avatar (hereinafter referred to as the second avatar) can interact. (S36). Then, the control unit 11 (dialogue processing unit 13a) of the second user terminal 10 starts a dialogue with the first avatar based on the second user model 12M (S37). Specifically, the control unit 11 of the second user terminal 10 generates an interaction message on behalf of the second user based on the second user model 12M, and transmits the generated interaction message to the first user terminal 10 ( S38).

On the other hand, when the control unit 11 (dialogue processing unit 13a) of the first user terminal 10 is notified from the second user terminal 10 that the second avatar can interact, the control unit 11 (dialogue processing unit 13a) generates a second A dialogue with the avatar is started (S39). Here, the control unit 11 of the first user terminal 10 generates an interaction message based on the first user model 12M instead of the first user, and transmits the generated interaction message to the second user terminal 10 (S40). . As a result, messages are exchanged between the first user terminal 10 (first user model 12M) and the second user terminal 10 (second user model 12M). Note that messages can be sent and received using, for example, a chat app or a message app, and the chat app or message app is stored in advance in the storage unit 12 of each user terminal 10, for example.

The control unit 11 of the first user terminal 10 inputs the dialog message received from the second user terminal 10 into the first user model 12M, and generates a reply message based on the output information from the first user model 12M. It is transmitted to the second user terminal 10. Similarly, the control unit 11 of the second user terminal 10 inputs the dialog message received from the first user terminal 10 into the second user model 12M, and generates a reply message based on the output information from the second user model 12M. and transmits it to the first user terminal 10. Note that the control unit 11 of the first user terminal 10 and the second user terminal 10 may select a response message to the received interactive message from a response message template prepared in advance for a predetermined message, for example. .

The control unit 11 of the first user terminal 10 determines whether to end the interaction with the second user terminal 10 (second avatar) (S41). For example, when the control unit 11 receives a message to end the dialogue from the second user terminal 10, it determines to end the dialogue. If the control unit 11 determines not to end the dialogue (S41: NO), it repeats the process of step S40, whereby the first user terminal 10 transmits the dialogue message or reply message to the second user terminal 10. , receives a reply message or a dialogue message from the second user terminal 10, and realizes a dialogue with the second user terminal 10 (second avatar). Similarly, the control unit 11 of the second user terminal 10 determines whether or not to end the interaction with the first user terminal 10 (first avatar) (S43), and if it is determined not to end the interaction (S43). : NO), the process of step S38 is repeated, whereby the second user terminal 10 transmits the dialogue message or reply message to the first user terminal 10, and receives the reply message or dialogue message from the first user terminal 10. , interaction with the first user terminal 10 (first avatar) can be realized.

When the control unit 11 of the first user terminal 10 determines to end the interaction with the second avatar (S41: YES), the control unit 11 stores the history information of the interaction with the second avatar in the interaction history DB 12b. (S42), the process ends. Similarly, when the control unit 11 of the second user terminal 10 determines to end the interaction with the first avatar (S43: YES), the control unit 11 stores the history information of the interaction with the first avatar in the interaction history DB 12b. (S44), and the process ends. Note that when a conversation is taking place between the first avatar and the second avatar, the configuration may be such that, for example, image (video) and audio data of each avatar is transmitted and received from each user terminal 10. In this case, the video and audio data of each avatar transmitted and received between the first user terminal 10 and the second user terminal 10 are stored as dialogue information in the dialogue history DB 12b. Thereby, history information of the dialogue performed between the first avatar and the second avatar is stored in the dialogue history DB 12b of each user terminal 10.

In the process described above, the control unit 11 of the first user terminal 10 generates an interaction message with the second user based on the first user model 12M, and the control unit 11 of the second user terminal 10 generates an interaction message with the second user model 12M. An interaction message with the first user is generated based on the message. As a result, the first user terminal 10 can realize a dialogue that is likely to be performed by the first user instead of the first user, and the second user terminal 10 can realize a dialogue that is likely to be performed by the second user instead of the second user. It is possible to achieve a high level of dialogue. Therefore, even in a situation where the first user and the second user cannot interact, the first user and the second user can interact in a pseudo manner by interacting with the first avatar and the second avatar. .

If it is determined in step S35 that the second user can interact (S35: YES), the control unit 11 of the second user terminal 10 notifies the first user terminal 10 that the second user can interact. (S45). When the control unit 11 of the first user terminal 10 is notified from the second user terminal 10 that the second user can interact, it transmits the first user model 12M to the second user terminal 10 (S46). Note that the control unit 11 of the first user terminal 10 deploys the first user model 12M to the second user terminal 10 so that the second user terminal 10 can perform processing based on the first user model 12M. Instruct.

When the control unit 11 of the second user terminal 10 receives the first user model 12M from the first user terminal 10, it can store the first user model 12M in the storage unit 12 and perform processing based on the first user model 12M. Deploy it like this. Then, the control unit 11 of the second user terminal 10 starts processing based on the first user model 12M (S47). Specifically, the control unit 11 of the second user terminal 10 determines whether a message from the second user has been received via the microphone 18 or the input unit 15 (S48). The second user who has consented to dialogue with the first avatar sends a message to the first user model 12M (first avatar) via the second user terminal 10 via voice input via the microphone 18 or via the input unit 15. Input by text input etc. When the control unit 11 of the second user terminal 10 determines that the message from the second user has not been accepted (S48: NO), the control unit 11 generates an interaction message to be communicated with the second user based on the first user model 12M. The dialog message is generated (S49), and the generated dialog message is output via the speaker 17 or the display unit 16 (S50). For example, the control unit 11 generates an interactive message such as “Hello” and speaks to the second user by outputting the generated interactive message from the speaker 17 or displaying it on the display unit 16. Thereby, the first user model 12M (first avatar) starts a dialogue with the second user via the second user terminal 10. Note that when outputting the dialog message, the control unit 11 displays the image of the first avatar corresponding to the first user model 12M on the display unit 16, as shown in FIG. The response message may be output as a voice from the speaker 17.

If it is determined that the message from the second user has been accepted (S48: YES), the control unit 11 of the second user terminal 10 generates a reply message to the accepted message based on the first user model 12M (S51). For example, the control unit 11 inputs the received message to the first user model 12M, and generates a reply message based on output information from the first user model 12M. The control unit 11 outputs the generated reply message via the speaker 17 or the display unit 16 (S52). Thereby, the second user terminal 10 can return a reply message to the received message from the second user, and can have a dialogue with the second user. Here too, the control unit 11 may display the image of the first avatar on the display unit 16 and output a response message from the speaker 17 using the voice of the first avatar.

After the processing in step S50 or step S52, the control unit 11 of the second user terminal 10 determines whether to end the interaction between the first avatar (first user model 12M) and the second user (S53). If it is determined that the dialogue is not to be ended (S53: NO), the control unit 11 returns to the process of step S48 and repeats the processes of steps S48 to S52. Thereby, the second user terminal 10 can return a response message to the dialogue message from the second user, and can output a dialogue message to be spoken to the second user, based on the first user model 12M. Dialogue with the second user can be realized.

If it is determined that the dialogue should end (S53: YES), the control unit 11 of the second user terminal 10 transmits dialogue information of the dialogue conducted between the first avatar and the second user to the first user terminal 10. (S54). Note that when the interaction processing unit 13a of the second user terminal 10 is interacting with the second user based on the first user model 12M, the control unit 11 (behavior processing unit 13b) uses the camera 19 and the microphone 18. The second user is photographed, and the acquired video and audio data of the second user and the image and audio data of the first avatar output using the speaker 17 and the display unit 16 are used as interaction information by the first user. Send to terminal 10. Further, the control unit 11 of the second user terminal 10 stores the transmitted dialogue information as dialogue history information in the dialogue history DB 12b (S55), and ends the process. On the other hand, when the control unit 11 of the first user terminal 10 receives dialogue information from the second user terminal 10, the control unit 11 of the first user terminal 10 uses the received dialogue information as history information of the dialogue that the first avatar has had with the second user. It is stored in the dialogue history DB 12b (S56), and the process ends.

Through the above-described process, the dialogue processing unit 13a of the second user terminal 10 generates a dialogue message to be output to the second user based on the first user model 12M, so that the dialogue processing unit 13a of the second user terminal 10 generates a dialogue message that is likely to be spoken by the first user. can be issued to the second user. Therefore, the second user can have a pseudo-dialogue with the first user by interacting with the first avatar. Furthermore, through the above-described process, the first avatar can interact with the second user or the second avatar instead of the first user, and the history information (dialogue information) indicating the interaction result is transmitted to the first user terminal 10. is stored in the dialogue history DB 12b. Therefore, the first user can confirm the content of the conversation that the first avatar acting as his/her representative has had with the second user or the second avatar by checking the history information of the conversation later. Therefore, when the first user does not have time to interact with the second user or when it is difficult to interact directly with the second user, the first user can have his or her first avatar interact with the second user, and the first user can The user can know the outcome of the interaction. Furthermore, if the second user does not have time to interact with the first avatar or does not want to interact directly with the first avatar, the second user can have his or her second avatar interact with the first avatar. .

FIG. 11 is a flowchart illustrating an example of an interaction processing procedure by the user terminal 10. The process shown in FIG. 11 is a process when the first user interacts with the second user model 12M (the second user's avatar) via the first user terminal 10. In FIG. 11, the left side shows the process performed by the first user terminal 10, and the right side shows the process performed by the second user terminal 10. The control unit 11 of the first user terminal 10 determines whether the first user interacts with the second avatar based on an instruction from the first user (S61). That is, the control unit 11 determines whether the first user wants to invite the second avatar to the first user terminal 10. The second avatar is an avatar (second user model 12M) of a user other than the first user (second user). For example, the first user inputs voice input such as "I want to chat with Mr. ○○'s avatar" through the input unit 15 of the first user terminal 10, and in this case, the control unit 11 determines that the first user is Mr. ○○'s avatar. (second avatar). If it is determined that the first user does not interact with the second avatar (S61: NO), the control unit 11 waits. If it is determined that the first user will interact with the second avatar (S61: YES), the control unit 11 reads the second user's destination information from the destination information DB 12a (S62). Based on the read destination information, the control unit 11 transmits a request signal requesting the second user terminal 10 of the second user to perform a dialogue with the second avatar (S63). That is, the control unit 11 transmits invitation information for inviting the second avatar to the first user terminal 10 to the second user terminal 10.

When the control unit 11 of the second user terminal 10 receives the request signal for interaction with the second avatar from the first user terminal 10, it transmits the second user model 12M to the first user terminal 10 (S64). When the control unit 11 of the first user terminal 10 receives the second user model 12M from the second user terminal 10, it can store the second user model 12M in the storage unit 12 and perform processing based on the second user model 12M. Deploy it like this. Then, the control unit 11 of the first user terminal 10 starts processing based on the second user model 12M (S65). As a result, the control unit 11 of the first user terminal 10 performs the same processing as the processing of steps S48 to S53 performed by the second user terminal 10 in the dialog processing shown in FIG. 9 (S66 to S71), and A dialogue between the user and the second user model 12M (second avatar) is realized. Specifically, the control unit 11 of the first user terminal 10 determines whether or not a message from the first user has been received (S66), and if it is determined that the message has not been received (S66: NO), the control unit 11 of the first user terminal 10 An interaction message to be interacted with the first user is generated based on the model 12M (S67), and the generated interaction message is output (S68). Further, if it is determined that the message from the first user has been accepted (S66: YES), the control unit 11 of the first user terminal 10 generates a reply message to the received message based on the second user model 12M (S69). , and output the generated reply message (S70). Thereby, the first user terminal 10 can issue an interactive message to the first user, can return a response message to the interactive message from the first user, and can interact with the second user model 12M (second avatar). 1 user can interact with each other.

After the processing in step S68 or step S70, the control unit 11 of the first user terminal 10 determines whether or not to end the interaction between the first user and the second avatar (S71), and if it is determined not to end ( S71: NO), the process returns to step S66, and the processes of steps S66 to S71 are repeated. As a result, the first user terminal 10 can return a response message to the dialog message from the first user, can output a dialog message to be spoken to the first user, and based on the second user model 12M. Dialogue with the first user can be realized.

If it is determined that the dialogue should end (S71: YES), the control unit 11 of the first user terminal 10 transmits dialogue information of the dialogue conducted between the first user and the second avatar to the second user terminal 10. (S72). Note that the dialogue information includes video and audio data captured by the first user, and image and audio data of the second avatar. Further, the control unit 11 of the first user terminal 10 stores the transmitted dialogue information as dialogue history information in the dialogue history DB 12b (S73), and ends the process. On the other hand, when the control unit 11 of the second user terminal 10 receives the dialogue information from the first user terminal 10, the control unit 11 of the second user terminal 10 uses the received dialogue information as history information of the dialogue that the second avatar has had with the first user. It is stored in the dialogue history DB 12b (S74), and the process ends.

Through the process described above, when the first user wants to interact with the second avatar, the first user terminal 10 downloads the second avatar (second user model 12M) from the second user terminal 10 to the first user terminal 10. It is possible to realize a dialogue with the second avatar. Since the first user terminal 10 performs dialogue based on the second user model 12M, the first user can perform a pseudo dialogue with the second user by interacting with the second avatar. Through the process described above, the second avatar interacts with the first user instead of the second user, so even if the second user is busy and cannot find time to interact, the first user can interact with the first user instead of the second user. The user can receive dialogue messages that are likely to be sent by the second user, and can enjoy dialogue with the second avatar without worrying about time.

12 and 13 are flowcharts illustrating an example of a behavior processing procedure by the user terminal 10. FIG. The process shown in FIGS. 12 and 13 is a process when the first avatar (first user model 12M) goes out to a place where the first user wants to go (installation location of the second user terminal 10). . In FIGS. 12 and 13, the left side shows the process performed by the first user terminal 10, and the right side shows the process performed by the second user terminal 10. The control unit 11 of the first user terminal 10 determines whether there is a place that the first user wants to go to that is away from the installation location of the first user terminal 10, based on an instruction from the first user or the first user model 12M. It is determined whether or not (S81). For example, if the first user says "I want to go to △△" during the interaction with the first user terminal 10 (first user model 12M), the control unit 11 controls the first user to go to △△. Decide that you want to go. For example, the control unit 11 inputs a message such as "Is there a place you want to go?" into the first user model 12M, and determines the place the first user wants to go based on the output information from the first user model 12M. You may decide whether or not it exists. In this way, by specifying a place that the first user wants to go to based on the first user model 12M, it is possible to specify a place that the first user is likely to want to go to. If it is determined that there is no place that the first user wants to go to (S81: NO), the control unit 11 waits.

If it is determined that there is a place that the first user wants to go to (S81: YES), the control unit 11 of the first user terminal 10 specifies the second user terminal 10 installed at the place that the first user wants to go to. (S82). Note that the control unit 11 of the present embodiment specifies one of the user terminals 10 registered in the destination information DB 12a as a place that the first user wants to go to, and selects the specified user terminal 10 (second user terminal 10). ) is read out from the destination information DB 12a (S83). Then, the control unit 11 transmits a request signal requesting lending to the second user terminal 10 based on the read destination information (S84). When the control unit 11 of the second user terminal 10 receives the rental request signal from the first user terminal 10, it determines whether the second user terminal 10 can be rented (S85). For example, if the second user terminal 10 has already been lent to another user (user model 12M), specifically, if the other user's user model 12M has been deployed to the second user terminal 10, the second user The control unit 11 of the user terminal 10 determines that the rental is not possible. Furthermore, if the second user is interacting with the second user model 12M using the second user terminal 10, it may be determined that the rental is not possible. Furthermore, if the user of the second user terminal 10 has set prohibition of lending the second user terminal 10, the control unit 11 of the second user terminal 10 may determine that lending is not possible. Note that if the user of the second user terminal 10 has set lending permission for the second user terminal 10, the control unit 11 of the second user terminal 10 may determine that lending is possible. When the control unit 11 of the second user terminal 10 determines that the rental is not possible (S85: NO), the control unit 11 of the second user terminal 10 notifies the first user terminal 10 that the rental is not possible (rental is not possible) (S86). When the control unit 11 of the first user terminal 10 receives the notification that lending is not possible from the second user terminal 10 (S87), the control unit 11 of the first user terminal 10 ends the process without performing anything. Note that the control unit 11 of the first user terminal 10 returns to the process of step S81, and determines whether there is another place that the first user wants to go to. The process may be repeated.

When the control unit 11 of the second user terminal 10 determines that the item can be rented (S85: YES), it notifies the first user terminal 10 that the item can be rented (S88). When the control unit 11 of the first user terminal 10 is notified from the second user terminal 10 that the model can be lent, it transmits the first user model 12M to the second user terminal 10 (S89). When the control unit 11 of the second user terminal 10 receives the first user model 12M from the first user terminal 10, it can store the first user model 12M in the storage unit 12 and perform processing based on the first user model 12M. Deploy it like this. Then, the control unit 11 of the second user terminal 10 starts processing based on the first user model 12M (S90). Here, the control unit 11 of the second user terminal 10 performs processing similar to the behavioral processing shown in FIG. 7 (S91 to S97), and the second user terminal 10 performs behavioral processing based on the first user model 12M. Accordingly, the first user model 12M (first avatar) can perform various actions using the second user terminal 10.

Specifically, the control unit 11 of the second user terminal 10 determines where the first user (first avatar) wants to go near the installation location of the second user terminal 10 based on the first user model 12M. It is determined whether there is a place to go, and it is determined whether or not to move depending on whether there is a place to go (S91). If the control unit 11 (behavior processing unit 13b) of the second user terminal 10 determines to move (S91: YES), it moves to the desired location using the moving mechanism 20 (S92), and if it determines not to move, the control unit 11 (behavior processing unit 13b) (S91: NO), the process of step S92 is skipped. Next, the control unit 11 of the second user terminal 10 determines whether or not the first user wants to take a picture or record a video based on the first user model 12M, and depending on whether or not the first user wants to take a picture or record a picture or record a video. Alternatively, it is determined whether or not to record (S93). If it is determined to photograph and/or record (S93: YES), the control unit 11 of the second user terminal 10 photographs and records the surroundings of the second user terminal 10 using the camera 19 and microphone 18 (S94). . Note that the behavior processing unit 13b of the second user terminal 10 identifies a subject that the first user is likely to want to photograph based on the first user model 12M, and performs photographing and audio recording of the identified subject. If it is determined that photographing and recording are not to be performed (S93: NO), the control unit 11 skips the process of step S94.

Next, the control unit 11 of the second user terminal 10 determines whether the first user wants to output audio or video based on the first user model 12M, and determines whether the first user wants to output audio or video. It is determined whether or not to output audio and/or video (S95). When the control unit 11 of the second user terminal 10 determines to output audio and/or video (S95: YES), it outputs the audio and/or video using the speaker 17 or the display unit 16 (S96), and outputs the audio and/or video using the speaker 17 or display unit 16 (S96). If it is determined that the video is not to be output (S95: NO), the process of step S26 is skipped. The control unit 11 of the second user terminal 10 determines whether to end the action based on the first user model 12M (S97). Note that if it is determined based on the first user model 12M that there is a high possibility that the first user wants to end the action (pseudo going out), it may be determined that the action is ended. Further, for example, a loanable time may be set on the second user terminal 10, and it may be determined that the activity is ended when the loanable time has elapsed.

If it is determined that the action is not to be completed (S97: NO), the control unit 11 of the second user terminal 10 returns to the process of step S91 and repeats the processes of steps S91 to S96. Thereby, the second user terminal 10 can perform actions that are highly likely to be performed by the first user based on the first user model 12M. When it is determined that the action is to be ended (S97: YES), the control unit 11 of the second user terminal 10 transfers the history information of the action performed by the second user terminal 10 based on the first user model 12M to the first user terminal 10. (S98). Note that the second user terminal 10 uses the camera 19 and the microphone 18 even when performing actions such as moving by the moving mechanism 20 and outputting audio and video by the speaker 17 and the display unit 16. The user acquires (photographs) video and audio of the surrounding area, and transmits such video and audio data to the first user terminal 10 as behavior information. When the control unit 11 of the first user terminal 10 receives behavior history information (behavior information) from the second user terminal 10, it stores it in the behavior history DB 12c (S99) and ends the process. Thereby, history information of actions performed at the second user terminal 10 based on the first user model 12M is stored in the action history DB 12c of the first user terminal 10.

Through the above-described process, behavior processing based on the second user model 12M is performed in the second user terminal 10 to which the first user model 12M has been transmitted, so that the second user performs an action that is likely to be performed by the first user. It can be executed on the terminal 10. Therefore, the first avatar can go to a place where the first user wants to go in a pseudo manner, and use the second user terminal 10 at the place to perform actions that are likely to be performed by the first user. Through the process described above, instead of the first user actually going out, the first avatar goes out in a pseudo manner, and the video and audio captured at the outside are stored in the action history DB 12c as history information. By checking this history information at a later date, the first user can trace the actions that the first avatar performed on the second user terminal 10, and the first user can feel as if he or she had experienced it himself. Therefore, even if the first user is busy and cannot find time to go out, the first user can simulate various actions by having the first avatar perform actions that the first user is likely to perform. You can experience it.

In the above-described process, the control unit 11 of the second user terminal 10 may be configured to return (move) to a predetermined position, for example, after completing the action based on the first user model 12M. In this case, even if the second user terminal 10 is lent to the first user model 12M, it can be returned to the predetermined position after being lent, so that the second user who is the owner of the second user terminal 10 can, for example, No additional effort is required to collect the second user terminal 10.

FIG. 14 is a flowchart showing an example of a history information reproduction processing procedure, and FIG. 15 is a schematic diagram showing an example of a screen on the user terminal 10. In the process shown in FIG. 14, dialogue information (dialogue history information) is accumulated in the dialogue history DB 12b by the dialogue processing shown in FIGS. 5, 8 to 9, and 11, and the actions shown in FIGS. This process is performed when history information is reproduced in the user terminal 10 where behavior information (behavior history information) has been accumulated in the behavior history DB 12c through the process. The control unit 11 of the user terminal 10 determines whether a request to reproduce any of the history information stored in the dialogue history DB 12b or the action history DB 12c has been received (S101). When the user of the user terminal 10 wishes to reproduce history information of conversations or actions, the user inputs a reproduction request for history information via the microphone 18 or the input unit 15, for example. If it is determined that the history information reproduction request has not been received (S101: NO), the control unit 11 waits until the reproduction request is received.

If it is determined that the request to reproduce history information has been received (S101: YES), the control unit 11 performs a reproduction request based on the dialogue history information stored in the dialogue history DB 12b and the action history information stored in the action history DB 12c. , as shown in FIG. 15, generates a history screen that displays information regarding each piece of history information (S102), and displays it on the display unit 16 (S103). The history screen shown in FIG. 15 displays thumbnail images and information indicating activity details regarding activities (dialogues and actions) performed by one's own avatar. Specifically, the interactions and actions performed based on the first user model 12M at the own user terminal 10 (first user terminal 10), and the interactions and actions performed by the first user at another user terminal 10 (second user terminal 10) Thumbnail images and information indicating activity details are displayed for conversations and actions performed based on the model 12M. Note that thumbnail images can be generated from images (videos) included in dialogue information or behavior information, and information on activity content includes information on the conversation partner and the time of conversation, and information on activities outside the home. Includes location information and time of departure. Further, the history screen displays a "past history" button for displaying thumbnail images and information indicating the activity contents of activities (dialogues and actions) performed by the user's avatar in the past. Therefore, when the "past history" button is operated via the input unit 15 on the history screen, the control unit 11 displays information regarding past history information accumulated in the dialogue history DB 12b and action history DB 12c on the history screen. It is configured to display.

The history screen is configured so that any of the displayed history information can be selected for playback, and includes a play button to instruct playback of the selected history information and a cancel button to end the playback process. is displayed. The control unit 11 determines whether or not a playback instruction for the selected history information has been received, depending on whether any history information has been selected via the input unit 15 on the history screen and the playback button has been operated. (S104). If it is determined that the playback instruction has not been received (S104: NO), the control unit 11 continues displaying the history screen and waits until the playback instruction is received. If it is determined that the playback instruction has been received (S104: YES), the control unit 11 reads the selected history information from the dialogue history DB 12b or the action history DB 12c (S105), and displays the read history information on the display unit 16 and the speaker 17. and outputs it (S106). For example, if the history information includes image data (video data), the control unit 11 displays the image data on the display unit 16, and if the history information includes audio data, the control unit 11 outputs the audio data from the speaker 17. Further, if the history information includes text data, the control unit 11 displays the text data on the display unit 16. As a result, the own avatar set on the user terminal 10 can reproduce the interaction process and the pseudo going out process performed on the own terminal (first user terminal 10) or another terminal (second user terminal 10). Can be done. Note that if the user terminal 10 includes a projector or is connected to a projector, the control unit 11 may output the history information using the projector.

Through the above-described processing, in this embodiment, the avatar set by the user on behalf of the user (himself) of the user terminal 10 can interact with other users or other avatars, and can also perform a pseudo going out. be able to. Therefore, the user (himself) of the user terminal 10 can check the content of the dialogue performed by the avatar and the simulated video and audio of the user outside by playing back the accumulated dialogue and action history information. , the interactions and actions performed by the avatar can be traced. Furthermore, the user can refer to the dialogues and actions performed by the avatar and take actual actions, which can be expected to expand the range of actions in the real world.

(Embodiment 2)
FIG. 16 is a block diagram illustrating a configuration example of an information processing system according to the second embodiment. In the information processing system of this embodiment, a lending-capable user model (hereinafter referred to as lending model), which is a learning model for lending, is prepared in advance, and each user downloads the lending model to his or her own user terminal 10. This is a system that enables dialogue and actions based on the lending model to be realized on the user's own user terminal 10. The information processing system of this embodiment includes a large number of user terminals 10 and a model management server 40 that manages rental models. Send and receive. Note that FIG. 16 shows only one user terminal 10, and the user terminal 10 has the same configuration as in the first embodiment and performs the same processing.

The model management server 40 is configured using a personal computer, a server computer, or the like. The model management server 40 includes a control section 41, a storage section 42, a communication section 43, an input section 44, a display section 45, a reading section 46, etc., and these sections are interconnected via a bus. The control unit 41, storage unit 42, communication unit 43, input unit 44, display unit 45, and reading unit 46 are the control unit 11, storage unit 12, communication unit 14, and input unit of the user terminal 10 described in the first embodiment. Since the configuration is similar to that of the unit 15, display unit 16, and reading unit 22, the description of the configuration will be omitted. Note that the storage unit 42 of the model management server 40 stores, in addition to the control program 42P, a plurality of rental models 42M1 to 42M3 and a model information DB 42a to be described later. In the example shown in FIG. 16, three rental models 42M1, 42M2, and 42M3 are stored, but the storage unit 42 stores a large number of rental models prepared in advance. In the following, the first lending model 42M1 to the third lending model 42M3... will be collectively referred to as the lending model 42M. The rental model 42M and model information DB 42a may be stored in an external storage device connected to the model management server 40, or may be stored in an external storage device with which the model management server 40 can communicate.

The lending model 42M has the same configuration as the user model 12M described in Embodiment 1, and the unique information of each user is collected (learned) based on the interaction history and action history of each user through the same learning process. It is a trained model. The rental model 42M is a user model that has learned unique information of users in various fields, such as athletes, celebrities, doctors, nurses, nutritionists, sports trainers, teachers, cooking teachers, lawyers, tax accountants, and the like. The rental model 42M is, for example, a user model 12M trained by each user on his or her own user terminal 10, transmitted to the model management server 40, and stored in the storage unit 42. Note that learning of the lending model 42M may be performed in the model management server 40, or may be performed in another learning device.

FIG. 17 is a schematic diagram showing a configuration example of the model information DB 42a. The model information DB 42a stores information regarding the rental model 42M registered in the storage unit 42 of the model management server 40. The model information DB 42a shown in FIG. 17 includes a rental model ID column, a name column, a profile column, a fee column, a loanable number column, a rental status column, and the like. The rental model ID column stores identification information assigned to each of the rental models 42M stored in the storage unit 42. The name column stores the name of the user corresponding to the rental model 42M in association with the rental model ID, and the profile column stores the profile of the user corresponding to the rental model 42M. The fee column stores information on fees (usage fees) when renting out the rental model 42M in association with the rental model ID. The fee information indicates, for example, a fee per predetermined time unit such as an hourly unit, a fee depending on the number of questions (number of times dialogue is performed) desired to be asked to the rental model 42M, and the like. The lentable number sequence stores, for example, the number of lentable items at the same time (the number of lending users) in association with the lent model ID. Note that the rental fee may be a fee that corresponds to the number of loanable devices (the number of user terminals 10 that can be deployed at the same time) set in the rental model 42M, and for example, a higher fee may be set as the number of loanable devices is smaller. good. The lending status column stores information indicating the lending status of the lending model 42M. The information indicating the rental status includes, for example, the date and time when the rental was performed, the date and time when the rental was reserved, and the like.

The rental model ID stored in the model information DB 42a is issued and stored by the control unit 41 when a new rental model 42M is stored in the storage unit 42. The name and profile stored in the model information DB 42a are acquired by the control unit 41 via the communication unit 43 and stored, for example, when a new rental model 42M is stored in the storage unit 42. The fee and the number of rental models stored in the model information DB 42a are set by, for example, a user corresponding to the rental model 42M, and are acquired by the control unit 41 via the communication unit 43 and stored or changed. The rental status stored in the model information DB 42a stores the date and time when the rental model 42M was rented or reserved by the control unit 41 when the rental model 42M was rented to the user terminal 10 or when the rental was reserved. The storage contents of the model information DB 42a are not limited to the example shown in FIG. 17, and various information regarding the rental model 42M can be stored.

Below, a process in which the user terminal 10 downloads the lending model 42M from the model management server 40 and executes dialogue and actions based on the lending model 42M on the user terminal 10 will be described. FIG. 18 is a flowchart showing an example of a lending process procedure of the lending model 42M, and FIG. 19 is a schematic diagram showing an example of a screen on the user terminal 10. In FIG. 18, the left side shows the processing performed by the user terminal 10, and the right side shows the processing performed by the model management server 40.

The control unit 11 of the user terminal 10 determines whether a request for lending the lending model 42M has been received from the user of the own terminal via the microphone 18 or the input unit 15 (S111). When a user wants to borrow a loanable model 42M managed by the model management server 40, the user inputs a message via the microphone 18, for example, to the effect that the user wants to borrow the loanable model 42M. Further, if a lending button for requesting lending of the lending model 42M is displayed on the display unit 16 of the user terminal 10, the user operates the lending button via the input unit 15. When the control unit 11 of the user terminal 10 determines that the lending request for the lending model 42M has not been received (S111: NO), it waits. If it is determined that the request for lending the lending model 42M has been accepted (S111: YES), the control unit 11 requests the model management server 40 for a list of the lending models 42M (S112). Note that the destination information of the model management server 40 is stored in the storage unit 12, for example.

When a list of rental models 42M is requested from any user terminal 10, the control unit 41 of the model management server 40 generates a list of rental models 42M that can be rented based on the stored contents of the model information DB 42a. It is transmitted to the user terminal 10 (S113). The control unit 11 of the user terminal 10 displays a list as shown in FIG. 19 on the display unit 16 based on the list of rental models 42M received from the model management server 40 (S114). The list shown in FIG. 19 classifies users corresponding to the rental model 42M by specialty field, and displays each user's name, profile, rental fee, etc. for each specialty field. In the list shown in FIG. 19, a list of rental models 42M that has learned specific information of users in the sports field (athletes), entertainment field (celebrities), and medical fields (doctors) is displayed as users of the rental models 42M. . Information on each rental model 42M is displayed in a selectable manner in the list, and the user of the user terminal 10 selects the rental model 42M he or she wishes to rent via the input unit 15 and operates the enter button. The control unit 11 of the user terminal 10 receives an instruction to rent out the selected rental model 42M when an arbitrary rental model 42M is selected and the determination button is operated. Therefore, the control unit 11 determines whether or not the rental instruction for the rental model 42M has been accepted (S115), and if it is determined that the instruction has not been accepted (S115: NO), it continues to display the list of the rental models 42M. (S114).

If it is determined that the lending instruction for the lending model 42M has been received (S115: YES), the control unit 11 requests the model management server 40 to lend the selected lending model 42M (S116). Note that the control unit 11 receives the rental time desired by the user, for example via the list of rental models 42M, and transmits to the model management server 40 a request signal that includes the rental time and requests rental of the rental model 42M. When a request for lending a lending model 42M is received from any user terminal 10, the control unit 41 (deployment unit) of the model management server 40 reads the requested lending model 42M from the storage unit 42 and transmits it to the user terminal 10. (Deploy) (S117). The control unit 41 of the model management server 40 checks the rental status and the number of rental models 42M requested to be rented, and transmits the rental model 42M to the user terminal 10 if the rental model 42M is available for rental.

The control unit 11 of the user terminal 10 receives the rental model 42M transmitted from the model management server 40, stores the received rental model 42M in the storage unit 12, and deploys it so that processing based on the rental model 42M can be performed. Note that the control unit 11 is configured so that when the lending model 42M is deployed, each hardware part of the user terminal 10 is occupied by processing based on the lending model 42M. This avoids a situation in which, for example, processing based on the user model 12M of the user terminal 10 and processing based on the lending model 42M coexist. The control unit 11 then starts processing based on the lending model 42M (S118). As a result, the control unit 11 of the user terminal 10 performs the same processing as the processing of steps S48 to S53 performed by the second user terminal 10 in the dialog processing shown in FIG. 9 (S119 to S124), and changes the rental model 42M to Based on this, interaction between the user of the user terminal 10 and the avatar of the rental model 42M is realized. Specifically, the control unit 11 of the user terminal 10 determines whether or not a message from the user of the user terminal 10 has been received (S119), and if it is determined that the message has not been received (S119: NO), the rental model 42M An interactive message to be interacted with the user is generated based on (S120), and the generated interactive message is output (S121). If it is determined that a message from the user has been accepted (S119: YES), the control unit 11 generates (specifies) a reply message to the received message based on the lending model 42M (S122), and outputs the generated reply message. (S123). Thereby, the user terminal 10 can perform a dialogue between the avatar of the lending model 42M based on the lending model 42M and the user of the user terminal 10.

After the processing in step S121 or step S123, the control unit 11 of the user terminal 10 determines whether or not to end the interaction between the avatar of the rental model 42M and the user (S124), and if it is determined not to end (S124: NO), the process returns to step S119, and the processes of steps S119 to S124 are repeated. If it is determined that the dialogue is to be ended (S124: YES), the control unit 11 stores the dialogue information of the dialogue between the user of the user terminal 10 and the avatar based on the lending model 42M in the storage unit 12 as dialogue history information. is stored in the dialogue history DB 12b (S125), and the process ends. Note that in step S124, the control unit 11 determines that the dialogue based on the lending model 42M will end when the lending time specified when the user terminal 10 requests the model management server 40 to lend the lending model 42M has elapsed. You can. Furthermore, the rental model 42M may be configured to be prohibited from being used when a specified rental time has elapsed. In this case, once the rental time has elapsed, the rental model 42M can no longer be used. Note that, for example, when the control unit 41 of the model management server 40 transmits the rental model 42M to the user terminal 10, or when the user terminal 10 finishes using the rental model 42M, the control unit 41 of the model management server 40 charges the rental fee (usage fee) of the rental model 42M. You may also perform billing processing. For example, the control unit 41 of the model management server 40 transmits request information requesting payment of the rental fee set for the rental model 42M to the user terminal 10, and the control unit 11 of the user terminal 10 transmits the request information requesting payment of the rental fee set for the rental model 42M. Fee payment processing may also be performed.

Through the above-described process, the user borrows the lending model 42M that can be lent via the model management server 40, and interacts with the avatar of the lending model 42M by executing processing based on the lending model 42M on his or her user terminal 10. It is possible to realize a pseudo dialogue with the user of the lending model 42M. Since the lending model 42M is used to interact with the user of the user terminal 10 instead of the user of the lending model 42M, the avatar of the lending model 42M can be used to interact with the user of the user terminal 10 instead of a busy celebrity or expert. can answer questions and provide advice. Specifically, for example, by borrowing a cooking teacher's avatar (rental model 42M) from the user terminal 10 and cooking while interacting with the cooking teacher's avatar, the user of the user terminal 10 can learn from the cooking teacher's avatar. You can cook while receiving advice. In addition, even if the user of the user terminal 10 has difficulty communicating directly with a celebrity or expert, he or she can easily consult with the celebrity or expert by interacting with the avatar of the celebrity or expert.

In this embodiment as well, the same effects as in Embodiment 1 can be obtained. In addition, in this embodiment, it is possible to lend out the lending model 42M that has learned the unique information of a celebrity or an expert, and each user can download the lending model 42M to his or her own user terminal 10 and execute it. Alternatively, conversations that each expert is likely to have can be realized on their own user terminals 10. Therefore, when a user wants to hear the opinion of an expert, etc., he or she can interact with the expert's avatar by borrowing the expert's lending model 42M and having his or her user terminal 10 execute a dialog process based on the lending model 42M. It is possible to have a simulated dialogue with an expert. In this case, there is a limit to the number of users that the experts themselves can respond to, but by lending out the rental model 42M, the experts can answer the questions of each user in a pseudo manner, solving the problems of many users. It is hoped that this will be possible. For example, if there is a favorite comedian, even if the user cannot go to the theater, the user can borrow the rental model 42M of this comedian and execute the process based on the rental model 42M on his/her own user terminal 10. You can interact with the avatar of your favorite entertainer, allowing you to create a simulated dialogue with the entertainer. Note that the user terminal 10 may execute not only dialogue processing based on the lent lending model 42M but also action processing, and in this case, the user terminal 10 realizes actions that are likely to be performed by the user of the lending model 42M. can. Therefore, the user of the user terminal 10 can realize a simulated behavior with the user of the lending model 42M via the avatar of the lending model 42M realized by the user terminal 10.

In the present embodiment, the lending model 42M is configured to relearn based on the content of the interaction process and behavior processing performed with the user of the lending destination user terminal 10 at the lending destination user terminal 10. You can. Thereby, information regarding the user of the lending model 42M can be further collected (learned) based on the history of interaction processing and behavior processing performed at the lending destination user terminal 10, and can be stored in the lending model 42M. Note that the lending model 42M relearned at the lending destination user terminal 10 is sent back to the model management server 40, and the lending model 42M stored in the storage unit 42 of the model management server 40 (the lending model 42M before relearning) is sent back to the model management server 40. ) may be stored separately or may be overwritten and saved. Thereby, the re-learned lending model 42M can be used as a model to be lent. Note that the lending model 42M may be configured to be able to select whether or not to perform relearning based on the history of interaction processing and behavior processing at the lending destination user terminal 10.

(Embodiment 3)
FIG. 20 is a block diagram illustrating a configuration example of an information processing system according to the third embodiment. In the information processing system of this embodiment, a rentable terminal (hereinafter referred to as a rental terminal), which is an information processing device for rental, is prepared in advance, and by downloading the user model 12M of each user to the rental terminal, This is a system that can realize dialogue and behavior based on the user model 12M of each user at the rental terminal. The information processing system of this embodiment includes, for example, a plurality of rental terminals 50 distributed in tourist spots and a terminal management server 70 that manages the rental terminals 50. Each of the rental terminals 50 and the terminal management server 70 has the following functions: They mutually send and receive information via network N. Note that although FIG. 20 shows an example in which three rental terminals 50 are arranged at tourist spots A and B, four or more rental terminals 50 may be arranged at each tourist spot.

FIG. 21 is a block diagram showing a configuration example of the rental terminal 50. The rental terminal 50 has the same configuration as the user terminal 10 of Embodiment 1, and includes a control section 51, a storage section 52, a processing section 53, a communication section 54, an input section 55, a display section 56, a speaker 57, and a microphone. 58, a camera 59, a moving mechanism 60, a sensor section 61, and the like. Each of the units 51 to 61 described above has the same configuration as each of the units 11 to 21 of the user terminal 10 of the first embodiment, so a description of the configuration will be omitted. Note that the storage unit 52 of the rental terminal 50 stores only the control program 52P. Furthermore, the lending terminal 50 may have a reading section similar to the user terminal 10 of the first embodiment. The rental terminal 50 of this embodiment may be a terminal having the same configuration as the user terminal 10 of Embodiment 1, like the rental terminal 50 on the left side at sightseeing spots A and B in FIG. Furthermore, it may be a terminal that does not have a movement mechanism, such as the central rental terminal 50 at tourist spots A and B in FIG. 20, or it may be a terminal that has a flight mechanism as a movement mechanism, such as the rental terminal 50 on the right side. You can. In addition, in each rental terminal 50, a display section 56, a speaker 57, a microphone 58, and a camera 59 are provided at appropriate positions according to the respective external shapes.

FIG. 22 is a schematic diagram showing a configuration example of the terminal management server 70. The terminal management server 70 is configured using a personal computer, a server computer, or the like, and includes a control section 71, a storage section 72, a communication section 73, an input section 74, a display section 75, a reading section 76, and the like. Each of the units 71 to 76 described above has the same configuration as each of the units 41 to 46 of the model management server 40 of the second embodiment, so a description of the configuration will be omitted. Note that the storage unit 72 of the terminal management server 70 stores a terminal information DB 72a and a lending status DB 72b, which will be described later, in addition to the control program 72P.

FIG. 23 is a schematic diagram showing a configuration example of DBs 72a to 72b stored in the terminal management server 70. FIG. 23A shows the terminal information DB 72a, and FIG. 23B shows the lending status DB 72b. The terminal information DB 72a stores information regarding rental terminals arranged at tourist spots and the like so that they can be rented. The terminal information DB 72a shown in FIG. 23 includes a rental terminal ID column, an installation location column, a resource column, a fee column, and the like. The lending terminal ID column stores identification information assigned to each of the lending terminals 50. The installation location column stores information regarding the installation location of the rental terminal 50 in association with the rental terminal ID. For example, an address, place name, landmark name, latitude and longitude, etc. can be used as the installation location information. The resource column stores information regarding the functions, hardware resources, etc. of the rental terminal 50 in association with the rental terminal ID. For example, information indicating the performance of the microphone 58 and camera 59 that the rental terminal 50 has, and the distance, time, movement method, etc. that can be moved by the movement mechanism 60 are stored in the resource column. Specifically, information such as the sound collection direction (directivity) of the microphone 58, whether the camera 59 is an omnidirectional camera, the type of lens of the camera 59, and whether the moving mechanism 60 is capable of flight is provided. be remembered. The fee column stores information on fees for renting the rental terminal 50 (rental fee) in association with the rental terminal ID. The fee information indicates, for example, a fee per predetermined time unit such as an hourly unit, a fee depending on the number of executions (number of times of processing) of processing based on the user model 12M deployed in the rental terminal 50, and the like. Note that the rental fee for the rental terminal 50 may be set depending on, for example, the quality of the functions that the rental terminal 50 has.

The rental terminal ID stored in the terminal information DB 72a is issued and stored by the control unit 71 when information about a new rental terminal 50 is stored in the storage unit 72. The information on the installation location and resources stored in the terminal information DB 72a is acquired by the control unit 71 via the communication unit 73, for example, when a new rental terminal 50 is registered or when the registered contents are changed. be remembered. The fee stored in the terminal information DB 72a is set, for example, by the owner of the rental terminal 50, and is acquired by the control unit 71 via the communication unit 73 and stored or changed. The storage contents of the terminal information DB 72a are not limited to the example shown in FIG. 23A, and various information regarding the rental terminal 50 can be stored.

The rental status DB 72b stores the rental status of the rental terminal 50 registered in the terminal information DB 72a. The rental status DB 72b shown in FIG. 23B includes a rental terminal ID column, a reservation information column, a reservation available time slot column, and the like. The rental terminal ID column stores identification information of the rental terminal 50. The reservation information column stores information indicating the rental status (reservation status) reserved for the rental terminal 50 in association with the rental terminal ID. Specifically, the reservation information column includes a reservation time slot column and a reservation person information column, the reservation time column stores information on the date and time when the use of the rental terminal 50 is reserved, and the reservation person information column stores information on the date and time when the use of the rental terminal 50 is reserved. Information about a user (reservator) who has reserved the use of the terminal 50 is stored. As the reservation person information, for example, information such as the reservation person's name, information for specifying the reservation person's user terminal 10, etc. can be used. The reservation available time slot column stores the available rental time slots set in the rental terminal 50 in association with the rental terminal ID.

As the rental terminal ID stored in the rental status DB 72b, for example, when information about a new rental terminal 50 is registered in the terminal information DB 72a, the rental terminal ID stored in the terminal information DB 72a is stored by the control unit 71. The reservation information stored in the rental status DB 72b is stored or stored by the control unit 71 when the control unit 71 receives a reservation for renting the rental terminal 50 or a change in the reservation from the user terminal 10, and the reservation or change in reservation is established. Updated. The available reservation time slots stored in the rental status DB 72b are set, for example, by the owner (user) of the rental terminal 50, and are acquired by the control unit 71 via the communication unit 73 and stored or changed. The storage contents of the rental status DB 72b are not limited to the example shown in FIG. 23B, and various information regarding the reservation status of the rental terminal 50 can be stored.

Below, the user of the user terminal 10 makes a reservation for lending the rental terminal 50, downloads his own user model 12M to the reserved rental terminal 50, and executes dialogue and actions based on his own user model 12M on the rental terminal 50. The process will be explained. FIG. 24 is a flowchart showing an example of a reservation processing procedure for lending the lending terminal 50, and FIG. 25 is a schematic diagram showing an example of a screen on the user terminal 10. In FIG. 24, the left side shows the processing performed by the user terminal 10, and the right side shows the processing performed by the terminal management server 70.

The control unit 11 of the user terminal 10 determines, based on the user model 12M, whether there is a place that the user wants to go to that is away from the installation location of the user terminal 10 (S131). By specifying a place that the user wants to go to based on the user model 12M, it is possible to specify a place that the user is likely to want to go to. If it is determined that there is no place that the user wants to go to (S131: NO), the control unit 11 waits. If it is determined that there is a place that the user wants to go to (S131: YES), the control unit 11 requests the terminal management server 70 for a list of rental terminals 50 (S132). Note that the destination information of the terminal management server 70 is stored in the storage unit 12, for example.

When a list of rental terminals 50 is requested from any user terminal 10, the control unit 71 of the terminal management server 70 generates a list of rental terminals 50 that can be rented based on the stored contents of the terminal information DB 72a. It is transmitted to the user terminal 10 (S133). The control unit 11 of the user terminal 10 displays a list as shown in FIG. 25 on the display unit 16 based on the list of rental terminals 50 received from the terminal management server 70 (S134). The list shown in FIG. 25 displays the installation location, terminal resources, rental fee, etc. of each rental terminal 50 for each installation location (for example, each tourist spot). Note that if a terminal name is added to each rental terminal 50, the terminal name of each rental terminal 50 may be displayed in the list. The list of rental terminals 50 also has an input field for inputting a reservation time slot in which the rental terminal 50 is to be reserved. The control unit 11 determines the reservation time slot based on the user model 12M, and inputs the reservation time slot into the input field. For example, the control unit 11 can input the message "When are you going?" to the user model 12M, and determine the reservation time slot based on the output information from the user model 12M. Note that the control unit 11 may receive an input of the reservation time slot into the input field from the user via the input unit 15, and in this case, inputs the received reservation time slot into the input field. Further, information on each rental terminal 50 is displayed in a selectable manner in the list, and the control unit 11 selects the rental terminal 50 corresponding to the place the user wants to go, which is specified based on the user model 12M. Note that the selection of the rental terminal 50 may also be accepted from the user via the input unit 15, and in this case, the control unit 11 selects the rental terminal 50 accepted via the input unit 15. When the control unit 11 of the user terminal 10 selects the rental terminal 50 and determines the reservation time slot, it determines whether to issue an instruction to rent the rental terminal 50 (S135). Note that when the user of the user terminal 10 selects the rental terminal 50 and inputs the reservation time slot, the user confirms the installation location and resources of each rental terminal 50 displayed in the list, and then selects the rental terminal 50 and inputs the reservation time slot. Select the rental terminal 50 you wish to rent, input the reservation time slot, and press the OK button. In this case, the control unit 11 receives an instruction to rent out the rental terminal 50 via the input unit 15 . If the control unit 11 determines not to instruct the rental terminal 50 (S135: NO), that is, if the selection of the rental terminal 50 and the determination of the reservation time slot have not been completed, the control unit 11 updates the list of the rental terminals 50. The display continues (S134). If it is determined that the lending terminal 50 is to be instructed (S135: YES), the control unit 11 requests the terminal management server 70 to reserve the selected lending terminal 50 (S136). Note that the control unit 11 sends a reservation application signal for reserving the rental of the rental terminal 50 to the terminal management server, including information on the rental terminal 50 selected through the list of rental terminals 50 and information on the specified reservation time slot. Send to 70.

When the control unit 71 of the terminal management server 70 receives an application for a rental reservation for any rental terminal 50 from any user terminal 10, the control unit 71 transmits the reservation time for the rental terminal 50 for which the reservation was applied. A band reservation is made (S137). Specifically, the control unit 71 associates the rental terminal ID of the rental terminal 50 for which the reservation was applied, with the reservation time slot for which the reservation was applied and information on the user (reserver) who applied for the reservation (reserver information). ) as reservation information in the rental status DB 72b and performs reservation processing. Note that the control unit 71 determines whether lending is possible during the reserved time slot for which the reservation was applied, and if it is possible, performs the reservation process. Specifically, the control unit 71 determines whether the reserved time slot for which the reservation was applied is within the reservable time slot of the rental terminal 50 for which the reservation was applied, and further determines whether the reserved time slot for which the reservation was applied is within the reservable time slot Determine whether there is a reservation by another user. Then, the control unit 71 determines that the reservation in the reservation time slot is possible if the reservation time slot is within the reservation possible time slot and there is no other user's reservation in the reservation time slot. Note that if it is determined that lending during the reserved time slot is not possible, that is, if the reserved time slot is not within the available reservation time slot, or if another user has already made a reservation for the reserved time slot, the control unit 71 may notify the user terminal 10 that the reservation (rental) cannot be made during the reserved time slot. In this case, the user who has been notified that the reservation cannot be made can change the reservation time slot and apply for the reservation again. After the reservation is completed, the control unit 71 transmits reservation information including information on the reserved rental terminal 50 and the reservation time period to the user terminal 10 (S138), and notifies the user terminal 10 that the reservation is completed. Note that during the reservation process, the control unit 71 generates authentication information such as a password for the rental terminal 50 to authenticate the user terminal 10 when renting out the reserved rental terminal 50, and the authentication information also includes the reservation information. and transmits it to the user terminal 10. Furthermore, the control unit 71 transmits reservation information including reservation person information, reservation time slot, and authentication information to the rental terminal 50.

The control unit 11 of the user terminal 10 receives reservation information including information on the reserved rental terminal 50, reservation time slot, and authentication information from the terminal management server 70, and stores the received reservation information in the storage unit 12 (S139). ), the reservation process ends. Through the above-described process, the user of the user terminal 10 can make a rental reservation for the rental terminal 50 that can be rented out via the terminal management server 70.

26 and 27 are flowcharts illustrating an example of the lending processing procedure of the lending terminal 50. In FIGS. 26 and 27, the left side shows the process performed by the user terminal 10, and the right side shows the process performed by the lending terminal 50. The control unit 11 of the user terminal 10 determines whether the reserved time for renting the rental terminal 50 has arrived (S151), and if it is determined that the reserved time has not arrived (S151: NO), it waits. If it is determined that the reserved time has arrived (S151: YES), the control unit 11 applies for rental to the reserved rental terminal 50 based on the reservation information received from the terminal management server 70 (S152). The control unit 11 transmits rental application information including authentication information included in the reservation information to the rental terminal 50 based on information (for example, destination information) of the rental terminal 50 included in the reservation information. When the control unit 51 of the lending terminal 50 receives the lending application information from the user terminal 10, it authenticates the user terminal 10 based on the authentication information included in the lending application information. Specifically, the control unit 51 determines whether the authentication information included in the rental application information matches the authentication information included in the reservation information received from the terminal management server 70, and if they match, the rental application is The user terminal 10 that sent the information is authenticated as a valid terminal, and if the information does not match, the authentication is not performed.

The control unit 51 determines whether or not the user terminal 10 that sent the rental application information has been authenticated (S153), and if it has not been authenticated (S153: NO), the control unit 51 informs the user that the rental terminal 50 cannot be rented. It is transmitted to the terminal 10 (S154). When the control unit 11 of the user terminal 10 receives a notification from the rental terminal 50 that has applied for rental that the rental is not possible (S155), the control unit 11 ends the process. When the control unit 51 of the rental terminal 50 determines that the user terminal 10 has been authenticated (S153: YES), it transmits to the user terminal 10 that the rental terminal 50 can be rented (S156). When the control unit 11 of the user terminal 10 receives from the rental terminal 50 that has applied for rental that it can be rented, it reads its own user model 12M from the storage unit 12 and transmits it to the rental terminal 50 (S157). Note that the control unit 11 transmits the user model 12M to the rental terminal 50 based on the information on the rental terminal 50 included in the reservation information received from the terminal management server 70.

When the control unit 51 of the lending terminal 50 receives the user model 12M from the user terminal 10, it stores the user model 12M in the storage unit 52 and deploys it so that processing based on the user model 12M can be performed. The control unit 51 then starts processing based on the user model 12M (S158). As a result, the control unit 51 of the lending terminal 50 performs processing similar to the behavior processing shown in FIG. 7 (S159 to S165), and the lending terminal 50 performs processing based on the user model 12M. avatar) can perform various actions using the rental terminal 50.

Specifically, the control unit 51 of the rental terminal 50 determines whether there is a place that the user of the user terminal 10 wants to go to near the installation location of the rental terminal 50, based on the user model 12M received from the user terminal 10. The user then determines whether or not to move depending on whether there is a place he or she wants to go to (S159). Note that movement here includes not only movement on the ground but also movement by flight. If the control unit 51 (behavior processing unit 53b) of the rental terminal 50 determines to move (S159: YES), it moves to the desired location using the movement mechanism 60 (S160), and if it determines not to move (S159). :NO), the process of step S160 is skipped. Next, the control unit 51 determines whether the user wants to take a picture or record based on the user model 12M, and determines whether to take a picture and/or record depending on whether the user wants to take a picture or record ( S161). If it is determined that photographing and/or recording is to be performed (S161: YES), the control unit 51 performs photographing and recording of the surroundings of the rental terminal 50 using the camera 59 and the microphone 58 (S162). Note that the behavior processing unit 53b of the rental terminal 50 identifies a subject that the user is likely to want to photograph based on the user model 12M, and performs photographing and recording of the identified subject. If it is determined that photographing and recording are not to be performed (S161: NO), the control unit 51 skips the process of step S162.

Next, the control unit 51 determines whether the user wants to output audio or video based on the user model 12M, and outputs audio and/or video depending on whether the user wants to output audio or video. It is determined whether or not (S163). When the control unit 51 determines that audio and/or video should be output (S163: YES), it outputs the audio and/or video using the speaker 57 or the display unit 56 (S164), and determines that the audio and/or video should not be output. If it is determined (S163: NO), the process of step S164 is skipped. The control unit 51 determines whether to end the behavior based on the user model 12M (S165). Note that the control unit 51 may determine to end the action when it is determined based on the user model 12M that there is a high possibility that the user wants to end the action (pseudo going out). Further, the control unit 51 may determine that the action is to be ended when the reservation time included in the reservation information received from the terminal management server 70 has expired. Furthermore, the rental terminal 50 may be configured to return to a predetermined standby position and end its operation when the reserved time has expired. In this case, once the reserved time has expired, the rental terminal 50 cannot be used. Note that, for example, when the rental terminal 50 is reserved, or when the user model 12M of the user terminal 10 is deployed to the rental terminal 50, the control unit 71 of the terminal management server 70 charges the rental fee (usage) of the rental terminal 50. Charge processing may also be performed. For example, the control unit 71 of the terminal management server 70 transmits request information requesting payment of the rental fee set in the rental terminal 50 to the user terminal 10, and the control unit 11 of the user terminal 10 sends the rental fee according to the request information. Fee payment processing may also be performed.

If it is determined that the action is not to be completed (S165: NO), the control unit 51 returns to the process of step S159 and repeats the processes of steps S159 to S164. Thereby, the lending terminal 50 can perform actions that are highly likely to be performed by the user of the user terminal 10 based on the user model 12M. That is, since the rental terminal 50 executes processing based on the deployed user model 12M, it performs different actions for each user model 12M, and realizes actions according to the hobbies, preferences, behavioral tendencies, etc. of the user of the user model 12M. can do. When it is determined that the action is to be ended (S165: YES), the control unit 51 of the lending terminal 50 transmits history information of the action performed by the lending terminal 50 based on the user model 12M to the user terminal 10 (S166). Note that the rental terminal 50 uses the camera 59 and the microphone 58 to display images of the surroundings of the rental terminal 50 even when the rental terminal 50 is performing actions such as moving using the moving mechanism 60 and outputting audio and video using the speaker 57 and the display unit 56. and audio are acquired (photographed), and such video and audio data is transmitted to the user terminal 10 as behavior information.

The control unit 11 of the user terminal 10 receives the behavior history information (behavior information, processing result) transmitted from the rental terminal 50, stores the received history information in the behavior history DB 12c (S167), and ends the process. . As a result, history information (processing results) of actions performed on the lending terminal 50 based on the user model 12M is stored in the action history DB 12c of the user terminal 10.

Through the above-described process, the behavior processing based on the user model 12M is performed at the rental terminal 50 to which the user model 12M has been transmitted from the user terminal 10, so that the behavior that the user is likely to perform is executed at the rental terminal 50. I can do it. As a result, instead of the user actually going out, the user's avatar goes out in a pseudo manner, and the video and audio captured while out are stored in the action history DB 12c as history information. By checking this history information (processing results) at a later date, the user can trace the actions taken by the avatar on the rental terminal 50, allowing the user to feel the avatar's experience as if it were his or her own experience, and various You can experience the action in a simulated way.

In this embodiment as well, the same effects as in Embodiment 1 can be obtained. Further, in this embodiment, it is possible to rent out the rental terminal 50 placed at a tourist spot, etc., and the user model 12M of each user is downloaded to the rental terminal 50, and the rental terminal 50 performs processing (dialogue and By performing the action), the lending terminal 50 can realize an action that each user is likely to perform. Thereby, the user can obtain the video, audio, etc. acquired by the rental terminal 50 based on the user model 12M without going to the tourist spot etc., and the user can go to the tourist spot in a pseudo manner. In addition, since the rental terminal 50 from which each user's user model 12M is downloaded is determined based on the user model 12M, the user does not need to do anything to download information about the place he/she wants to go (video and audio). etc.) can be collected. Note that the lending terminal 50 to which the user model 12M has been downloaded may perform not only behavioral processing but also interaction processing based on the user model 12M. In this case, the rental terminal 50 can realize the dialogue that the user of the user model 12M is likely to have at the tourist spot, etc., and the user's avatar can have a dialogue with the local person at the rental terminal 50. Users can interact with local people in a simulated manner.

In this embodiment, for example, if the rental terminal 50 is installed (fixed) in a place where you can see a spectacular view, each user can download (deploy) his or her own user model 12M to the rental terminal 50, A spectacular view can be photographed via the terminal 50. Therefore, each user can go out and travel in a pseudo manner via the rental terminal 50 that performs processing based on the user model 12M.

In this embodiment, between the user terminal 10 and the rental terminal 50 (i.e., the terminals 10, 50 located in close positions) that exist within a predetermined range, different user models 12M are deployed to the respective terminals 10, 50. , the avatars can interact on behalf of the user of the user model 12M. Therefore, for example, when a meeting is held with multiple people and there is an absentee, the absentee's user model 12M can be downloaded to the terminal (rental terminal 50 or user terminal 10) in the conference room in place of the absentee. The avatar of the absent person can join the meeting instead. For example, if there are two absentees in a conference with five people, the user models 12M of the two absentees (for example, the first user model 12M and the second user model 12M) are connected to terminals installed in the conference room. (the first information processing device and the second information processing device). Then, the first information processing device performs processing based on the first user model 12M, and the second information processing device performs processing based on the second user model 12M, so that the avatar of the absentee attends the meeting in place of the absentee. Participants can interact with the three participants (users) and the two absentee avatars. Therefore, even if it is difficult to coordinate the time of all the participants in the meeting, the meeting can proceed smoothly by having the avatars of the absent persons attend the meeting. In addition, historical information (captured images and collected audio data) of meetings held in this way is sent to the user terminals 10 of absentees, so absentees can understand the content of the conference by checking the historical information. You can participate in a virtual meeting.

In each of the embodiments described above, since the unique information of each user is stored in the user model 12M, it is possible to reproduce interactions and actions that are likely to be performed by the user based on the user model 12M. Therefore, even if the user himself/herself is unable to interact due to illness or the like, for example, what the user himself/herself wants to say or do can be predicted by processing based on the user model 12M.

In each of the embodiments described above, the user terminal 10, the second user terminal 10 to which the user model 12M has been deployed, and the lending terminal 50 perform processing based on the user model 12M, so it is highly likely that the user of the user model 12M will perform the processing. Enabling dialogue and action. In addition to this configuration, interactions and actions that are unlikely to be performed by the user of the user model 12M may be identified, such interactions and actions may be performed with a predetermined probability, and historical information may be provided to the user. In addition, the user terminal 10 or the rental terminal 50 identifies what is popular or popular at the installation location of its own terminal, executes what is popular or what is trending based on the user model 12M, and Information may be provided to the user. With such a configuration, the user can experience a simulated conversation and behavior with a conversation partner that he or she would not normally do, and new discoveries can be expected from the user.

In each of the embodiments described above, the user terminal 10 and the rental terminal 50 may be configured such that, for example, the arrangement positions (arrangement heights) of the display sections 16 and 56 can be changed. FIG. 28 is a schematic diagram showing a modified example of the configuration of the user terminal 10 and the rental terminal 50. The example shown in FIG. 28A shows a modification of the user terminal 10 shown in FIG. 1, and as shown on the left side of FIG. It may have an adjustable height adjustment mechanism 10e. Further, as shown on the right side of FIG. 28A, the user terminal 10 may include a height adjustment mechanism 10e that can adjust the vertical positions of the head 10c and the body 10d. Furthermore, as shown in FIG. 28B, the user terminal 10 and the rental terminal 50 may be configured to be able to adjust the vertical position of the display units 16, 56 with respect to the moving mechanisms 20, 60 that are in contact with the floor or the ground. good. In the user terminal 10 and rental terminal 50 having such configurations, when performing processing based on the own user model 12M or the deployed user model 12M, the display unit 16 is , 56 can be changed in the vertical direction. As a result, the display units 16 and 56 can be placed at a height that matches the height of the user of the user model 12M, and the avatars displayed on the display units 16 and 56 can be interacted with as being closer to the user. can. Note that the user model 12M of each user also learns information regarding the physical characteristics of each user.

Furthermore, in addition to the configuration in which the heights of the display units 16 and 56 can be changed, the width of the body part 10d in the second user's user terminal 10 shown in FIG. 1 may be changed, for example. For example, by configuring the torso portion 10d to have a structure that allows switching between a plurality of widths, and switching the width to match the physique of the user of the user model 12M, the user corresponding to the avatar displayed on the display units 16, 56 can be A user terminal 10 that can be imagined more easily can be realized. Further, the width of the torso portion 10d can be set in advance according to the type of user, such as a female type, a male type, a rakugo performer type, a thin type, a chubby type, etc., and the width can be changed to match the user type of the user model 12M. It may be configured as follows. Furthermore, the user terminal 10 may be configured to be able to change the color of its outer surface. For example, by providing a liquid crystal display on the outer surface of the user terminal 10 or the outer surface of the body portion 10d, the outer surface can be colored in a color that matches the user's preference or image of the user model 12M. In this case as well, it is possible to realize the user terminal 10 that allows the user to better imagine the user corresponding to the displayed avatar.

In each of the embodiments described above, when a certain user model 12M (including the lending model 42M) is deployed to a certain user terminal 10 (including the lending terminal 50), the deployed user is displayed on the display unit 16 of the user terminal 10. It may be configured to display information regarding the user of the model 12M (for example, the user's name, nickname, etc.). For example, when the first user's user model 12M is deployed on the second user's user terminal 10, the first user's information may be displayed on the display unit 16 of the second user's user terminal 10. With such a configuration, a user who is interacting with an avatar via the user terminal 10 can easily understand whose avatar is displayed on the display unit 16 of the user terminal 10. Therefore, the user who interacts with the avatar can be expected to feel closer to the avatar. Further, for example, in a meeting where there are a plurality of absentees, it is possible to easily know whose avatar is the one who attended in place of each absentee, so that the meeting can be held smoothly.

In each of the embodiments described above, the unique information of each user is stored in the user model 12M, but it may also be stored in a table or DB, for example. In this case, for example, the control unit 11 of the user terminal 10 extracts user-specific information from the content of the interaction with the user by the interaction processing unit 13a, and stores it in a table or DB. Even in such a configuration, the same processing as in each of the embodiments described above is possible, and the same effects can be obtained.

The embodiments disclosed herein are illustrative in all respects and should not be considered restrictive. The scope of the present invention is indicated by the claims rather than the above-mentioned meaning, and is intended to include meanings equivalent to the claims and all changes within the scope.

10 User terminal (computer)
11 Control unit 12 Storage unit 13 Processing unit 15 Input unit 16 Display unit 17 Speaker 18 Microphone 19 Camera 20 Movement mechanism 40 Model management server 50 Rental terminal 70 Terminal management server 12M User model 42M Rental model 13a, 53a Dialogue processing unit 13b, 53b behavior processing unit

Claims (12)

storing a first learning model learned based on interaction history information of the first user in a storage unit;
accepting a designation for an information processing device capable of deploying the first learning model;
deploying the first learning model to the specified information processing device;
The information processing device to which the first learning model is deployed operates based on the deployed first learning model,
When a second learning model learned based on interaction history information of a second user different from the first user is deployed to the information processing device, the first learning model by the information processing device Actions based on this are prohibited.
Information processing method. The information processing method according to claim 1 , wherein when the second learning model deployed to the information processing device is undeployed, the information processing device resumes operation based on the first learning model. The information processing according to claim 1 or 2 , wherein a second learning model learned based on interaction history information of a second user different from the first user is deployed to the information processing device of the first user. Method. accepting a selection for one of a plurality of learning models for lending learned based on interaction history information of each of a plurality of users different from the first user;
The information processing method according to any one of claims 1 to 3 , wherein the received learning model for lending is deployed to the first user's information processing device. storing a first learning model learned based on interaction history information of the first user in a storage unit;
accepting a designation for an information processing device capable of deploying the first learning model;
deploying the first learning model to the specified information processing device;
The information processing device to which the first learning model is deployed operates based on the deployed first learning model,
The learning model uses teacher data including dialogue history information and emotions, and dialogue information and behavior information corresponding to the dialogue history information and emotions, and when the dialogue history information and emotions are input, the learning model It has been trained to output dialogue information and behavior information according to history information and emotions,
receiving interaction information from the first user through the first user's information processing device;
analyzing the emotions of the first user through the first user's information processing device based on the image of the first user acquired from the imaging unit;
By inputting the received dialogue information from the first user and the analyzed emotions of the first user into a learning model of a user different from the first user, a dialogue output from the learning model of the different user. outputting information or behavior information through the first user's information processing device;
Information processing method. storing a first learning model learned based on interaction history information of the first user in a storage unit;
accepting a designation for an information processing device capable of deploying the first learning model;
deploying the first learning model to the specified information processing device;
The information processing device to which the first learning model is deployed operates based on the deployed first learning model,
accepting a selection for one of a plurality of learning models for lending learned based on interaction history information of each of a plurality of users different from the first user;
deploying the received learning model for lending to the first user's information processing device;
The learning model for lending is adjusted according to the time for deploying the learning model for lending to the information processing device, or the number of times the information processing device executes processing based on the deployed learning model for lending. Charge rental fee
Information processing method. storing a first learning model learned based on interaction history information of the first user in a storage unit;
accepting a designation for an information processing device capable of deploying the first learning model;
deploying the first learning model to the specified information processing device;
The information processing device to which the first learning model is deployed is configured to
It operates based on the learning model of
accepting a selection for one of a plurality of learning models for lending learned based on interaction history information of each of a plurality of users different from the first user;
deploying the received learning model for lending to the first user's information processing device;
For each of the learning models for lending, the number of devices that can be simultaneously deployed is set,
A rental fee is charged according to the number of devices that can be simultaneously deployed, which is set for each of the learning models for rental.
Information processing method. storing a first learning model learned based on interaction history information of the first user in a storage unit;
accepting a designation for an information processing device capable of deploying the first learning model;
deploying the first learning model to the specified information processing device;
The information processing device to which the first learning model is deployed operates based on the deployed first learning model,
The information processing device to which the first learning model is deployed selects one of a plurality of distributed information processing devices for lending based on the first learning model,
deploying the first learning model to the selected rental information processing device;
causing the lending information processing device to output processing results executed based on the first learning model to the first user's information processing device;
Information processing method. storing a first learning model learned based on interaction history information of the first user in a storage unit;
accepting a designation for an information processing device capable of deploying the first learning model;
deploying the first learning model to the specified information processing device;
The information processing device to which the first learning model is deployed operates based on the deployed first learning model,
The information processing device includes a display section and a height adjustment mechanism that changes the arrangement height of the display section,
Changing the height adjustment mechanism of the display unit through the information processing device based on the physical characteristics of the user corresponding to the learning model deployed to the information processing device.
Information processing method. Accepts selection of one of a plurality of learning models learned based on interaction history information of each of a plurality of users,
receiving the selected learning model;
storing the received learning model in a storage unit;
operates based on the stored learning model ;
A program that causes a computer to execute a process of charging a rental fee according to the number of devices that can be rented out at the same time, which is set for the learning model . An information processing device capable of executing the program according to claim 10 . a storage unit that stores a plurality of learning models learned based on interaction history information of each of the plurality of users;
a deployment unit that deploys the requested learning model to the information processing device when one of the plurality of learning models is requested from the information processing device ;
The number of devices that can be simultaneously deployed is set for each of the plurality of learning models,
a charging unit that charges a rental fee according to the number of devices that can be simultaneously deployed, which is set in the learning model deployed to the information processing device;
An information processing device further comprising :

Images