JP2022176415A - Information processing device and program - Google Patents

Abstract

To facilitate a conversation among a plurality of users.SOLUTION: In an information processing system 1, a host device 30 which acquires a first face image and a first voice of a first user, acquires a second face image and a second voice of a second user, generates a first prediction face image of the first user U1 on the basis of the first face image and generates a second prediction face image of the second user U2 on the basis of the second face image, comprises: means which presents the first prediction face image to the second user via a second client device; means which presents the second prediction face image to the first user via a first client device; means which generates a message that urges at least one of the first user and the second user to utter; means which presents the message to at least one of the first user and the second user; means which presents the second voice according to the message to the first user via the first client device; and means which presents the first voice according to the message to the second user via the second client device.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information processing apparatus and program.

In recent years, attention has been focused on techniques for promoting conversations between users.
For example, Patent Literature 1 discloses a technique for promoting conversation based on factual information (preference information, similarity, and action history).

Re-table 2012-017786

The actual conversation develops interactively according to the combination of reactions during the conversation of each participant in the conversation (for example, the expression of the conversation partner and the meaning of the words uttered by the conversation partner).

However, the technique of Patent Document 1 refers only to factual information that has been established before the conversation, so it does not affect the development of the conversation according to the combination of reactions during the conversation. Therefore, it does not contribute to promotion of conversation among multiple users.

It is an object of the present invention to facilitate multi-user conversations.

One aspect of the present invention is
An information processing device that supports conversation of a plurality of users,
comprising means for acquiring a first face image of the first user;
comprising means for acquiring a first voice of the first user;
comprising means for acquiring a second face image of the second user;
comprising means for acquiring a second voice of the second user;
means for generating a first predicted face image of the first user at a first designated age that is older or younger than the age of the first user, based on the first face image;
means for generating a second predicted face image of the second user at a second specified age that is older or younger than the age of the second user, based on the second face image;
means for presenting the first predicted face image to the second user via a second client device;
means for presenting the second predicted face image to the first user via the first client device;
means for generating a message prompting at least one of the first user and the second user to speak;
means for presenting the message to at least one of the first user and the second user;
means for presenting a second voice corresponding to the message to the first user via the first client device;
means for presenting a first voice corresponding to the message to the second user via the second client device;
It is an information processing device.

ADVANTAGE OF THE INVENTION According to this invention, a conversation of several users can be promoted.

1 is a block diagram showing the configuration of an information processing system according to an embodiment; FIG. 2 is a functional block diagram of the first client device of FIG. 1; FIG. 3 is a functional block diagram of a second client device of FIG. 2; FIG. 2 is a functional block diagram of the host device of FIG. 1; FIG. 2 is a diagram showing an example of the form of the information processing system of FIG. 1; FIG. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram of the outline of this embodiment; 4 is a sequence diagram of information processing according to the embodiment; FIG. 8 is a diagram showing an example of a screen displayed in the information processing of FIG. 7; FIG. 8 is a diagram showing an example of a screen displayed in the information processing of FIG. 7; FIG. 8 is a diagram showing an example of a screen displayed in the information processing of FIG. 7; FIG. 8 is a diagram showing an example of a screen displayed in the information processing of FIG. 7; FIG. 8 is a diagram showing an example of a screen displayed in the information processing of FIG. 7; FIG.

An embodiment of the present invention will be described in detail below with reference to the drawings. In the drawings for describing the embodiments, in principle, the same constituent elements are denoted by the same reference numerals, and repeated description thereof will be omitted.

(1) Configuration of information processing system The configuration of the information processing system will be described. FIG. 1 is a block diagram showing the configuration of the information processing system of this embodiment.

As shown in FIG. 1 , the information processing system 1 includes a first client device 10 , a second client device 20 and a host device 30 .
The host device 30 is communicably connected to the first client device 10 and the second client device 20 .

The first client device 10 is a computer (an example of an “information processing device”) that transmits a request to the host device 30 . The first client device 10 is, for example, a smart phone, a tablet terminal, or a personal computer.

The host device 30 is a computer (an example of an “information processing device”) that provides the first client device 10 with a response in response to a request sent from the first client device 10 . The host device 30 is, for example, a web host device.

(1-1) Configuration of First Client Device The configuration of the first client device 10 will be described. FIG. 2 is a functional block diagram of the first client device of FIG.

As shown in FIG. 2, the first client device 10 includes a storage device 11, a processor 12, an input/output interface 13, a communication interface 14, a camera 15, a display 16, a microphone 17, a speaker 18, Prepare.

Storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of ROM (Read Only Memory), RAM (Random Access Memory), and storage (eg, flash memory or hard disk).

Programs include, for example, the following programs.
・OS (Operating System) program ・Application (for example, web browser) program that executes information processing

The data includes, for example, the following data.
・Databases referenced in information processing ・Data obtained by executing information processing (that is, execution results of information processing)

The processor 12 is configured to implement the functions of the first client device 10 by running programs stored in the storage device 11 . The processor 12 is, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

The input/output interface 13 is configured to acquire user instructions from an input device connected to the first client device 10 and to output information to an output device connected to the first client device 10 .
Input devices are, for example, hard keys, keyboards, pointing devices, touch panels, or combinations thereof. Input devices also include camera 15 and display 16 .
Output devices include display 16 and speakers 18 .

The communication interface 14 is configured to control communication between the first client device 10 and the host device 30 .

Camera 15 is configured to acquire images.

Display 16 is configured to display images generated by processor 12 .

Microphone 17 is configured to pick up sound.

Speaker 18 is configured to output audio generated by processor 12 .

(1-2) Configuration of Second Client Device The configuration of the second client device 20 will be described. 3 is a functional block diagram of the second client device of FIG. 2. FIG.

As shown in FIG. 3, the second client device 20 includes a storage device 21, a processor 22, an input/output interface 23, a communication interface 24, a camera 25, a display 26, a microphone 27, a speaker 28, Prepare.

Storage device 21 is configured to store programs and data. The storage device 21 is, for example, a combination of ROM, RAM and storage.

Programs include, for example, the following programs.
- OS program - Application (for example, web browser) program that executes information processing

The data includes, for example, the following data.
・Databases referenced in information processing ・Data obtained by executing information processing (that is, execution results of information processing)

The processor 22 is configured to implement the functions of the second client device 20 by running programs stored in the storage device 21 . Processor 22 is, for example, a CPU, ASIC, FPGA, or a combination thereof.

The input/output interface 23 is configured to acquire user instructions from an input device connected to the second client device 20 and to output information to an output device connected to the second client device 20 .
Input devices are, for example, hard keys, keyboards, pointing devices, touch panels, or combinations thereof. Input devices also include a camera 25 and a microphone 27 .
Output devices include display 26 and speakers 28 .

Communication interface 24 is configured to control communication between second client device 20 and host device 30 .

Camera 25 is configured to acquire images.

Display 26 is configured to display images generated by processor 22 .

A microphone 27 is configured to pick up sound.

Speaker 28 is configured to output audio generated by processor 22 .

(1-3) Configuration of Host Device The configuration of the host device 30 will be described. FIG. 4 is a functional block diagram of the host device of FIG. 1;

As shown in FIG. 4, the host device 30 includes a storage device 31, a processor 32, an input/output interface 33, and a communication interface .

Storage device 31 is configured to store programs and data. Storage device 31 is, for example, a combination of ROM, RAM, and storage (eg, flash memory or hard disk).

Programs include, for example, the following programs.
・OS program ・Application program that executes information processing

The data includes, for example, the following data.
・Databases referenced in information processing ・Execution results of information processing

The processor 32 is configured to implement the functions of the host device 30 by activating programs stored in the storage device 31 . Processor 32 is, for example, a CPU, ASIC, FPGA, or a combination thereof.

The input/output interface 33 is configured to acquire user instructions from an input device connected to the host device 30 and to output information to an output device connected to the host device 30 .
Input devices are, for example, keyboards, pointing devices, touch panels, or combinations thereof.
An output device is, for example, a display. Output devices also include printers.

The communication interface 34 is configured to control communication between the host device 30 and the first client device 10 and the second client device 20 .

(1-4) Form of Information Processing System An example of the form of the information processing system 1 of the present embodiment will be described. FIG. 5 is a diagram showing an example of the form of the information processing system of FIG.

As shown in FIG. 5, the information processing system 1 includes a first space SP1 to a third space SP3. The third space SP is arranged between the first space SP1 and the second space SP2.

The first space SP1 is a space into which the first user U1 enters. The first user U1 can enter the first space SP1 through the entrance ENT1. A camera 15, a display 16, a microphone 17, and a speaker 18 are arranged in the first space SP1. A shielding portion (for example, a wall) W1 is arranged at the boundary between the first space SP1 and the third space SP. A display 16 is placed on the shield W1 so as to face the X-direction. The first user U1 can visually recognize the image displayed on the display 16 by directing the line of sight in the X+ direction.

The second space SP2 is a space into which the second user U2 enters. The second user U2 can enter the second space SP2 through the entrance ENT2. A camera 25, a display 26, a microphone 27, and a speaker 28 are arranged in the second space SP2. A shield W2 is arranged at the boundary between the second space SP2 and the third space SP. The display 26 is arranged on the shield W2 so as to face the X+ direction. The second user U2 can visually recognize the image displayed on the display 26 by directing the line of sight in the X-direction.

That is, the shield W1 is configured to visually isolate the first space SP1 and the second space SP2. Therefore, the first user U1 and the second user U2 cannot directly recognize each other's voice and expression.

A host device 30 is arranged in the third space SP.

(2) Outline of Embodiment An outline of the present embodiment will be described. FIG. 6 is an explanatory diagram of the outline of this embodiment.

The host device 30 in FIG. 1 is an information processing device that supports conversations of a plurality of users.

As shown in FIG. 6, the host device 30 has the following means.
-Means for acquiring the first facial image (a plurality of still images or moving images) of the first user -Means for acquiring the first voice of the first user -Second facial image of the second user (a plurality of still images or moving images) Means for acquiring the second voice of the second user Based on the first face image, the first voice of the first user at the first designated age that has aged or decreased from the age of the first user Means for generating a predicted face image Means for generating a second predicted face image of the second user at a second designated age that is older or younger than the age of the second user, based on the second face image First prediction Means for presenting the face image to the second user via the second client device 20 Means for presenting the second predicted face image to the first user via the first client device 10 First user and second user Means for generating a message prompting at least one person to speak Means for presenting a message to at least one of the first user and the second user A second voice corresponding to the message is generated by the first client device means for presenting to the first user via 10;
- Means for presenting the first voice corresponding to the message to the second user via the second client device 20

(3) Information processing Information processing according to this embodiment will be described. FIG. 7 is a sequence diagram of information processing according to this embodiment. 8 is a diagram showing an example of a screen displayed in the information processing of FIG. 7. FIG. 9 is a diagram showing an example of a screen displayed in the information processing of FIG. 7. FIG. FIG. 10 is a diagram showing a screen example displayed in the information processing of FIG. FIG. 11 is a diagram showing a screen example displayed in the information processing of FIG. FIG. 12 is a diagram showing a screen example displayed in the information processing of FIG.

As shown in FIG. 7, the first client device 10 receives user information (S110).
Specifically, processor 12 displays screen P10 ( FIG. 8 ) on display 16 .

The screen P10 includes field objects F10a to F10e and an operation object B10.

The field object F10a is an object that receives user input regarding the name of the first user.

The field object F10b is an object that receives user input regarding the relationship between the first user and the second user. Relationships include, for example, at least one of the following:
・Family (for example, at least one of parent and child, husband and wife, siblings, and relatives)
Relationships (i.e. couples)
・Friends ・Colleagues at work

The field object F10c is an object that receives user input regarding the duration of the relationship between the first user and the second user (hereinafter referred to as "relationship period"). The relationship period is, for example, any of the following.
・Less than 5 years ・More than 5 years

The field object F10d is an object that receives user input regarding the gender of the first user.

The field object F10e is an object that receives user input regarding the date of birth of the first user.

The operation object B10 is an object that receives a user instruction for confirming user input to the field objects F10a to F10e.

When the first user gives user inputs to the field objects F10a to F10e and operates the operation object B10, the camera 15 acquires an image of the first user's face (hereinafter referred to as "first user image").
The processor 12 transmits to the host device 30 a combination of the information input to the field objects F10a to F10e and the first user image (hereinafter referred to as “first user information”).

The second client device 20 receives user information (S120).
Specifically, the processor 22 displays the screen P20 (FIG. 8) on the display 26. FIG.

The screen P20 includes field objects F20a to F20e and an operation object B20.
The field object F20a is an object that receives user input regarding the name of the second user.
The field object F20b is an object that receives user input regarding the relationship between the first user and the second user.
The field object F20c is an object that receives user input regarding the relationship period between the first user and the second user.
The field object F20d is an object that receives user input regarding the gender of the second user.
The field object F20e is an object that receives user input regarding the date of birth of the second user.
The operation object B20 is an object that receives a user instruction for confirming user input to the field objects F20a to F20e.

When the second user gives user inputs to the field objects F20a to F20e and operates the operation object B20, the camera 25 acquires an image of the second user's face (hereinafter referred to as a "second user image").
The processor 22 transmits to the host device 30 a combination of the information input to the field objects F20a to F20e and the second user image (hereinafter referred to as “second user information”).

After steps S110 and S120, the host device 30 transfers user information (S130).
Specifically, the processor 32 acquires first user information from the first client device 10 .
The processor 32 acquires second user information from the second client device 20 .
Processor 32 calculates the value of the current age of the first user (hereinafter referred to as "first current age") based on the date of birth included in the first user information.
Processor 32 calculates the value of the second user's current age (hereinafter referred to as "second current age") based on the date of birth included in the second user information.
Processor 32 transmits the first user information to second client device 20 .
Processor 32 transmits the second user information to first client device 10 .

After step S131, the first client device 10 executes reception of the specified age (S111).
Specifically, processor 12 displays screen P11 ( FIG. 9 ) on display 16 .

The screen P11 includes a display object A11, an image object IMG11, field objects F11a to F11b, and an operation object B11.
The display object A11 displays a message prompting the first user to have a conversation (for example, a message "Please select the age of the other party."). This message is a predetermined message.
Image object IMG11 is the second user image included in the second user information transmitted from host device 30 .
The field object F11a is an object that receives user input for designating the fictitious age of the first user (hereinafter referred to as "first specified age"). The initial value of the field object F11a is the value of the first current age.
The field object F11b is an object that accepts user input for specifying the fictitious age of the second user (hereinafter referred to as "second specified age"). The initial value of the field object F11b is the second current age value.
The operation object B11a is an object that receives a user instruction for confirming user input of the field objects F11a to F11b.

When the first user inputs an arbitrary value to the field object F11b and operates the operation object B11a, the processor 12 transmits the value given to the field object F11b to the host device 30. FIG.

After step S131, the second client device 20 executes acceptance of the specified age (S121).
Specifically, processor 22 displays screen P21 ( FIG. 9 ) on display 26 .

The screen P21 includes a display object A21, an image object IMG21, field objects F21a-F21b, and operation objects B21a-B21b.
The display object A21 displays a message prompting the second user to have a conversation (for example, a message "Please select the age of the other party."). This message is a predetermined message.
Image object IMG21 is the first user image included in the first user information transmitted from host device 30 .
The field object F21a is an object that receives user input for designating the first designated age. The initial value of the field object F21a is the value of the first current age. The value of field object F21a is synchronized with the value of field object F11a.
The field object F21b is an object that receives user input for specifying the second specified age. The initial value of the field object F21b is a value (that is, the current age of the first user) calculated from the date of birth entered in the field object F20e (FIG. 8). The value of field object F21b is synchronized with the value of field object F11b.
The operation object B21a is an object that receives a user instruction for confirming user input of the field objects F21a to F21b.
The operation object B21b is an object that receives a user instruction to end the conversation.

When the second user inputs an arbitrary value (for example, the fictitious age value of the first user) into the field object F21b and operates the operation object B21a, the processor 12 outputs the value given to the field object F21b ( That is, the fictitious age of the first user (hereinafter referred to as “first specified age”) is transmitted to the host device 30 .

After steps S111 and S121, the host device 30 executes morphing (S131).
Specifically, a conversion model is stored in the storage device 31 . The transformation model defines a correlation between a combination of a person's current age and a person's fictitious age, and a transformation filter for transforming the person's facial image.
The processor 32 obtains from the first client device 10 the value of the second specified age given to the field object F11b.
The processor 32 obtains from the second client device 20 the value of the first specified age given to the field object F21a.
The processor 32 provides the value of the first current age obtained in step S130 and the value of the first specified age obtained in step S111 to the conversion model to obtain the combination of the first current age and the first specified age. Select a conversion filter that corresponds to .
The processor 32 applies the selected transform filter to the first user image to generate an image of the face of the first user corresponding to the first specified age (hereinafter referred to as "first predicted face image").
The processor 32 provides the value of the second current age obtained in step S130 and the value of the second specified age obtained in step S121 to the conversion model, thereby obtaining a combination of the second current age and the second specified age. Select a conversion filter that corresponds to .
The processor 32 applies the selected conversion filter to the second user image to generate a face image of the second user corresponding to the second specified age (hereinafter referred to as a "second predicted face image").

After step S131, the host device 30 executes message transmission (S132).

Specifically, the processor 32 transmits the first message data to the first client device 10 . The first message data includes the following information.
・Second predicted face image ・Predetermined initial message

Processor 32 transmits the second message data to second client device 20 . The second message data includes the following information.
・First predicted face image ・Predetermined initial message

After step S132, the first client device 10 executes reception of conversation information (S112).

Specifically, the processor 12 displays a screen P12a (FIG. 10) on the display 16. FIG. Screen P12a corresponds to the first message data.

The screen P12a includes a display object A12a, an image object IMG12, field objects F11a to F11b, and an operation object B11.
The initial message included in the first message data is displayed in the display object A12a.
Image object IMG12 is the second predicted face image included in the first message data.

While the first user sees the message displayed on the display object A12a and the image object IMG12 (that is, the image of the face of the second user corresponding to the second designated age), the second user output from the speaker 18 while listening to the voice of the second user, the microphone 17 acquires the first user's voice (hereinafter referred to as "first uttered voice").
The camera 15 acquires a facial image of the first user during conversation (hereinafter referred to as "first facial image").
Processor 12 transmits conversation information to host device 30 . Conversation information includes the following information.
・First face image acquired by camera 15 ・First utterance voice acquired by microphone 17

After step S132, the second client device 20 executes reception of conversation information (S122).

Specifically, the processor 22 displays a screen P22a (FIG. 10) on the display 26. FIG. Screen P22a corresponds to the second message data.

The screen P22a includes a display object A22a, an image object IMG22, field objects F21a-F21b, and operation objects B21a-B21b.
The initial message included in the second message data is displayed in the display object A22a.
Image object IMG22 is the first predicted face image included in the second message data.

While the second user sees the message displayed on the display object A22a and the image object IMG22 (that is, the image of the face of the first user corresponding to the first designated age), the first user output from the speaker 28 while listening to the voice of the first user, the microphone 27 acquires the second user's voice (hereinafter referred to as "second uttered voice").
The camera 25 acquires a facial image of the second user during conversation (hereinafter referred to as "second facial image").
Processor 22 transmits conversation information to host device 30 . Conversation information includes the following information.
・Second face image acquired by camera 25 ・Second speech voice acquired by microphone 27

After steps S112 and S122, the host device 30 performs emotion analysis (S133).

In the first example of step S133, the storage device 31 stores an emotion analysis model that defines the correlation between the person's facial expression and the person's emotion.
The processor 32 identifies the emotion of the first user by inputting the first uttered voice transmitted in step S112 into the emotion analysis model.
The processor 32 identifies the emotion of the second user by inputting the second utterance voice transmitted in step S122 to the emotion analysis model.

In the second example of step S133, the storage device 31 stores an emotion analysis model that defines the correlation between the person's uttered voice and the person's emotion.
The processor 32 identifies the emotion of the first user by inputting the first uttered voice transmitted in step S112 into the emotion analysis model.
The processor 32 identifies the emotion of the second user by inputting the second utterance voice transmitted in step S122 to the emotion analysis model.

After step S133, the host device 30 generates a message (S134).
Specifically, the storage device 31 stores a message generation model. The message generation model defines the correlation between a person's emotion and a message for prompting the next conversation.

In the first example of step S133, the processor 32 generates a message to be presented to the first user by inputting the emotion of the first user obtained in step S133 into the message generation model, and presents the message to the first user. and transmitting the first message data containing the first message data to the first client device 10 .
The processor 32 generates a message to be presented to the second user by inputting the emotion of the second user obtained in step S133 into the message generation model, and transmits the second message data including the message to the second user. Send to the client device 20 .

In a second example of step S133, the processor 32 inputs the emotion of the first user and the emotion of the second user obtained in step S133 into the message generation model to present the first user and the second user. A message is generated and message data including the message is transmitted to the first client device 10 and the second client device 20 .

After step S134, the first client device 10 executes conversation information reception (S112) again.
Specifically, the processor 12 displays a screen P12b (FIG. 11) on the display 16. FIG. Screen P12b corresponds to the message data transmitted from host device 30 in step S132.

The screen P12b includes a display object A12b, an image object IMG12, field objects F11a-F11b, and an operation object B11.
The display object A12b displays the message sent from the host device 30 in step S133.

While the first user sees the message displayed on the display object A12b and the image object IMG12 (that is, the image of the face of the second user corresponding to the second specified age), the second user output from the speaker 18 When speaking to the second user while listening to the voice of , the microphone 17 acquires the first uttered voice.
Processor 12 transmits conversation information to host device 30 . Conversation information includes the following information.
・First face image acquired by camera 15 ・First utterance voice acquired by microphone 17

After step S134, the second client device 20 executes conversation information reception (S122) again.
Specifically, the processor 22 displays the screen P22b (FIG. 11) on the display 26. FIG. Screen P22b corresponds to the message data transmitted from host device 30 in step S134.

The screen P22b includes a display object A22b, an image object IMG22, field objects F21a-F21b, and operation objects B21a-B21b.
A message included in the second message data is displayed in the display object A22b.

While the second user sees the message displayed on the display object A22b and the image object IMG22 (that is, the image of the face of the first user corresponding to the first specified age), the first user output from the speaker 28 When speaking to the first user while listening to the voice of , the microphone 27 acquires the second uttered voice.
Processor 22 transmits conversation information to host device 30 . Conversation information includes the following information.
・Second face image acquired by camera 25 ・Second speech voice acquired by microphone 27

When the conversation end condition is satisfied, the first client device 10 acquires a commemorative image (S113).
Specifically, when the end condition of the conversation is satisfied, the processor 12 displays the screen P13 (FIG. 12) on the display.
Conversation termination conditions are, for example, at least one of the following.
- A predetermined time (eg, 10 minutes) has passed since the start of the conversation (eg, the end of steps S111 and S121) - A message has been presented a predetermined number of times (that is, step 134 has been executed) - First A user instruction (for example, at least one of a voice instruction and an operation on an object displayed on the screen) to end the conversation is given by the user or the second user

Screen P13 includes an image object IMG13 and an operation object B13. Image object IMG13 is a preview image acquired by camera 25 at the time of execution of step S113.
The operation object B13 is an object that receives a user instruction to acquire a preview image corresponding to the image object IMG13 as a commemorative image of the second user (hereinafter referred to as "second commemorative image").

When the first user operates the operation object B13, the processor 12 transmits the second commemorative image to the host device 30 and transmits the end notification of step S113 to the second client device 20. FIG.

After step S113, the second client device 20 acquires a commemorative image (S123).
Specifically, when the first user operates the operation object B13, the processor 12 displays the screen P23 (FIG. 12) on the display.

Screen P23 includes an image object IMG23 and an operation object B23. Image object IMG23 is a preview image acquired by camera 15 at the time of execution of step S123.
The operation object B23 is an object that receives a user instruction to acquire a preview image corresponding to the image object IMG23 as a commemorative image of the first user (hereinafter referred to as "first commemorative image").

When the second user operates the operation object B23, the processor 22 transmits the first commemorative image and the end notification of step S123 to the host device 30. FIG.

After step S123, the host device 30 saves the log data (S135).
Specifically, the processor 32 associates and stores the following information in the storage device 31 .
- The first user information transmitted from the first client device 10 in step S110 - The second user information transmitted from the second client device 20 in step S120 - The output request data transmitted from the first client device 10 in step S113 The first predicted face image and the first voice included in the second predicted face image and the second voice included in the output request data transmitted from the second client device 20 in step S123 From execution of steps S111 and S121 to step S113 and at least one execution time of S123 (that is, the elapsed time of the conversation)
- The second commemorative image transmitted from the first client device 10 in step S113 - The first commemorative image transmitted from the second client device 20 in step S123

The processor 32 generates a group commemorative image by synthesizing the first commemorative image and the second commemorative image.
The processor 32 transmits the first commemorative image, the second commemorative image, and the group commemorative image to the first client device 10 and the second client device 20 .

After step S135, the first client device 10 executes the output request (S114).
Specifically, processor 12 displays screen P14 ( FIG. 12 ) on display 16 .

The screen P14 includes an image object IMG14 and operation objects B14a to B14b.
The image object IMG14 is a group commemorative image.
The operation object B14a is an object that receives a user's instruction to share the group commemorative image on the network service. Network services are, for example, at least one of social network services, chat services, and image sharing services.
The operation object B14b is an object that receives a user's instruction to print a commemorative image of the gathering.

When the first user operates the operation object B14a, the processor 12 transmits a user instruction for sharing the commemorative image of the gathering to the host device.

After step S135, the second client device 20 executes the output request (S124).
Specifically, when the second user operates the operation object B21b (FIGS. 10 and 11), the processor 22 displays the screen P24 (FIG. 12) on the display.

The screen P24 includes an image object IMG24 and operation objects B24a to B24b.
The image object IMG24 is a group commemorative image.
The operation object B24a is an object that receives a user's instruction to share the group commemorative image on the network service. Network services are, for example, at least one of social network services, chat services, and image sharing services.
The operation object B24b is an object that receives a user's instruction to print a commemorative image of the gathering.

When the second user operates the operation object B24b, the processor 22 transmits to the host device 30 a user instruction for printing a commemorative image of the gathering.

After step S135, the host device 30 outputs a commemorative image (S136).
Specifically, the processor 32 transmits the gathering commemorative image to the web server that provides the network service in accordance with the user instruction transmitted from the first client device 10 in step S113. As a result, the group commemorative image is shared with users other than the first user (for example, at least one of the second user and the third party) via the network service.
The processor 32 transmits the gathering commemorative image to the printer in accordance with the user instruction transmitted from the second client device 20 in step S123. As a result, the gathering commemorative image is printed via the printer.

According to this embodiment, each user is presented with a predicted face image of a conversation partner and a message for promoting the conversation during conversation. As a result, it is possible to promote a conversation according to the information in the conversation.

(4) Modification A modification of the present embodiment will be described.

(4-1) Modification 1
Modification 1 will be described. Modification 1 is an example of presenting a message according to at least one of the context of the first voice and the context of the second voice to at least one of the first user and the second user.

In step S134 (FIG. 7), the host device 30 of Modification 1 generates a message according to at least one of the contexts of the first voice and the second voice.
Specifically, the message generation model defines the correlation between the context of the speech and the message for prompting the next conversation.

The processor 32 of the first example of the modified example 1 generates a message to be presented to the first user by inputting the first voice obtained in step S133 to the message generation model, and generates a first speech including the message. 1 message data is sent to the first client device 10;
The processor 32 generates a message to be presented to the second user by inputting the second voice obtained in step S133 to the message generation model, and transmits second message data including the message to the second client device. 20.

The processor 32 of the second example of the modified example 1 inputs the combination of the first voice and the second voice obtained in step S133 to the message generation model to present the message to the first user and the second user. and send message data including the message to the first client device 10 and the second client device 20 .

According to Modification 1, a message prompting conversation is presented according to the context of the voice uttered by at least one of the first user and the second user. Thereby, the conversation between the first user and the second user can be facilitated.

(4-2) Modification 2
Modification 2 will be described. Modification 2 is an example in which at least one of the first user and the second user is presented with a message according to at least one of the environment of the first user and the environment of the second user.

In step S134 (FIG. 7), the host device 30 of Modification 2 collects information about the first user's environment (hereinafter referred to as "first environment information") and information about the second user's environment (hereinafter referred to as "second environment information"). ) to generate a message according to at least one of
Specifically, the message generation model prescribes the correlation between the environment and the message for prompting the next conversation.
The information processing system 1 includes sensors that acquire environmental information. Environmental information includes at least one of the following:
・Temperature information about the temperature of the first space SP1 ・Temperature information about the temperature of the second space SP2 ・Humidity information about the humidity of the first space SP1 ・Humidity information about the temperature of the second space SP2

The processor 32 of the first example of the modified example 2 generates a message to be presented to the first user by inputting the first environmental information obtained by the sensor into the message generation model, and generates the first environment information including the message. 1 message data is sent to the first client device 10;
The processor 32 generates a message to be presented to the second user by inputting the second environmental information obtained by the sensor into the message generation model, and transmits second message data including the message to the second client device. 20.

The processor 32 of the second example of the modified example 2 inputs the combination of the first environmental information and the second environmental information obtained by the sensor into the message generation model to generate messages to be presented to the first user and the second user. and send message data including the message to the first client device 10 and the second client device 20 .

In the information processing system 1 of the third example of modification 2, at least one of the first space SP1 and the second space SP2 is outdoors.
Environmental information includes at least one of the following:
・Temperature information about the temperature of the first space SP1 ・Temperature information about the temperature of the second space SP2

The host device 30 may acquire at least one of the first environmental information and the second environmental information from a device other than the sensor (for example, a server that provides environmental information) instead of the sensor.

According to Modification 2, a message prompting conversation is presented according to the environment of at least one of the first user and the second user. Thereby, the conversation between the first user and the second user can be facilitated.

(4-3) Modification 3
Modification 3 will be described. Modification 3 is an example of presenting a message according to at least one of the attributes of the first user and the attributes of the second user to at least one of the first user and the second user.

In step S134 (FIG. 7), the host device 30 of Modification 3 sends information about the attribute of the first user (hereinafter referred to as "first attribute information") and information about the attribute of the second user (hereinafter referred to as "second attribute information"). ) to generate a message according to at least one of
Specifically, the message generation model defines the correlation between attributes and messages for prompting the next conversation. Attributes include at least one of the following:
- Current age of the first user - Current age of the second user - Gender of the first user - Gender of the second user

The processor 32 of the first example of the modification 3 inputs the attribute of the first user included in the first user information transmitted from the first client device 10 in step S110 to the message generation model, thereby A message to be presented is generated, and first message data including the message is sent to the first client device 10 .
The processor 32 generates a message to be presented to the second user by inputting the attributes of the second user included in the second user information transmitted from the second client device 20 in step S120 into the message generation model, Then, it transmits second message data including the message to the second client device 20 .

The processor 32 of the second example of the modified example 3 acquires the attribute of the first user included in the first user information transmitted from the first client device 10 in step S110, and the attribute of the first user transmitted from the second client device 20 in step S120. By inputting a combination of attributes of the second user included in the second user information into the message generation model, a message to be presented to the first user and the second user is generated, and message data including the message is generated It transmits to the first client device 10 and the second client device 20 .

According to Modification 3, a message prompting conversation is presented according to the attribute of at least one of the first user and the second user. Thereby, the conversation between the first user and the second user can be facilitated.

(4-4) Modification 4
Modification 4 will be described. Modification 4 is an example in which a message corresponding to the relationship between the first user and the second user is presented to at least one of the first user and the second user.

In step S134 (FIG. 7), the host device 30 of Modification 4 generates a message according to the relationship between the first user and the second user.
Specifically, the message generation model defines the correlation between the relationships of multiple users and messages for prompting the next conversation.
The processor 32 converts the relationship included in at least one of the first user information transmitted from the first client device 10 in step S110 and the second user information transmitted from the second client device 20 in step S120 into the message generation model. By inputting, a message to be presented to the first user and the second user is generated, and message data including the message is transmitted to the first client device 10 and the second client device 20 .

According to Modification 4, a message prompting conversation is presented according to the relationship between the first user and the second user. Thereby, the conversation between the first user and the second user can be facilitated.

(4-5) Modification 5
Modification 5 will be described. Modification 5 is an example of presenting a message according to at least one of the first specified age and the second specified age to at least one of the first user and the second user.

In step S134 (FIG. 7), the host device 30 of Modification 5 generates a message according to at least one of the first specified age and the second specified age.
Specifically, the message generation model defines a correlation between age and a message for prompting the next conversation.

The processor 32 of the first example of modification 5 calculates the value of the first designated age transmitted from the first client device 10 in step S111 and the value of the second designated age transmitted from the second client device 20 in step S121. Determine the magnitude relationship of values.
The processor 32 generates a message to be presented to the first user and the second user by inputting the magnitude relation into the message generation model, and transmits message data including the message to the first client device 10 and the second client device 10 . Send to the client device 20 .

The processor 32 of the second example of the modified example 5 stores the value of the first designated age transmitted from the first client device 10 in step S111 and the value of the second designated age transmitted from the second client device 20 in step S121. Calculate the difference in values (hereinafter referred to as the "specified age difference").
The processor 32 generates a message to be presented to the first user and the second user by inputting the specified age difference into the message generation model, and transmits message data including the message to the first client device 10 and the second user. 2 to the client device 20;

According to Modification 5, a message prompting conversation is presented according to the specified age of at least one of the first user and the second user. Thereby, the conversation between the first user and the second user can be facilitated.

(4-6) Modification 6
Modification 6 will be described. Modification 6 is an example of reproducing the replay videos of the first user and the second user.

The processor 12 or 22 of Modification 6 transmits replay request data for requesting reproduction of the replay video to the host device 30 according to the user's instruction.

The processor 32 generates a replay video by synthesizing the following information stored in the storage device 31 according to the replay request data transmitted from the first client device 10 or the second client device 20.
・First face image ・First predicted face image ・First voice ・Second face image ・Second predicted face image ・Second voice

Processor 32 transmits the replay video to first client device 10 or second client device 20 .

The processor 12 or 22 reproduces the replay video transmitted from the first client device 10 or the second client device 20. FIG. As a result, the display 16 or 26 displays the image of the replay moving image. The sound of the replay video is output from the speaker 18 or 28 .

According to Modified Example 6, a replay video that is a recording of conversations between the first user and the second user is provided. This makes it possible to give the user motivation for conversation.

Note that in Modification 6, the processor 12 or 22 may store the file of the replay movie in the host device 30 .
If the host device 30 is a web server, the host device 30 assigns a URL (Uniform Resource Locator) to the replay movie file.
The host device 30 is used by the user who made the browsing request in response to the browsing request of the first user, the second user, and a third user (that is, a third party) different from the first user and the second user. A replay video is presented via a client device that The viewing request is, for example, an access to an address (for example, a URL (Uniform Resource Locator)) of a web server in which a replay movie file is stored.
Thereby, at least one of the first user to the third user can be provided with a replay video after the end of the conversation.

(5) Summary of this embodiment This embodiment will be summarized.

The first aspect of this embodiment is
An information processing device (for example, a host device 30) that supports conversations of a plurality of users,
Means for acquiring the first face image of the first user (processor 32 for executing the process of step S130),
Means for acquiring the first voice of the first user (processor 32 for executing the process of step S133),
Means for acquiring the second face image of the second user (processor 32 for executing the process of step S130),
A means for acquiring the second voice of the second user (processor 32 for executing the process of step S133),
Means (processor 32 for executing the process of step S131) for generating, based on the first facial image, a first predicted facial image of the first user at a first specified age that is older or younger than the age of the first user prepared,
means for generating a second predicted face image of the second user at a second designated age that is older or younger than the age of the second user (processor 32 that executes the process of step S131) based on the second face image; prepared,
means for presenting the first predicted face image to the second user via the second client device 20 (processor 32 for executing the process of step S132);
means for presenting the second predicted face image to the first user via the first client device 10 (processor 32 for executing the process of step S132);
Means for generating a message prompting at least one of the first user and the second user to speak (processor 32 for executing the process of step S132);
Means for presenting a message to at least one of the first user and the second user (processor 32 for executing the process of step S132);
means for presenting a second voice corresponding to the message to the first user via the first client device 10 (processor 32 for executing the process of step S133);
means for presenting the first voice corresponding to the message to the second user via the second client device 20 (processor 32 for executing the process of step S133);
It is an information processing device.

According to the first aspect, the first predicted image corresponding to the first specified age of the first user is presented to the second user, and the second predicted image corresponding to the second specified age of the second user is presented to the first user. and a message prompting at least one of the first user and the second user to speak. Thereby, the conversation between the first user and the second user can be promoted.

A second aspect of the present embodiment is
Based on the first facial image, means for analyzing the emotion of the first user (processor 32 for executing the process of step S133),
Based on the second face image, means for analyzing the emotion of the second user (processor 32 for executing the process of step S133),
the means for generating a message generates a message according to at least one of the emotion of the first user and the emotion of the second user;
It is an information processing device.

According to the second aspect, at least one of the first user and the second user is presented with a message according to at least one of the first user's emotion and the second user's emotion. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The third aspect of this embodiment is
Based on the first voice, means for analyzing the emotion of the first user (processor 32 for executing the process of step S133);
Based on the second voice, means for analyzing the emotion of the second user (processor 32 for executing the process of step S133);
the means for generating a message generates a message according to at least one of the emotion of the first user and the emotion of the second user;
It is an information processing device.

According to the third aspect, the message is presented according to at least one of the emotions of the first user and the emotions of the second user. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The fourth aspect of this embodiment is
Based on the first speech, means for analyzing the context of the first speech (for example, the processor 32 of Variation 1);
Based on the second speech, means for analyzing the context of the second speech (for example, the processor 32 of the first variant);
the means for generating a message generates a message responsive to at least one of the context of the first audio and the context of the second audio;
It is an information processing device.

According to the fourth aspect, the message is presented according to the context of the voice uttered by at least one of the first user and the second user. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The fifth aspect of this embodiment is
means for obtaining first environmental information about the first user's environment;
means for obtaining second environmental information about the second user's environment;
the means for generating a message generates a message according to at least one of the first environmental information and the second environmental information;
It is an information processing device.

According to the fifth aspect, the message is presented according to the environment of at least one of the first user and the second user. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

A sixth aspect of the present embodiment is
the first environmental information and the second environmental information include at least one of temperature information, humidity information, and ultraviolet information;
It is an information processing device.

According to the sixth aspect, a message is presented according to at least one of the temperature, humidity, and ultraviolet rays in the first user's environment and the temperature, humidity, and ultraviolet rays in the second user's environment. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The seventh aspect of this embodiment is
means for generating a message generates a message according to a combination of attributes of the first user and attributes of the second user;
It is an information processing device.

According to the seventh aspect, a message is presented according to the combination of the attributes of the first user and the attributes of the second user. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The eighth aspect of the present embodiment is
the attributes include at least one of age and gender of the user;
It is an information processing device.

According to the seventh aspect, a message is presented according to a combination of at least one of the first user's age and sex and at least one of the second user's age and sex. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

A ninth aspect of the present embodiment is
the means for generating a message generates a message according to the relationship between the first user and the second user;
It is an information processing device.

According to the ninth aspect, a message is presented according to the relationship between the first user and the second user. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

A tenth aspect of the present embodiment is
The means for generating a message generates a message according to a combination of the first designated age and the second designated age;
It is an information processing device.

According to the tenth aspect, a message is presented according to the combination of the first specified age of the first user and the second specified age of the second user. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The eleventh aspect of this embodiment is
The means for generating the message presents a message according to at least one of the magnitude relationship between the first specified age and the second specified age and the specified age difference.
It is an information processing device.

According to the eleventh aspect, a message is presented according to at least one of the size relationship between the first specified age of the first user and the second specified age of the second user and the specified age difference. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The twelfth aspect of this embodiment is
A shielding part W1 arranged at a position that blocks the first user's line of sight to the second user and the second user's line of sight to the first user,
It is an information processing device.

According to the twelfth aspect, the shield W1 constructs an environment in which the first user and the second user cannot directly perceive each other's facial expressions. The first user and the second user are guided to have a conversation while viewing the second predicted image and the first predicted image, respectively. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The thirteenth aspect of this embodiment is
Log data including a combination of a first facial image, a second facial image, a first predicted facial image, a second predicted facial image, a first voice, and a second voice are stored in association with the elapsed time of the conversation. means (e.g., processor 32 for performing step S135);
Means for generating a replay video using log data associated with elapsed time (for example, processor 32 of modification 6);
means for presenting to at least one of the first user, the second user, and a third user other than the first user and the second user (e.g., the processor 32 of Variation 6);
It is an information processing device.

According to the thirteenth aspect, a replay movie is provided. This allows at least one of the first to third users to view the state of the conversation after the conversation ends.

The fourteenth aspect of this embodiment is
means for acquiring a first commemorative image of the first user and a second commemorative image of the second user after the end of the conversation (e.g., the processor 32 executing step S135);
It is an information processing device.

According to the fourteenth aspect, a combination of the first commemorative image of the first user and the second commemorative image of the second user is obtained. As a result, a commemorative image of the conversation can be saved.

The fifteenth aspect of this embodiment is
means for presenting the first predicted face image and the second predicted image to the first user and the second user via the first client device 10 and the second client device 20 (eg, the processor 32 executing step S130); ,
It is an information processing device.

According to the fifteenth aspect, the combination of the first predicted image and the second predicted image is presented to the first user and the second user. Thereby, conversation of a 1st user and a 2nd user can be promoted more.

The sixteenth aspect of this embodiment is
A program for causing a computer (for example, at least one of processors 12, 22, and 32) to function as each of the means described above.

(6) Other Modifications Other modifications will be described.

The storage device 11 may be connected to the first client device 10 via the network NW. The storage device 31 may be connected to the host device 30 via the network NW.

Each step of the host device 30 for the above information processing can also be executed by the first client device 10 and the second client device 20 . In this case, the first client device 10 and the second client device 20 function as standalone information processing devices without sending requests to the host device 30 .

The host device 30 may be a server (eg, web server). In this case, the first client device 10 and the second client device 20 are connected to the server via the Internet, for example.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the above embodiments. Also, the above embodiments can be modified and modified in various ways without departing from the gist of the present invention. Also, the above embodiments and modifications can be combined.

1: information processing system 10 : first client device 11 : storage device 12 : processor 13 : input/output interface 14 : communication interface 15 : camera 16 : display 17 : microphone 18 : speaker 20 : second client device 21 : storage device 22 : Processor 23 : Input/output interface 24 : Communication interface 25 : Camera 26 : Display 27 : Microphone 28 : Speaker 30 : Host device 31 : Storage device 32 : Processor 33 : Input/output interface 34 : Communication interface

Claims (16)

An information processing device that supports conversation of a plurality of users,
comprising means for acquiring a first face image of the first user;
comprising means for acquiring a first voice of the first user;
comprising means for acquiring a second face image of the second user;
comprising means for acquiring a second voice of the second user;
means for generating a first predicted face image of the first user at a first designated age that is older or younger than the age of the first user, based on the first face image;
means for generating a second predicted face image of the second user at a second specified age that is older or younger than the age of the second user, based on the second face image;
means for presenting the first predicted face image to the second user via a second client device;
means for presenting the second predicted face image to the first user via the first client device;
means for generating a message prompting at least one of the first user and the second user to speak;
means for presenting the message to at least one of the first user and the second user;
means for presenting a second voice corresponding to the message to the first user via the first client device;
means for presenting a first voice corresponding to the message to the second user via the second client device;
Information processing equipment. comprising means for analyzing the emotion of the first user based on the first facial image;
comprising means for analyzing the emotion of the second user based on the second face image;
the means for generating the message generates a message according to at least one of the emotion of the first user and the emotion of the second user;
The information processing device according to claim 1 . comprising means for analyzing the emotion of the first user based on the first voice;
comprising means for analyzing the emotion of the second user based on the second voice;
the means for generating the message generates a message according to at least one of the emotion of the first user and the emotion of the second user;
The information processing apparatus according to claim 1 or 2. means for analyzing the context of the first speech based on the first speech;
means for analyzing the context of the second speech based on the second speech;
the means for generating a message generates a message according to at least one of the context of the first sound and the context of the second sound;
The information processing apparatus according to any one of claims 1 to 3. comprising means for obtaining first environmental information relating to the first user's environment;
comprising means for acquiring second environmental information relating to the second user's environment;
the means for generating the message generates a message according to at least one of the first environmental information and the second environmental information;
The information processing apparatus according to any one of claims 1 to 4. wherein the first environmental information and the second environmental information include at least one of temperature information, humidity information, and ultraviolet information;
The information processing device according to claim 5 . The means for generating the message generates a message according to a combination of attributes of the first user and attributes of the second user.
The information processing apparatus according to any one of claims 1 to 6. the attributes include at least one of age and gender of the user;
The information processing apparatus according to any one of claims 1 to 7. the means for generating the message generates a message according to the relationship between the first user and the second user;
The information processing apparatus according to any one of claims 1 to 8. The means for generating the message generates a message according to the combination of the first specified age and the second specified age.
The information processing apparatus according to any one of claims 1 to 9. The means for generating the message presents a message according to at least one of the magnitude relationship between the first specified age and the second specified age and the specified age difference.
The information processing apparatus according to claim 10. A shielding unit arranged at a position that blocks the line of sight of the first user with respect to the second user and the line of sight of the second user with respect to the first user,
The information processing apparatus according to any one of claims 1 to 11. Log data including a combination of the first face image, the second face image, the first predicted face image, the second predicted face image, the first voice, and the second voice, and the elapsed time of the conversation , and means for storing in association with
comprising means for generating a replay video using log data associated with the elapsed time;
Means for presenting to at least one of the first user, the second user, and a third user other than the first user and the second user;
The information processing apparatus according to any one of claims 1 to 12. After the end of the conversation, means for acquiring a first commemorative image of the first user and a second commemorative image of the second user,
The information processing apparatus according to any one of claims 1 to 13. means for presenting the first predicted face image and the second predicted image to the first user and the second user via the first client device and the second client device;
The information processing apparatus according to any one of claims 1 to 14. A program for causing a computer to function as each means according to any one of claims 1 to 15.

Images