JP7371770B2 - Avatar control program, avatar control method, and information processing device - Google Patents

Description

The present invention relates to an avatar control program, an avatar control method, and an information processing device.

In recent years, there have been automated response systems that utilize technology called chatbots. A chatbot is a computer system that uses artificial intelligence to interact and exchange messages with humans. In addition, in a conversation with a chatbot, if specific conditions are met, the conversation may be switched from chatbot support to manned conversation.

The prior art involves acquiring information about the user, estimating the user's state from the acquired information, generating information for making the character take action and/or speaking based on the estimation result, and using the generated information. There is something to output. In addition, the information processing terminal receives an instruction to start a conversation with the chatbot as well as the user's attribute information acquired by the information processing terminal, and uses script information determined based on the user's attribute information to There are techniques to start a conversation with a chatbot in . It also determines whether the phone call should be transferred from the bot to the human based on analysis of the real-time conversation, and in response to the determination that the phone call should be transferred, transfers the phone call from the bot to the human. There is technology.

Japanese Patent Application Publication No. 2019-139170 Patent No. 6648876 Special table 2019-522914 publication

However, in the conventional technology, when an operator responds to a user through an avatar displayed on a digital board or the like, there is a problem in that the movements of the avatar become stereotypical.

In one aspect, the present invention aims to control the output of an avatar in accordance with the external environment.

In one embodiment, an avatar that moves in response to an operation by a first user on a first information processing terminal is output to a second information processing terminal, and a second user on the second information processing terminal is detected. In this case, when information prompting an operation corresponding to the avatar is output to the first information processing terminal, and an operation corresponding to the outputted information is received at the first information processing terminal, the received operation and the An avatar control program is provided that controls output of the avatar based on external environment information detected by the second information processing terminal.

According to one aspect of the present invention, it is possible to control the output of the avatar in accordance with the external environment.

FIG. 1 is an explanatory diagram showing an example of a system configuration of a conversation control system 100. FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus 101. FIG. 3 is a block diagram showing an example of the hardware configuration of the information processing terminals 102 and 103. FIG. 4 is an explanatory diagram showing an example of the storage contents of the option management table 120. FIG. 5 is an explanatory diagram showing an example of the stored contents of the avatar management table 130. FIG. 6 is an explanatory diagram showing an example of the stored contents of the script table 140. FIG. 7 is a block diagram showing an example of the functional configuration of the information processing device 101. As shown in FIG. FIG. 8 is an explanatory diagram showing one embodiment of the conversation control system 100. FIG. 9 is an explanatory diagram (part 1) showing an example of the talk screen displayed on the second information processing terminal 103. FIG. 10 is an explanatory diagram showing an example of the operator support screen displayed on the first information processing terminal 102. FIG. 11 is an explanatory diagram (part 2) showing an example of the talk screen displayed on the second information processing terminal 103. FIG. 12 is a flowchart (part 1) illustrating an example of the avatar control processing procedure of the information processing apparatus 101. FIG. 13 is a flowchart (Part 2) illustrating an example of the avatar control processing procedure of the information processing apparatus 101. FIG. 14 is a flowchart illustrating an example of a specific procedure for changing the avatar output.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an avatar control program, an avatar control method, and an information processing apparatus according to the present invention will be described in detail below with reference to the drawings.

(Embodiment)
First, a system configuration example of the conversation control system 100 including the information processing device 101 according to the embodiment will be described. The conversation control system 100 is applied, for example, to services such as store guidance, career counseling in companies, etc., using automatic responses by chatbots and manned responses by operators.

FIG. 1 is an explanatory diagram showing an example of a system configuration of a conversation control system 100. In FIG. 1, conversation control system 100 includes an information processing device 101, a first information processing terminal 102, and a second information processing terminal 103. In the conversation control system 100, an information processing device 101, a first information processing terminal 102, and a second information processing terminal 103 are connected via a wired or wireless network 110. The network 110 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like. The conversation control system 100 includes, for example, a plurality of first information processing terminals 102 and a plurality of second information processing terminals 103.

Here, the information processing device 101 is a computer that controls the output of the avatar. The avatar is a character that moves in response to the first user's operation on the first information processing terminal 102, and is displayed as an alter ego of the operator in a virtual space on the network, for example. The information processing device 101 is, for example, a server.

The information processing device 101 includes, for example, an option management table 120, an avatar management table 130, and a script table 140. The stored contents of the option management table 120, the avatar management table 130, and the script table 140 will be described later using FIGS. 4 to 6.

The first information processing terminal 102 is a computer used by a first user, and is capable of operating an avatar displayed on the second information processing terminal 103. The first user is, for example, an operator who responds to users who use the second information processing terminal 103. At the first information processing terminal 102, the first user can have a conversation with the second user through the avatar displayed on the second information processing terminal 103. The first information processing terminal 102 is, for example, a PC (Personal Computer), a tablet PC, or the like.

The second information processing terminal 103 is a computer used by a second user. The second information processing terminal 103 is, for example, a digital board installed in a store or facility and used by an unspecified number of people. Further, the second information processing terminal 103 may be a smartphone, a tablet PC, etc. rented out at a store or facility, or owned by the user. The facility is, for example, a commercial facility, a corporate facility, a public facility, or the like.

Further, the second information processing terminal 103 may be available for use by the second user when having a chatbot conversation. On the second information processing terminal 103, the second user interacts with the chatbot operator, searching for a desired answer or using the service, by inputting a message or selecting options, for example. You can do it. The answers are, for example, FAQs (Frequently Asked Questions).

FAQ means "frequently asked questions" and is a collection of questions and answers to anticipated questions. The question may ask how to solve some problem, or may be a spoken question. The answers include answers that show how to solve the posed problem and answers to spoken questions.

Although not shown, the information processing device 101 includes, for example, an FAQ master, a chat log DB (Database), and the like. The FAQ master stores FAQs. The chat log DB stores chat logs. The chat log is a conversation history regarding conversations with the second user. The chat log is stored in the chat log DB in association with, for example, a room ID. According to the room ID, a series of conversations with a chatbot or an operator can be specified.

Here, by having the operator respond to the user through an avatar, it can be expected that the user will be able to receive a more detailed response than an automatic response provided by a chatbot. On the other hand, when an operator responds to a user through an avatar, if the avatar's movements become routine, it is difficult to give the user the feeling that he or she is talking to a human being.

For this reason, it is conceivable to have the avatar make movements such as waving, nodding, or smiling in response to the operator's operations. Thereby, the movement of the avatar can be changed depending on the content of the response to the user. However, the preferred movement of the avatar may differ depending on the time, location, weather, etc.

For example, it may be preferable to move brightly and with a larger motion during daytime hours than during nighttime hours. Also, depending on the location, it may be preferable to move in a polite and calm manner, or it may be preferable to move in a bright and loud motion for children.

Therefore, in the embodiment, when an operator responds to a user through an avatar, the movement of the avatar is controlled by taking into account not only the content of the response to the user but also the external environment such as time, location, and weather. , we will explain an avatar control method that realizes a wide variety of avatar movements.

(Example of hardware configuration of information processing device 101)
Next, an example of the hardware configuration of the information processing device 101 will be described.

FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus 101. In FIG. 2, the information processing device 101 includes a CPU (Central Processing Unit) 201, a memory 202, a disk drive 203, a disk 204, a communication I/F (Interface) 205, and a portable recording medium I/F 206. , and a portable recording medium 207. Further, each component is connected to each other by a bus 200.

Here, the CPU 201 is in charge of overall control of the information processing apparatus 101. CPU 201 may have multiple cores. The memory 202 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash ROM, and the like. Specifically, for example, a flash ROM stores an OS (Operating System) program, a ROM stores an application program, and a RAM is used as a work area for the CPU 201. The program stored in the memory 202 is loaded into the CPU 201 and causes the CPU 201 to execute the coded processing.

The disk drive 203 controls reading/writing of data to/from the disk 204 under the control of the CPU 201 . The disk 204 stores data written under the control of the disk drive 203. Examples of the disk 204 include a magnetic disk and an optical disk.

The communication I/F 205 is connected to a network 110 through a communication line, and is connected to an external computer (for example, the first information processing terminal 102 and the second information processing terminal 103 shown in FIG. 1) via the network 110. Ru. The communication I/F 205 serves as an interface between the network 110 and the inside of the device, and controls data input/output from an external computer. For example, a modem or a LAN adapter can be used as the communication I/F 205.

The portable recording medium I/F 206 controls reading/writing of data to/from the portable recording medium 207 under the control of the CPU 201 . The portable recording medium 207 stores data written under the control of the portable recording medium I/F 206. Examples of the portable recording medium 207 include a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disk), and a USB (Universal Serial Bus) memory.

Note that the information processing device 101 may include, for example, an SSD (Solid State Drive), an input device, a display, etc. in addition to the above-described components. Further, the information processing apparatus 101 does not need to include, for example, the disk drive 203, the disk 204, the portable recording medium I/F 206, and the portable recording medium 207 among the above-mentioned components.

(Example of hardware configuration of first information processing terminal 102 and second information processing terminal 103)
Next, an example of the hardware configuration of the first information processing terminal 102 and the second information processing terminal 103 will be described. However, here, the first information processing terminal 102 and the second information processing terminal 103 are referred to as "information processing terminals 102, 103."

FIG. 3 is a block diagram showing an example of the hardware configuration of the information processing terminals 102 and 103. In FIG. 3, the information processing terminals 102 and 103 include a CPU 301, a memory 302, a display 303, an input device 304, a communication I/F 305, a camera 306, a speaker 307, a microphone 308, and a GPS (Global Positioning System) unit 309. Further, each component is connected to each other by a bus 300.

Here, the CPU 301 is in charge of overall control of the information processing terminals 102 and 103. CPU 301 may have multiple cores. The memory 302 is a storage unit including, for example, ROM, RAM, and flash ROM. Specifically, for example, a flash ROM or ROM stores various programs, and a RAM is used as a work area for the CPU 301. The program stored in the memory 302 is loaded into the CPU 301 and causes the CPU 301 to execute the coded processing.

The display 303 is a display device that displays data such as cursors, icons, tool boxes, documents, images, and functional information. As the display 303, for example, a liquid crystal display, an organic EL (electroluminescence) display, or the like can be used.

The input device 304 has keys for inputting characters, numbers, various instructions, etc., and inputs data. The input device 304 may be a touch panel type input pad, a numeric keypad, or the like, or may be a keyboard, a mouse, or the like.

The communication I/F 305 is connected to the network 110 through a communication line, and is connected to an external computer (for example, the information processing apparatus 101, the first information processing terminal 102, and the second information processing terminal shown in FIG. 1) via the network 110. 103). The communication I/F 305 serves as an interface between the network 110 and the inside of the device itself, and controls input/output of data from external devices.

The camera 306 is a photographing device that photographs images (still images or moving images) and outputs image data. The speaker 307 converts the electrical signal into sound and outputs the sound. Microphone 308 receives audio and converts it into an electrical signal.

The GPS unit 309 receives radio waves from GPS satellites and outputs position information of the information processing terminals 102 and 103. The location information of the information processing terminals 102 and 103 is, for example, information specifying one point on the earth, such as latitude and longitude. Further, as the satellite, for example, a satellite of the quasi-zenith satellite system may be used.

In addition to the above-mentioned components, the information processing terminals 102 and 103 also include, for example, an HDD (Hard Disk Drive), an SSD, a short-range wireless communication I/F, a portable recording medium I/F, a portable recording medium, etc. You may decide to have The second information processing terminal 103 does not need to have the GPS unit 309.

(Memory contents of various tables 120, 130, 140)
Next, the stored contents of the various tables 120, 130, and 140 included in the information processing apparatus 101 will be explained using FIGS. 4 to 6. The various tables 120, 130, and 140 are realized by storage devices such as the memory 202 and the disk 204 shown in FIG. 2, for example.

FIG. 4 is an explanatory diagram showing an example of the storage contents of the option management table 120. In FIG. 4, the option management table 120 has fields for conversation content, option 1, option 2, option 3, option 4, and option 5, and by setting information in each field, option management information (for example, option Management information 400-1 to 400-3) are stored as records.

The conversation content is information that specifies the content of the conversation with the operator through the avatar or the conversation with the chatbot. For example, the conversation content indicates options selected, information input, etc. in a conversation with an operator through an avatar or a conversation with a chatbot.

Options 1 to 5 are options corresponding to avatars, and are prepared in association with conversation content. Options 1 to 5 are presented to an operator who can operate the avatar, for example. That is, the options presented to the operator are changed depending on the content of the conversation with the operator through the avatar or the chatbot.

For example, option management information 400-1 indicates options ch1 to ch4 corresponding to conversation content C1. In addition, although "5" is given as an example and is demonstrated here as the maximum number of options corresponding to the content of a conversation, it is not limited to this. For example, the maximum number of options may be less than 5, or may be 6 or more.

FIG. 5 is an explanatory diagram showing an example of the stored contents of the avatar management table 130. In FIG. 5, the avatar management table 130 has fields for operator ID, options, face data, external environment, and body data, and by setting information in each field, avatar management information (for example, avatar management information 500- 1,500-2) is stored as a record.

Here, the options correspond to options (for example, options 1 to 5 shown in FIG. 4) that are selectably displayed on the first information processing terminal 102. Face data is information that specifies the appearance of the avatar's face. For example, face data is prepared in association with options. The external environment represents the external environment of the second information processing terminal 103 where the avatar is displayed. For example, the external environment represents time (time of day, time zone, date, etc.), location, season, weather, temperature, humidity, and the like.

Body data is information that specifies the appearance of the avatar's body parts. For example, body data is prepared in association with the external environment. The operator ID is an identifier that uniquely identifies the operator corresponding to the option. For example, the operator ID may be an identifier that uniquely identifies the first information processing terminal 102 used by the operator.

For example, avatar management information 500-1 indicates face data fc1 and operator OP1 corresponding to option ch1, and body data bd1 corresponding to external environment E1.

FIG. 6 is an explanatory diagram showing an example of the stored contents of the script table 140. In FIG. 6, the script table 140 has user attribute and script fields, and by setting information in each field, script management information (for example, script management information 600-1 to 600-3) is stored as a record. do.

Here, the user attribute is information representing the user's characteristics. For example, user attributes include age, generation, gender, language used, clothing, presence or absence of companions, and the like. The script is information (scenario) that defines the conversation flow of the chatbot, and is prepared in association with user attributes. The script is information that defines chat processing.

The talk process is a process for realizing the operation (behavior) of the chatbot. For example, the talk process includes the process of speaking, the process of displaying options, the process of selecting options according to user operations, the process of performing procedures according to the selected options and input information, and the process of terminating the talk process. Including.

That is, in the script table 140, scripts corresponding to user attributes are prepared in advance. For example, frequently selected FAQs tend to change depending on the age group. For example, users in their 30s and 40s are more likely to choose FAQs related to career advancement than users in other age groups. Frequently selected FAQs also tend to change depending on the language. For example, when taking a course, people who speak a language other than Japanese (eg, foreigners) are more likely to choose FAQs regarding Japanese courses than people who speak Japanese.

For example, script management information 600-1 indicates script SP1 corresponding to user attribute UA1. Note that frequently selected FAQs may change depending on the time or location. For this reason, scripts corresponding to combinations of user attributes and external environments may be prepared in advance in the script table 140.

(Functional configuration example of information processing device 101)
FIG. 7 is a block diagram showing an example of the functional configuration of the information processing device 101. As shown in FIG. In FIG. 7, the information processing device 101 includes a first output control section 701, a second output control section 702, a communication section 703, a conversation control section 704, and a storage section 710. Specifically, for example, the first output control unit 701 to conversation control unit 704 cause the CPU 201 to execute a program stored in a storage device such as the memory 202, the disk 204, or the portable recording medium 207 shown in FIG. This function is realized by the communication I/F 205 or by the communication I/F 205. The processing results of each functional unit are stored in a storage device such as the memory 202 or the disk 204, for example. The storage unit 710 is realized by a storage device such as the memory 202 and the disk 204, for example. Specifically, for example, the storage unit 710 stores the option management table 120 shown in FIG. 4, the avatar management table 130 shown in FIG. 5, the script table 140 shown in FIG. 6, the FAQ master, the chat log DB, etc. do.

The first output control unit 701 outputs an avatar that moves according to the first user's operation on the first information processing terminal 102 to the second information processing terminal 103 . Here, the first user is, for example, an operator who responds to users who use the second information processing terminal 103. The avatar is, for example, a character displayed as an alter ego of the operator.

Specifically, for example, the first output control unit 701 outputs an avatar to the second information processing terminal 103 in response to the activation of the second information processing terminal 103. Further, the first output control unit 701 causes the second information processing terminal 103 to display the avatar in response to the first user's activation operation using the input device 304 (see FIG. 3) of the second information processing terminal 103. You may decide to output .

Further, the first output control unit 701 outputs the avatar's message and options to the second information processing terminal 103, for example, in response to the first user's operation on the first information processing terminal 102. The avatar's messages and options may be displayed, for example, on the display 303 (see FIG. 3), or may be output as audio from the speaker 307 (see FIG. 3).

Furthermore, when the second information processing terminal 103 is multilingual, when the first user inputs a message in Japanese, the first output control unit 701 converts the input message into second information. The information may be translated into a foreign language selected by the processing terminal 103 and output to the second information processing terminal 103. Furthermore, when the second user inputs a message in a foreign language, the first output control unit 701 translates the input message into Japanese and outputs it to the first information processing terminal 102. You can also do this.

Further, the first output control unit 701 may change the facial expression of the avatar in accordance with, for example, the movement of the operator's (speaker's) face detected by a motion sensor. Note that the type of avatar to be output when the second information processing terminal 103 is activated is set in advance, for example.

When the second output control unit 702 detects the second user at the second information processing terminal 103 , it outputs information prompting an operation corresponding to the avatar to the first information processing terminal 102 . Here, the second user is a user who uses the second information processing terminal 103. For example, in the case where the second information processing terminal 103 is a digital board installed in a commercial facility or a store, the second user is, for example, a customer.

Moreover, taking as an example the case where the second information processing terminal 103 is a digital board installed in a corporate facility, the second user is, for example, an employee. Moreover, taking as an example the case where the second information processing terminal 103 is a digital board installed in a public facility, the second user is, for example, a tourist.

The second user is detected, for example, at the second information processing terminal 103. Specifically, for example, the second information processing terminal 103 detects the second user when a person is detected from image information captured by the camera 306 of the own terminal (see FIG. 3). Further, the second information processing terminal 103 may detect the second user when a temperature sensor (not shown) detects a person.

Further, the second information processing terminal 103 detects the second user when a conversation start button or the like is pressed by a user operation using the input device 304 (see FIG. 3) of the own terminal. Good too. When the second information processing terminal 103 detects the second user, it transmits information indicating that the second user has been detected to the information processing apparatus 101.

Information indicating that the second user has been detected may be transmitted from the second information processing terminal 103 to the information processing apparatus 101, for example, as an instruction to start a chat. Here, the chat start instruction is for starting a conversation with the operator through the avatar. Further, the chat start instruction may be for starting a conversation with a chatbot.

A chat start instruction from the second information processing terminal 103 is received by the communication unit 703. The second output control unit 702 determines that the second user at the second information processing terminal 103 has been detected, for example, when the communication unit 703 receives an instruction to start a chat. Thereby, the information processing apparatus 101 can determine that the second user at the second information processing terminal 103 has been detected.

However, the second user detection process may be performed in the information processing apparatus 101. For example, the information processing apparatus 101 receives image information taken by the camera 306 from the second information processing terminal 103 and detects a person from the received image information, thereby detecting the second user. Good too.

The operation corresponding to the avatar is, for example, an operation that specifies the movement of the avatar. Examples of the avatar's movements include waving, nodding, bowing, and being happy. Further, the operation corresponding to the avatar may be an operation for specifying an operator who will respond to the user through the avatar. Further, the operation corresponding to an avatar may be an operation for specifying an avatar.

The information that prompts the operation corresponding to the avatar includes, for example, an option for specifying what kind of movement the avatar should make, an option for specifying an operator who will respond to the user through the avatar, and the like. The information prompting the user to perform an operation corresponding to the avatar may include a message prompting the user to select one of the options.

Here, the information output to the first information processing terminal 102 that prompts the operation corresponding to the avatar may be predetermined information. For example, options for specifying basic avatar actions such as waving, nodding, and being happy may be predetermined.

Further, the second output control unit 702 may decide the information to be output based on the second user's attribute information. Here, the information to be output is information that prompts an operation corresponding to the avatar output to the first information processing terminal 102. The second user attribute information is information that specifies the second user's attributes.

For example, the second user's attribute information is information that can specify the second user's age, generation, gender, language, clothing, presence or absence of a wheelchair, presence or absence of a companion, and the like. The second user's attribute information is acquired from the second information processing terminal 103. Specifically, for example, the communication unit 703 receives a chat start instruction from the second information processing terminal 103 and also receives attribute information of the second user acquired by the second information processing terminal 103. You can also do this.

To explain in more detail, for example, the second output control unit 702 specifies the user attribute based on the received attribute information of the second user. Next, the second output control unit 702 refers to the storage unit 710 that stores information indicating the correspondence between user attributes and information prompting operations corresponding to the avatars, and outputs the avatars corresponding to the identified user attributes. The information that prompts the corresponding operation is determined as the information to be output. Then, the second output control unit 702 outputs the determined output target information to the first information processing terminal 102.

Thereby, it is possible to output information that prompts an operation corresponding to the avatar, taking into account the characteristics of the second user. For example, when the user attribute "under 10 years old" is specified, there are options that allow you to specify movements for children, options that allow you to specify an operator who is good at interacting with children, and options that allow you to specify an avatar that is suitable for children. etc. can be presented to the operator.

Note that, in the second information processing terminal 103, the second user's attribute information may be input by a user operation using the input device 304, for example. In this case, the first information processing terminal 102 transmits the input attribute information of the second user to the information processing apparatus 101 along with an instruction to start a chat.

Further, the second information processing terminal 103 analyzes the image information captured by the camera 306 and obtains attribute information such as the second user's age, gender, clothing, presence or absence of a wheelchair, and presence or absence of a companion. It's okay. Further, the second information processing terminal 103 may analyze audio information received by the microphone 308 (see FIG. 3) to obtain attribute information such as gender and language of the second user.

The communication unit 703 may also receive image information captured by the camera 306 and audio information received by the microphone 308 as the second user's attribute information. In this case, the information processing apparatus 101 may, for example, acquire information representing the user's characteristics from the received image information or audio information. That is, the process of analyzing image information and audio information may not be performed by the second information processing terminal 103, but may be performed by the information processing apparatus 101.

Note that any existing technology may be used as the technology for acquiring information representing the user's characteristics from image information and audio information. For example, the second information processing terminal 103 or the information processing device 101 uses a method based on machine learning such as deep learning to determine the age, gender, clothing, language, presence or absence of a wheelchair, and the presence or absence of a companion from image and audio features. Information such as presence/absence may be extracted.

Further, the second output control unit 702 may decide the information to be output based on information received in a conversation through an avatar at the second information processing terminal 103. Here, the information received in the conversation is information representing the content of the conversation between the first user and the second user through the avatar.

For example, information received in a conversation may include information indicating a selected option or information input by the second user's operation using the input device 304 (see FIG. 3) of the second information processing terminal 103. It is. That is, the second output control unit 702 determines the information to be output, taking into consideration the content of the conversation through the avatar in the second information processing terminal 103 .

When the conversation control unit 704 detects the second user at the second information processing terminal 103, the conversation control unit 704 may start a conversation using a chatbot at the second information processing terminal 103. Specifically, for example, when the conversation control unit 704 receives a chat start instruction for starting a chatbot conversation, the conversation control unit 704 selects a talk to be applied to the chatbot conversation on the second information processing terminal 103. Decide on a script.

To explain in more detail, for example, the conversation control unit 704 identifies the user attribute based on the second user's attribute information received from the second information processing terminal 103 together with the chat start instruction. Next, the conversation control unit 704 refers to the script table 140 shown in FIG. 6 and determines the script corresponding to the specified user attribute as the talk script.

Then, the conversation control unit 704 starts a chatbot conversation on the second information processing terminal 103 using the determined talk script. Thereby, the conversation in the chatbot can be controlled according to the characteristics of the second user who uses the second information processing terminal 103.

Further, the second output control unit 702 may determine information to be output based on information received in a chatbot conversation on the second information processing terminal 103. That is, the second output control unit 702 determines the information to be output, taking into consideration the content of the conversation in the chatbot on the second information processing terminal 103.

To explain in more detail, for example, the second output control unit 702 specifies the content of the conversation based on information received in a conversation through an avatar or a conversation with a chatbot. Then, the second output control unit 702 refers to the option management table 120 shown in FIG. 4 and determines the option corresponding to the specified conversation content as the information to be output.

As an example, the conversation content specified from information received in a conversation through an avatar or a conversation with a chatbot is referred to as "conversation content C1." The conversation content C1 is, for example, a combination of options selected in a conversation through an avatar or a conversation with a chatbot. In this case, the second output control unit 702, for example, refers to the option management table 120 and determines options ch1 to ch4 corresponding to the identified conversation content C1 as information to be output. Options ch1 to ch4 are, for example, options for specifying the movement of the avatar, options for specifying the operator who responds to the user through the avatar, and the like.

Note that information received in a conversation through an avatar or a conversation with a chatbot is specified, for example, from a chat log DB (not shown). Further, an example of the output of information that prompts an operation corresponding to an avatar, which is output to the first information processing terminal 102, will be described later using FIG. 10.

When the first information processing terminal 102 receives an operation corresponding to the output information, the first output control unit 701 performs a control based on the received operation and the external environment information detected by the second information processing terminal 103. to control the output of the avatar. Here, the external environment information is information representing the external environment of the second information processing terminal 103 on which the avatar is displayed.

For example, external environment information represents time, location, season, weather, temperature, humidity, etc. The time is, for example, a time of day, a time zone, a date, or the like. External environment information is acquired from the second information processing terminal 103. Specifically, for example, the communication unit 703 may receive a chat start instruction from the second information processing terminal 103 and may also receive external environment information detected by the second information processing terminal 103. .

Note that the second information processing terminal 103 detects external environmental information representing time, season, weather, temperature, humidity, etc., using an unillustrated environmental sensor or by inquiring an external server. Further, the second information processing terminal 103 may detect, by the GPS unit 309 (see FIG. 3), position information indicating the current position of the second information processing terminal 103 as external environment information indicating the location.

Specifically, for example, the first output control unit 701 determines the movement of the avatar based on the received operation and the external environment information detected by the second information processing terminal 103. Then, the first output control unit 701 changes the output of the avatar based on the determined movement.

Here, assume that the first information processing terminal 102 receives an operation by the first user using the input device 304 to select an option for specifying the avatar's movement of "waving hand". . In this case, the first output control unit 701 receives an operation from the first information processing terminal 102 to select an option that specifies the avatar's movement of "waving hand."

Then, the first output control unit 701 determines how the avatar will "wave" according to the external environment specified from the external environment information detected by the second information processing terminal 103, Change avatar output. For example, how to adjust the movement of the avatar according to the external environment is set in advance.

To explain in more detail, for example, the first output control unit 701 refers to the storage unit 710 and determines the content of the avatar's movement of “waving” according to the external environment. For example, the storage unit 710 stores, for each movement of the avatar, information indicating the correspondence between the content of the movement and the external environment.

For example, it is assumed that the external environment specified from the external environment information detected by the second information processing terminal 103 is "time zone: daytime". In this case, the first output control unit 701 may change the output of the avatar so that the avatar waves in a brighter and louder motion than during nighttime hours, for example.

Further, the external environment specified from the external environment information detected by the second information processing terminal 103 is "temperature: below freezing." In this case, the first output control unit 701 may change the output of the avatar so that the avatar waves its hands as if it is shivering from the cold, for example.

Further, the external environment specified from the external environment information detected by the second information processing terminal 103 is assumed to be "location: hotel". In this case, the first output control unit 701 changes the output of the avatar so that, for example, the hotelier waves his hand as if he is seeing off a guest.

Further, the first output control unit 701 may change the output of the face portion of the avatar based on the received operation. Here, assume that the first information processing terminal 102 receives an operation by the first user using the input device 304 to select an option for specifying the operator OP.

In this case, the first output control unit 701 receives an operation from the first information processing terminal 102 to select an option for specifying the operator OP. Then, the first output control unit 701 changes the face part of the avatar to the face of the avatar corresponding to the specified operator OP. Thereby, the face of the avatar can be changed according to the operator who responds to the user.

Furthermore, the first output control unit 701 determines to switch the first information processing terminal 102 to which the second information processing terminal 103 is connected when the first user is switched based on the received operation. good. For example, when the first user switches from "operator OP1" to "operator OP2", the first output control unit 701 switches the first information processing terminal 102 to which the second information processing terminal 103 is connected to the operator Switch to the first information processing terminal 102 used by OP2.

Further, the first output control unit 701 may change the output of the avatar's body part based on external environment information detected by the second information processing terminal 103. Here, it is assumed that the external environment specified from the external environment information detected by the second information processing terminal 103 is "season: summer". In this case, the first output control unit 701 changes the body part of the avatar to a body wearing summer clothes, for example.

Further, the external environment specified from the external environment information detected by the second information processing terminal 103 is assumed to be "location: hotel". In this case, the first output control unit 701 changes the body part of the avatar to one wearing a hotel uniform, for example. This allows the avatar's clothing to be changed depending on the location where the second information processing terminal 103 is installed and the season.

To explain in more detail, for example, the first output control unit 701 refers to the avatar management table 130 shown in FIG. 5 and identifies avatar management information corresponding to the received operation. As an example, assume that an operation to select option ch1 is accepted. Further, the external environment specified from the external environment information detected by the second information processing terminal 103 is assumed to be "E2".

In this case, the first output control unit 701 refers to the avatar management table 130 and specifies the avatar management information 500-2 that corresponds to the option ch1 and corresponds to the external environment E2. Then, the first output control unit 701 refers to the avatar management information 500-2 and changes the face part of the avatar based on the face data fc1.

Further, the first output control unit 701 refers to the avatar management information 500-2 and changes the body part of the avatar based on the body data bd2. The first output control unit 701 refers to the avatar management information 500-2 and controls the first information processing terminal 102 to which the second information processing terminal 103 is connected to the first information processing unit used by the operator OP2. Switch to terminal 102.

However, if operator switching does not occur, switching of the first information processing terminal 102 to which the second information processing terminal 103 is connected also does not occur. Furthermore, if the avatar management information is not specified from the avatar management table 130, the first output control unit 701 may decide not to change the output of the avatar's face and body parts.

Further, when the first output control unit 701 detects the second user at the second information processing terminal 103, the first output control unit 701 may determine whether there is history information corresponding to the second user. . If there is history information corresponding to the second user, the first output control unit 701 may control the output of the avatar based on the avatar information included in the history information.

Here, the history information is information indicating that the second user used the second information processing terminal 103. The history information includes, for example, identification information that uniquely identifies the second user, and information that allows identification of the avatar output when the second user uses the second information processing terminal 103.

For example, assume that the second information processing terminal 103 is installed in a commercial facility or store, and when using the second information processing terminal 103, the member ID of the commercial facility or store is input. In this case, the identification information that uniquely identifies the second user is, for example, a member ID. Further, it is assumed that the second information processing terminal 103 is installed in a corporate facility and that an employee ID is input when using the second information processing terminal 103. In this case, the identification information that uniquely identifies the second user is, for example, an employee ID.

For example, the member ID and employee ID are included in the chat start instruction. Note that the user attribute may be specified from the member ID or employee ID. For example, the second user's gender, age, years of service, annual income, etc. can be specified as user attributes from the employee information associated with the employee ID.

The information that allows the avatar to be specified includes, for example, information that specifies the face and body parts of the avatar. The information specifying the face and body parts of the avatar is, for example, the face data and body data of the avatar that were displayed when the conversation through the avatar on the second information processing terminal 103 ended.

Further, the information that can identify the avatar may include information that identifies the operator corresponding to the avatar. The information identifying the operator is, for example, the operator ID of the operator who was responding to the second user immediately before the end of the conversation. The history information is stored in a history information DB (not shown) included in the information processing device 101, for example, when the conversation through the avatar on the second information processing terminal 103 ends.

As an example, assume that a chat start instruction including an employee ID is received. In this case, the first output control unit 701 refers to the history information DB and determines whether or not there is history information including the employee ID included in the chat start instruction. Here, if there is no history information including the employee ID, the first output control unit 701 does not change the output of the avatar based on the history information.

On the other hand, if there is history information including the employee ID, the first output control unit 701 changes the facial part of the avatar to be output to the second information processing terminal 103 based on the face data included in the history information. Further, the first output control unit 701 changes the body part of the avatar based on the body data 2 included in the history information. Further, the first output control unit 701 switches the first information processing terminal 102 to which the second information processing terminal 103 is connected to the first information processing terminal 102 corresponding to the operator ID included in the history information. . This allows the same operator as last time to respond to the second user through an avatar with the same appearance as last time, and it is possible to maintain character consistency.

Further, when the first output control unit 701 detects a second user existing around the second information processing terminal 103, the first output control unit 701 outputs information prompting the second user to operate the second information processing terminal 103. The data may be output to the information processing terminal 103. The information prompting the user to operate the second information processing terminal 103 may be, for example, a message that makes the user aware of the existence of the second information processing terminal 103, such as "Hello, you there!".

The second user existing around the second information processing terminal 103 can, for example, use the image information taken by the camera 306 of the second information processing terminal 103 or the temperature sensor of the second information processing terminal 103. (not shown). Specifically, for example, the first output control unit 701 causes the speaker 307 of the second information processing terminal 103 to output a message such as "Hello, you there!" as the avatar's utterance. Good too. Thereby, for example, it is possible to actively call out to customers of a commercial facility or store, and increase opportunities for customer service using the second information processing terminal 103.

Note that each functional unit of the information processing device 101 described above may be realized by a plurality of computers within the conversation control system 100. For example, the information processing device 101, the first information processing terminal 102, and the second information processing terminal 103 may share the functions and realize the functional units of the information processing device 101 described above.

(One embodiment of conversation control system 100)
Next, one embodiment of the conversation control system 100 will be described using FIG. 8.

FIG. 8 is an explanatory diagram showing one embodiment of the conversation control system 100. The information processing device 101 outputs an avatar av that moves according to the operation of the operator OP1 on the first information processing terminal 102 to the second information processing terminal 103. When the information processing device 101 detects the user U1 at the second information processing terminal 103, it outputs options #1 to #3 for prompting the operation corresponding to the avatar av to the first information processing terminal 102.

Options #1 to #3 are options for specifying the movement of the avatar av and the operator who responds to the user U1 through the avatar av. When the information processing device 101 receives an operation to select the output options #1 to #3 at the first information processing terminal 102, the information processing device 101 compares the received operation with the external environment information detected by the second information processing terminal 103. Based on this, the output of the avatar AV is controlled.

For example, the information processing device 101 determines the movement of the avatar av based on the received operation and the external environment information detected by the second information processing terminal 103, and outputs the avatar av based on the determined movement. Control. For example, if the option to specify the avatar's movement of ``waving hands'' is selected and the external environment specified from the external environment information is ``temperature: below freezing'', the avatar's hands may appear as if it is shivering from the cold. You can control the movement of the avatar AV, such as shaking it.

Furthermore, the information processing device 101 changes the output of the face portion fc of the avatar av based on the received operation. For example, when an option to specify an operator OP is selected, the face portion fc of the avatar av can be changed to the face of the avatar av corresponding to the specified operator OP.

Further, the information processing device 101 changes the output of the body part bd of the avatar av based on the external environment information detected by the second information processing terminal 103. For example, when the external environment specified from the external environment information is "temperature: below freezing", the avatar's body part bd can be changed to a body wearing winter gear.

(Screen example of talk screen displayed on second information processing terminal 103)
Next, an example of the talk screen displayed on the display 303 of the second information processing terminal 103 will be described using FIGS. 9 and 10.

Here, it is assumed that the second information processing terminal 103 is a digital board installed in a corporate facility and used by employees. In addition, we assume that the escalation from chatbot support involves manned support using avatars. Further, it is assumed that the user attribute specified from the second user's attribute information acquired by the second information processing terminal 103 is “30s x male”.

In this case, the information processing device 101 refers to the script table 140 and starts a chatbot conversation on the second information processing terminal 103 using the talk script corresponding to the user attribute "30s x male". . The talk script corresponding to the user attribute "30s x male" is, for example, a script for having a conversation about career advancement.

FIG. 9 is an explanatory diagram (part 1) showing an example of the talk screen displayed on the second information processing terminal 103. In FIG. 9, a talk screen 900 is a talk screen displayed on the second information processing terminal 103 when a second user at the second information processing terminal 103 is detected and starts a conversation with the chatbot. This is an example. In FIG. 9, avatar av represents a chatbot character or a character that moves in response to an operator's operation.

On the talk screen 900, selection buttons 901 and 902 regarding career advancement are displayed in a conversational format along with the message m1 of the avatar AV. On the talk screen 900, when the second user selects the selection button 901 through an operation input using the input device 304 of the second information processing terminal 103, a conversation about courses popular among people in their 30s can be held.

Furthermore, by selecting a selection button 902 on the talk screen 900, a conversation regarding a course on DX (Digital Transformation) can be held. Furthermore, by selecting language buttons b1, b2, b3, and b4 on the talk screen 900, the language to be used can be switched. For example, by selecting the language button b2, the language to be used can be switched from Japanese to English.

Furthermore, by selecting the microphone button b5 on the talk screen 900, a message can be input by voice. On the talk screen 900, information indicating the selected option and input information are transmitted from the second information processing terminal 103 to the information processing apparatus 101. As a result, the talk screen 900 is updated.

Here, it is assumed that the selection button 901 is selected on the talk screen 900. In this case, on the talk screen 900, along with the message m2 of the avatar av, selection buttons 911 to 919 for asking the second user's desired content are displayed in a conversational format.

On the talk screen 900, the second user can select the desired content of the course he or she wants to take by selecting options such as selection buttons 911 to 919. When the second user finishes selecting the desired content of the course he/she wants to attend, the chatbot response is switched to the manned response via the avatar AV.

Here, it is assumed that the options "for engineers", "English", and "intermediate" are selected on the talk screen 900. In this case, an operator support screen 1000 as shown in FIG. 10 is displayed on the display 303 of the first information processing terminal 102, and the chatbot support is switched to the manned support using the avatar AV.

FIG. 10 is an explanatory diagram showing an example of the operator support screen displayed on the first information processing terminal 102. In FIG. 10, an operator support screen 1000 is an example of an operation screen that supports operations by an operator. The operator is the first user who uses the first information processing terminal 102.

Operator support screen 1000 includes talk screen 900 and conversation history 1010. The talk screen 900 is a talk screen displayed on the second information processing terminal 103, and the display content changes in synchronization with the talk screen displayed on the second information processing terminal 103. The conversation history 1010 is information indicating the content of the conversation that took place at the second information processing terminal 103.

The operator (first user) can check the screen content displayed on the second information processing terminal 103 using the talk screen 900 within the operator support screen 1000. Further, the operator can check the contents of the conversation that took place on the second information processing terminal 103 based on the conversation history 1010.

The operator support screen 1000 also includes operation buttons b11 to b13. On the operator support screen 1000, when the operator selects the action button b11 using the input device 304 of the first information processing terminal 102, the avatar av can be made to wave. At this time, the waving motion of the avatar av is controlled according to the external environment of the second information processing terminal 103.

Furthermore, when the action button b12 is selected on the operator support screen 1000, the avatar av can be made to nod. At this time, the nodding movement of the avatar av is controlled according to the external environment of the second information processing terminal 103.

Furthermore, by selecting the action button b13 on the operator support screen 1000, the avatar av can be made to make movements that make the user happy. At this time, the happy movements of the avatar av are controlled according to the external environment of the second information processing terminal 103.

The operator support screen 1000 also includes a voice synthesis button b14 and a QR sending button b15. When the voice synthesis button b14 is selected on the operator support screen 1000, a voice synthesis process can be performed in which the text input on the first information processing terminal 102 is converted into voice and output to the second information processing terminal 103. .

Thereby, the text input by the operator (or the text obtained by recognizing the operator's voice) can be converted into voice and output as a voice message of the avatar AV.

Furthermore, when the QR sending button b15 is selected on the operator support screen 1000, a QR (Quick Response) code can be output to the second information processing terminal 103. The QR code is information that specifies, for example, a script that introduces recommended courses by a chatbot. QR code is a registered trademark.

As a result, the second user can, for example, read the QR code displayed on the digital board (second information processing terminal 103) with his or her own smartphone (second information processing terminal 103), You can start a conversation with a chatbot. Note that the QR code output to the second information processing terminal 103 can be arbitrarily selected, for example, according to the operation of the first user on the first information processing terminal 102.

Further, on the operator support screen 1000, options 1021 to 1023 are displayed along with a message 1020. Here, the message 1020 and the options 1021 to 1023 are examples of information output to the second information processing terminal 103 that prompts an operation corresponding to the avatar.

The message 1020 and the options 1021 to 1023 are information determined according to the content of the conversation on the talk screen 900 (for example, the desired content of the course that the second user wants to take). Message 1020 is a message prompting the user to specify an operator who will respond to the second user.

Options 1021 to 1023 are options for specifying the operator who will respond to the second user. Here, operator candidates who are knowledgeable about courses that match the content desired by the second user are presented as options 1021 to 1023.

By selecting option 1021 on operator support screen 1000, it is possible to designate "person in charge of information technology courses" as the operator. Furthermore, by selecting option 1022 on the operator support screen 1000, a "person in charge of foreign language courses" can be designated as the operator. Furthermore, by selecting the option 1023 on the operator support screen 1000, it is possible to designate a "person in charge of dissertation courses" as the operator.

The operator can, for example, refer to the talk screen 900, conversation history 1010, etc. to determine which operator should respond. Although not shown, the operator support screen 1000 may display an image of the second user taken by the camera 306 of the second information processing terminal 103. In this case, the operator can check what kind of person the second user is and determine which operator should respond to the second user.

Here, it is assumed that option 1022 is selected on operator support screen 1000. In this case, after a message such as "You will be connected to the corresponding operator" is displayed on the talk screen 900, a talk screen 1100 as shown in FIG. 11 is displayed on the display 303 of the second information processing terminal 103. .

FIG. 11 is an explanatory diagram (part 2) showing an example of the talk screen displayed on the second information processing terminal 103. In FIG. 11, a talk screen 1100 is an example of a talk screen displayed on the second information processing terminal 103 when a "person in charge of foreign language courses" is designated as the operator.

Note that when the "person in charge of foreign language courses" is designated as the operator, the first information processing terminal 102 to which the second information processing terminal 103 is connected will be the first information processing terminal 102 used by the "person in charge of foreign language courses". 1 information processing terminal 102. Further, on the talk screen 1100, information 1110 indicating that the service is manned is displayed.

In FIG. 11, avatar av represents a character that moves in response to the operations of an operator who is a "person in charge of foreign language courses." The face part of the avatar av has been changed based on the face data (for example, see FIG. 5) corresponding to the option 1022 selected on the operator support screen 1000 shown in FIG. 10.

The body parts of avatar av have been changed based on body data (for example, see FIG. 5) corresponding to the external environment specified from the external environment information detected by the second information processing terminal 103. Here, since the external environment specified from the external environment information is "season: summer", the body is changed to one wearing summer clothes.

The operator who is the "person in charge of foreign language courses" can have a conversation with the second user through the avatar AV. At this time, since the face and body parts of the avatar av shown in FIG. 9 have been changed, it is possible to make the second user feel as if they are having a conversation with another person.

In the example of FIG. 11, a message m11 from an operator who is a "person in charge of foreign language courses" is displayed through the avatar av. Furthermore, in response to the question from the operator via message m11, a message m12 from a second user is input. Furthermore, in response to the second user's message m12, a message m13 from an operator who is a "person in charge of foreign language courses" is displayed through the avatar av.

In this way, the second user uses the avatar AV to talk to an operator who has a wealth of knowledge about courses that match his or her desired content, and consults with the operator about what kind of courses he or she should take. be able to.

(Avatar control processing procedure of information processing device 101)
Next, the avatar control processing procedure of the information processing device 101 will be explained using FIGS. 12 and 13. Here, we will explain an example of a case in which a chatbot response is escalated to a manned response using an avatar.

12 and 13 are flowcharts illustrating an example of an avatar control processing procedure of the information processing apparatus 101. In FIG. 12, first, the information processing apparatus 101 outputs an avatar that moves according to the first user's operation on the first information processing terminal 102 to the second information processing terminal 103 (step S1201).

Next, the information processing device 101 determines whether or not it has received from the second information processing terminal 103 the chat start instruction as well as the second user's attribute information and external environment information detected by the second information processing terminal 103. (Step S1202). However, the chat start instruction is a chat start instruction for starting a conversation with the chatbot.

Here, the information processing apparatus 101 waits to receive the second user's attribute information and external environment information together with the chat start instruction (step S1202: No). When the information processing device 101 receives the second user's attribute information and external environment information together with the chat start instruction (step S1202: Yes), the information processing device 101 determines the user attributes based on the received second user's attribute information. Specify (step S1203).

Next, the information processing apparatus 101 refers to the script table 140 and determines the script corresponding to the identified user attribute as the talk script (step S1204). The talk script is a script applied to a chatbot conversation on the second information processing terminal 103.

Then, the information processing apparatus 101 starts a chatbot conversation on the second information processing terminal 103 using the determined talk script (step S1205). Next, the information processing apparatus 101 determines whether a specific condition is satisfied based on the information received in the chatbot conversation on the second information processing terminal 103 (step S1206).

Specifically, for example, the information processing device 101 sets a specific condition when the conversation content in the chatbot on the second information processing terminal 103 corresponds to any conversation content in the option management table 120. It is determined that it satisfies the requirements. On the other hand, if the conversation content does not correspond to any of the conversation contents in the option management table 120, the information processing device 101 determines that the specific condition is not satisfied.

In step S1205, if the specific condition is not satisfied (step S1206: No), the information processing device 101 determines whether to end the conversation with the chatbot on the second information processing terminal 103 (step S1207). . Note that the conversation in the chatbot ends, for example, according to a talk script or in response to an end operation by the second user.

Here, if the conversation with the chatbot is not ended (step S1207: No), the information processing apparatus 101 returns to step S1206. On the other hand, if the conversation with the chatbot is to be ended (step S1207: Yes), the information processing apparatus 101 ends the series of processes according to this flowchart.

Further, in step S1206, if a specific condition is satisfied (step S1206: Yes), the information processing apparatus 101 switches from chatbot-based conversation to manned-based conversation (step S1208), and moves to step S1301 shown in FIG. 13. .

In the flowchart of FIG. 13, the information processing device 101 first refers to the option management table 120 and selects options corresponding to the content of the chatbot conversation on the second information processing terminal 103 from the first information processing terminal 103. (Step S1301). Specifically, for example, the information processing apparatus 101 displays options corresponding to the content of the chatbot conversation on an operator support screen 1000 as shown in FIG. 10 .

Next, the information processing apparatus 101 determines whether any of the output options has been selected (step S1302). Here, the information processing apparatus 101 waits for one of the options to be selected (step S1302: No). If any option is selected (step S1302: Yes), the information processing apparatus 101 executes avatar output change processing (step S1303).

The avatar output change process is a process for controlling the output of the avatar displayed on the second information processing terminal 103 according to the selected option. The specific processing procedure of the avatar output change processing will be described later using FIG. 14.

Next, the information processing apparatus 101 determines whether a specific condition is satisfied based on the information received in the conversation through the avatar at the second information processing terminal 103 (step S1304). Specifically, for example, the information processing device 101 determines that a specific condition is satisfied when the content of the conversation through the avatar by the manned person corresponds to the content of any conversation in the option management table 120. On the other hand, if the conversation content does not correspond to any of the conversation contents in the option management table 120, the information processing device 101 determines that the specific condition is not satisfied.

Here, if the specific condition is not satisfied (step S1304: No), the information processing device 101 determines whether to end the conversation through the avatar by the manned response at the second information processing terminal 103 (step S1304: No). S1305). Note that the conversation through the avatar with the attendant ends, for example, in response to the second user's end operation.

Here, if the conversation through the avatar with the attendant is not ended (step S1305: No), the information processing apparatus 101 returns to step S1304. On the other hand, if the conversation through the avatar by the manned person is to be ended (step S1305: Yes), the information processing apparatus 101 ends the series of processes according to this flowchart.

In addition, in step S1304, if the specific condition is satisfied (step S1304: Yes), the information processing device 101 refers to the option management table 120, and the information processing device 101 refers to the option management table 120 to determine whether the second information processing terminal 103 uses the manned avatar. The options corresponding to the content of the conversation are output to the first information processing terminal 102 (step S1306), and the process returns to step S1302.

Thereby, the conversation in the chatbot can be controlled according to the characteristics of the second user who uses the second information processing terminal 103. Additionally, escalation from chatbot support allows for manned support through avatars.

Next, the specific processing procedure of the avatar output changing process in step S1303 shown in FIG. 13 will be described using FIG. 14.

FIG. 14 is a flowchart illustrating an example of a specific procedure for changing the avatar output. In the flowchart of FIG. 14, first, the information processing apparatus 101 identifies the external environment of the second information processing terminal 103 based on the external environment information received in step S1202 shown in FIG. 12 (step S1401).

Next, the information processing apparatus 101 determines whether the option selected in step S1302 shown in FIG. 13 specifies the movement of the avatar or the operator (step S1402).

Here, if the movement of the avatar is to be specified (step S1402: movement of the avatar), the information processing apparatus 101 determines the movement of the avatar based on the specified movement and the specified external environment ( Step S1403). The information processing apparatus 101 then changes the output of the avatar displayed on the second information processing terminal 103 based on the determined movement (step S1404), and returns to the step that called the avatar output change process.

On the other hand, if an operator is to be specified (step S1402: operator), the information processing apparatus 101 refers to the avatar management table 130 and specifies avatar management information corresponding to the selected option (step S1405). Next, the information processing apparatus 101 refers to the specified avatar management information and specifies the face data of the avatar (step S1406).

Next, the information processing apparatus 101 refers to the specified avatar management information and specifies the body data of the avatar corresponding to the specified external environment (step S1407). Next, the information processing apparatus 101 refers to the specified avatar management information and switches the first information processing terminal 102 to which the second information processing terminal 103 is connected (step S1408).

Then, the information processing device 101 changes the output of the avatar displayed on the second information processing terminal 103 based on the specified face data and body data (step S1409), and returns to the step that called the avatar output change process. return.

Thereby, the movement of the avatar can be controlled in accordance with the external environment of the second information processing terminal 103 in accordance with the operation of the operator who responds to the user. Furthermore, the face and body parts of the avatar can be changed depending on the operator who responds to the user.

As described above, according to the information processing apparatus 101 according to the embodiment, an avatar that moves in response to the first user's operation on the first information processing terminal 102 is output to the second information processing terminal 103. be able to. Further, according to the information processing device 101, when the second user at the second information processing terminal 103 is detected, information that prompts an operation corresponding to the avatar can be output to the first information processing terminal 102. The information that prompts an operation corresponding to the avatar is, for example, information that prompts an operation to specify an action of the avatar, or information that prompts an operation to specify an operator who will respond to the second user through the avatar. According to the information processing device 101, when an operation corresponding to the output information is received at the first information processing terminal 102, based on the received operation and the external environment information detected by the second information processing terminal 103, You can control the output of your avatar.

Thereby, the output of the avatar can be controlled in accordance with the external environment of the second information processing terminal 103 in accordance with the operation of the operator who responds to the user.

Further, according to the information processing device 101, the movement of the avatar is determined based on the received operation and the external environment information detected by the second information processing terminal 103, and the output of the avatar is determined based on the determined movement. Can be changed.

Thereby, the movement of the avatar can be controlled in accordance with the external environment of the second information processing terminal 103 in accordance with the operation of the operator who responds to the user. For example, when the avatar's movement of "waving hands" is specified and the external environment of the second information processing terminal 103 is "temperature: below freezing", the avatar will wave its hands as if it is shivering from the cold. You can control the movement of your avatar. Furthermore, when the avatar's movement of "bowing" is specified and the external environment of the second information processing terminal 103 is "location: hotel", the avatar's movement is changed to bow in the manner of a hotel worker. AV movement can be controlled.

Further, according to the information processing device 101, the output of the avatar's face can be changed based on the received operation, and the output of the avatar's body can be changed based on external environment information.

Thereby, the face part (appearance) of the avatar can be changed according to the operator who responds to the user. Further, the body parts (appearance) of the avatar can be changed according to the external environment of the second information processing terminal 103. For example, the avatar's clothing can be changed depending on the location where the second information processing terminal 103 is installed or the season. For example, if the external environment of the second information processing terminal 103 is "location: hotel", the avatar's body parts can be changed to clothes like a hotel worker. Further, if the external environment of the second information processing terminal 103 is "season: summer", the body part of the avatar can be changed to summer clothes.

According to the information processing apparatus 101, when the second user at the second information processing terminal 103 is detected, it is determined whether or not there is history information corresponding to the second user, and the process is performed to correspond to the second user. If there is history information, the output of the avatar can be controlled based on the avatar information included in the history information.

As a result, for example, the same operator can respond to the second user through an avatar that looks the same as when the second user used the second information processing terminal 103 last time. It can be consistent.

According to the information processing device 101, when the second user at the second information processing terminal 103 is detected, the first information processing process generates information prompting an operation corresponding to the avatar based on the attribute information of the second user. It can be output to the terminal 102.

As a result, it is possible to present options suitable for the second user, taking into account the characteristics of the second user, for example, options for specifying the operator who will perform the avatar's movements and respond to the second user. Able to support reception duties. For example, if the second user is a child, it is possible to present options for actions that children would enjoy, or to connect the user to an operator who is good at dealing with children.

According to the information processing device 101, when the second user at the second information processing terminal 103 is detected, the information processing device 101 responds to the avatar based on the information received in the conversation through the avatar at the second information processing terminal 103. Information prompting operation can be output to the first information processing terminal 102.

This makes it possible to present options suitable for the second user, taking into account the content of the conversation through the avatar by the manned person at the second information processing terminal 103, and to support the operator's reception work. For example, it is possible to find out what the second user wants in a conversation through the avatar and present options for connecting to an operator who is knowledgeable about the content.

According to the information processing device 101, when detecting the second user at the second information processing terminal 103, it starts a conversation with the chatbot on the second information processing terminal 103, and receives a message received in the chatbot conversation. Based on the information, information that prompts an operation corresponding to the movement of the avatar can be output to the first information processing terminal 102.

This makes it possible to present options suitable for the second user in consideration of the content of the conversation in the chatbot on the second information processing terminal 103, thereby supporting the operator's response work. For example, it is possible to find out what the second user wants in a conversation with a chatbot and present options for connecting the second user with an operator who is knowledgeable about the content.

Therefore, according to the conversation control system 100 and the information processing device 101, it is possible to realize a wide variety of avatar movements and to improve the quality of the service by providing an appropriate response from an operator that suits the user. can.

Note that the avatar control method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This avatar control program is recorded on a computer-readable recording medium such as a hard disk, flexible disk, CD-ROM, DVD, or USB memory, and is executed by being read from the recording medium by the computer. Further, this avatar control program may be distributed via a network such as the Internet.

Further, the information processing device 101 described in this embodiment can also be realized by a specific application IC such as a standard cell or a structured ASIC (Application Specific Integrated Circuit), or a PLD (Programmable Logic Device) such as an FPGA.

Regarding the embodiments described above, the following additional notes are further disclosed.

(Additional Note 1) Outputting an avatar that moves in accordance with the operation of the first user on the first information processing terminal to the second information processing terminal,
If a second user at the second information processing terminal is detected, outputting information prompting an operation corresponding to the avatar to the first information processing terminal;
When the first information processing terminal receives an operation corresponding to the outputted information, the output of the avatar is controlled based on the received operation and external environment information detected by the second information processing terminal. do,
An avatar control program characterized by causing a computer to perform processing.

(Additional note 2) The process to be controlled is
A movement of the avatar is determined based on the received operation and external environment information detected by the second information processing terminal, and an output of the avatar is changed based on the determined movement. The avatar control program according to supplementary note 1.

(Additional Note 3) The process to be controlled is
changing the output of the facial part of the avatar based on the received operation;
changing the output of the body part of the avatar based on the external environment information;
The avatar control program according to supplementary note 1.

(Supplementary Note 4) When a second user in the second information processing terminal is detected, determining whether there is history information corresponding to the second user,
If there is history information corresponding to the second user, controlling the output of the avatar based on information about the avatar included in the history information;
The avatar control program according to supplementary note 1, which causes the computer to execute the process.

(Supplementary Note 5) When a second user at the second information processing terminal is detected, information prompting the user to perform an operation corresponding to the avatar is sent to the first information processing terminal based on the attribute information of the second user. output to,
The avatar control program according to supplementary note 1, which causes the computer to execute the process.

(Supplementary Note 6) When a second user is detected at the second information processing terminal, an operation corresponding to the avatar is performed based on information received in a conversation through the avatar at the second information processing terminal. outputting prompting information to the first information processing terminal;
The avatar control program according to supplementary note 1, which causes the computer to execute the process.

(Additional Note 7) When detecting a second user on the second information processing terminal, start a conversation with a chatbot on the second information processing terminal,
outputting information that prompts an operation corresponding to the movement of the avatar to the first information processing terminal based on information received in the conversation with the chatbot;
The avatar control program according to supplementary note 1, which causes the computer to execute the process.

(Supplementary note 8) The avatar control program according to supplementary note 1, wherein the information prompting an operation corresponding to the avatar is information prompting an operation specifying an action of the avatar.

(Supplementary note 9) The avatar according to supplementary note 3, wherein the information prompting an operation corresponding to the avatar is information prompting an operation to specify an operator who will respond to the second user through the avatar. control program.

(Additional Note 10) Outputting an avatar that moves according to the operation of the first user on the first information processing terminal to the second information processing terminal,
If a second user at the second information processing terminal is detected, outputting information prompting an operation corresponding to the avatar to the first information processing terminal;
When the first information processing terminal receives an operation corresponding to the outputted information, the output of the avatar is controlled based on the received operation and external environment information detected by the second information processing terminal. do,
An avatar control method characterized in that processing is executed by a computer.

(Additional Note 11) A first output control unit that outputs an avatar that moves according to the operation of the first user on the first information processing terminal to the second information processing terminal;
a second output control unit that outputs information prompting an operation corresponding to the avatar to the first information processing terminal when a second user at the second information processing terminal is detected;
The first output control section includes:
When the first information processing terminal receives an operation corresponding to the information output by the second output control unit, the received operation and the external environment information detected by the second information processing terminal controlling the output of the avatar based on;
An information processing device characterized by:

100 Conversation control system 101 Information processing device 102 First information processing terminal 103 Second information processing terminal 110 Network 120 Choice management table 130 Avatar management table 140 Script table 200,300 Bus 201,301 CPU
202,302 Memory 203 Disk Drive 204 Disk 205,305 Communication I/F
206 Portable recording medium I/F
207 Portable recording medium 303 Display 304 Input device 306 Camera 307 Speaker 308 Microphone 309 GPS unit 701 First output control section 702 Second output control section 703 Communication section 704 Conversation control section 710 Storage section 900, 1100 Talk screen 1000 Operator Support screen

Claims (8)

outputting an avatar that moves in response to an operation of a first user on a first information processing terminal to a second information processing terminal;
If a second user at the second information processing terminal is detected, outputting information prompting an operation corresponding to the avatar to the first information processing terminal;
When the first information processing terminal receives an operation corresponding to the outputted information, the output of the avatar is controlled based on the received operation and external environment information detected by the second information processing terminal. do,
An avatar control program characterized by causing a computer to perform processing. The process to be controlled is
A movement of the avatar is determined based on the received operation and external environment information detected by the second information processing terminal, and an output of the avatar is changed based on the determined movement. The avatar control program according to claim 1. The process to be controlled is
changing the output of the facial part of the avatar based on the received operation;
changing the output of the body part of the avatar based on the external environment information;
The avatar control program according to claim 1, characterized in that: When detecting a second user in the second information processing terminal, determining whether there is history information corresponding to the second user;
If there is history information corresponding to the second user, controlling the output of the avatar based on information about the avatar included in the history information;
The avatar control program according to claim 1, which causes the computer to execute the process. If a second user at the second information processing terminal is detected, outputting information prompting an operation corresponding to the avatar to the first information processing terminal based on attribute information of the second user;
The avatar control program according to claim 1, which causes the computer to execute the process. When a second user on the second information processing terminal is detected, information prompting the avatar to perform a corresponding operation is sent to the second information processing terminal based on information received in a conversation through the avatar on the second information processing terminal. Output to the first information processing terminal,
The avatar control program according to claim 1, which causes the computer to execute the process. outputting an avatar that moves in response to an operation of a first user on a first information processing terminal to a second information processing terminal;
If a second user at the second information processing terminal is detected, outputting information prompting an operation corresponding to the avatar to the first information processing terminal;
When the first information processing terminal receives an operation corresponding to the outputted information, the output of the avatar is controlled based on the received operation and external environment information detected by the second information processing terminal. do,
An avatar control method characterized in that processing is executed by a computer. a first output control unit that outputs an avatar that moves in response to a first user's operation on the first information processing terminal to a second information processing terminal;
a second output control unit that outputs information prompting an operation corresponding to the avatar to the first information processing terminal when a second user at the second information processing terminal is detected;
The first output control section includes:
When the first information processing terminal receives an operation corresponding to the information output by the second output control unit, the received operation and the external environment information detected by the second information processing terminal controlling the output of the avatar based on;
An information processing device characterized by:

Images