JP2021110768A - Information processing device, information processing system, information processing method, and program - Google Patents

Abstract

To provide a device and a method that allow processing according to a user's utterance to be executed on a device other than a device that has input the user's utterance.SOLUTION: When a data processing unit causes an external second information processing device to execute processing corresponding to a user's utterance, the data processing unit analyzes the user's utterance to generate user utterance interpretation data, converts the generated user utterance interpretation data to generate conversion data which the second information processing device can understand, and transmits the conversion data to the second information processing device. The user utterance interpretation data has intent corresponding to the intention of the user's utterance and a slot corresponding to element information included in the user's utterance. The data processing unit converts the user utterance interpretation data including the intent and the slot into data including intent and a slot which the second information processing device can understand.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to information processing devices, information processing systems, information processing methods, and programs. More specifically, the present invention relates to an information processing device, an information processing system, an information processing method, and a program that execute processing and response according to a user's utterance.

In recent years, the use of voice dialogue systems that perform voice recognition of user utterances and perform various processes and responses based on the recognition results is increasing.
In this voice dialogue system, the user's utterance input via the microphone is analyzed, and processing is performed according to the analysis result.

For example, when the user utters "Tell me the weather tomorrow", the weather information is acquired from the weather information providing server, a system response based on the acquired information is generated, and the generated response is output from the speaker. Specifically, for example
System utterance = "Tomorrow's weather will be sunny, but there may be thunderstorms in the evening."
Output such a system utterance.

Information processing devices that interact with users in this way are called agent devices and smart speakers.
As the prior art disclosed about the agent device that interacts with the user, there are, for example, Patent Document 1 (Japanese Patent Laid-Open No. 2008-090545), Patent Document 2 (Japanese Patent Laid-Open No. 2018-081444) and the like.

In recent years, various manufacturers have provided inexpensive agent devices, and many users have multiple agent devices at home.
However, the individual agent devices cannot execute all the processes requested by the user, and the processes that they are good at are limited. For example, agent device A that controls TVs and air conditioners in the house in response to requests from users, agent device B that is good at providing information on news and weather information, restaurant search, and meal delivery services. There are various types such as an agent device C that is good at request processing and the like.

For example, in order for a user having a plurality of agent devices A, B, and C having different processes, which are the above-mentioned specialties, to request a meal delivery service,
"Order a delicious pizza"
It is necessary to speak to the agent device C, which is good at restaurant search, request processing of meal delivery service, and the like.
There is a problem that the user's request is not executed even if the above-mentioned user utterance is made to the other agent devices A and B.

Japanese Unexamined Patent Publication No. 2008-090545 Japanese Unexamined Patent Publication No. 2018-081444

The present disclosure has been made in view of the above problems, for example, and makes it possible for another agent device to execute processing for an utterance made by a user on one agent device, thereby reducing the burden on the user. It is an object of the present invention to provide an information processing device, an information processing system, an information processing method, and a program that realize the above.

The first aspect of the disclosure is
A voice input unit for inputting user utterances,
It has a data processing unit that executes the analysis of the user utterance and generates the user utterance interpretation data.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
The information processing device converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and transmits the converted data to the second information processing device.

Further, the second aspect of the present disclosure is
It is an information processing system that has multiple information processing devices.
The first information processing device is
A voice input unit for inputting user utterances,
It has a data processing unit that executes the analysis of the user utterance and generates the user utterance interpretation data.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
The user utterance interpretation data is converted to generate conversion data that can be understood by the second information processing device, and the data is transmitted to the second information processing device.
The second information processing device is
The information processing system executes processing corresponding to the user's utterance based on the converted data received from the first information processing apparatus.

Further, the third aspect of the present disclosure is
It is an information processing method executed in an information processing device.
The voice input section inputs the user's utterance,
The data processing department
Data processing that executes the analysis of the user utterance and generates the user utterance interpretation data is executed.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
It is an information processing method that converts the user utterance interpretation data, generates conversion data that can be understood by the second information processing apparatus, and transmits the converted data to the second information processing apparatus.

Further, the fourth aspect of the present disclosure is
It is an information processing method executed in an information processing system having a plurality of information processing devices.
The first information processing device
When the external second information processing device executes the processing corresponding to the input user utterance
The analysis of the user utterance is executed to generate the user utterance interpretation data, and the user utterance interpretation data is generated.
A process of converting the user utterance interpretation data, generating converted data that can be understood by the second information processing device, and transmitting the converted data to the second information processing device is executed.
The second information processing device
It is an information processing method that executes a process corresponding to the user's utterance based on the converted data received from the first information processing apparatus.

Further, the fifth aspect of the present disclosure is
A program that executes information processing in an information processing device.
The program is installed in the data processing unit.
Analyze user utterances to generate user utterance interpretation data
When the external second information processing device executes the process corresponding to the user's utterance,
It is in a program that converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and causes the second information processing device to transmit the converted data.

The program of the present disclosure is, for example, a program that can be provided by a storage medium or a communication medium that is provided in a computer-readable format to an information processing device or a computer system that can execute various program codes. By providing such a program in a computer-readable format, processing according to the program can be realized on an information processing device or a computer system.

Yet other objectives, features and advantages of the present disclosure will become apparent in more detailed description based on the examples of the present disclosure and the accompanying drawings described below. In the present specification, the system is a logical set configuration of a plurality of devices, and the devices having each configuration are not limited to those in the same housing.

According to the configuration of one embodiment of the present disclosure, a device and a method capable of executing a process according to a user utterance in a device other than the device in which the user utterance is input are realized.
Specifically, for example, when the external second information processing device is made to execute the process corresponding to the user utterance, the user utterance analysis is executed to generate the user utterance interpretation data, and the generated user utterance interpretation data is converted. Then, the conversion data that can be understood by the second information processing device is generated and transmitted to the second information processing device. The user utterance interpretation data has an intent corresponding to the intention of the user utterance and a slot corresponding to the element information included in the user utterance, and the data processing unit uses the user utterance interpretation data including the intent and the slot as a second. Converts the data to include intents and slots that the information processor can understand.
With this configuration, a device and a method that enable a device other than the device that input the user utterance to execute the process according to the user utterance are realized.
The effects described in the present specification are merely exemplary and not limited, and may have additional effects.

It is a figure explaining the example of the agent apparatus which is an information processing apparatus which performs a response and processing based on a user's utterance. It is a figure explaining the use example and the problem of a plurality of agent devices. It is a figure explaining the use example and the problem of a plurality of agent devices. It is a figure explaining the specific example of the process executed by the agent apparatus of this disclosure. It is a figure explaining an example of mapping data. It is a figure explaining the sequence of processing using a plurality of agent devices. It is a figure explaining the sequence of processing using a plurality of agent devices. It is a figure explaining the specific example of the agent apparatus list. User utterance interpretation data A (intent, slot) generated by one agent device A and user utterance interpretation data X (intent, slot) required for various agent devices X other than agent device A to execute processing. , Slot) is a diagram showing an example of corresponding data. It is a figure explaining the specific processing example of the agent apparatus A, 10 corresponding to the example shown in FIG. 9 (5). It is a figure which shows the flowchart explaining the processing sequence executed by the agent apparatus. It is a figure which shows the flowchart explaining the processing sequence executed by the agent apparatus. It is a figure which shows the flowchart explaining the processing sequence executed by the agent apparatus. It is a figure which shows the flowchart explaining the processing sequence executed by the agent apparatus. It is a figure which shows the data example of the dialogue history information (context) recorded in a storage part. It is a figure which shows the data example of the dialogue history information (context) recorded in a storage part. It is a figure explaining the system configuration example using three or more agent devices. It is a figure explaining the configuration example of the agent apparatus which is the information processing apparatus of this disclosure. It is a figure explaining the configuration example and the use example of the agent apparatus which is an information processing apparatus of this disclosure. It is a figure explaining the hardware configuration example of the agent apparatus which is the information processing apparatus of this disclosure.

Hereinafter, the details of the information processing apparatus, the information processing system, the information processing method, and the program of the present disclosure will be described with reference to the drawings. The explanation will be given according to the following items.
1. 1. About the outline of the agent device 2. Usage examples and problems of multiple agent devices 3. 4. Processing executed by the information processing device (agent device) of the present disclosure. 4. Processing sequence using multiple agent devices. 6. Examples of different user utterance interpretation data for each agent device. Processing flow executed by the agent device 7. 8. Processing example when there is not enough data to execute the processing. Example of management processing of multiple user utterances 9. About other examples 10. Configuration example of agent device (information processing device) 11. Hardware configuration example of agent device (information processing device) 12. Summary of the structure of this disclosure

[1. About the outline of the agent device]
First, an outline of an agent device corresponding to the information processing device of the present disclosure will be described with reference to FIGS. 1 and 1 and below.

FIG. 1 is a diagram showing an example of processing of an agent device 10 that recognizes a user's utterance uttered by the user 1 and makes a response.
The agent device 10 is a user's utterance, for example.
User utterance = "Tell me the weather in Osaka tomorrow and afternoon"
The voice recognition process of this user utterance is executed.

Further, the agent device 10 executes a process based on the voice recognition result of the user's utterance.
In the example shown in FIG. 1, the speaker 14 is used to acquire data for responding to user utterance = "Tell me the weather tomorrow and afternoon in Osaka" and generate a response based on the acquired data. Output via.
In the example shown in FIG. 1, the agent device 10 makes the following system response.
System response = "Tomorrow in Osaka, the weather will be fine in the afternoon, but there may be a shower in the evening."
The agent device 10 executes voice synthesis processing (TTS: Text To Speech) to generate and output the above system response.

The agent device 10 generates and outputs a response by using the knowledge data acquired from the storage unit in the device or the knowledge data acquired via the network.
The agent device 10 shown in FIG. 1 includes a camera 11, a microphone 12, a display unit 13, and a speaker 14, and has a configuration capable of audio input / output and image input / output.

The voice recognition process and the semantic analysis process for the user's utterance may be performed in the agent device 10 or may be executed in the server on the cloud side.

The agent device 10 recognizes the utterance of the user 1 and performs a response based on the utterance of the user. In addition, for example, various processes according to the utterance of the user, for example, control of an external device such as a television or an air conditioner in the house. Execute.
For example, when the user utterance is a request such as "change the TV channel to 1" or "set the temperature of the air conditioner to 20 degrees", the agent device 10 externally recognizes the voice recognition result of the user utterance. A control signal (Wi-Fi, infrared light, etc.) is output to the device to execute control according to the user's utterance.

[2. Usage examples and problems of multiple agent devices]
Next, usage examples and problems of a plurality of agent devices will be described.

In recent years, various manufacturers have provided inexpensive agent devices, and many users have multiple agent devices at home.
However, as described above, the individual agent devices cannot execute all the processes requested by the user, and the processes that they are good at are limited. For example, agent device A that controls TVs and air conditioners in the house in response to requests from users, agent device B that is good at providing information on news and weather information, restaurant search, and meal delivery services. There are various types such as an agent device C that is good at request processing and the like.

FIG. 2 With reference to the following, an example of using the agent device by a user who owns a plurality of agent devices and problems will be described.

As shown in FIG. 2, user 1 is on the second floor of the house. Agent devices A and 10 are placed on the second floor where the user is located, and other agent devices B and 20 are placed on the first floor.
The main functions that can be executed by the agent devices A and 10 on the second floor are various information provision processes such as news, weather information, and traffic information.
On the other hand, in the agent devices B and 20 on the first floor, the main functions that can be executed are control of electric appliances (television, air conditioner, etc.) in the house, restaurant search, request for home delivery (delivery) service, and the like.

Two agent devices with different characteristics are placed on the first floor and the second floor, respectively.
Here, as shown in FIG. 3, it is assumed that the user 1 on the second floor makes the following user utterance to the agent devices A and 10 in front of him.
"I want to order pizza. Deliver at 12:00, please Margherita."

This user utterance is input to the agent devices A and 10 via the microphones of the agent devices A and 10 in front of the user 1.
The agent devices A and 10 perform voice analysis of the user's utterance to understand the user's request, but the agent devices A and 10 do not have a delivery service request function. As a result, as shown in FIG. 3, the agent devices A and 10 make the following system utterances to, for example, the user 1.
"Sorry, I can't do it."

As described above, even if the pizza order request is uttered to the agent devices A and 10 that do not have the delivery service request function, the processing is not performed.
As a result, the user 1 must move to the first floor where the agent devices B and 20 having the delivery service request function are located, and make the above user utterance to the agent devices B and 20 again. Must be.
The information processing device (agent device) of the present disclosure eliminates such a burden on the user.

[3. Processing executed by the information processing device (agent device) of the present disclosure]
Next, the processing executed by the information processing apparatus (agent apparatus) of the present disclosure will be described.

The outline of the process executed by the agent apparatus of the present disclosure will be described with reference to FIG.
FIG. 4 shows the same settings as described above with reference to FIGS. 2 and 3. That is, the user 1 is on the second floor of the house, the agent devices A and 10 are on the same second floor, and another agent devices B and 20 are placed on the first floor.
The main functions that can be executed by the agent devices A and 10 on the second floor are various information provision processes such as news, weather information, and traffic information.
On the other hand, in the agent devices B and 20 on the first floor, the main functions that can be executed are control of electric appliances (television, air conditioner, etc.) in the house, restaurant search, request for home delivery (delivery) service, and the like.
Two agent devices having such different characteristics are on the first floor and the second floor, respectively.

Here, as shown in FIG. 4, it is assumed that the user 1 on the second floor makes the following user utterance to the agent devices A and 10 in front of him.
"I want to order pizza. Deliver at 12:00, please Margherita."

This user utterance is input to the agent devices A and 10 via the microphones of the agent devices A and 10 in front of the user 1.
The agent devices A and 10 execute voice analysis of the user's utterance to understand the user's request.
This process is the process shown in FIG. 4 (step S01).

In (step S01) shown in FIG. 4, the agent devices A and 10 interpret the user utterance by performing voice recognition processing, semantic analysis processing, and dialogue state estimation processing for the user utterance.
The agent devices A and 10 estimate the intention of the user utterance (Intent) and the element information (slot) which is a meaningful element (significant element) included in the utterance as the interpretation result of the user utterance. ..
The intent and slot estimation processing is executed according to the user utterance interpretation algorithm peculiar to the agent devices A and 10.

The agent devices A and 10 shown in FIG. 4 generate, for example, the following user utterance interpretation data in (step S01).
(Intent a) Delivery (Slot a1) Delivery time = 12:00
(Slot a2) Food = Pizza (Slot a3) Type = Margherita

The agent devices A and 10 understand that the user 1 is requesting the delivery request of the pizza (Margherita) based on the above-mentioned user utterance interpretation data.
However, the agent devices A and 10 do not have a pizza delivery request function.

In this case, the agent devices A and 10 of the present disclosure transfer the user utterance interpretation data to another agent device having a pizza delivery request function.
In the example shown in FIG. 4, the user utterance interpretation data is transferred to the agent devices B and 20 on the first floor.

However, even if the agent devices A and 10 directly transfer the user utterance interpretation data (intent a and slot a) generated in the above (step S01) to the agent devices B and 20, the agent devices B and 20 still have these. It may not be possible to execute the process based on the user utterance interpretation data (intent a, slot a), that is, the pizza order process.

The reason for this is that the user utterance interpretation processing algorithm A executed by the agent devices A and 10 may be different from the user utterance interpretation processing algorithm B executed by the agent devices B and 20.

The agent devices A and 10 generate the user utterance interpretation data A (intent a, slot a) according to the user utterance interpretation processing algorithm A of the agent devices A and 10, and the user utterance interpretation data according to the user utterance interpretation processing algorithm A. Processing (for example, a response to a user) is executed according to A (intent a, slot a).

On the other hand, the agent devices B and 20 generate the user utterance interpretation data B (intent b, slot b) according to the user utterance interpretation processing algorithm B of the agent devices B and 20, and the user utterance according to the user utterance interpretation processing algorithm B. The process is executed according to the interpretation data B (intent b, slot b).

Therefore, even if the user utterance interpretation data A (intent a, slot a) according to the user utterance interpretation processing algorithm A generated by the agent devices A and 10 is transmitted to the agent devices B and 20, the agent device B , 20 may not be able to execute correct processing (processing according to user request) based on user utterance interpretation data A (intent a, slot a).

Therefore, the agent devices A and 10 execute the process shown in FIG. 4 (step S02).
In (step S02), the agent devices A and 10 can use the user utterance interpretation data A (intent a, slot a) generated in (step S01) in the user utterance interpretation data B (intent a, slot a) that the agent devices B and 20 can understand. The process of converting to the intent b and the slot b) (mapping process) is executed.

The user utterance interpretation data B (intent b, slot b) corresponds to the user utterance interpretation data (intent b, slot b) generated according to the user utterance interpretation processing algorithm B executed by the agent devices B and 20.

The agent devices A and 10 store the mapping data required for this data conversion in the storage unit, and refer to this mapping data to generate the user utterance interpretation data A (intent a, slot) in (step S01). A process (mapping process) of converting a) into user utterance interpretation data B (intent b, slot b) that can be understood by the agent devices B and 20 is executed.

Specifically, for example, the agent devices A and 10 generate the following conversion data (user utterance interpretation data B (intent b, slot b)) shown in FIG. 4 (step S02).
(Intent b) Pizza order (Slot b1) Delivery time = 12:00
(Slot b2) Type = Margherita

This conversion data is executed with reference to the mapping data stored in the storage units of the agent devices A and 10, specifically, the mapping data as shown in FIG. 5, for example.

In the above conversion data (user utterance interpretation data B (intent b, slot b)), the agent devices B and 20 make the following user utterances, that is,
"I want to order pizza. Deliver at 12:00, please Margherita."
This corresponds to the user utterance interpretation data B (intent b, slot b) generated according to the user utterance interpretation processing algorithm B executed by the agent devices B and 20 when the user utterance is directly input.

FIG. 5 shows an example of mapping data stored in the storage units of the agent devices A and 10.
As shown in FIG. 5, the mapping data has a configuration in which the following data are associated and registered.
(A) User utterance interpretation data A (intent a, slot a) generated according to the user utterance interpretation processing algorithm A executed by the agent devices A and 10.
(B) User utterance interpretation data B (intent b, slot b) required for the agent devices B and 20 to execute the process.
The data (B) corresponds to data capable of causing the agent device B to execute the process by applying the API (Application Programming Interface) of the agent devices B and 20.
For example, the data is allowed as input data by the API provided by the agent devices B, 20 or the management server of the agent devices B, 20.

Next, the agent devices A and 10 have the conversion data generated in (step S02) in (step S03).
(Intent b) Pizza order (Slot b1) Delivery time = 12:00
(Slot b2) Type = Margherita This converted data is transmitted to the agent devices B and 20 on the first floor via the communication unit.
For example, in the agent devices A and 10, the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S02 is input by the agent devices B, 20 or the management server of the agent devices B, 20. It can be input to the agent devices B and 20 by the processing using the provided API (Call processing of API).

The agent devices B and 20 on the first floor receive the above conversion data from the agent devices A and 10 on the second floor.
In (step S04), the agent devices B and 20 execute processing based on the conversion data received from the agent devices A and 10.
That is, the process of ordering pizza is performed according to the user's request.

The conversion data received from the agent devices A and 10 is the following user utterance, that is, the conversion data received by the agent devices B and 20.
"I want to order pizza. Deliver at 12:00, please Margherita."
This is the same as the user utterance interpretation data B (intent b, slot b) generated according to the user utterance interpretation processing algorithm B executed by the agent devices B and 20 when the user utterance is directly input.
Therefore, the agent devices B and 20 can accurately understand the intention of the user 1 and execute accurate processing as in the case where the user utterance is directly input from the user 1.

Further, in the agent devices B and 20, it is not necessary to perform user utterance interpretation processing such as voice recognition processing, semantic analysis processing, and dialogue state estimation processing for the user utterance, and the processing can be smoothly performed without delay. It becomes.

[4. About the sequence of processing using multiple agent devices]
Next, a processing sequence using a plurality of agent devices will be described.

A processing sequence using a plurality of agent devices will be described with reference to FIGS. 6 and 7.
In FIGS. 6 and 7, from the left,
(1) User who speaks
(2) Agent devices A, 10,
(3) Agent devices B, 20,
These are shown.
The processes are executed in the order of steps S21 to S27 shown in FIGS. 6 and 7.
Hereinafter, the processing of each step will be described in sequence.

(Step S21)
First, in step S21, the user 1 speaks to the agent devices A and 10, and the dialogue process is executed between the user 1 and the agent devices A and 10.

(Step S22)
Next, in step S22, the agent devices A and 10 execute the user utterance interpretation process by the voice recognition process, the semantic analysis process, and the dialogue state estimation process of the user utterance acquired in the dialogue process of step S21.

This processing is performed according to the user utterance interpretation algorithm A executed by the data processing unit (voice analysis unit) of the agent devices A and 10.
The agent devices A and 10 are composed of the intention of the user utterance (Intent) and the element information (slot) which is a meaningful element (significant element) included in the utterance as a result of the user utterance interpretation process. User utterance interpretation data A is generated.

Specifically, for example, the following user utterance interpretation data A described above with reference to FIG. 4 is generated.
(Intent a) Delivery (Slot a1) Delivery time = 12:00
(Slot a2) Food = Pizza (Slot a3) Type = Margherita

(Steps S23 to S24)
The agent devices A and 10 determine whether or not the request of the user 1 can be processed based on the user utterance interpretation data acquired in step S22, and if the processing can be executed, perform the processing.
However, this sequence diagram shows a sequence when the agent devices A and 10 determine that the request of the user 1 cannot be processed.

For example, the agent devices A and 10 understand that the user 1 requests the delivery request of the pizza (Margherita) based on the user utterance interpretation data acquired in step S22, but the agent devices A and 10 have the agent devices A and 10 requesting the delivery request. , Does not have a pizza delivery request function.

In this case, the agent devices A and 10 output the following system utterances in step S23.
"Cannot process"
The user 1 makes the following user utterance in response to this system response.
"Transfer to agent device B"
The agent devices A and 10 interpret the user utterance and perform a process for transferring the user request to the agent device B according to the user request.
That is, the processing of step S25 or less shown in FIG. 7 is performed.

In the sequence diagram shown in FIG. 6, in steps S23 to S24, the agent devices A and 10 inform the user 1 that the user request cannot be processed, and the user 1 makes a request to transfer the user request to the agent device B as a response. An example is shown.

A process in which the agent devices A and 10 search for another communicable agent device capable of processing the user request without performing such a dialogue process with the user, and transfer the user request to the searched agent device. It may be configured to perform.

However, when this process is performed, it is necessary to hold, for example, a list of agent devices as shown in FIG. 8 in the storage units of the agent devices A and 10. Refer to this list to select the agent device to which the user request is forwarded.

The agent device list shown in FIG. 8 is a list of agent devices recorded in association with identifiers of other agent devices capable of communication, functions of the agent devices, and communication addresses.
The agent devices A and 10 may be configured to perform a process of selecting an agent device as a user request transfer destination by referring to the agent device list shown in FIG. 8 stored in the storage unit of the agent devices A and 10.

(Step S25)
When the agent device to which the user request is transferred is determined, the agent devices A and 10 execute the process of step S25.
It is assumed that the agent devices to which the user request is transferred are the agent devices B and 20.

In step S25, the agent devices A and 10 perform a process of converting (mapping) the "user utterance interpretation data A" generated in step S22 into "user utterance interpretation data B" that the agent devices B and 20 can understand. ..
That is, the agent devices B and 20 perform a process of converting (mapping) the user utterance interpretation data B (intent, slot) required for executing the process.

As described above with reference to FIG. 4, the agent devices B and 20 generate user utterance interpretation data (intent b, slot b) according to the user utterance interpretation processing algorithm B of the agent devices B and 20. The process is executed according to the generated user utterance interpretation data B (intent b, slot b).
Therefore, even if the user utterance interpretation data A (intent a, slot a) according to the user utterance interpretation processing algorithm A generated by the agent devices A and 10 is transmitted to the agent devices B and 20, the agent device B , 20 may not be able to execute correct processing (processing according to user request).

In order to avoid such a situation, the agent devices A and 10 change the "user utterance interpretation data A" generated in step S22 into the "user utterance interpretation data B" that the agent devices B and 20 can understand in step S25. Performs conversion (mapping) processing.
That is, the agent devices B and 20 perform a process of generating the user utterance interpretation data B (intent, slot) required for executing the process.

The agent devices A and 10 store the mapping correspondence data applied to the mapping processing in the storage unit, refer to the mapping correspondence data, and refer to the "user utterance interpretation data A" and the "user utterance interpretation data B". Performs the process of converting to.
The corresponding data for mapping stored in the storage unit is
"Intents and slots that make up user utterance interpretation data A",
It is composed of data corresponding to "tents and slots constituting the user utterance interpretation data B".

In step S25, the agent devices A and 10 refer to the mapping correspondence data stored in the storage unit, and the agent devices B and 20 can understand the "user utterance interpretation data A" generated in step S22. Performs a process of converting (mapping) into "user utterance interpretation data B".

Specifically, for example, as described above with reference to FIG. 4, the following conversion data (user utterance interpretation data B (intent b, slot b)) shown in FIG. 4 (step S02) is generated.
(Intent b) Pizza order (Slot b1) Delivery time = 12:00
(Slot b2) Type = Margherita

In the conversion data (user utterance interpretation data B (intent b, slot b)), the agent devices B and 20 have the following user utterances, that is,
"I want to order pizza. Deliver at 12:00, please Margherita."
This corresponds to the user utterance interpretation data B (intent b, slot b) generated according to the user utterance interpretation processing algorithm B executed by the agent devices B and 20 when the user utterance is directly input.

(Step S26)
Next, in step S26, the agent devices A and 10 transmit the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S25 to the agent devices B and 20.

The conversion data (user speech interpretation data B (intent b, slot b)) generated by the agent devices A and 10 in step S25 is, for example, the agent devices B, 20 or the management server of the agent devices B, 20. This is the data allowed as input data by the API (Application Programming Interface) provided by the above.
The agent devices A and 10 transfer the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S25 to the agent devices B and 20 by processing using this API (Call processing of the API). You can enter it.

(Step S27)
The agent devices B and 20 that have input the conversion data (user utterance interpretation data B (intent b, slot b)) from the agent devices A and 10 in step S26 then move the input conversion data (user) in step S27. The process based on the utterance interpretation data B (intent b, slot b)) is executed.

In the case of the embodiment described above with reference to FIG. 4, for example, the conversion data received from the agent devices A and 10 is such that the agent devices B and 20 make the following user utterances, that is,
"I want to order pizza. Deliver at 12:00, please Margherita."
This is the same as the user utterance interpretation data B (intent b, slot b) generated according to the user utterance interpretation processing algorithm B executed by the agent devices B and 20 when the user utterance is directly input.
Therefore, the agent devices B and 20 can accurately understand the intention of the user 1 and execute accurate processing as in the case where the user utterance is directly input from the user 1.

Further, in the agent devices B and 20, it is not necessary to perform user utterance interpretation processing such as voice recognition processing, semantic analysis processing, and dialogue state estimation processing for the user utterance, and the processing can be smoothly performed without delay. It becomes.

[5. Example of different user utterance interpretation data for each agent device]
Next, an example of different user utterance interpretation data for each agent device will be described.

As described above, the user utterance interpretation processing algorithm executed by each of the agent devices often differs depending on the type of the agent device.

For example, even if the same user utterance is input to the three agent devices A to C, if the user utterance interpretation processing algorithm executed by each agent device is different, the user utterance generated by each agent device A to C is generated. Interpretation data (intents, slots) may differ.

In FIG. 9, the user utterance interpretation data A (intent, slot) generated by one agent device A and various agent devices X other than the agent device A execute processing for various user utterances. An example of the corresponding data of the user utterance interpretation data X (intent, slot) required for this is shown.
The agent device X is not a single agent device, but various agent devices different from the agent device A.

(1) shown in FIG. 9 is
User utterance = "Set the washing machine timer to 12:00"
Correspondence between the user utterance interpretation data A (intent, slot) generated by the agent device A for the above user utterance and the user utterance interpretation data X (intent, slot) required for the agent device X to execute the process. Shows the data.
In this example (1), the intents and slots corresponding to these two agent devices A and X are the same as follows.
(Intent) Washing machine timer setting (Slot) Start time = 12:00

In the case of this example (1), the agent device A corresponds to the agent device X as it is without converting the user utterance interpretation data A (intent, slot) generated by applying the user utterance interpretation algorithm A. By inputting data to the agent device X via the API, the agent device X can be made to execute the correct process according to the user's utterance.

(2) and (3) shown in FIG. 9 are the same as in (1).
User utterance = "Set the washing machine timer to 12:00"
Correspondence between the user utterance interpretation data A (intent, slot) generated by the agent device A for the above user utterance and the user utterance interpretation data X (intent, slot) required for the agent device X to execute the process. Shows the data.
The example of (2) is a similar example in which the intents and slots corresponding to the agent devices A and X do not completely match.
The example of (3) is an example in which the intents corresponding to the agent devices A and X match, but the number of slots does not match.

In these cases, the agent device A does not convert the user utterance interpretation data A (intent, slot) generated by applying the user utterance interpretation algorithm A, and the agent device A directly passes through the agent device X-compatible API. If input to X, the agent device X may not be able to perform correct processing according to the user's utterance.

In such a case, the agent device A converts the user utterance interpretation data A (intent, slot) generated by applying the user utterance interpretation algorithm A, and enters each of the entries (2) and (3) in FIG. It is necessary to generate conversion data corresponding to the user utterance interpretation data X (intent, slot) of the above and input it to the agent device X via the API corresponding to the agent device X.
By performing such conversion data generation and input processing, it is possible to cause the agent device X to execute correct processing according to the user's utterance.

9 (4) and 9 (5) show the user utterance interpretation data A (intent, slot) generated by the agent device A for a certain user utterance, and the user required for the agent device X to execute the process. This is an example in which the utterance interpretation data X (intent, slot) is different.
These are examples in which the intent generated by the agent device A is lower-level data of the intent required for the agent device X to execute the process.

In FIGS. 9 (6) and 9 (7), the user utterance interpretation data A (intent, slot) generated by the agent device A for a certain user utterance and the user required for the agent device X to execute the process are also shown. This is an example in which the utterance interpretation data X (intent, slot) is different.
These are examples in which the intent generated by the agent device A is higher-level data of the intent required for the agent device X to execute the process.

Also in the case of the settings of these examples (4) to (7), the agent device A converts the user utterance interpretation data A (intent, slot) generated by applying the user utterance interpretation algorithm A, and shows the figure. Generate conversion data corresponding to the user utterance interpretation data X (intent, slot) of each entry of 9 (4) to (7) and input it to the agent device X via the API corresponding to the agent device X. Is required.
By performing such conversion data generation and input processing, it is possible to cause the agent device X to execute correct processing according to the user's utterance.

In the examples shown in FIGS. 9 (6) and 9 (7), time information not included in the user's utterance is set in the slot required for the agent device X to execute the process.
As described above, when the slot required for the agent device X to execute the process contains information not included in the user's speech, the agent A is required for the agent device X to execute the process. Performs processing such as asking a user a question in order to acquire information corresponding to the slot.
An example of this processing will be described later.

FIG. 10 is a diagram illustrating a specific processing example of the agent devices A and 10 corresponding to the example shown in FIG. 9 (5).

Each step of steps S21 to S27 shown in FIG. 10 corresponds to each step of steps S21 to S27 shown in the sequence diagram of FIGS. 6 and 7 described above.

First, in step S21, a dialogue process is executed between the user 1 and the agent devices A and 10, and the user 1 executes the following user utterance.
User utterance = "Searching for a Japanese restaurant and opening it in Osaki from 17:00"

Next, in step S22, the agent devices A and 10 execute the user utterance interpretation process by the voice recognition process, the semantic analysis process, and the dialogue state estimation process of the user utterance acquired in the dialogue process of step S21.

This processing is performed according to the user utterance interpretation algorithm A executed by the data processing unit (voice analysis unit) of the agent devices A and 10.
The agent devices A and 10 generate the following user utterance interpretation data A as a result of the user utterance interpretation process.
(Intent) Japanese restaurant search (Slot 1) Rotation time = 17:00
(Slot 2) Location = Osaki

Next, in step S25, the agent devices A and 10 convert (map) the "user utterance interpretation data A" generated in step S22 into "user utterance interpretation data B" that the agent devices B and 20 can understand. Perform processing.
That is, the agent devices B and 20 perform a process of generating the user utterance interpretation data B (intent, slot) required for executing the process.

The agent devices A and 10 generate the following conversion data (user utterance interpretation data B (intent, slot)) in step S25.
(Intent) Restaurant search (Slot 1) Opening time = 17:00
(Slot 2) Location = Osaki (Slot 3) Genre = Japanese food

Next, in step S26, the agent devices A and 10 transmit the conversion data (user utterance interpretation data B (intent, slot)) generated in step S25 to the agent devices B and 20.
For example, it is input to the agent devices B and 20 by processing using the API provided by the agent devices B and 20 or the management server of the agent devices B and 20 (Call processing of the API).

The agent devices B and 20 that have input the conversion data (user utterance interpretation data B (intent, slot)) from the agent devices A and 10 in step S26 then in step S27, input conversion data (user utterance interpretation). Process based on data B (intent, slot)) is executed.

Specifically, the process of searching for a Japanese restaurant that is open from 17:00 in Osaki is executed.
The processing result is transmitted from the agent devices B and 20 to the agent devices A and 10, and is presented to the user 1 via the output unit (display unit and speaker) of the agent devices A and 10.

[6. Processing flow executed by the agent device]
Next, the processing flow executed by the agent device will be described.

FIG. 11 The processing sequence executed by the agent device that directly executes the dialogue with the user will be described with reference to the following flowcharts.
Most of the processes shown in the flowcharts shown in FIGS. 11 and 11 are the processes executed by the agent devices A and 10 that directly execute the dialogue with the user, that is, the examples described with reference to FIGS. 4 to 10. be.
A part also includes an agent device that does not directly interact with the user, that is, a process executed by the agent devices B and 20 in the examples described with reference to FIGS. 4 to 10.

In the flow shown in FIG. 11 and below, the processing executed by the agent device A (corresponding to the agent devices A and 10) that directly interacts with the user is shown by a solid line, and the agent device B (agent) that does not directly interact with the user. The processing executed by the devices B and 20) is shown by the dotted line.

FIG. 11 The process according to the flowchart shown below is executed according to the program stored in the storage unit of the agent device A. For example, it can be executed as a program execution process by a processor such as a CPU having a program execution function.

First, the processing of each step of the flow shown in FIG. 11 will be described.

(Step S101)
First, the agent device A, which directly interacts with the user, inputs the user's utterance in step S101.

(Step S102)
Next, in step S102, the agent device A executes the user utterance interpretation process by the voice recognition process, the semantic analysis process, and the dialogue state estimation process of the user utterance input in step S101.

This processing is performed according to the user utterance interpretation algorithm A executed by the data processing unit (voice analysis unit) of the agent device A.
In this user utterance interpretation process, the context (dialogue history information) stored in the storage unit of the agent device A is referred to. In the storage unit of the agent device A, for example, the past context of each user, that is, the dialogue history information is stored. By referring to the context, for example, the user (U1) often talks about cooking. Alternatively, the user (U2) can acquire data such as a lot of stories about cars and travel, and can analyze the user's intent with high accuracy based on these data.

(Step S103)
Next, in step S103, the agent device A is an element that is a meaningful element (significant element) included in the user utterance intention (Intent) or the utterance as a result of the user utterance interpretation process in step S102. User utterance interpretation data A composed of information (slot: Slot) is generated.

Specifically, for example, the following user utterance interpretation data A described above with reference to FIG. 4 is generated.
(Intent a) Delivery (Slot a1) Delivery time = 12:00
(Slot a2) Food = Pizza (Slot a3) Type = Margherita

(Steps S104 to S105)
Next, in steps S104 to S105, the agent device A determines whether or not the agent device A itself can execute the process according to the user utterance based on the user utterance interpretation data acquired in step S103.
If it is determined that the processing by the agent device A itself is possible, the process proceeds to step S106.
On the other hand, if it is determined that the processing by the agent device A itself is not possible, the process proceeds to step S201.

(Step S106)
If it is determined in step S105 that the process corresponding to the user's utterance can be executed by the agent device A itself, the process proceeds to step S106.
In this case, the agent device A executes the process according to the user's utterance in step S106.

(Step S107)
After executing the process according to the user's utterance in step S106, the agent device A stores the context such as the dialogue history information between the agent device A and the user in the storage unit in step S107.

Next, in step S105, a process when it is determined that the process according to the user's utterance cannot be executed by the agent device A itself will be described with reference to the flowchart shown in FIG.

(Step S201)
If it is determined in step S105 that the process corresponding to the user's utterance cannot be executed by the agent device A itself, the agent device A searches for another communicable agent device capable of processing the user request in step S201.

For example, an agent device list as shown in FIG. 8 described above is held in the storage unit of the agent device A, and the agent device of the user request transfer destination is selected with reference to this list.

Note that, as in steps S23 to S24 of the sequence described above with reference to FIGS. 6 and 7, the agent device A notifies the user that it cannot be executed, and the user transfers the request to another agent device. The request transfer process may be started on the condition that the request is made.

(Step S202)
Next, in step S202, the agent device A determines whether or not another communicable agent device capable of processing the user request is detected in the search process in step S201.
If detected, the process proceeds to step S204.
On the other hand, if it is not detected, the process proceeds to step S203.

(Step S203)
If it is determined in the determination process in step S202 that no other communicable agent device capable of processing the user request has been detected, the agent device A informs the user that the process based on the user's utterance cannot be executed in step S203. Generates a system utterance for the user and outputs it to the user.

(Step S204)
On the other hand, if it is determined in the determination process in step S202 that another communicable agent device (agent device B) capable of processing the user request is detected, the agent device A executes the following process in step S204.

The agent device A converts (maps) the "user utterance interpretation data A" generated in the previous step S103 into "user utterance interpretation data B" that can be understood by the communicable agent device B that can process the user request. To do.
That is, the agent devices B and 20 perform a process of converting (mapping) the user utterance interpretation data B (intent, slot) required for executing the process.

As described above, the agent device A stores the mapping correspondence data applied to this mapping process in the storage unit, refers to the mapping correspondence data, and converts the “user utterance interpretation data A” into the “user utterance interpretation data A”. Performs the process of converting to "data B".
The corresponding data for mapping stored in the storage unit is
"Intents and slots that make up user utterance interpretation data A",
It is composed of data corresponding to "tents and slots constituting the user utterance interpretation data B".

In step S204, the agent device A refers to the mapping correspondence data stored in the storage unit, and the agent device B can understand the “user utterance interpretation data A” generated in step S103 as “user utterance interpretation data”. Performs the process of converting (mapping) to "B".

(Step S205)
Next, in step S205, the agent device A transmits the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S204 to the agent device B.

The conversion data (user speech interpretation data B (intent b, slot b)) generated by the agent device A in step S204 is, for example, an API (Application) provided by the agent device B, the management server of the agent device B, or the like. This is the data allowed by the Programming Interface) as input data.
The agent device A may input the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S204 to the agent device B by a process using this API (Call process of API). can.

(Step S206)
The process of step S206 is a process executed by the agent device B.
In step S205, the agent device B that has input the conversion data (user utterance interpretation data B (intent b, slot b)) from the agent device A has input conversion data (user utterance interpretation data B (intent)) in step S206. b, the process based on slot b)) is executed.

The conversion data (user utterance interpretation data B (intent b, slot b)) input from the agent device A is the user utterance executed by the agent device B when the agent device B directly inputs the user utterance in step S101. This is data corresponding to user utterance interpretation data B (intent b, slot b) generated according to the interpretation processing algorithm B.
Therefore, the agent device B can accurately understand the user's intention and execute accurate processing as in the case where the user's utterance is directly input from the user.

Further, in the agent device B, it is not necessary to perform user utterance interpretation processing such as voice recognition processing, semantic analysis processing, and dialogue state estimation processing for the user utterance, and the processing can be smoothly performed without delay. ..

(Step S207)
Next, in step S207, the agent device A receives the processing completion notification from the agent device B.

(Step S208)
Finally, the agent device A notifies the user that the process is completed in step S208. For example, a system utterance for notifying the user that the process according to the user's utterance is completed, such as "process completed" or "I ordered pizza", is generated and output to the user.

In this way, by performing the processing according to the flowcharts shown in FIGS. 11 and 12, even if the agent device to be spoken by the user cannot execute the processing according to the user's utterance, the other agent device can be used. It is possible to execute the process.
As a result, the user does not have to move to a place where another agent device is located and repeat the same utterance, and the burden on the user is reduced.
That is, if the user makes one utterance to one agent device, the user can perform processing using all the functions of the plurality of agent devices.

[7. About processing example when there is not enough data to execute processing]
Next, a processing example when there is insufficient data for executing the processing will be described.

As described with reference to the flowcharts shown in FIGS. 11 and 12, if the agent device A that directly inputs the user utterance cannot execute the process according to the user utterance, another communicable agent device B Will be requested to process.

In this case, the agent device A converts the user utterance interpretation data A (intent a, slot a) generated based on the user utterance, and executes the process. The understandable user utterance interpretation data B ( Intent b, slot b) is generated.

However, the agent device B cannot execute the process only with the data included in the user utterance interpretation data B (intent b, slot b) generated by converting the user utterance interpretation data A (intent a, slot a). In some cases.

For example, the data shown in FIG. 9 described above, that is, the user utterance interpretation data A (intent, slot) generated by one agent device A for various user utterances, and various data other than the agent device A. It is the data of the entry (3) and the entries (6) and (7) in the corresponding data example of the user utterance interpretation data X (intent, slot) required for the agent device X to execute the process.

For example, FIG. 9 (3) shows
User utterance = "Set the washing machine timer to 12:00"
Correspondence between the user utterance interpretation data A (intent, slot) generated by the agent device A for the above user utterance and the user utterance interpretation data X (intent, slot) required for the agent device X to execute the process. Shows the data.

Here, the user utterance interpretation data X (intent, slot) required for the agent device X of the entry (4) to execute the process is the following data.
(Intent) Washing machine timer setting (Slot 1) Start time = 12:00
(Slot 2) Start mode =?

Among these intents, slots 1 and 2, intents and slots 1 are obtained from the user utterance "Set the timer of the washing machine to 12:00".
(Slot 2) Start mode This slot is data that is not acquired from the above user utterance.
In order to perform accurate processing (= timer setting processing of the washing machine) in the agent device X,
(Slot 2) Start mode This slot data is required, and this data cannot be obtained from the above user utterance "Set the timer of the washing machine to 12:00".

Similarly, FIGS. 9 (6) and 9 (7) are shown in FIGS. 9 (6) and 9 (7), respectively.
User utterance = "Search Italian and search in Osaki"
User utterance = "Search for a Japanese restaurant and look for it in Osaki"
Correspondence between the user utterance interpretation data A (intent, slot) generated by the agent device A for the above user utterance and the user utterance interpretation data X (intent, slot) required for the agent device X to execute the process. Shows the data.

The data shown in FIG. 9 includes user utterance interpretation data X (intents, slots) required for these agent devices X to execute processing.
(Slot 1) Opening hours = 17: 00-23: 00
Although this slot is included, this (slot) is data that is not acquired from the above user utterance.
In order to perform accurate processing (restaurant search processing) in the agent device X,
(Slot 1) Opening hours = 17: 00-23: 00
This slot data is required and this data cannot be obtained from user utterances.

In this way, the agent device A that inputs the user utterance converts the user utterance interpretation data A (intent a, slot a) generated based on the user utterance, and the user utterance interpretation data B (intent b, slot a). Even if b) is generated and transmitted to the agent device B, the agent device B may not be able to execute the process.

In such a case, the agent device B requests the agent device A to acquire the information necessary for executing the process.
The agent device A asks a question to the user in response to the information acquisition request from the agent device B, acquires a response from the user, and transmits data based on this response to the agent device B.
By performing this additional information transmission processing, the agent device B can accurately execute the processing according to the user's utterance.

This processing sequence will be described with reference to the flowchart shown in FIG.
The flowchart shown in FIG. 13 shows a process to be executed following step S205 of the flow described above with reference to FIGS. 11 and 12.
In the flow shown in FIG. 13, the step shown by the solid line is the process executed by the agent device A that directly inputs the user utterance, and the step shown by the dotted line is the process that communicates with the agent device A and responds to the user utterance. This is a process executed by the agent device B that executes the above.

The processing of each step of the flow shown in FIG. 13 will be described.
(Step S205)
The process of step S205 is the process described above with reference to FIG. That is, in step S205, the agent device A transmits the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S204 to the agent device B.

(Step S211)
The processes of steps S211 to S213 are processes executed by the agent device B.
First, the agent device B executes a verification process of the conversion data (user utterance interpretation data B (intent b, slot b)) received from the agent device A in step S204.

Specifically, it is determined whether or not all the data required for processing in the agent device B is included in the received data from the agent device A. That is, it is determined whether or not there is insufficient information in the user utterance interpretation data B (intent b, slot b) which is the conversion data received from the agent device A.
If it is determined that there is insufficient information, the process proceeds to step S212.

On the other hand, if it is determined that there is no insufficient information and the process can be executed, the process proceeds to step S206. In this case, the processes of steps S206 to S208 described above with reference to FIG. 12 are executed.

(Step S212)
If the agent device B determines in step S211 that the user utterance interpretation data B (intent b, slot b), which is the conversion data received from the agent device A, has insufficient information, the agent device B processes the process in step S212. To execute.

In step S212, the agent device B requests the agent device A to ask a question to the user for acquiring the shortage information.

In response to the request from the agent device B, the agent device A generates a question for acquiring the shortage information and outputs the question to the user. That is, the question is executed, and the process for the user's utterance in response to the user's response to the question is executed.
The process executed for the new user utterance is the same process as the processes of steps S102 to S103 and steps S204 to S205 described above with reference to FIGS. 11 and 12.

(Step S213)
Next, the agent device B acquires additional information from the agent device in step S213, and further returns to step S211 to determine whether or not there is insufficient information.

If there is still insufficient information even after inputting additional information, the processes of steps S211 to S213 are repeated.
If it is determined in step S211 that there is no insufficient information, the process proceeds to step S206 to execute the process.

In this way, when the agent device B determines that the information required for executing the process is insufficient, the agent device B requests the agent device A to acquire the missing information and retransmits the missing information. , Align the data required for processing and execute the processing.
By performing this additional information transmission processing, the agent device B can accurately execute the processing according to the user's utterance.

In the process described with reference to the flowchart shown in FIG. 13, the agent device B that executes the process according to the user's utterance determines whether or not there is insufficient information required for the process execution.
In addition to such a processing mode, it is also possible for the agent device A to determine whether or not the information required for processing by the agent device B is insufficient.
An example of the processing sequence when this processing is performed will be described with reference to the flowchart shown in FIG.

Steps S204 and S205 to S208 in the flow shown in FIG. 14 are the same processes as steps S204 and S205 to S208 of the flow described above with reference to FIG.
Steps S221 to S224 are newly added processes. The processing of steps S221 to S224 is executed by the agent device A following the processing of step S204.
The processing of steps S204 and S221 to S224 will be described.

(Step S204)
When it is determined that another communicable agent device (agent device B) capable of processing the user request is detected in the determination process in step S202 described above with reference to FIG. 12, the agent device A determines in step S204. Execute the following processing.

The agent device A converts (maps) the "user utterance interpretation data A" generated in the previous step S103 into "user utterance interpretation data B" that can be understood by the communicable agent device B that can process the user request. To do.
That is, the agent devices B and 20 perform a process of converting (mapping) the user utterance interpretation data B (intent, slot) required for executing the process.

As described above, the agent device A stores the mapping correspondence data applied to the mapping process in the storage unit, and refers to the mapping correspondence data to refer to the “user utterance interpretation data A” and the “user utterance interpretation data A”. Perform the process of converting to "B".

(Step S211)
Next, in step S211 of the agent device A, the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S204 is sufficient as information necessary for processing in the agent device B. Or determine if there is a shortage.

It should be noted that it is possible to determine whether or not there is a shortage by referring to, for example, the mapping data stored in the storage unit described above.
If it is determined that there is insufficient information, the process proceeds to step S222.

On the other hand, if it is determined that there is no insufficient information and the process can be executed, the process proceeds to step S206. In this case, the processes of steps S206 to S208 described above with reference to FIG. 12 are executed.

(Step S222)
If it is determined in step S221 that the conversion data (user utterance interpretation data B (intent b, slot b)) generated in step S204 is insufficient as information necessary for processing in the agent device B, the agent device A executes the process of step S222.

In step S222, the agent device A generates a question for acquiring the missing information and outputs it to the user, and inputs a new user utterance as a response to the question from the user.

(Steps S223 to S224)
Next, the agent device A executes the processes of steps S223 to S224 for a new user utterance as a response to the question.

This process is the same as the process of steps S102 to S103 described above with reference to FIGS. 11 and 12.
That is, in step S223, the user utterance interpretation process by the voice recognition process, the semantic analysis process, and the dialogue state estimation process of the new user utterance for the question input in step 222 is executed.
Further, in step S224, the user utterance interpretation data A (intent, Intent) composed of the intention of the user utterance (Intent) and the element information (slot: Slot) which is a meaningful element (significant element) included in the utterance. Slot) is created.

After these processes, the process returns to step S204, and the user utterance interpretation data A (intent, slot) additionally generated in step S224 is used as the user utterance interpretation data A (intent, slot) generated based on the previous user utterance. ), And based on the new user utterance interpretation data A (intent, slot) after the addition, the process of generating (mapping) the "user utterance interpretation data B" is performed.

Further, in step S221, it is determined whether or not the newly generated "user utterance interpretation data B" is sufficient as information necessary for processing in the agent device B. If there is a shortage, the processes of steps S222 to S224 and S204 to S221 are repeated.

If it is determined that the "user utterance interpretation data B" newly generated in step S221 is sufficient as the information necessary for processing in the agent device B, the process proceeds to step S205, and the conversion data (user utterance interpretation data B) is obtained. ) Is transmitted to the agent device B.

After that, in step S206, the agent device B executes a process based on the transmitted data.
The agent device B can execute the process after acquiring all the information necessary for executing the process, and can accurately execute the process according to the user's utterance.

[8. About management processing example of multiple user utterances]
Next, an example of management processing of a plurality of user utterances will be described.

The agent device A that directly inputs the user utterance may continuously input the user utterance from the same user or a plurality of different users.

The agent device A must manage each of the plurality of user utterances input in this way by distinguishing whether the utterance is about one common task (process) or another task (process). , There is a possibility that wrong processing will be performed.

In order to prevent the occurrence of such an erroneous process, the agent device records the dialogue history information in which the task ID is set for each user utterance, that is, the context in the storage unit.
FIG. 15 is a diagram showing a data example of dialogue history information (context) recorded in the storage unit.

FIG. 15 shows an example of registration data related to the following two user utterances input at the time (t1) and (t2).
User utterance at time t1 = I want to order pizza. User utterance at time t2 at 12:00 delivery = one Margherita is registered data of dialogue history information (context) related to these two user utterances.

The data processing unit of the agent device A for inputting these user utterances generates the registration data of the dialogue history information (context) shown in FIG. 15 and stores it in the storage unit.
That is, the following corresponding data is generated and recorded for each user's utterance.
(A) Task ID
(B) User utterance (text data)
(C) Spoken user identifier (d) User utterance interpretation data (intent, slot)
(E) Processing execution agent identifier

If the task ID is an utterance related to one task, the same ID is set.
The example shown in FIG. 15 is
User utterance at time t1 = I want to order pizza. This is an example in which the data processing unit of the agent device A determines that these two user utterances are the same task with one user utterance at time t2 at 12:00 and the same task ID is set.

As a criterion for determining whether or not to set the same task ID, for example, the time interval of each utterance and the similarity of the domain corresponding to the user utterance estimated in the utterance interpretation process are used.

Specifically, when the time interval between a plurality of user utterances is equal to or less than a predetermined threshold time, these user utterances are determined to be the same task, and the same task ID is set.
Further, when the domains corresponding to the user utterances estimated in the utterance interpretation process executed for each user utterance are similar, it is determined that these user utterances are the same task, and the same task ID is set.

The domain is a semantic area of user utterance, and is a superordinate concept of the intent described above.
for example,
User utterance = An example of data obtained as utterance interpretation data of this user utterance by setting the timer of the washing machine to 12:00 is
(Intent) Washing machine timer setting (Slot) Start time = 12:00
These data, but also as (domain) indicating the semantic area of the superordinate concept of (intent)
(Domain) Home appliance control It is estimated that such a domain is also used.

That is, the agent device determines the utterance interpretation data corresponding to the domain, the intention, and the slot in the utterance interpretation process for the user's utterance.

The agent device executes the user utterance interpretation process for the input user utterance, and if the plurality of user utterance intervals are within the threshold time and the domains are similar, the utterances related to the same task are used. It is determined that there is, and the same task ID is set and recorded.
The two user utterances shown in FIG. 15 are data examples of dialogue history information (context) related to such utterances.

FIG. 16 is a data example of dialogue history information (context) related to utterance different from that of FIG.
FIG. 16 shows an example of registration data related to the following two user utterances input at the times (t1) and (t2).
User utterance at time t1 = I want to order a pizza with tomatoes. User utterance at time t2 with delivery at 12:00 = timer is set at 8:00 tomorrow morning, and it is the registration data of dialogue history information (context) related to these two user utterances.

When these user utterances are continuously input to the agent device, the agent device executes the user utterance interpretation process for the input user utterances.
If the utterance intervals of a plurality of users are within the threshold time and the domains are similar, it is determined that the utterances are related to the same task, and the same task ID is set and recorded.

However, in the case of the example shown in FIG.
User utterance at time t1 = I want to order a pizza with tomatoes. User utterances at time t2 with delivery at 12:00 = timers are set at 8:00 tomorrow morning, and the two utterance-compatible domains obtained as a result of the utterance interpretation processing for these two user utterances are different domains.
for example,
User utterance at time t1 = I want to order a pizza with tomatoes. Delivered at 12:00 This user-spoken domain is, for example,
(Domain) Delivery processing.

on the other hand,
User utterance at time t2 = Set a timer at 8:00 tomorrow morning, and the domain of this user utterance is, for example.
(Domain) Timer setting.

Therefore, the data processing unit of the agent device determines that these two user utterances are not utterances related to the same task, and sets different task IDs for these two utterances.

The processing for a plurality of utterances for which the same task ID is set is basically executed by one agent device. The processing for a plurality of utterances in which different task IDs are set may be performed by an agent device that executes the processing for each task ID unit.

[9. About other examples]
Next, other examples will be described.

In the above-described embodiment, there are two units, an agent device A that directly inputs the user utterance and an agent device B that communicates with the agent device A and executes processing based on the user utterance without directly inputting the user utterance. An example based on a system using an agent device has been described.

In addition, for example, as shown in FIG. 17, a system configuration example using three or more agent devices may be used.
The system shown in FIG. 17 includes agent devices A and 10 that directly input user utterances, and agent devices that communicate with agent devices A and 10 without directly inputting user utterances and execute processing based on user utterances. This is a system that uses three agent devices, B, 20 and agent devices C, 30.

In the storage unit of each of the agent devices A to C, a list of agent devices as described above with reference to FIG. 8 is recorded. For example, the agent devices A and 10 refer to this list to select the agent device to which the user request is forwarded.

As described above, the agent device list shown in FIG. 8 is a list of agent devices recorded in association with identifiers of other agent devices capable of communication, functions of the agent devices, and communication addresses.
The agent devices A and 10 can perform a process of selecting an agent device as a user request transfer destination by referring to the agent device list shown in FIG. 8 stored in the storage unit of the agent devices A and 10.

Further, in the above-described embodiment, the agent devices A and 10 that directly input the user utterance do not directly input the user utterance, but communicate with the agent devices A and 10 to execute the process based on the user utterance. An embodiment of transmitting "user utterance interpretation data B" that can be understood by the agent devices B and 20 to the devices B and 20 has been described.

The processing executed by the agent devices A and 10 is the user utterance interpretation data A (intent a, slot a) generated according to the user utterance interpretation processing algorithm A, and the user utterance interpretation data B (intent a, slot a) that the agent devices B and 20 can understand. This is a process (mapping process) of converting to the intent b) and the slot b).
The agent devices A and 10 transmit the user utterance interpretation data B (intent b, slot b) that the agent devices B and 20 can understand to the agent devices B and 20.

The conversion data (user speech interpretation data B (intent b, slot b)) generated by the agent devices A and 10 is, for example, an API provided by the agent devices B and 20 or the management server of the agent devices B and 20. This is the data allowed as input data by the Application Programming Interface).
The agent devices A and 10 input the generated conversion data (user utterance interpretation data B (intent b, slot b)) to the agent devices B and 20 by a process using this API (Call process of API). Can be done.

For example, if this API allows not only the conversion data (user utterance interpretation data B (intent b, slot b)) but also the user utterance data itself (text data) as input data, the agent device A, 10 may transmit the user utterance data itself (text data) to the agent devices B and 20 without generating the conversion data (user utterance interpretation data B (intent b, slot b)).

The data format input from the agent devices A and 10 to the agent devices B and 20 may be any data allowed by the API used for data input. For example, the following data formats are used depending on the API settings. It may be possible.
1. 1. User utterance itself (text data)
2. Normalized data of user utterances (for example, data obtained by converting hiragana into kanji, data obtained by correcting particle omissions and notational fluctuations, etc.)
4. Data converted into easy-to-interpret sentences according to the characteristics of the destination agent device

Further, not only the data based on the user's utterance but also the user's face image, the sensor detection information by other sensors, and the like may be transmitted together.

Further, in the above-described embodiment, when the agent devices A and 10 that directly input the user utterance cannot execute the process according to the user utterance, the agent devices B and 20 are requested to perform the process. did.

In addition, for example, the processing for the user's utterance may be distributed according to the ability of each agent device for natural language processing (NLP: Natural Language Processing).

For example, assume that there are the following three agent devices that can be used by the user.
Agent device A
Agent device B
Agent device C
Each of these agent devices has the ability of natural language processing (NLP: Natural Language Processing), the ability of speech recognition processing (ASR: Automatic Speech Recognition), the ability of speech synthesis processing (TTS: Text To Speech), and the equipment. The type of sensor is different.

In the case of such a setting, the optimum agent device that executes the process corresponding to the user utterance is selected according to the content of the user utterance and the like, and the selected agent device executes the process.
With such a configuration, processing corresponding to user utterance can be executed in a more optimal agent device.

In order to realize this process, it is necessary to record in the storage unit of each agent device a list obtained by further expanding the agent list shown in FIG. 8 described above. That is, a list recording information such as the ability of each agent device for natural language processing (NLP), speech recognition processing (ASR), speech synthesis processing (TTS), and the type of attached sensor is stored in the storage unit. Select the agent device to store and refer to this list to execute the process corresponding to the user's speech.

[10. About the configuration example of the agent device (information processing device)]
Next, a configuration example of the information processing device of the present disclosure, that is, the agent device will be described.

FIG. 18 is a diagram showing a configuration example of an agent device 100 (information processing device) that recognizes a user utterance and performs a process or a response corresponding to the user utterance.

As shown in FIG. 18, the agent device 100 includes an input unit 110, an output unit 120, a data processing unit 130, a communication unit 140, and a storage unit 150.

The input unit 110 includes a voice input unit (microphone) 111, an image input unit (camera) 112, and a sensor 113.
The output unit 120 includes an audio output unit (speaker) 121 and an image output unit (display unit) 122.

The voice input unit (microphone) 111 corresponds to, for example, the microphone 12 of the agent device 10 shown in FIG.
The image input unit (camera) 112 corresponds to the camera 11 of the agent device 10 shown in FIG.
The audio output unit (speaker) 121 corresponds to the speaker 14 of the agent device 10 shown in FIG.
The image output unit (display unit) 122 corresponds to the display unit 13 of the agent device 10 shown in FIG.
The image output unit (display unit) 122 can be configured by, for example, a projector or the like, or can be configured by using the display unit of a television of an external device.

The data processing unit 130 includes an input data analysis unit 160 and a response processing execution unit 170.
The input data analysis unit 160 includes a voice analysis unit 161, an image analysis unit 162, a sensor information analysis unit 163, a user utterance correspondence processing execution control unit 164, and a data conversion unit 165.
The data processing execution unit 170 includes an output voice generation unit 171, a display information generation unit 172, and a user utterance response processing execution unit 173.

The user's spoken voice is input to a voice input unit 111 such as a microphone.
The voice input unit (microphone) 111 inputs the input user-spoken voice to the voice analysis unit 161.
The voice analysis unit 161 has, for example, an ASR (Automatic Speech Recognition) function, and converts voice data into text data composed of a plurality of words.
Further, the utterance semantic analysis process for the text data is executed.
The voice analysis unit 161 has, for example, a natural language understanding function such as NLU (Natural Language Understanding), and from text data, the intention (intent) of the user's utterance and the meaningful element (significant element) included in the utterance. ) (Slot: Slot) is estimated. That is, the user utterance interpretation process is executed to generate the user utterance interpretation data (intent, slot).

If the intention (intent) and the element (slot) can be accurately estimated and acquired from the user utterance, the agent device 100 can perform accurate processing for the user utterance.

The user utterance analysis information acquired by the voice analysis unit 161 is stored in the storage unit 150 and output to the user utterance correspondence processing execution control unit 164.

The image input unit 112 captures an image of the speaking user and its surroundings and inputs the image to the image analysis unit 162.
The image analysis unit 162 analyzes the facial expression of the utterance user, the user's behavior, the line-of-sight information, the surrounding information of the utterance user, etc. Output to 164.

The sensor 113 is composed of a sensor that acquires data necessary for analyzing, for example, temperature, atmospheric pressure, user's line of sight, and the like. The sensor acquisition information is input to the sensor information analysis unit 163.
The sensor information analysis unit 163 acquires data such as temperature, atmospheric pressure, and the user's line of sight based on the sensor acquisition information, stores the analysis result in the storage unit 150, and also stores the analysis result in the storage unit 150, and also stores the user speech response processing execution control unit 164. Output to.

The user utterance correspondence processing execution control unit 164 includes user utterance interpretation data (intents, slots) generated by the voice analysis unit 161, image analysis information generated by the image analysis unit 162, and a sensor generated by the sensor information analysis unit 163. By inputting the analysis information, it is determined whether the process corresponding to the user's utterance is executed by the own device or another communicable agent device.

When the user utterance correspondence processing execution control unit 164 decides to execute the processing corresponding to the user utterance by another agent device, the data conversion unit 165 determines the user utterance interpretation data A (intent) generated by the voice analysis unit 161. A process (mapping process) of converting a, slot a) into user utterance interpretation data B (intent b, slot b) that can be understood by other agent devices B, 20 that execute user utterance correspondence process is executed.
The converted data generated by the data conversion unit 165 is transmitted to another agent device B via the communication unit 140.

The storage unit 150 stores the content of the user's utterance, learning data based on the user's utterance, context information such as dialogue history information with the user, display data to be output to the image output unit (display unit) 122, and the like. ..

Further, in the storage unit 150, the user utterance interpretation data B (intent b, slot b) in which the user utterance interpretation data A (intent a, slot a) generated by the voice analysis unit 161 can be understood by another agent device. Mapping data for executing the process of converting to (mapping process) is stored.
Further, the storage unit 150 stores a list of agent devices in which functional information, access information (communication address), and the like of other communicable agent devices described above with reference to FIG. 8 are recorded.

The response processing execution unit 170 includes an output voice generation unit 171, a display information generation unit 172, and a user utterance response processing execution unit 173.
The output voice generation unit 171 generates system utterances for the user based on the user utterance analysis data which is the analysis result of the voice analysis unit 161.
The response voice information generated by the output voice generation unit 171 is output via the voice output unit 121 such as a speaker.

The display information generation unit 172 displays the text information of the system utterance to the user and other presentation information.
For example, when the user makes a user utterance to show the world map, the world map is displayed.
The world map can be obtained from, for example, a service providing server.

The user utterance response processing execution unit 173 executes the processing for the user utterance.
When the user utterance correspondence processing execution control unit 164 of the input data analysis unit 160 decides to execute the processing corresponding to the user utterance in its own device, the user utterance response processing execution unit 173 executes the processing for the user utterance.

for example,
User utterance = play music and show user utterance = interesting moving image In such an utterance, the user utterance correspondence processing execution unit 173 performs processing for user utterance, that is, music reproduction processing and moving image reproduction processing.

However, when the user utterance correspondence processing execution control unit 164 of the input data analysis unit 160 decides to execute the processing corresponding to the user utterance by another agent device, the data conversion unit 165 generates the data as described above. The converted data is transmitted to another agent device B via the communication unit 140, and the other agent device B executes a process corresponding to the user's utterance.

The communication unit 140 performs communication processing with other agent devices, communication processing with an external server, for example, a server that provides various information such as news information, weather information, music information, and a server that executes voice analysis processing. It is used for.
A part of the processing executed by the input data analysis unit 160 of the data processing unit 130 shown in FIG. 18, such as voice analysis processing and semantic analysis processing, may be configured to be executed by using an external server.

As shown in FIG. 19, the agent device (information processing device) 100 of the present disclosure is not limited to the so-called smart speaker type agent device, and can be in various device forms such as a smartphone or a PC. be.

The agent device (information processing device) 100 recognizes the utterance of the user 1 and performs a response based on the utterance of the user. For example, the agent device (information processing device) 100 also controls an external device 250 such as a television or an air conditioner in response to the utterance of the user.
For example, when the user utterance is a request such as "change the TV channel to 1" or "set the temperature of the air conditioner to 20 degrees", the agent device (information processing device) 100 determines the voice recognition result of the user utterance. A control signal (Wi-Fi, infrared light, etc.) is output to the external device 250 based on the above, and control according to the user's utterance is executed.

Further, the agent device (information processing device) 100 is connected to a server 200 that performs various data processing and information provision via a network. The agent device (information processing device) 100 can acquire information necessary for generating a response to a user's utterance from the server 200. Further, the server may be configured to perform voice recognition processing and semantic analysis processing.

[11. About hardware configuration example of agent device (information processing device)]
Next, a hardware configuration example of the agent device (information processing device) will be described with reference to FIG.
The hardware described with reference to FIG. 20 is one specific hardware configuration example of the agent device (information processing device) 100 described above with reference to FIG. 18, and also with reference to FIG. It is also an example of the hardware configuration of the information processing apparatus constituting the server 200 described above.

The CPU (Central Processing Unit) 301 functions as a control unit or a data processing unit that executes various processes according to a program stored in the ROM (Read Only Memory) 302 or the storage unit 308. For example, the process according to the sequence described in the above-described embodiment is executed. The RAM (Random Access Memory) 303 stores programs and data executed by the CPU 301. These CPU 301, ROM 302, and RAM 303 are connected to each other by a bus 304.

The CPU 301 is connected to the input / output interface 305 via the bus 304, and the input / output interface 305 is connected to an input unit 306 consisting of various switches, a keyboard, a mouse, a microphone, a sensor, etc., and an output unit 307 consisting of a display, a speaker, and the like. Has been done. The CPU 301 executes various processes in response to a command input from the input unit 306, and outputs the process results to, for example, the output unit 307.

The storage unit 308 connected to the input / output interface 305 is composed of, for example, a hard disk or the like, and stores a program executed by the CPU 301 and various data. The communication unit 309 functions as a transmission / reception unit for Wi-Fi communication, Bluetooth (registered trademark) (BT) communication, and other data communication via a network such as the Internet or a local area network, and communicates with an external device.

The drive 310 connected to the input / output interface 305 drives a removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card, and records or reads data.

[12. Summary of the structure of this disclosure]
As described above, the examples of the present disclosure have been described in detail with reference to the specific examples. However, it is self-evident that one of ordinary skill in the art can modify or substitute the examples without departing from the gist of the present disclosure. That is, the present invention has been disclosed in the form of an example, and should not be construed in a limited manner. In order to judge the gist of this disclosure, the column of claims should be taken into consideration.

The technology disclosed in the present specification can have the following configuration.
(1) A voice input unit for inputting user utterances and
It has a data processing unit that executes the analysis of the user utterance and generates the user utterance interpretation data.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
An information processing device that converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and transmits the converted data to the second information processing device.

(2) The user utterance interpretation data generated by the data processing unit is
The information processing device according to (1), which is data having an intent corresponding to the intention of the user's utterance and a slot corresponding to the element information included in the user's utterance.

(3) The data processing unit
Described in (2), the process of converting the user utterance interpretation data including the intents and slots generated by the data processing unit into the converted data including the intents and slots that can be understood by the second information processing apparatus is executed. Information processing device.

(4) The data processing unit
Map pink data in which intents and slots included in user speech interpretation data generated according to a user speech interpretation processing algorithm executed by the data processing unit are associated with intents and slots that can be understood by the second information processing apparatus. The information processing apparatus according to (2) or (3), which executes the conversion data generation process with reference to.

(5) The information processing device is
The information processing apparatus according to (4), which has a storage unit that stores the mapping data.

(6) The data processing unit
The information processing device according to any one of (1) to (5), wherein the converted data is input to an API (Application Programming Interface) that allows the second information processing device to input data that can be understood.

(7) The data processing unit
It is determined whether or not the process corresponding to the user's utterance can be executed by the own device, and if it is determined that the process cannot be executed, the converted data is generated and transmitted to the second information processing apparatus (1) to (6). ) The information processing device described in any of the above.

(8) The data processing unit
When the user inputs a request utterance to execute the process corresponding to the user utterance on another device,
The information processing apparatus according to any one of (1) to (7), which generates the converted data and transmits the converted data to the second information processing apparatus.

(9) The data processing unit
The external device according to any one of (1) to (8) is determined by referring to a list in which functional information and access information of each of the other communicable information processing devices are registered to execute a process corresponding to the user's utterance. Information processing device.

(10) The data processing unit is
When the generated conversion data does not include at least a part of necessary information for executing the process corresponding to user utterance in the second information processing apparatus, and there is insufficient information.
The information processing device according to any one of (1) to (9), which asks a user a question for acquiring the lack information.

(11) The data processing unit is
The information processing device according to any one of (1) to (10), wherein a task ID corresponding to a user utterance is set and a user utterance management process is executed.

(12) The data processing unit is
When a plurality of user utterances are input at time intervals within the threshold time and the domains of the plurality of user utterances are similar, the same task ID is set for the plurality of user utterances (11). ).

(13) An information processing system having a plurality of information processing devices.
The first information processing device is
A voice input unit for inputting user utterances,
It has a data processing unit that executes the analysis of the user utterance and generates the user utterance interpretation data.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
The user utterance interpretation data is converted to generate conversion data that can be understood by the second information processing device, and the data is transmitted to the second information processing device.
The second information processing device is
An information processing system that executes processing corresponding to the user's utterance based on the converted data received from the first information processing apparatus.

(14) The first information processing device is
User utterance interpretation data having an intent corresponding to the intention of the user utterance and a slot corresponding to the element information included in the user utterance is generated.
The information processing system according to (13), wherein the conversion data having an intent and a slot that the second information processing apparatus can understand is generated and transmitted to the second information processing apparatus.

(15) An information processing method executed in an information processing device.
The voice input section inputs the user's utterance,
The data processing department
Data processing that executes the analysis of the user utterance and generates the user utterance interpretation data is executed.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
An information processing method that converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and transmits the converted data to the second information processing device.

(16) An information processing method executed in an information processing system having a plurality of information processing devices.
The first information processing device
When the external second information processing device executes the processing corresponding to the input user utterance
The analysis of the user utterance is executed to generate the user utterance interpretation data, and the user utterance interpretation data is generated.
A process of converting the user utterance interpretation data, generating converted data that can be understood by the second information processing device, and transmitting the converted data to the second information processing device is executed.
The second information processing device
An information processing method that executes processing corresponding to the user's utterance based on the converted data received from the first information processing apparatus.

(17) A program that executes information processing in an information processing device.
The program is installed in the data processing unit.
Analyze user utterances to generate user utterance interpretation data
When the external second information processing device executes the process corresponding to the user's utterance,
A program that converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and causes the second information processing device to transmit the converted data.

In addition, the series of processes described in the specification can be executed by hardware, software, or a composite configuration of both. When executing processing by software, install the program that records the processing sequence in the memory in the computer built in the dedicated hardware and execute it, or execute the program on a general-purpose computer that can execute various processing. It can be installed and run. For example, the program can be pre-recorded on a recording medium. In addition to installing on a computer from a recording medium, the program can be received via a network such as LAN (Local Area Network) or the Internet and installed on a recording medium such as a built-in hard disk.

The various processes described in the specification are not only executed in chronological order according to the description, but may also be executed in parallel or individually as required by the processing capacity of the device that executes the processes. Further, in the present specification, the system is a logical set configuration of a plurality of devices, and the devices having each configuration are not limited to those in the same housing.

As described above, according to the configuration of the embodiment of the present disclosure, a device and a method capable of executing a process according to the user's utterance in a device other than the device in which the user's utterance is input are realized. ..
Specifically, for example, when the external second information processing device is made to execute the process corresponding to the user utterance, the user utterance analysis is executed to generate the user utterance interpretation data, and the generated user utterance interpretation data is converted. Then, the conversion data that can be understood by the second information processing device is generated and transmitted to the second information processing device. The user utterance interpretation data has an intent corresponding to the intention of the user utterance and a slot corresponding to the element information included in the user utterance, and the data processing unit uses the user utterance interpretation data including the intent and the slot as a second. Converts the data to include intents and slots that the information processor can understand.
With this configuration, a device and a method that enable a device other than the device that input the user utterance to execute the process according to the user utterance are realized.

10, 20, 30 Agent device 11 Camera 12 Microphone 13 Display unit 14 Speaker 110 Input unit 111 Audio input unit 112 Image input unit 113 Sensor 120 Output unit 121 Audio output unit 122 Image output unit 130 Data processing unit 140 Communication unit 150 Storage unit 160 Input data analysis unit 161 Voice analysis unit 162 Image analysis unit 163 Sensor information analysis unit 164 User speech response processing execution control unit 165 Data conversion unit 170 Response processing execution unit 171 Output voice generation unit 172 Display information generation unit 173 User speech response processing Execution unit 301 CPU
302 ROM
303 RAM
304 Bus 305 Input / output interface 306 Input unit 307 Output unit 308 Storage unit 309 Communication unit 310 Drive 311 Removable media

Claims (17)

A voice input unit for inputting user utterances,
It has a data processing unit that executes the analysis of the user utterance and generates the user utterance interpretation data.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
An information processing device that converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and transmits the converted data to the second information processing device. The user utterance interpretation data generated by the data processing unit is
The information processing device according to claim 1, which is data having an intent corresponding to the intention of the user's utterance and a slot corresponding to the element information included in the user's utterance. The data processing unit
The second aspect of claim 2 is to execute a process of converting user utterance interpretation data including intents and slots generated by the data processing unit into conversion data including intents and slots that can be understood by the second information processing apparatus. Information processing device. The data processing unit
Map pink data in which intents and slots included in user speech interpretation data generated according to a user speech interpretation processing algorithm executed by the data processing unit are associated with intents and slots that can be understood by the second information processing apparatus. The information processing apparatus according to claim 2, wherein the conversion data generation process is executed with reference to the above. The information processing device
The information processing device according to claim 4, further comprising a storage unit that stores the mapping data. The data processing unit
The information processing device according to claim 1, wherein the converted data is input to an API (Application Programming Interface) that allows the second information processing device to input data that can be understood. The data processing unit
The first aspect of claim 1, wherein it is determined whether or not the process corresponding to the user's utterance can be executed by the own device, and when it is determined that the process cannot be executed, the conversion data is generated and transmitted to the second information processing apparatus. Information processing device. The data processing unit
When the user inputs a request utterance to execute the process corresponding to the user utterance on another device,
The information processing device according to claim 1, wherein the converted data is generated and transmitted to the second information processing device. The data processing unit
The information processing device according to claim 1, wherein an external device that executes a process corresponding to the user's utterance is determined by referring to a list in which functional information and access information of each of the other communicable information processing devices are registered. The data processing unit
When the generated conversion data does not include at least a part of necessary information for executing the process corresponding to user utterance in the second information processing apparatus, and there is insufficient information.
The information processing device according to claim 1, wherein the user is asked a question for acquiring the lack information. The data processing unit
The information processing device according to claim 1, wherein a task ID corresponding to the user utterance is set and the user utterance management process is executed. The data processing unit
A claim in which a plurality of user utterances are input at time intervals within a threshold time, and when the domains of the plurality of user utterances are similar, the same task ID is set for the plurality of user utterances. 11. The information processing apparatus according to 11. It is an information processing system that has multiple information processing devices.
The first information processing device is
A voice input unit for inputting user utterances,
It has a data processing unit that executes the analysis of the user utterance and generates the user utterance interpretation data.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
The user utterance interpretation data is converted to generate conversion data that can be understood by the second information processing device, and the data is transmitted to the second information processing device.
The second information processing device is
An information processing system that executes processing corresponding to the user's utterance based on the converted data received from the first information processing apparatus. The first information processing device is
User utterance interpretation data having an intent corresponding to the intention of the user utterance and a slot corresponding to the element information included in the user utterance is generated.
The information processing system according to claim 13, wherein the conversion data having an intent and a slot that the second information processing apparatus can understand is generated and transmitted to the second information processing apparatus. It is an information processing method executed in an information processing device.
The voice input section inputs the user's utterance,
The data processing department
Data processing that executes the analysis of the user utterance and generates the user utterance interpretation data is executed.
The data processing unit
When the external second information processing device executes the process corresponding to the user's utterance,
An information processing method that converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and transmits the converted data to the second information processing device. It is an information processing method executed in an information processing system having a plurality of information processing devices.
The first information processing device
When the external second information processing device executes the processing corresponding to the input user utterance
The analysis of the user utterance is executed to generate the user utterance interpretation data, and the user utterance interpretation data is generated.
A process of converting the user utterance interpretation data, generating converted data that can be understood by the second information processing device, and transmitting the converted data to the second information processing device is executed.
The second information processing device
An information processing method that executes processing corresponding to the user's utterance based on the converted data received from the first information processing apparatus. A program that executes information processing in an information processing device.
The program is installed in the data processing unit.
Analyze user utterances to generate user utterance interpretation data
When the external second information processing device executes the process corresponding to the user's utterance,
A program that converts the user utterance interpretation data to generate conversion data that can be understood by the second information processing device, and causes the second information processing device to transmit the converted data.

Images