JP2018120203A - Information processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing method and a program to reduce the response time of a plurality of dialogs.SOLUTION: The information processing method, outputs a first phrase information generated from a first voice of a user to a server via a network, acquires a first response information indicating a first response message for the first voice from the server, makes a speaker output the first response message, acquires a second response information from the server, the second response information contains a conditional branch instruction for branching to a plurality of instructions according to the phrase information concerning a selection candidate for a response to the first response message, acquires a second voice containing a response of the user to the first response message after the output of the first response message, and make at least one of speakers and apparatus execute predetermined operation according to the instruction of a conditional branch instruction determined by referencing the phrase information to a second phrase information generated from the second voice.SELECTED DRAWING: Figure 13A

Description

  The present disclosure relates to an information processing method and a program.

  For example, Patent Document 1 discloses an example of a voice recognition technique. The device and method of Patent Document 1 perform voice control of a device related to consumer electronics using a voice command. This apparatus and method uses a voice recognition command frequently spoken by a user and a control command corresponding thereto in order to reduce the time taken from the command recognition to the execution of an operation when a user command is recognized using a server. Information is stored in a local “save unit”.

JP 2014-71457 A

  In the voice dialogue agent represented by the apparatus and method of Patent Document 1, only one round-trip dialogue is assumed as one question one answer.

  The present disclosure provides an information processing method and program for reducing response time of a plurality of dialogues.

  An information processing method according to an aspect of the present disclosure is an information processing method executed by a processor that controls at least one device through a dialog with a user, and includes a first voice indicating the first voice of the user input from a microphone. 1 voice information is acquired, the 1st phrase information generated from the 1st voice information is outputted to a server via a network, and the 1st response information according to the 1st phrase information is sent via the network Obtained from the server, the first response information indicates a first response message for the first voice, and based on the first response information, causes the speaker to output the first response message, and on the server Second response information associated with the first response information is acquired from the server via the network, and the second response information is equal to or greater than one. Including a conditional branch instruction that branches into a plurality of different instructions according to the lasing information, wherein each of the one or more pieces of phrase information relates to a selection candidate of the user's response to the first response message, and the first response After the message is output, second voice information indicating the second voice of the user input from the microphone is acquired, and the second voice includes a response of the user to the first response message, and the conditional branch Of the commands, at least one of the speaker and the at least one device according to a command determined by collating the one or more phrase information and the second phrase information generated from the second audio information. During the period from when the second sound information is acquired until the predetermined operation is performed. Both of information and the second phrase information is not output to the server.

  A program according to an aspect of the present disclosure causes the processor to execute the information processing method.

  An interaction processing method according to an aspect of the present disclosure is an information processing method executed by a second processor on a server, and the second processor controls at least one device through interaction with a user. The first phrase information about the first voice of the user input from the microphone is acquired from the first processor via the network, and the first phrase information is At least one of a character string corresponding to one voice and semantic information of the character string, and outputting first response information corresponding to the first phrase information to the first processor via the network; 1 response information shows the 1st response message with respect to the said 1st audio | voice output from a speaker, The said 1st response on the said server Conditional branch instruction that outputs second response information associated with the information to the first processor via the network, and the second response information branches to a plurality of different instructions according to one or more phrase information Each of the one or more pieces of phrase information relates to a selection candidate of the user's response to the first response message.

  A program according to an aspect of the present disclosure causes the second processor to execute the information processing method.

  According to the information processing method and program of the present disclosure, it is possible to reduce the response time of a plurality of dialogues.

FIG. 1A is a diagram illustrating an example of an environment in which a voice dialogue agent system including a dialogue processing apparatus according to an embodiment is arranged, and a diagram illustrating an overview of services provided by an information management system including a voice dialogue agent system It is. FIG. 1B is a diagram illustrating an example in which the data center operating company in FIG. 1A corresponds to a device manufacturer. FIG. 1C is a diagram illustrating an example in which the data center operating company in FIG. 1A corresponds to either or both of an equipment manufacturer and a management company. FIG. 2 is a schematic diagram showing the configuration of the voice interaction agent system according to the embodiment. FIG. 3 is a diagram illustrating an example of a hardware configuration of the voice input / output device according to the embodiment. FIG. 4 is a diagram illustrating an example of a hardware configuration of the device according to the embodiment. FIG. 5 is a diagram illustrating an example of a hardware configuration of the local server according to the embodiment. FIG. 6 is a diagram illustrating an example of a hardware configuration of the cloud server according to the embodiment. FIG. 7 is a diagram illustrating an example of a system configuration of the voice input / output device according to the embodiment. FIG. 8 is a diagram illustrating an example of a system configuration of a device according to the embodiment. FIG. 9 is a diagram illustrating an example of a system configuration of the local server according to the embodiment. FIG. 10 is a diagram illustrating an example of a system configuration of the cloud server according to the embodiment. FIG. 11 is a specific example of the local dictionary DB according to the embodiment. FIG. 12A is a specific example of the dialogue rule DB according to the embodiment. FIG. 12B is a specific example of the offload instruction generation DB according to the embodiment. FIG. 13A is a sequence diagram of communication processing for shortening the response time of the voice interaction agent system according to the embodiment. FIG. 13B is a sequence diagram of communication processing for shortening the response time of the voice interaction agent system according to the embodiment. FIG. 14 is a diagram illustrating an overall image of a service provided by an information management system in service type 1 (in-house data center type cloud service) to which the voice interaction agent system according to the embodiment is applicable. FIG. 15 is a diagram illustrating an overall image of a service provided by an information management system in service type 2 (cloud service using IaaS) to which the voice interaction agent system according to the embodiment is applicable. FIG. 16 is a diagram showing an overall image of a service provided by an information management system in service type 3 (PaaS use type cloud service) to which the voice interaction agent system according to the embodiment is applicable. FIG. 17 is a diagram illustrating an overall image of a service provided by an information management system in service type 4 (SaaS-based cloud service) to which the voice interaction agent system according to the embodiment is applicable.

[Knowledge that became the basis of the technology of this disclosure]
The present inventors have found that the following problems occur in the prior art as disclosed in Patent Document 1. The method and apparatus of the above-mentioned Patent Document 1 is supposed to be a one-on-one dialogue only. Such methods and devices may increase response time or fail to respond to interactions that exceed one question. The present inventors have studied a technique for reducing the response time of the local side and cloud side devices for a plurality of dialogues. The present inventors give a simple recognition command and a corresponding control command to the local side in a cloud-type voice interaction agent that performs device control by a plurality of conversations, that is, the load is reduced. It has been found that the response time of the voice interaction agent can be shortened. Therefore, the present inventors examined the following improvement measures.

  A first information processing method according to an aspect of the present disclosure is an information processing method executed by a processor that controls at least one device through interaction with a user, and the first voice of the user input from a microphone Is obtained, and the first phrase information (first speech recognition result) generated from the first voice information is output to a server via the network, and the first phrase information is determined according to the first phrase information. First response information is obtained from the server via the network, and the first response information indicates a first response message for the first voice, and the first response information is sent to a speaker based on the first response information. Outputting a response message, obtaining second response information associated with the first response information on the server from the server via the network; The second response information includes a conditional branch instruction (processing information) that branches into a plurality of different instructions according to one or more phrase information, and each of the one or more phrase information includes the user for the first response message. The second voice information indicating the second voice of the user inputted from the microphone is acquired after the first response message is output, and the second voice is the second voice information. A second response information (second speech recognition result) generated from the one or more pieces of phrase information (estimated response information) and the second speech information in the conditional branch instruction ) And causing at least one of the speaker and the at least one device to perform a predetermined operation in accordance with an instruction determined by checking During the period from the acquisition of the second audio information to thereby perform a predetermined operation, both of the second audio information and the second phrase information is not output to the server.

  According to the above aspect, after the acquisition of the first voice of the user, the first response information according to the first voice and the second response information according to the user's response to the first response message of the first response information are: Determined at the server. Then, the first response information and the second response information are obtained from the server. For this reason, the 1st response message according to a user's 1st voice can be outputted based on the 1st response information. Further, based on the second response information, it is possible to output a message and control the device by a speaker in accordance with a user response to the first response message. Therefore, multiple dialogues with the user are possible. Furthermore, since the second response information includes the conditional branch instruction, it is possible to respond to the first response message according to a plurality of different responses from the user. Therefore, after acquiring the user's response to the first response message, communication with the server for generating response information corresponding to the response is unnecessary. Accordingly, it is possible to shorten the response time in a plurality of dialogues.

  In addition, a second information processing method according to an aspect of the present disclosure is an information processing method executed by a second processor (interaction processing device) on a server, and the second processor passes through a dialog with a user. The first processor is capable of communicating with a first processor that controls at least one device via a network, and first phrase information (first speech recognition result) regarding the first voice of the user input from a microphone is used as the first processor. And the first phrase information indicates at least one of a character string corresponding to the first voice and semantic information of the character string, and a first response corresponding to the first phrase information Information is output to the first processor via the network, and the first response information is output from a speaker in response to a first sound corresponding to the first sound. A response message is displayed, and second response information associated with the first response information on the server is output to the first processor via the network, and the second response information includes one or more pieces of phrase information A conditional branch instruction (processing information) that branches into a plurality of different instructions according to (estimated response information), and each of the one or more phrase information relates to a selection candidate of the user's reply to the first response message It is.

  According to the above aspect, the server responds to the first response information corresponding to the first voice and the user's response to the first response message of the first response information after obtaining the first phrase information about the first voice of the user. The second response information is determined. Then, the first response information and the second response information are output from the server to the first processor. The first processor can output a first response message corresponding to the first voice of the user based on the first response information. In addition, the first processor can perform message output and speaker control by a speaker in accordance with a user response to the first response message based on the second response information. Therefore, multiple dialogues with the user are possible. Furthermore, since the second response information includes a conditional branch instruction, the first processor can respond to a plurality of different responses from the user to the first response message. Therefore, after the user's response to the first response message is acquired, communication between the first processor and the server for generating response information corresponding to the response is unnecessary. Accordingly, it is possible to shorten the response time in a plurality of dialogues.

  For example, in the first information processing method according to an aspect of the present disclosure, the first response information and the second response information may be simultaneously acquired from the server.

  According to the above aspect, the communication frequency between the server and the processor is reduced, and the response time to the user's voice can be shortened. Specifically, before confirming the user's response to the first response message indicated by the first response information, the second response information for the user's response is obtained from the server. This eliminates the need for communication for the server to confirm the user's response. Furthermore, although a response to the first response message has been received from the user, the second response information has not yet been acquired, and processing delays such that the next operation cannot be performed can be prevented.

  For example, in the first information processing method according to an aspect of the present disclosure, after the first phrase information is output to the server, the second response information includes the first phrase information and the first response information. According to at least one, it may be selected from a plurality of second response information stored on the server.

  For example, in the first information processing method according to one aspect of the present disclosure, after the predetermined operation is executed, an execution notification indicating that the predetermined operation is executed may be output to the server. Good.

  According to the above aspect, the server can end the operation related to the control of the device through the dialogue with the user by acquiring the execution notification. Therefore, unnecessary standby and operation of the server can be reduced.

  For example, in the first information processing method according to an aspect of the present disclosure, at least one of the plurality of commands may include a command to output a second response message for the second sound from the speaker.

  According to the above aspect, by acquiring the second response information, it is possible to take a response using voice according to the content of the second voice including the user's response to the first response message.

  For example, in the first information processing method according to an aspect of the present disclosure, at least one of the plurality of instructions may include an instruction that causes a control command to be transmitted to the at least one device.

  According to the said aspect, by acquiring 2nd response information, an apparatus can be controlled according to the content of the 2nd audio | voice containing the user's reply with respect to a 1st response message.

  For example, in the first information processing method according to an aspect of the present disclosure, the network is the Internet, and the information processing method is executed on a local server that can communicate with the at least one device without using the Internet. May be.

  According to the above aspect, an increase in response time due to communication between the local server and at least one device can be suppressed.

  A first program according to an aspect of the present disclosure causes the processor to execute the first information processing method.

  For example, in the second information processing method according to an aspect of the present disclosure, the first processor includes second phrase information generated from the one or more phrase information and the second audio information in the conditional branch instruction. And at least one of the speaker and the at least one device may perform a predetermined operation in accordance with an instruction determined by comparing

  According to the above aspect, the first processor can select an instruction suitable for the content of the user's second voice from the conditional branch instructions. As a result, an increase in response time due to incompatibility between the content of the user's second voice and the operation of the first processor based on the command is suppressed.

  For example, in the second information processing method according to an aspect of the present disclosure, the second processor is configured to perform the predetermined operation after the first processor acquires the user's response. The second audio information and the second phrase information may not be output to the first processor.

  For example, the second information processing method according to an aspect of the present disclosure may further acquire an execution notification indicating that the predetermined operation has been executed from the first processor.

  According to the above aspect, the second processor of the server can end the operation related to the control of the device through the dialogue with the user by acquiring the execution notification. Therefore, unnecessary standby and operation of the second processor can be reduced.

  For example, in the second information processing method according to an aspect of the present disclosure, the first response information and the second response information may be simultaneously output to the first processor.

  According to the above aspect, the communication frequency between the second processor and the first processor is reduced, and the response time to the user's voice can be shortened. Specifically, the second response information for the user response is output to the first processor before confirming the user response to the first response message indicated by the first response information. For this reason, the second processor does not need to perform communication for obtaining the user's response. Furthermore, although the user responds to the first response message, since the first processor has not yet acquired the second response information, it is possible to prevent a delay in processing that the next operation cannot be performed.

  For example, in the second information processing method according to an aspect of the present disclosure, at least one of the plurality of instructions may include an instruction for outputting a second response message for the second sound from the speaker.

  According to the above aspect, the first processor can take a response using the voice according to the content of the second voice including the user's response to the first response message by acquiring the second response information. .

  For example, in the second information processing method according to an aspect of the present disclosure, at least one of the plurality of instructions may include an instruction that causes the at least one device to output a control command.

  According to the said aspect, the 1st processor can control an apparatus according to the content of the 2nd audio | voice containing the user's reply with respect to a 1st response message by acquiring 2nd response information.

  For example, in the second information processing method according to one aspect of the present disclosure, on the server, a plurality of semantic information, a plurality of first response information respectively associated with the plurality of semantic information, and the plurality of the plurality of semantic information A database including a plurality of second response information respectively associated with the first response information, and after obtaining the first phrase information, referring to the database, Identifying the semantic information corresponding to the first voice, identifying the first response information associated with the identified semantic information from the plurality of first response information, You may identify the 2nd response information linked | related with the identified said 1st response information from 2nd response information.

  According to the said aspect, the production | generation process of 1st response information and 2nd response information from 1st phrase information is possible only in the server side. The server has a high processing capacity and a large data capacity. When the server performs the generation process, the processing time can be shortened.

  Further, a second program according to an aspect of the present disclosure causes the second processor to execute the second information processing method.

  These comprehensive or specific modes may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. The system, method, integrated circuit, computer program Also, any combination of recording media may be realized.

  Hereinafter, embodiments will be specifically described with reference to the drawings. Note that each of the embodiments described below shows a specific example of the technology of the present disclosure. Numerical values, shapes, components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements. In all the embodiments, the contents can be combined.

[Embodiment]
[1. Overview of services provided]
First, with reference to FIG. 1A to FIG. 1C, an overview of services provided by an information management system in which a voice interaction agent system 1 including an interaction processing apparatus according to an embodiment is provided will be described. FIG. 1A is a diagram illustrating an example of an environment in which a voice dialogue agent system 1 including a dialogue processing apparatus according to an embodiment is arranged, and illustrates an overall image of a service provided by an information management system including the voice dialogue agent system FIG. FIG. 1B is a diagram illustrating an example in which the data center operating company in FIG. 1A corresponds to a device manufacturer. FIG. 1C is a diagram illustrating an example in which the data center operating company in FIG. 1A corresponds to either or both of an equipment manufacturer and a management company. The interactive processing device may be a home gateway (also referred to as a local server) 102 described later, may be a cloud server 111, or may include the home gateway 102 and the cloud server 111.

  As shown in FIG. 1A, the information management system 4000 includes a group 4100, a data center operating company 4110, and a service provider 4120. The group 4100 is, for example, a company, an organization, a home, or the like, and may be of any size. The group 4100 includes a plurality of devices 101 including a first device 101a and a second device 101b, and a home gateway 102. The plurality of devices 101 are home appliances, for example. The plurality of devices 101 may include devices that can be connected to a communication network such as the Internet such as a smartphone, a personal computer (PC), or a television, for example, such as a lighting, a washing machine, a refrigerator, etc. A device that cannot be connected to the communication network may be included. The plurality of devices 101 may include devices that can be connected to a communication network such as the Internet via the home gateway 102 even if they cannot be connected to a communication network such as the Internet. The user 5100 uses a plurality of devices 101 in the group 4100.

  The data center operating company 4110 includes a cloud server 111. The cloud server 111 is a virtualization server that cooperates with various devices via a communication network such as the Internet. The cloud server 111 mainly manages huge data (big data) that is difficult to handle with a normal database management tool or the like. The data center operating company 4110 performs management of data, management of the cloud server 111, operation of the data center that performs them, and the like. Details of services performed by the data center operating company 4110 will be described later. In the following description, it is assumed that the Internet is used as the communication network, but the communication network is not limited to the Internet.

  Here, the data center operating company 4110 is not limited to a company that only manages data or manages the cloud server 111. For example, as shown in FIG. 1B, when a device manufacturer that develops or manufactures one of a plurality of devices 101 performs data management or cloud server 111 management, the device manufacturer This corresponds to the data center operating company 4110. Further, the data center operating company 4110 is not limited to one company. For example, as shown in FIG. 1C, when the device manufacturer and the management company jointly or share the management of the data or the cloud server 111, both or one of them corresponds to the data center operating company 4110. .

  The service provider 4120 includes a server 121. The server 121 here is not limited in scale, and includes, for example, a memory in a personal PC. In addition, the service provider 4120 may not include the server 121.

  In the information management system 4000 described above, the home gateway 102 is not essential. For example, when the cloud server 111 manages all data, the home gateway 102 becomes unnecessary. Further, there may be a case where there is no device that cannot be connected to the Internet by itself, as in the case where all the devices 101 in the home are connected to the Internet.

  Next, the information flow in the information management system 4000 will be described. First, the first device 101a or the second device 101b of the group 4100 transmits each log information to the cloud server 111 of the data center operating company 4110, respectively. The cloud server 111 accumulates log information of the first device 101a or the second device 101b (arrow 131 in FIG. 1A). Here, the log information is information indicating, for example, driving conditions and operation dates / times of the plurality of devices 101. For example, the log information may include TV viewing history, recorder recording reservation information, washing machine operation date / time, amount of laundry, refrigerator opening / closing date / time, refrigerator opening / closing frequency, etc., but is not limited thereto. Various information that can be acquired from various devices 101 may be included. Note that the log information may be provided directly to the cloud server 111 from the plurality of devices 101 themselves via the Internet. The log information may be temporarily accumulated in the home gateway 102 from a plurality of devices 101 and provided to the cloud server 111 from the home gateway 102.

  Next, the cloud server 111 of the data center operating company 4110 provides the collected log information to the service provider 4120 in a certain unit. Here, the fixed unit may be a unit that can organize and provide the information collected by the data center operating company 4110 to the service provider 4120, or may be a unit that the service provider 4120 requests. Further, the log information is provided in a fixed unit. However, the log information may not be provided in a fixed unit, and the amount of information provided may vary depending on the situation. The log information is stored in the server 121 held by the service provider 4120 as necessary (arrow 132 in FIG. 1A).

  Then, the service provider 4120 organizes the log information into information suitable for the service provided to the user and provides it to the user. The user to whom information is provided may be a user 5100 using a plurality of devices 101 or an outside user 5200. As a method for providing information to the users 5100 and 5200, for example, information may be provided directly from the service provider 4120 to the users 5100 and 5200 (arrows 133 and 134 in FIG. 1A). As a method for providing information to the user 5100, for example, a method in which information is provided to the user 5100 through the cloud server 111 of the data center operating company 4110 again (arrows 135 and 136 in FIG. 1A) may be used. Further, the cloud server 111 of the data center operating company 4110 may organize the log information into information suitable for the service provided to the user and provide it to the service provider 4120. Note that the user 5100 may be the same as or different from the user 5200.

[2-1. Configuration of Spoken Dialogue Agent System According to Embodiment]
Hereinafter, the configuration of the voice interaction agent system 1 according to the embodiment will be described. The voice interaction agent system 1 is a system that shortens a response time in response to a user's voice instruction. In this embodiment, the voice interaction agent system 1 is a cloud-type voice interaction agent system that performs device control in a plurality of conversations, and by offloading a simple recognition instruction and a corresponding control instruction to the local side. , Reduce its response time.

  First, regarding the configuration of the voice interaction agent system 1, the configuration of the voice interaction agent system, the hardware configuration of the voice input / output device, the hardware configuration of the device, the hardware configuration of the local server, the hardware configuration of the cloud server, and the voice input / output The functional block of the device, the functional block of the device, the functional block of the local server, and the functional block of the cloud server will be described in order. Further, regarding the configuration of the voice interaction agent system 1, a specific example of the local dictionary DB, a specific example of the interaction rule DB, and a specific example of the offload instruction generation DB will be described. Then, regarding the operation of the voice interaction agent system 1, a processing sequence for shortening the response time by the voice interaction agent system 1 will be described.

  With reference to FIG. 2, the structure of the spoken dialogue agent system 1 including the dialogue processing apparatus according to the embodiment will be described. FIG. 2 is a schematic diagram showing a configuration of the voice interaction agent system 1 according to the embodiment. The voice interaction agent system 1 includes a voice input / output device 240, a plurality of devices 101, a local server 102, an information communication network 220, and a cloud server 111. The local server 102 is an example of a home gateway. The information communication network 220 is, for example, the Internet, and is an example of a communication network. In the present embodiment, the plurality of devices 101 includes a television 243, an air conditioner 244, and a refrigerator 245. In addition, the devices constituting the plurality of devices 101 are not limited to the television 243, the air conditioner 244, and the refrigerator 245, and may be arbitrary devices. The voice input / output device 240, the plurality of devices 101, and the local server 102 are arranged in a group 4100. Here, the local server 102 may constitute a dialogue processing device, the cloud server 111 may constitute a dialogue processing device, and both the local server 102 and the cloud server 111 constitute a dialogue processing device. Also good.

  In the example shown in FIG. 2, a user 5100 who is a human exists in a group 4100 in which the voice interaction agent system 1 is arranged. Further, it is assumed that the user 5100 is a speaker for the voice interaction agent system 1.

  The voice input / output device 240 is an example of a sound collection unit that acquires voice in the group 4100 and is also an example of a voice output unit that outputs voice in the group 4100. The voice input / output device 240 may acquire voice via a microphone and may output voice via a speaker. The microphone and the speaker may be provided in the voice input / output device 240, may be provided in a device on which the voice input / output device 240 is mounted, or may be provided in a device separate from the voice input / output device 240 and the above device. Good. The group 4100 is a space in which the voice input / output device 240 can provide information to the user by voice. The voice input / output device 240 recognizes the voice of the user 5100 in the group 4100, presents voice information from the voice input / output device 240 and controls the device 101 in accordance with an instruction of the user 5100 by the recognized voice input. . More specifically, the voice input / output device 240 displays content according to an instruction of the user 5100 by voice input, answers a question of the user 5100, or controls the device 101.

  In addition, here, a wired or wireless connection can be used for the connection between the voice input / output device 240, the plurality of devices 101, and the local server 102. Various wireless communications can be applied to the wireless connection. For example, a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark) may be applied, and short-range wireless communication such as Bluetooth (registered trademark) or ZigBee (registered trademark) is applied. Also good.

  In addition, at least some of the voice input / output device 240, the device 101, and the local server 102 may be integrated. For example, the function of the local server 102 may be incorporated in the voice input / output device 240, and the voice input / output device 240 may function as a local terminal that communicates with the cloud server 111 itself. Alternatively, the voice input / output device 240 may be incorporated in each of the plurality of devices 101 or one of the plurality of devices 101. In the latter case, the device 101 in which the voice input / output device 240 is incorporated may control another device 101. Alternatively, at least the function of the local server 102 among the function of the voice input / output device 240 and the function of the local server 102 may be incorporated into each of the plurality of devices 101 or one of the plurality of devices 101. . In the former case, each device 101 may function as a local terminal that communicates with the cloud server 111 by itself. In the latter case, each device 101 passes through one device 101 that is a local terminal in which the function of the local server 102 is incorporated. Other devices 101 may communicate with the cloud server 111.

  Further, the voice input / output device 240, the device 101, the local server 102, and the cloud server 111 will be described from the viewpoint of hardware configuration. FIG. 3 shows an example of a hardware configuration of the voice input / output device 240 according to the embodiment. As shown in FIG. 3, the audio input / output device 240 includes a processing circuit 300, a sound collection circuit 301, an audio output circuit 302, and a communication circuit 303. The processing circuit 300, the sound collection circuit 301, the audio output circuit 302, and the communication circuit 303 are connected to each other via a bus 330, and can exchange data and commands with each other. Here, the cloud server 111 is an example of a server.

  The processing circuit 300 can be realized by a combination of a CPU (Central Processing Unit) 310 and a memory 320 that stores a device ID 341 and a computer program 342. The CPU 310 controls the operation of the voice input / output device 240, but may also control the operation of each device 101 connected via the local server 102. In this case, the processing circuit 300 transmits a control command for each device 101 via the local server 102, but may directly transmit the control command to each device 101. CPU 310 executes an instruction group described in computer program 342 expanded in memory 320. Thus, the CPU 310 can realize various functions. The computer program 342 describes a group of instructions for realizing the operation of the voice input / output device 240 described later. The computer program 342 described above may be stored in advance in the memory 320 of the voice input / output device 240 as a product. Alternatively, the computer program 342 is recorded on a recording medium such as a CD-ROM and distributed as a product to the market, or transmitted through an electric communication line such as the Internet, and acquired through the recording medium or the electric communication line. May be stored in the memory 320.

  Alternatively, the processing circuit 300 may be realized by dedicated hardware configured to realize the operation described below. The device ID 341 is an identifier uniquely assigned to the device 101. The device ID 341 may be uniquely assigned by the manufacturer of the device 101, or may be a physical address (so-called MAC (Media Access Control) address) uniquely assigned on the network in principle.

  In FIG. 3, it is assumed that the device ID 341 is stored in the memory 320 in which the computer program 342 is stored. However, this is an example of the configuration of the processing circuit 300. For example, the computer program 342 may be stored in a RAM (Random Access Memory) or ROM (Read Only Memory), and the device ID 341 may be stored in a flash memory.

  The sound collection circuit 301 collects the user's voice to generate an analog voice signal, converts the analog voice signal into digital data, and transmits the digital data to the bus 330.

  The audio output circuit 302 converts the digital data received through the bus 330 into an analog audio signal and outputs the analog audio signal.

  The communication circuit 303 is a circuit that communicates with another device (for example, the local server 102) via wired communication or wireless communication. Although not limited thereto, in the present embodiment, the communication circuit 303 communicates with other devices via a network, for example, via a wired LAN such as a network compliant with the Ethernet (registered trademark) standard. I do. The communication circuit 303 transmits log information and ID information generated by the processing circuit 300 to the local server 102. In addition, the communication circuit 303 transmits a signal received from the local server 102 to the processing circuit 300 through the bus 330.

  The voice input / output device 240 may include other components for realizing the functions required for the voice input / output device 240 in addition to the illustrated components.

  FIG. 4 shows an example of a hardware configuration of the device 101 according to the embodiment. A television 243, an air conditioner 244, and a refrigerator 245 shown in FIG. 2 are examples of the device 101. As illustrated in FIG. 4, the device 101 includes an input / output circuit 410, a communication circuit 450, and a processing circuit 470. The input / output circuit 410, the communication circuit 450, and the processing circuit 470 are connected to each other through a bus 460, and can exchange data and commands with each other.

  The processing circuit 470 can be realized by a combination of the CPU 430 and the memory 440 storing the device ID 441 and the computer program 442. The CPU 430 controls the operation of the device 101. The CPU 430 can execute a group of instructions described in the computer program 442 expanded in the memory 440 to realize various functions. The computer program 442 describes a group of instructions for realizing the operation of the device 101. The computer program 442 described above may be stored in advance in the memory 440 of the device 101 as a product. Alternatively, the computer program 442 is recorded on a recording medium such as a CD-ROM and distributed as a product to the market, or transmitted through an electric communication line such as the Internet, and acquired through the recording medium or the electric communication line. May be stored in the memory 440.

  Alternatively, the processing circuit 470 may be realized by dedicated hardware configured to realize the operation described below. The device ID 441 is an identifier uniquely assigned to the device 101. The device ID 441 may be uniquely assigned by the manufacturer of the device 101, or may be a physical address (so-called MAC address) uniquely assigned on the network in principle.

  In FIG. 4, it is assumed that the device ID 441 is stored in the memory 440 in which the computer program 442 is stored. However, this is an example of the configuration of the processing circuit 470. For example, the computer program 442 may be stored in RAM or ROM, and the device ID 441 may be stored in flash memory.

  The input / output circuit 410 outputs the result processed by the processing circuit 470. The input / output circuit 410 converts the input analog signal into digital data and transmits the digital data to the bus 330.

  The communication circuit 450 is a circuit that communicates with another device (for example, the local server 102) via wired communication or wireless communication. Although not limited thereto, in the present embodiment, the communication circuit 450 communicates with other devices via a network, for example, via a wired LAN such as a network compliant with the Ethernet (registered trademark) standard. I do. The communication circuit 450 transmits the log information and ID information generated by the processing circuit 470 to the local server 102. Further, the communication circuit 450 transmits the signal received from the local server 102 to the processing circuit 470 through the bus 460.

  The device 101 may include other components for realizing the functions required for the device 101 in addition to the illustrated components.

  FIG. 5 shows an example of the hardware configuration of the local server 102. The local server 102 constitutes a gateway between the voice input / output device 240, the device 101, and the information communication network 220. As shown in FIG. 5, the local server 102 includes a first communication circuit 551, a second communication circuit 552, a processing circuit 570, an acoustic model DB (database) 580, a language model DB 581, and a voice. A segment DB 582, a prosody control DB 583, and a local dictionary DB 584 are provided as components. These components are connected to each other by a bus 560, and can exchange data and commands with each other.

  The processing circuit 570 is connected to the acoustic model DB 580, the language model DB 581, the speech segment DB 582, the prosody control DB 583, and the local dictionary DB 584, and can acquire and edit management information stored in these DBs. In this embodiment, the acoustic model DB 580, the language model DB 581, the speech segment DB 582, the prosody control DB 583, and the local dictionary DB 584 are internal components of the local server 102, but are provided outside the local server 102. May be. In that case, in addition to the bus 560, the connection means between each DB and the components of the local server 102 may include a communication line such as an Internet line, a wired line, or a wireless LAN.

  The first communication circuit 551 is a circuit that communicates with other devices (for example, the voice input / output device 240 and the device 101) via wired communication or wireless communication. Although not limited, in the present embodiment, the first communication circuit 551 communicates with other devices via a network and, for example, via a wired LAN such as a network compliant with the Ethernet (registered trademark) standard. To communicate. The first communication circuit 551 transmits the log information and ID information generated by the processing circuit 570 to the voice input / output device 240 and the device 101. The first communication circuit 551 transmits the signal received from the voice input / output device 240 and the device 101 to the processing circuit 570 through the bus 560.

  The second communication circuit 552 is a circuit that communicates with the cloud server 111 via wired communication or wireless communication. The second communication circuit 552 is connected to a communication network via wired communication or wireless communication, and further communicates with the cloud server 111 via the communication network. In the present embodiment, the communication network is the information communication network 220. The second communication circuit 552 performs communication via a wired LAN such as a network conforming to the Ethernet (registered trademark) standard, for example. The second communication circuit 552 transmits / receives various information to / from the cloud server 111.

  The processing circuit 570 can be realized by a combination of the CPU 530 and a memory 540 storing a uniquely identifiable gateway ID (hereinafter also referred to as GW-ID) 541 and a computer program 542. The CPU 530 controls the operation of the local server 102, but may also control the operations of the voice input / output device 240 and the device 101. The gateway ID 541 is an identifier uniquely assigned to the local server 102. The gateway ID 541 may be uniquely assigned by the manufacturer of the local server 102, or may be a physical address (so-called MAC address) uniquely assigned on the network in principle. The CPU 530 can execute a group of instructions described in the computer program 542 expanded in the memory 540 to realize various functions. The computer program 542 describes a group of instructions for realizing the operation of the local server 102. The computer program 542 described above may be stored in advance in the memory 540 of the local server 102 as a product. Alternatively, the computer program 542 is recorded on a recording medium such as a CD-ROM and distributed as a product to the market, or transmitted through an electric communication line such as the Internet, and acquired through the recording medium or the electric communication line. May be stored in the memory 540. Here, the processing circuit 570 or the CPU 530 is an example of a processor or a first processor.

  Alternatively, the processing circuit 570 may be realized by dedicated hardware configured to realize the operation described below. In addition to the illustrated components, the local server 102 may include other components for realizing the functions required for the local server 102.

  In FIG. 5, it is assumed that the gateway ID 541 is stored in the memory 540 in which the computer program 542 is stored. However, this is an example of the configuration of the processing circuit 570. For example, the computer program 542 may be stored in RAM or ROM, and the gateway ID 541 may be stored in flash memory.

  The acoustic model DB 580 registers various acoustic models including frequency patterns such as speech waveforms and character strings corresponding to speech. The language model DB 581 registers various language models including words and how to arrange them. The speech unit DB 582 registers various speech units that express phonetic features in units of phonemes and the like. The prosody control DB 583 registers various information for controlling the prosody of the character string. The local dictionary DB 584 registers various character strings and meaning tags corresponding to the character strings in association with each other. The character string is composed of phrases such as words and phrases. A semantic tag refers to a logical expression representing the meaning of a character string. For example, the same meaning tag is commonly set to a plurality of character strings having similar character string meanings. For example, the semantic tag indicates the name of the task target, the task content for the task target, and the like as keywords. For example, referring to FIG. 11, an example of a combination of a character string and a semantic tag corresponding to the character string is shown. Here, the semantic tag is an example of semantic information.

  FIG. 6 shows an exemplary hardware configuration of the cloud server 111. As illustrated in FIG. 6, the cloud server 111 includes a communication circuit 650, a processing circuit 670, an interaction rule DB 691, and an offload instruction generation DB 692 as components. These components are connected to each other by a bus 680 and can exchange data and commands with each other.

  The processing circuit 670 includes a CPU 671 and a memory 672 that stores a program 673. The CPU 671 controls the operation of the cloud server 111. The CPU 671 executes an instruction group described in the computer program 673 expanded in the memory 672. Thus, the CPU 671 can realize various functions. The computer program 673 describes a group of instructions for the cloud server 111 to realize an operation described later. The computer program 673 described above can be recorded on a recording medium such as a CD-ROM and distributed as a product to the market, or can be transmitted through an electric communication line such as the Internet. A device (for example, a PC) having hardware shown in FIG. 6 can function as the cloud server 111 according to the present embodiment by reading the computer program 673. Here, the processing circuit 670 or the CPU 671 is an example of a second processor.

  The processing circuit 670 is connected to the dialogue rule DB 691 and the offload instruction generation DB 692 and can acquire and edit management information stored in these DBs. In the present embodiment, the interaction rule DB 691 and the offload instruction generation DB 692 are components inside the cloud server 111, but may be provided outside the cloud server 111. In that case, in addition to the bus 680, the connection means between each DB and the components of the cloud server 111 may include a communication line such as an Internet line, a wired line, or a wireless LAN. Although details will be described later, the dialogue rule DB 691 registers various semantic tags and conditions related to the respective semantic tags in association with each other. The offload command generation DB 692 registers the response message of the voice interaction agent system 1 and the command data for the local server 102 corresponding to the response message (in this embodiment, offload command data) in association with each other. The offload command data is command data corresponding to a plurality of types of dialogue between the user and the voice dialogue agent system 1, and is command data configured to reduce the load on the local server 102. The offload command data is also command data that can correspond to a plurality of dialogues between the user and the voice dialogue agent system 1. Here, the dialogue rule DB 691 and the offload instruction generation DB 692 are examples of databases.

  The communication circuit 650 is a circuit that communicates with another device (for example, the local server 102) via wired communication or wireless communication. The communication circuit 650 is connected to a communication network via wired communication or wireless communication, and further communicates with another device (for example, the local server 102) via the communication network. In the present embodiment, the communication network is the information communication network 220. The communication circuit 650 performs communication via a wired LAN such as a network conforming to the Ethernet (registered trademark) standard, for example.

  Next, the voice input / output device 240, the device 101, the local server 102, and the cloud server 111 will be described from the viewpoint of the system configuration. FIG. 7 is a block diagram illustrating an example of a system configuration of the voice input / output device 240. As illustrated in FIG. 7, the voice input / output device 240 includes a sound collection unit 700, a voice detection unit 710, a voice segment cutout unit 720, a communication unit 730, and a voice output unit 740.

  The sound collection unit 700 corresponds to the sound collection circuit 301 in FIG. The sound collection unit 700 collects a user's voice to generate an analog voice signal, converts the generated analog voice signal into digital data, and generates a voice signal from the converted digital data.

  The voice detection unit 710 and the voice segment cutout unit 720 are realized by the processing circuit 300 in FIG. The CPU 310 that has executed the computer program 342 functions as, for example, a voice detection unit 710 at a certain point in time, and functions as a voice segment cutout unit 720 at another different point in time. Note that at least one of these two components may be realized by hardware that performs dedicated processing such as a DSP (Digital Signal Processor).

  The sound detection unit 710 determines whether sound is detected. For example, when the detected sound level is equal to or lower than a predetermined value, the sound detection unit 710 determines that no sound is detected. The voice segment cutout unit 720 detects a segment in which voice is present from the acquired voice signal. For example, the section is a time section.

  The communication unit 730 corresponds to the communication circuit 303 in FIG. The communication unit 730 communicates with another device (for example, the local server 102) of the voice input / output device 240 via wired communication such as a network or wireless communication. The communication unit 730 performs communication via a wired LAN such as a network conforming to the Ethernet (registered trademark) standard, for example. The communication unit 730 transmits the audio signal of the audio segment detected by the audio segment cutout unit 720 to another device. In addition, the communication unit 730 passes an audio signal received from another device to the audio output unit 740.

  The audio output unit 740 corresponds to the audio output circuit 302 of FIG. The audio output unit 740 converts the audio signal received by the communication unit 730 into an analog audio signal and outputs the analog audio signal.

  FIG. 8 is a block diagram illustrating an example of a system configuration of the device 101. As illustrated in FIG. 8, the device 101 includes a communication unit 800 and a device control unit 810.

  The communication unit 800 corresponds to the communication circuit 450 in FIG. The communication unit 800 communicates with other devices (for example, the local server 102) of the device 101 through wired communication such as a network or wireless communication. The communication unit 800 performs communication via a wired LAN such as a network conforming to the Ethernet (registered trademark) standard, for example.

  The device control unit 810 corresponds to the input / output circuit 410 and the processing circuit 470 of FIG. The device control unit 810 reads the control data received by the communication unit 800 and controls the operation of the device 101. In addition, the device control unit 810 controls output of a processing result in controlling the operation of the device 101. For example, the device control unit 810 performs reading and processing of the control data received by the communication unit 800 by the processing circuit 470, input / output control of the input / output circuit 410, and the like.

  FIG. 9 is a block diagram illustrating an example of a system configuration of the local server 102. As shown in FIG. 9, the local server 102 includes a communication unit 900, a received data analysis unit 910, a speech recognition unit 920, a local dictionary collation unit 930, a control unit 940, a speech synthesis unit 950, and a transmission. A data generation unit 960 and a command recording unit 970 are provided.

  The communication unit 900 corresponds to the first communication circuit 551 and the second communication circuit 552 in FIG. The communication unit 900 communicates with other devices (for example, the voice input / output device 240 and the device 101) via the wired communication such as a network or wireless communication. The communication unit 900 is also connected to a communication network such as the information communication network 220 via wired communication or wireless communication, and further communicates with the cloud server 111 via the communication network. The communication unit 900 performs communication via a wired LAN such as a network conforming to the Ethernet (registered trademark) standard, for example. The communication unit 900 passes data received from another device, the cloud server 111, and the like to the received data analysis unit 910. In addition, the communication unit 900 transmits the data generated by the transmission data generation unit 960 to other devices, the cloud server 111, and the like.

  The reception data analysis unit 910 corresponds to the processing circuit 570 in FIG. The reception data analysis unit 910 analyzes the type of data received by the communication unit 900. Also, the received data analysis unit 910 determines whether further processing is to be performed within the local server 102 or data should be transmitted to another device as a result of analyzing the type of received data. In the former case, the reception data analysis unit 910 delivers the received data to the voice recognition unit 920 or the like. In the latter case, the reception data analysis unit 910 determines a combination of a device to be transmitted next and data to be transmitted to the device.

  The speech recognition unit 920 is realized by the processing circuit 570, the acoustic model DB 580, and the language model DB 581 shown in FIG. The voice recognition unit 920 converts the voice signal into character string data. Specifically, the speech recognition unit 920 acquires information on the acoustic model registered in advance from the acoustic model DB 580, and converts the speech data into phoneme data from the acoustic model and the frequency characteristics of the speech data. Furthermore, the speech recognition unit 920 acquires information on a language model registered in advance from the language model DB 581 and converts the phoneme data into specific character string data based on how the language model and the phoneme data are arranged. The voice recognition unit 920 delivers the converted character string data to the local dictionary collation unit 930.

  The local dictionary collation unit 930 is realized by the processing circuit 570 in FIG. 5 and the local dictionary DB 584. The local dictionary collation unit 930 converts character string data into a semantic tag. Specifically, the semantic tag is a keyword indicating a device to be controlled, task contents, and the like. The local dictionary collating unit 930 collates the received character string data with the local dictionary DB 584 to extract a semantic tag that matches the character string data. The local dictionary DB 584 stores character strings such as words and meaning tags corresponding to the character strings in association with each other. By searching the local dictionary DB 584 for a character string that matches the received character string, a semantic tag that matches, that is, matches, the received character string is extracted.

  The control unit 940 corresponds to the processing circuit 570 in FIG. The control unit 940 performs a process for determining whether or not there is offload instruction data recorded in the command recording unit 970 and a process for comparing the offload instruction data recorded in the command recording unit 970 with the semantic tag.

  The speech synthesis unit 950 is realized by the processing circuit 570, the speech segment DB 582, and the prosody control DB 583 in FIG. The voice synthesizer 950 converts character string data into a voice signal. Specifically, the speech synthesizer 950 acquires information on the speech unit model and the prosody control model registered in advance from the speech unit DB 582 and the prosody control DB 583, respectively, and the speech unit model, the prosody control model, and the character From the column data, the character string data is converted into a specific audio signal.

  The transmission data generation unit 960 corresponds to the processing circuit 570 in FIG. The transmission data generation unit 960 generates transmission data from the combination of the device to be transmitted next and the data to be transmitted to the device determined by the reception data analysis unit 910.

  The command recording unit 970 corresponds to the memory 540 in FIG. The command recording unit 970 records offload command data transmitted from the cloud server 111 to the local server 102.

  FIG. 10 is a block diagram illustrating an example of a system configuration of the cloud server 111. As illustrated in FIG. 10, the cloud server 111 includes a communication unit 1000, a dialogue rule matching unit 1020, and a response generation unit 1030.

  The communication unit 1000 corresponds to the communication circuit 650 in FIG. The communication unit 1000 is connected to a communication network such as the information communication network 220 via wired communication or wireless communication such as a network, and further communicates with another device (for example, the local server 102) via the communication network. . The communication unit 1000 performs communication via a wired LAN such as a network conforming to the Ethernet (registered trademark) standard, for example.

  The dialogue rule matching unit 1020 is realized by the processing circuit 670 of FIG. 6 and the dialogue rule DB 691. The dialogue rule collation unit 1020 collates the semantic tag and its condition, and converts it into a processing ID for referring to the offload instruction generation DB.

  The response generation unit 1030 is realized by the processing circuit 670 of FIG. 6 and the offload instruction generation DB 692. The response generation unit 1030 collates the offload instruction generation DB 692 and generates a control signal for controlling the device 101 to be controlled. Furthermore, the response generation unit 1030 generates character string data of text information to be provided to the user 5100 based on the collation result.

  FIG. 11 is a diagram showing a specific example of registered contents in the local dictionary DB 584. As shown in FIG. In the local dictionary DB 584, character strings such as words and semantic tag information are stored in association with each other. A semantic tag refers to a logical expression representing the meaning of a certain character string. The semantic tag can hold a value as a variable. As shown in the table of FIG. 11, for example, the local dictionary DB 584 holds a character string “reserved” and a semantic tag <reserve> in association with each other. Further, for example, the local dictionary DB 584 holds the character string “volume 20” and the semantic tag “[<volume>] = 20” representing an expression in association with each other. <Volume> is a semantic tag representing the meaning of the character string “volume”, but the semantic tag enclosed in square brackets “[]” can serve as a variable that can hold a value.

  Referring to FIGS. 12A and 12B respectively, specific examples of registration contents of the dialogue rule DB 691 and the offload instruction generation DB 692 are shown. For example, FIG. 12A shows a specific example of the dialogue rule DB 691. In the dialogue rule DB 691, semantic tags, semantic tag value conditions, and processing ID information are stored in association with each other. For example, when a character string includes a plurality of words, such as a meaning tag <reserve> <program> corresponding to a character string “program reservation”, the meaning tag corresponding to such a character string includes a plurality of meaning tags. May be a combination. The condition of the semantic tag value is a condition set in the semantic tag, and is a condition regarding the value assigned to the semantic tag. The process ID is a number uniquely assigned to each item in the dialogue rule DB 691.

  FIG. 12B shows a specific example of the offload instruction generation DB 692. In the offload instruction generation DB 692, a process ID, a system response message, and offload instruction data are stored in association with each other. The system response message is a response message issued to the user in the dialog between the user and the voice interaction agent system 1. The offload command data is command data for processing in response to a user response to the system response message, and is command data corresponding to a plurality of dialogues between the user and the voice interaction agent system 1. One offload command data is configured to correspond to a plurality of user responses. Such offload command data reduces communication between the cloud server 111 and the local server 102 and reduces the load on the local server 102. Furthermore, in this example, since the offload instruction data includes a simple recognition instruction and a control instruction corresponding to the simple recognition instruction, processing based on the offload instruction data is simple. Here, the system response message is an example of a first response message, and the offload instruction data is an example of a conditional branch instruction. A plurality of dialogues between the user and the voice interaction agent system 1 are an example of selection candidates for user responses to the first response message.

  The process ID is a number uniquely assigned to each item in the offload instruction generation DB 692. The process ID of the offload instruction generation DB 692 corresponds to the process ID of the dialogue rule DB 691. When the process IDs are the same, the system response message and offload command data in the offload command generation DB 692 correspond to the semantic tag and semantic tag value conditions of the dialogue rule DB 691. For example, the system response message of the offload instruction generation DB 692 is a response message to the semantic tag of the dialogue rule DB 691. Further, the content of the offload instruction data in the offload instruction generation DB 692 is related to the meaning tag and the meaning tag value condition of the dialogue rule DB 691.

[2-2. Operation of Spoken Dialogue Agent System According to Embodiment]
Next, regarding the operation of the voice interaction agent system 1, the flow of processing for shortening the response time of the voice interaction agent system 1 will be described. FIG. 13A and FIG. 13B show a series of processes for shortening the response time of the voice interaction agent system 1. This sequence is started when the user 5100 starts some instruction to the voice input / output device 240 by voice.

  As shown in FIG. 13A, when the user 5100 inputs an instruction to the voice input / output device 240 by voice from a microphone or the like, the voice input / output device 240 acquires the voice data of the user 5100 in step S1501. The communication circuit 303 of the voice input / output device 240 transmits the acquired voice data to the local server 102. The local server 102 receives the audio data. The following description will be given on the assumption that the above instruction is the voice that is initially issued in a series of conversations performed by the user 5100 and the voice conversation agent system 1 over a plurality of times. Therefore, FIG. 13A shows a sequence of the voice interactive agent system 1 when a voice is first uttered. Here, the user's voice is an example of the first voice, and the voice data is an example of the first voice information.

  Next, in step S1502, the local server 102 receives voice data from the voice input / output device 240, and performs voice recognition processing on the voice data. The voice recognition process is a process of recognizing the user's voice by the voice recognition unit 920 included in the local server 102. Specifically, the local server 102 holds information on acoustic models and language models registered in the acoustic model DB 580 and the language model DB 581. When the user 5100 inputs voice to the voice input / output device 240, the CPU 530 of the local server 102 extracts the frequency characteristics from the voice of the user 5100, and corresponds to the extracted frequency characteristics from the acoustic model held in the acoustic model DB 580. Phoneme data to be extracted. Next, CPU 530 converts phoneme data into specific character string data by collating which character string data of the language model stored in language model DB 581 is the closest to the arrangement of the extracted phoneme data. . As a result, the voice data is converted into character string data.

  Next, in step S1503, the local server 102 performs local dictionary collation processing of character string data. The local dictionary collation process is a process in which character string data is converted into a semantic tag by the local dictionary collation unit 930 included in the local server 102. Specifically, the local server 102 holds dictionary information registered in the local dictionary DB 584. The CPU 530 of the local server 102 collates the character string data converted in step S1502 with the local dictionary DB 584, and outputs a semantic tag that matches the character string data, that is, a semantic tag corresponding to the character string data. Here, the output semantic tag is an example of first phrase information.

  Further, in step S1504, the local server 102 determines whether or not the offload command data received from the cloud server 111 is recorded in the command recording unit 970. If it is not recorded (No in step S1504), the local server 102 combines the semantic tag and its gateway ID and transmits it to the cloud server 111, and proceeds to step S1505. Note that the processing from step S1501 to S1503 described above is processing for the voice that the user 5100 first utters in a series of conversations between the user 5100 and the voice interaction agent system 1, and therefore, offload command data. Has not yet been generated and is not recorded in the command recording unit 970. For this reason, a series of processes in steps S1501 to S1508 performed before outputting the system response message to the user 5100 are continuously selected for the subsequent processes.

  However, in the middle of a series of conversations between the user 5100 and the voice interaction agent system 1, offload instruction data may be generated and recorded in the command recording unit 970. As described above, when the offload instruction data is recorded (Yes in Step S1504), the subsequent processing as shown in FIG. 13B is performed after outputting the system response message to the user 5100, Steps S1509 to 1514. A series of processes is selected. In this case, the processing of the local server 102 proceeds from step S1504 to step S1513.

  In step S <b> 1505 following step S <b> 1504, the cloud server 111 performs a dialogue rule matching process for the received semantic tag. The dialogue rule matching process is a process of converting the semantic tag into a process ID by the dialogue rule matching unit 1020 of the cloud server 111. Specifically, the cloud server 111 holds information on the dialogue rules held in the dialogue rule DB 691. As shown in FIG. 12A, the dialogue rule information includes a semantic tag, a semantic tag value condition, and a process ID. The CPU 671 of the cloud server 111 collates the meaning tag converted in step S1503 with the dialogue rule DB 691, confirms the condition of the meaning tag value held in the dialogue rule DB 691 and corresponding to the meaning tag, and the meaning The tag is converted into an appropriate process ID corresponding to the tag. For example, when the semantic tag is <reserve> <program> shown in FIG. 12A, this semantic tag is converted into the process ID “S001”.

  Next, in step S1506, the cloud server 111 performs an offload instruction generation process from the process ID. The offload command generation process is a process of generating a system response message and offload command data by the response generation unit 1030 of the cloud server 111. Specifically, the cloud server 111 holds information on the system response message and offload command data held in the offload command generation DB 692. The CPU 671 of the cloud server 111 collates the process ID converted in step S1505 with the offload instruction generation DB 692, and outputs a system response message and offload instruction data corresponding to the process ID. For example, in the example of FIG. 12B, when the process ID is “S001”, the system response message “Can I make a reservation?” And offload instruction data “if (<yes>) then cmd = 0x10010f0a and reply“ Then, the reservation is made. “Else reply“ Reservation is canceled ”” is output. The cloud server 111 transmits the generated system response message, offload command data, and gateway ID to the local server 102 in combination.

  In step S1507, the local server 102 associates the system response message received from the cloud server 111, the offload command data, and the gateway ID, and records them in the command recording unit 970. Specifically, the CPU 530 of the local server 102 stores the system response message, the offload command data, and the gateway ID in the memory 540 corresponding to the command recording unit 970 in association with each other. Here, the system response message and the gateway ID are examples of first response information, and the offload command data and the gateway ID are examples of second response information.

  In the next step S1508, the local server 102 performs speech synthesis processing. The voice synthesis process is a process in which the voice synthesis unit 950 included in the local server 102 converts the system response message into voice data. Specifically, the local server 102 holds information on speech units registered in the speech unit DB 582 and prosody information registered in the prosody control DB. The CPU 530 of the local server 102 reads the speech unit information registered in the speech unit DB 582 and the prosody information registered in the prosody control DB, and converts the character string data of the system response message into specific speech data. . Then, the local server 102 transmits the audio data converted in step S1508 to the audio input / output device 240. Furthermore, the voice input / output device 240 that has received the voice data outputs a system response message as voice to the user 5100 from a speaker or the like.

  Referring to FIG. 13B, after that, when the user 5100 inputs voice to the voice input / output device 240, the voice input / output device 240 acquires voice data of the user 5100 in step S <b> 1509. The voice input / output device 240 transmits the acquired voice data to the local server 102, and the local server 102 receives the voice data. The voice data may or may not contain an appropriate answer to the system response message. This determination is made in later processing. Thereafter, a series of processing in steps S1509 to 1514 performed after outputting the system response message to the user 5100 is performed. If the voice input / output device 240 does not detect voice input for a predetermined time or more, for example, a series of processes including the processes of steps S1501 to S1508 may be terminated. Here, the user's voice is an example of the second voice, and the voice data is an example of the second voice information.

  Next, in step S1510, the local server 102 receives voice data from the voice input / output device 240, and further performs voice recognition processing for converting the voice data into character string data in the same manner as the processing in step S1502. Further, in step S1511, the local server 102 performs a local dictionary collation process for converting the converted character string data into a semantic tag, as in the process of step S1503.

  In the next step S1512, the local server 102 determines whether or not the offload command data received from the cloud server 111 is recorded in the command recording unit 970. If it is recorded (Yes in step S1512), the local server 102 proceeds to the process of step S1513. The recorded offload instruction data is the data recorded in step S1507. In this example, since the offload instruction data is recorded in the command recording unit 970 in step S1507, and the system response message is output as voice in step S1508, the process in step S1513 is performed.

  However, when the offload command data is not recorded (No in step S1512), for example, a series of processes in steps S1501 to S1508 before outputting a system response message to the user 5100 is selected for the subsequent processes. Will be.

  In the next step S1513, the local server 102 determines whether or not the meaning tag converted in step S1511 matches the condition included in the offload instruction data recorded in the command recording unit 970. If a match is found (Yes in step S1513), the local server 102 transmits a control command conforming to the condition among the control commands described in the offload instruction data to the device 101, and is described in the offload instruction data. The character example data of the message when conforming to the relevant condition is output. Further, the local server 102 combines the execution notification of the control command and the gateway ID, and transmits them to the cloud server 111. The received cloud server 111 recognizes the completion of the control of the device 101 involving a plurality of dialogues between the user 5100 and the voice dialogue agent system 1. Then, the local server 102 proceeds to the speech synthesis process in step S1514. Here, the meaning tag is an example of second phrase information.

  On the other hand, if there is no match (No in step S1513), the local server 102 outputs the character string data of the message when the condition is not met among the messages described in the offload command data. Further, the local server 102 outputs a control command at the time of nonconformity to the condition among the control commands described in the offload instruction data, but does not output the control command in this example as shown in FIG. 12B. In the offload instruction data, a control command at the time of non-conformity different from that at the time of conforming to the condition may be described, and the local server 102 may output this control command. Then, the local server 102 combines the non-execution notification of the control command and the gateway ID, and transmits them to the cloud server 111. Further, the local server 102 proceeds to the speech synthesis process in step S1514.

  For example, “if (<yes>) then cmd = 0x10010f0a and reply“ I will reserve ”else reply“ Cancel reservation ”” shown in FIG. 12B is recorded in the command recording unit 970. Suppose that Further, in step S1508 prior to step S1513, a system response message “Can I make a reservation?” Is output from the voice input / output device 240. When the user 5100 utters “Yes”, “No”, or other phrases, the local server 102 converts the voice data into the character strings “Yes”, “No”, or “phrase contents” in the voice recognition process in step S1510. To "". Thereafter, in the local dictionary collation process in step S1511, the local server 102 converts the character string “Yes”, “No”, or “phrase content” into the meaning tag <yes>, <no>, or <phrase content keyword>. Convert to In step S 1512, the local server 102 determines that offload command data is recorded in the command recording unit 970.

  In step S1513, the local server 102 determines whether or not the semantic tag <yes>, <no>, or <phrase content keyword> matches the condition of the offload instruction data. In this example, the condition of the offload instruction data is “if (<yes>)”. Therefore, when the semantic tag is <yes>, the semantic tag matches the condition of the offload instruction data. When the semantic tag is other than <no> or <phrase content keyword>, that is, <yes>, the semantic tag does not match the condition of the offload instruction data. In the case of a match, the local server 102 transmits a control command “cmd = 0x10010f0a” to the device 101, and outputs character string data of a message “I will make a reservation”. If there is no match, the local server 102 outputs only the character string data of the message “Cancel reservation”. As described above, the offload instruction data is an example of a conditional branch instruction and includes a plurality of different instructions that branch according to one or more pieces of phrase information including semantic tags <yes> and <no>. Further, the message “I will make a reservation” and the message “Cancel reservation” are examples of the second response message.

  In step S1514, the local server 102 performs speech synthesis processing. In this voice synthesis process, the local server 102 converts the message output in relation to the process in step S1513 into voice data, as in the process in step S1508. For example, the character string data “Now make a reservation” or “Cancel reservation” of the message exemplified above is converted into voice data. The local server 102 transmits the audio data converted in step S1514 to the audio input / output device 240. Furthermore, the voice input / output device 240 that has received the voice data outputs the message as voice to the user 5100 from a speaker or the like.

  As described above, in the processing of steps S1501 to S1514, the processing related to a plurality of dialogues between the user 5100 and the voice dialogue agent system 1 is performed by the local server 102, and the offload command data is generated by the cloud server. 111. For this reason, the communication between the local server 102 and the cloud server 111 is performed when offload command data is generated, and does not need to be performed every time the user 5100 and the voice interaction agent system 1 interact. For this reason, the frequency | count of communication between the local server 102 and the cloud server 111 is restrained low, and communication time becomes short. The voice interaction agent system 1 according to such an embodiment provides a voice interaction agent system by offloading a simple recognition command and a corresponding control command to the local side, that is, so as to reduce the burden. 1 response time is shortened. In addition, since the cloud server 111 that can include a large-capacity database generates offload instruction data, the load on the local server 102 can be kept low.

  Further, the above description of the operation of the voice interaction agent system 1 has been described with respect to the case where the first sound of the user 5100 is emitted in step S1501, but the time of the conversation in step S1501 is not specified. May be.

  In the embodiment, the offload instruction data including one condition and an instruction corresponding to the condition has been exemplified, but the present invention is not limited to this. The offload instruction data may include two or more conditions and an instruction corresponding to each condition. Thereby, it is possible to control the device 101 corresponding to various answers of the user 5100.

  Further, in the embodiment, one offload command data is used for the process related to the dialogue between the user 5100 and the voice dialogue agent system 1, but two or more offload command data are used. May be. For example, when the condition of the first offload instruction data is not satisfied, the process may be performed so that the second offload instruction data is used.

  In the spoken dialogue agent system 1 according to the embodiment, the local server 102 transmits the semantic tag to the cloud server 111 in step S1504, but the present invention is not limited to this. The local server 102 may transmit the character string data instead of or in addition to the semantic tag. Such character string data is an example of first phrase information. In this case, if the cloud server 111 includes a dictionary DB such as the local dictionary DB 584, the cloud server 111 may convert the received character string data into a semantic tag. Since the cloud server 111 can have a dictionary DB having a larger capacity than the local server 102, it is possible to convert a wide variety of character string data into semantic tags.

  In the voice interaction agent system 1 according to the embodiment, the cloud server 111 transmits the system response message and the offload command data together to the local server 102, but transmits them separately with a time difference. Also good. For example, the cloud server 111 may transmit the offload command data after transmitting the system response message, and the transmission timing of the offload command data is determined by performing the process of step S1512 after the transmission of the system response message in step S1506. Any time during the period before doing. For example, the cloud server 111 may estimate the period or store the period estimated in advance, and transmit offload command data within the estimated period. The transmission of the offload command data is preferably performed before the voice input / output device 240 outputs the system response message by voice. Such timing is estimated, and the offload command data is based on the estimated timing. May be sent.

[3. Effect]
As described above, the cloud server 111 which is an aspect of the dialog processing device according to the embodiment of the present disclosure includes the communication unit 1000 as an acquisition unit that acquires a speech recognition result and the dialog rule as a dialog processing unit. A collation unit 1020 and a response generation unit 1030 are provided. The dialogue rule matching unit 1020 and the response generation unit 1030 determine response information based on the speech recognition result acquired by the communication unit 1000, and determine processing information corresponding to the response information. Note that the processing information includes estimated response information regarding a user's response estimated with respect to the response information, and operation control information corresponding to the estimated response information. The communication unit 1000 transmits the determined response information and processing information. For example, the response information may be a system response message. For example, the estimated response information may be information related to a condition included in the offload command data, and the operation control information may include a control command and a message corresponding to the condition included in the offload command data.

  In the above configuration, the cloud server 111 determines and transmits response information based on the voice recognition result and processing information corresponding to the response information. Then, a device such as the local server 102 that receives the response information and the processing information from the cloud server 111 can output a response message for the user's voice recognition result to the user based on the response information. Based on the estimated response information and the operation control information, the operation control corresponding to the user's response to the response message can be performed. The estimated response information is information estimated based on the response information, and the operation control information is information set corresponding to the estimated response information. Such processing information is not information generated in response to a response after receiving a response from the user, but information set in advance. Thus, when the device receives a response from the user, the device does not need to communicate with the cloud server 111 in order to acquire information corresponding to the response. Therefore, when a plurality of dialogues with the user are performed, the number of communication between the device and the cloud server 111 is reduced, and the response time of the device and the cloud server 111 can be shortened.

  In cloud server 111 of one aspect of the dialog processing device according to the embodiment, communication unit 1000 receives operation control result information indicating a result of operation control performed using the operation control information. For example, the operation control result information may be a control command execution notification. In the above-described configuration, the cloud server 111 can end the operation related to the interactive process between the user and the device such as the local server 102 by receiving the operation control result information. Therefore, unnecessary standby and operation of the cloud server 111 can be reduced.

  In the cloud server 111 of one aspect of the interactive processing device according to the embodiment, the communication unit 1000 transmits both response information and processing information. In the above configuration, the communication frequency of the cloud server 111 is reduced, and the response time of devices such as the cloud server 111 and the local server 102 can be shortened. Specifically, since the cloud server 111 transmits the processing information without confirming the user's response to the response information, communication for confirming the user's response is unnecessary. Furthermore, although a device such as the local server 102 has received a response to the response information from the user, the processing information has not yet been received and processing delays such that the next operation cannot be performed can be prevented.

  In the cloud server 111 of one aspect of the dialog processing device according to the embodiment, a speech recognition result is acquired via the communication unit 1000. In the configuration described above, the cloud server 111 acquires a speech recognition result from a device such as the local server 102 at a remote location. Therefore, the cloud server 111 can be configured without being affected by the environment of the voice acquisition location.

  The local server 102 of another aspect of the interactive processing device according to the embodiment includes a speech recognition unit 920 as an acquisition unit that acquires a speech recognition result, a communication unit 900, and a control unit 940 as an operation control unit. Prepare. The communication unit 900 receives response information based on the first voice recognition result acquired by the voice recognition unit 920 and processing information corresponding to the response information. The processing information includes estimated response information about the user's response estimated from the response information and operation control information corresponding to the estimated response information. The control unit 940 performs an operation based on the operation control information according to the collation result between the second speech recognition result acquired by the speech recognition unit 920 and the estimated response information.

  In the above-described configuration, the local server 102 can output a response message to the first voice recognition result to the user based on the response information based on the first voice recognition result of the user's voice. Based on the estimated response information and the operation control information, the operation control corresponding to the user's response to the response message can be performed. Furthermore, the local server 102 can perform an operation based on the operation control information corresponding to the second voice recognition result of the user's voice. The processing information is information that is set in advance, not information that is generated in response to the response after receiving the response from the user. Thus, when the local server 102 acquires the second speech recognition result, it is not necessary to communicate in order to acquire information for corresponding to the second speech recognition result. Therefore, the response time of the local server 102 can be shortened when the user interacts a plurality of times.

  In local server 102 of another aspect of the interactive processing device according to the embodiment, communication unit 900 receives both response information and processing information. In the above configuration, the communication frequency of the local server 102 is reduced, and the response time of the local server 102 can be shortened. Specifically, since the local server 102 receives the processing information before obtaining the user's response to the response information, there is no need to communicate in response to the user's response. Furthermore, although the local server 102 has received a response to the response information from the user, the processing information has not yet been received, and processing delays such that the next operation cannot be performed can be prevented.

  In local server 102 of another aspect of the interactive processing device according to the embodiment, communication unit 900 transmits a first speech recognition result. In the configuration described above, the processing based on the first speech recognition result is performed by an apparatus different from the local server 102 such as the cloud server 111. Thereby, the processing load on the local server 102 is reduced, and the processing speed can be improved.

  In local server 102 of another aspect of the dialog processing device according to the exemplary embodiment, the second speech recognition result is a speech recognition result after a response based on the response information received by communication unit 900 is reproduced. . In the above configuration, the second speech recognition result may include a user response to the response message based on the response information. Therefore, the local server 102 enables processing appropriately corresponding to the user's response.

  In the cloud server 111 and the local server 102 according to the embodiment, the processing information includes a plurality of sets of estimated response information and operation control information. In the above configuration, control corresponding to various responses of the user is possible.

  In the cloud server 111 and the local server 102 according to the embodiment, the operation control information includes second response information with respect to a user response and operation instruction information including an operation instruction to the control target. For example, the user response may include a message, the estimated response information may include message information corresponding to the content of the message, and the operation control information may include second response information and operation instruction information corresponding to the message information. . In the above-described configuration, in the operation based on the operation control information, it is possible to control the control target while presenting a response to the user and receiving confirmation. For example, the message information may be a semantic tag corresponding to the semantic tag of the user's message and included in the condition in the offload command data. The second response information according to the message information may be a message according to the condition in the offload instruction data, and the operation instruction information according to the message information is a control command according to the condition in the offload instruction data. It may be.

  In addition, the conversation processing method of the cloud server 111 according to an aspect of the embodiment acquires a speech recognition result, determines response information based on the acquired speech recognition result, and determines processing information according to the response information. Then, the determined response information and processing information are transmitted, and the processing information includes estimated response information regarding a user's response estimated with respect to the response information, and operation control information corresponding to the estimated response information.

  Further, the dialogue processing method of the local server 102 according to another aspect of the embodiment acquires and transmits the first speech recognition result, and response information based on the transmitted first speech recognition result; Processing information corresponding to the response information is received. Here, the processing information includes estimated response information about the user's response estimated from the response information, and operation control information corresponding to the estimated response information. Further, the dialogue processing method obtains the second voice recognition result, and performs an operation based on the action control information according to the collation result between the obtained second voice recognition result and the estimated response information.

  According to the above-described dialogue processing method, the same effect as that obtained by the dialogue processing apparatus according to the embodiment can be obtained. Note that the above method may be realized by an MPU (Micro Processing Unit), a CPU, a processor, a circuit such as an LSI (Large Scale Integration), an IC card (Integrated Circuit Card), or a single module. Good.

  The processing in the embodiment may be realized by a software program or a digital signal made up of a software program. For example, the processing in the embodiment is realized by the following program or a digital signal including the program.

  The program that realizes the processing of the cloud server 111 according to an aspect of the embodiment is a program that causes a computer to execute the following functions, acquires a speech recognition result, and response information based on the acquired speech recognition result And processing information corresponding to the response information is determined, and transmission of the determined response information and processing information is executed. Note that the processing information includes estimated response information regarding a user's response estimated with respect to the response information, and operation control information corresponding to the estimated response information.

  A program that realizes processing of the local server 102 according to another aspect of the embodiment is a program that causes a computer to execute the following functions, obtains a speech recognition result, and obtains the first speech recognition result And receiving the response information based on the response information and the processing information corresponding to the response information. Here, the processing information includes estimated response information about the user's response estimated from the response information, and operation control information corresponding to the estimated response information. The program further causes the computer to execute control based on the operation control information in accordance with the collation result between the acquired second speech recognition result and the estimated response information.

[Others]
As described above, the dialogue processing apparatus according to the embodiment has been described as an example of the technology disclosed in the present application, but the present disclosure is not limited to the embodiment. The technology in the present disclosure can also be applied to a modified example of the embodiment in which modifications, replacements, additions, omissions, and the like are appropriately performed or other embodiments. In addition, it is possible to combine the constituent elements described in the embodiment to form a new embodiment or modification.

  As described above, the comprehensive or specific aspect of the present disclosure may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. The comprehensive or specific aspect of the present disclosure may be realized by any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

  For example, each processing unit included in the dialogue processing apparatus according to the above embodiment is typically realized as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

  Further, the circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  In the above embodiment, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  Furthermore, the technology of the present disclosure may be a program or a non-transitory computer-readable recording medium on which the program is recorded. Needless to say, the program can be distributed via a transmission medium such as the Internet.

  Further, the numbers such as the ordinal numbers and the quantities used in the above are examples for specifically explaining the technology of the present disclosure, and the present disclosure is not limited to the illustrated numbers. In addition, the connection relationship between the constituent elements is exemplified for specifically explaining the technology of the present disclosure, and the connection relationship for realizing the functions of the present disclosure is not limited thereto.

  In addition, division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, a single functional block can be divided into a plurality of functions, or some functions can be transferred to other functional blocks. May be. In addition, functions of a plurality of functional blocks having similar functions may be processed in parallel or time-division by a single hardware or software.

  As mentioned above, although the interactive processing apparatus etc. which concern on one aspect were demonstrated based on embodiment, this indication is not limited to embodiment. Unless it deviates from the gist of the present disclosure, forms in which various modifications conceived by those skilled in the art have been made in the embodiments and forms constructed by combining components in different embodiments are also included in the scope of one aspect. May be.

  It should be noted that the present disclosure is applicable as long as it relates to a dialogue between a voice interaction agent system and a user. For example, it is effective when a user operates home appliances or the like using a voice interaction agent system. For example, it is assumed that the user gives an instruction “warm up” when operating a microwave oven or oven corresponding to voice operation. At this time, the voice interaction agent system can ask the user to return a specific instruction such as “How many minutes do you want to warm up?” Or “How many times do you want to warm up?”. In contrast to this, only users who initially instructed to “warm up” are users who can respond (users who receive instructions from the agent system in response to hearing back).

  In addition to this, the present disclosure can be applied to an operation in which the voice interaction agent system listens back to specific contents in response to an abstract instruction from the user. The content that the voice interaction agent system asks the user to return may be confirmation of operation execution.

  Note that in the above aspect, voice input from the user may be performed by a microphone provided in the system or each home appliance. Further, the user's feedback from the voice interaction agent system may be transmitted to the user from a speaker or the like included in the system or each home appliance. In the present disclosure, the “predetermined operation” may be an operation of outputting sound to the user via a speaker, for example. That is, in the present disclosure, the “device” to be controlled may be a voice input / output device (for example, a speaker). In the present disclosure, the “processor”, “microphone”, and / or “speaker” may be incorporated in a “device” to be controlled, for example. In the present disclosure, “phrase information” is information indicating a character string or its meaning. “Character string data” and “meaning tag” in the above embodiment are examples of phrase information.

  In addition, the technique demonstrated in the said aspect can be implement | achieved in the following types of cloud services, for example. However, the types of cloud services in which the technology described in the above aspect is realized are not limited to these.

  Hereinafter, an overview of services provided by an information management system using service type 1 (in-house data center type cloud service), and services provided by an information management system using service type 2 (cloud service using IaaS) Overview of services provided by an information management system using a service type 3 (PaaS-based cloud service), and information services provided by an information management system using a service type 4 (SaaS-based cloud service) The overall image will be described sequentially.

[Service type 1: In-house data center type cloud service]
FIG. 14 is a diagram illustrating an overall image of a service provided by an information management system in service type 1 (in-house data center type cloud service) to which the voice interaction agent system according to the embodiment is applicable. As shown in FIG. 14, in this type, the service provider 4120 acquires information from the group 4100 and provides a service to the user. In this type, the service provider 4120 has a function of a data center operating company. That is, the service provider 4120 has a cloud server 111 that manages big data. Therefore, there is no data center operating company.

  In this type, the service provider 4120 operates and manages a data center (cloud server) 4203. The service provider 4120 manages an operating system (OS) 4202 and an application 4201. The service provider 4120 provides a service using the OS 4202 and the application 4201 managed by the service provider 4120 (arrow 204).

[Service type 2: Cloud service using IaaS]
FIG. 15 is a diagram illustrating an overall image of a service provided by an information management system in service type 2 (cloud service using IaaS) to which the voice interaction agent system according to the embodiment is applicable. Here, IaaS is an abbreviation for infrastructure as a service, and is a cloud service provision model that provides a base for constructing and operating a computer system as a service via the Internet.

  As shown in FIG. 15, in this type, a data center operating company 4110 operates and manages a data center (cloud server) 4203. The service provider 4120 manages the OS 4202 and the application 4201. The service provider 4120 provides a service using the OS 4202 and the application 4201 managed by the service provider 4120 (arrow 204).

[Service type 3: Cloud service using PaaS]
FIG. 16 is a diagram showing an overall image of a service provided by an information management system in service type 3 (PaaS use type cloud service) to which the voice interaction agent system according to the embodiment is applicable. Here, PaaS is an abbreviation for Platform as a Service, and is a cloud service provision model that provides a platform serving as a foundation for constructing and operating software as a service via the Internet.

  As shown in FIG. 16, in this type, the data center operating company 4110 manages the OS 4202 and operates and manages the data center (cloud server) 4203. The service provider 4120 manages the application 4201. The service provider 4120 provides a service using the OS 4202 managed by the data center operating company 4110 and the application 4201 managed by the service provider 4120 (arrow 204).

[Service type 4: Cloud service using SaaS]
FIG. 17 is a diagram illustrating an overall image of a service provided by an information management system in service type 4 (SaaS-based cloud service) to which the voice interaction agent system according to the embodiment is applicable. Here, SaaS is an abbreviation for software as a service. The SaaS-based cloud service is, for example, an application provided by a platform provider who owns a data center (cloud server), or a user such as a company or individual who does not have a data center (cloud server) on the Internet. This is a cloud service provision model that has functions that can be used via other networks.

  As shown in FIG. 17, in this type, the data center operating company 4110 manages the application 4201, manages the OS 4202, and operates and manages the data center (cloud server) 4203. The service provider 4120 provides a service using the OS 4202 and the application 4201 managed by the data center operating company 4110 (arrow 204).

  As described above, in any cloud service type, the service provider 4120 provides a service. In addition, for example, the service provider or the data center operating company may develop an OS, an application, or a big data database, or may be outsourced to a third party.

  The technology of the present disclosure can be applied to a voice interaction agent.

101, 101a, 101b equipment 102 local server 111 cloud server 240 voice input / output device 300 processing circuit of voice input / output device 301 sound collecting circuit of voice input / output device 302 voice output circuit of voice input / output device 303 communication of voice input / output device Circuit 310 CPU of voice input / output device
320 Memory of voice input / output device 330 Bus of voice input / output device 341 Device ID of voice input / output device
342 Voice input / output device program 410 Device input / output circuit 430 Device CPU
440 Device memory 441 Device ID of device
442 Device program 450 Device communication circuit 460 Device bus 470 Device processing circuit 530 Local server CPU
540 Local server memory 541 Local server gateway ID
542 Local server program 551 Local server first communication circuit 552 Local server second communication circuit 560 Local server bus 570 Local server processing circuit 580 Local server acoustic model DB
581 Local server language model DB
582 Local server speech segment DB
583 Prosody control DB of local server
584 Local dictionary DB of local server
650 Cloud server communication circuit 670 Cloud server processing circuit 671 Cloud server CPU
672 Cloud server memory 680 Cloud server bus 691 Cloud server conversation rule DB
692 Cloud Server Offload Instruction Generation DB
700 Sound Collection Unit of Audio Input / Output Device 710 Audio Detection Unit of Audio Input / Output Device 720 Audio Section Extraction Unit of Audio Input / Output Device 730 Audio Communication Unit of Audio Input / Output Device 740 Audio Output Unit of Audio Input / Output Device 800 Device Communication Unit 810 Device control unit of device 900 Local server communication unit 910 Local server received data analysis unit 920 Local server speech recognition unit 930 Local server local dictionary collation unit 940 Local server control unit 950 Local server speech synthesis unit 960 Local Server transmission data generation unit 970 Local server command recording unit 1000 Cloud server communication unit 1020 Cloud server interaction rule verification unit 1030 Cloud server response generation unit

Claims (17)

An information processing method executed by a processor that controls at least one device through dialogue with a user,
Obtaining first voice information indicating the first voice of the user input from a microphone;
Outputting the first phrase information generated from the first voice information to a server via a network;
First response information corresponding to the first phrase information is acquired from the server via the network, and the first response information indicates a first response message for the first voice,
Based on the first response information, the speaker outputs the first response message,
Second response information associated with the first response information on the server is acquired from the server via the network, and the second response information includes a plurality of instructions that differ according to one or more pieces of phrase information Each of the one or more pieces of phrase information relates to a selection candidate of the user's response to the first response message,
After the first response message is output, second voice information indicating the second voice of the user input from the microphone is acquired, and the second voice includes the user's response to the first response message. ,
Of the conditional branch instructions, in response to an instruction determined by collating the one or more phrase information and the second phrase information generated from the second audio information, the speaker and the at least one device Causing at least one to perform a predetermined action,
Neither the second audio information nor the second phrase information is output to the server until the second operation is performed after the second audio information is acquired.
Information processing method. The first response information and the second response information are acquired from the server at the same time.
The information processing method according to claim 1. After the first phrase information is output to the server, the second response information is a plurality of second information stored on the server according to at least one of the first phrase information and the first response information. Selected from the response information,
The information processing method according to claim 1 or 2. Furthermore, after executing the predetermined operation, an execution notification indicating that the predetermined operation has been executed is output to the server.
The information processing method according to any one of claims 1 to 3. At least one of the plurality of instructions includes an instruction to output a second response message for the second sound from the speaker.
The information processing method according to any one of claims 1 to 4. At least one of the plurality of instructions includes an instruction that causes a control command to be transmitted to the at least one device.
The information processing method according to any one of claims 1 to 5. The network is the Internet;
The information processing method is executed on a local server capable of communicating with the at least one device without going through the Internet.
The information processing method according to any one of claims 1 to 6.   The program which makes the said processor perform the information processing method as described in any one of Claim 1 to 7. An information processing method executed by a second processor on a server, wherein the second processor is capable of communicating via a network with a first processor that controls at least one device through interaction with a user.
First phrase information related to the first voice of the user input from a microphone is acquired from the first processor via the network, and the first phrase information includes a character string and the character corresponding to the first voice. Indicates at least one of the column semantic information,
First response information corresponding to the first phrase information is output to the first processor via the network, and the first response information is a first response message for the first sound output from a speaker. Show
Second response information associated with the first response information on the server is output to the first processor via the network, and the second response information is different depending on one or more pieces of phrase information. Each of the one or more pieces of phrase information relates to a selection candidate of the user's response to the first response message.
Information processing method. The first processor, in the conditional branch instruction, according to an instruction determined by collating the one or more phrase information and second phrase information generated from second audio information, the speaker and the Causing at least one of the at least one device to perform a predetermined operation;
The information processing method according to claim 9. The second processor outputs the second voice information and the second phrase information to the first processor from when the first processor acquires the user's response until the predetermined operation is executed. do not do,
The information processing method according to claim 10. Further, an execution notification indicating that the predetermined operation has been executed is acquired from the first processor.
The information processing method according to claim 10 or 11. The first response information and the second response information are simultaneously output to the first processor.
The information processing method according to any one of claims 9 to 12. At least one of the plurality of instructions includes an instruction to output a second response message to the second sound from the speaker.
The information processing method according to any one of claims 9 to 13. At least one of the plurality of instructions includes an instruction that causes the at least one device to output a control command.
The information processing method according to any one of claims 9 to 14. On the server, a plurality of semantic information, a plurality of first response information respectively associated with the plurality of semantic information, and a plurality of second response information respectively associated with the plurality of first response information And a database containing
After obtaining the first phrase information, referring to the database,
Identifying semantic information corresponding to the first voice from the plurality of semantic information;
Identifying the first response information associated with the identified semantic information from the plurality of first response information;
Identifying the second response information associated with the identified first response information from the plurality of second response information;
The information processing method according to any one of claims 9 to 15.   A program causing the second processor to execute the information processing method according to any one of claims 9 to 16.

Images