JP2021173880A - Information processing unit, program and information processing method - Google Patents

Abstract

To respond to user's voice even when communication with a device which recognizes voice is disabled.SOLUTION: In an interaction system 1, an interaction device 100 is an information processing device which connects with a voice recognition system recognizing voice through a network, and comprises: a voice acquisition part which acquires user's voice; a determination part which determines whether communication with the voice recognition system is available; a transmission part which transmits voice data on the acquired voice to the voice recognition system if the communication with the voice recognition system is available; a reception part which receives first recognition information representing a recognition result of the voice data from the voice recognition system; a voice recognition part which recognizes the acquired voice and generates second recognition information representing a recognition result if the communication with the voice recognition system is not available; a response generation part which generates first response information for responding to voice based upon the first recognition information or second recognition information; and an output part which outputs a response to the voice based upon the first response information.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device, a program, and an information processing method.

Conventionally, there is known a technique of a so-called smart speaker, which is a device that acquires a user's voice and performs various operations in response to the acquired voice.

In the smart speaker disclosed in Patent Document 1 below, voice information indicating a user's voice is input, and this voice information is transmitted to a voice output device connected via a network. The voice output device recognizes the voice based on the received voice information, and generates utterance data for speaking to the user's voice based on the recognition result. The voice output device transmits this utterance data to the smart speaker, and the smart speaker outputs voice based on the utterance data.

Japanese Unexamined Patent Publication No. 2020-21040

The smart speaker of Patent Document 1 has a problem that it cannot respond to the user's voice when the communication of the voice output device becomes impossible.

Therefore, an object of the present invention is to provide an information processing device, a program, and an information processing method capable of responding to a user's voice even when communication with a device that recognizes voice becomes impossible.

The information processing device according to one aspect of the present invention is an information processing device that connects to a voice recognition system that recognizes voice via a network, and is a communication between a voice acquisition unit that acquires a user's voice and the voice recognition system. When communication with the voice recognition system is possible, the voice data recognition result from the transmission unit that transmits the voice data of the acquired voice to the voice recognition system and the voice recognition system. When communication between the receiving unit that receives the first recognition information indicating the above and the voice recognition system is impossible, the voice recognition unit that recognizes the acquired voice and generates the second recognition information indicating the recognition result, and the first It includes a response generation unit that generates first response information for responding to voice based on one recognition information or second recognition information, and an output unit that outputs a response to voice based on the first response information. ..

The program according to one aspect of the present invention has a voice acquisition function for acquiring a user's voice and communication with the voice recognition system in an information processing device connected to a voice recognition system for recognizing voice via a network. The first, which indicates the recognition result of the voice data from the voice recognition system, and the transmission function of transmitting the voice data of the acquired voice to the voice recognition system when communication with the voice recognition system is possible. When communication with the voice recognition system is not possible with the reception function that receives the recognition information, the voice recognition function that recognizes the acquired voice and generates the second recognition information indicating the recognition result, and the first recognition information or A response generation function that generates a first response information for responding to a voice based on the second recognition information, and an output function that outputs a response to the voice based on the first response information are realized.

In the information processing method according to one aspect of the present invention, the information processing device connected to the voice recognition system that recognizes voice via a network acquires the user's voice and determines whether or not communication with the voice recognition system is possible. Then, when communication with the voice recognition system is possible, the voice data of the acquired voice is transmitted to the voice recognition system, and the first recognition information indicating the recognition result of the voice data is received from the voice recognition system to perform voice recognition. When communication with the system is impossible, it recognizes the acquired voice, generates the second recognition information indicating the recognition result, and responds to the voice based on the first recognition information or the second recognition information. The first response information is generated, and the response to the voice is output based on the first response information.

According to the above aspect, even when communication between the information processing device and the voice recognition system is impossible, the voice recognition unit in the information processing device can recognize the user's voice. Therefore, the information processing device can respond to the user's voice even when communication with the voice recognition system is impossible.

According to the present invention, it is possible to provide an information processing device, a program, and an information processing method capable of responding to a user's voice even when communication with a device that recognizes voice becomes impossible.

It is a figure for demonstrating the system configuration example of the dialogue system which concerns on 1st Embodiment. It is a figure for demonstrating the outline of the dialogue system which concerns on 1st Embodiment. It is a figure for demonstrating the outline of the dialogue system which concerns on 1st Embodiment. It is a figure which shows an example of the functional structure of the dialogue apparatus which concerns on 1st Embodiment. It is a figure explaining an example of the relationship between a local instruction / remote instruction and its execution / queuing which concerns on 1st Embodiment. It is a figure which shows an example of the functional structure of the server apparatus which concerns on 1st Embodiment. It is a figure which shows the operation example of the dialogue apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the hardware composition of the dialogue apparatus and the server apparatus which concerns on 1st Embodiment. It is a figure for demonstrating the outline of the dialogue system which concerns on 2nd Embodiment. It is a figure for demonstrating the outline of the dialogue system which concerns on 2nd Embodiment. It is a figure which shows the operation example of the dialogue apparatus which concerns on 2nd Embodiment.

Preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each figure, those having the same reference numerals have the same or similar configurations.

In the present embodiment, the "part", "means", "device", and "system" do not simply mean physical means, but the "part", "means", "device", and "system". Including the case where the function of is realized by software. Further, even if the functions of one "part", "means", "device", or "system" are realized by two or more physical means or devices, two or more "parts" or "means", The functions of "device" and "system" may be realized by one physical means or device.

[First Embodiment]
The first embodiment of the present invention (hereinafter, referred to as “the present embodiment”) will be described. In the present embodiment, the dialogue system 1 according to the present embodiment (1) interacts with the user, (2) records the user's voice in the minutes, and (3) operates a device such as a home appliance according to the user's voice instruction. It will be explained using an example of controlling, but the purpose is not limited to this.

<1. System configuration>
A system configuration example of the dialogue system 1 will be described with reference to FIG. The dialogue system 1 is a system that operates in response to a user's voice. The dialogue system 1 provides the functions (1) to (3) above to the user.

As shown in FIG. 1, the dialogue system 1 includes a dialogue device 100 and a server device 200. Further, the dialogue system 1 is connected to the voice recognition system 300 via the first network N1. Further, the dialogue device 100 of the dialogue system 1 is connected to the local device 400a via the second network N2. Further, the dialogue system 1 is connected to the remote device 400b via the first network N1.

The first network N1 is a network of a wide area communication network, and includes, for example, the Internet, a mobile communication network, a telephone line, and the like. The first network N1 is, for example, a 3G (3rd generation mobile communication system) line, a 4G (4th generation mobile communication system) line, a 5G (5th generation mobile communication system) line, or LTE (registered trademark) (registered trademark). A wireless communication method using a Long Term Evolution) line or the like may be used.

The second network N2 is a communication network independently constructed for a predetermined facility or room, and is a LAN (Local Area Network). In other words, the dialogue device 100 and the local device 400a are installed in the same LAN. The second network N2 can use any communication method as long as the dialogue device 100 and the local device 400a can communicate with each other by wire and / or wirelessly. Further, the second network N2 may use a plurality of communication methods. The second network N2 may include, for example, a wireless LAN conforming to the Wi-Fi (registered trademark) standard, and these interconnections may be realized by relaying by a router.

The second network N2 may be, for example, a network for directly connecting to the local device 400a. The second network N2 may include, for example, a network that realizes short-range wireless communication of about 10 m such as Bluetooth (registered trademark) and infrared communication.

The dialogue device 100 is an information processing device capable of communicating with the server device 200, the voice recognition system 300, or the device 400. The dialogue device 100 is a so-called smart speaker that acquires a user's voice and responds to the acquired voice by dialogue or the like. The dialogue device 100 may be, for example, a general-purpose tablet terminal or a smartphone. The dialogue device 100 may use the tablet terminal as the dialogue device 100 by, for example, installing a dedicated program on a general-purpose tablet terminal and executing this program.

The server device 200 is an information processing device capable of communicating with the dialogue device 100 and managing minutes. By executing a predetermined program, the server device 200 cooperates with the dialogue device 100 to newly register, change, and delete the response to the user's voice and the minutes (hereinafter, these processes are collectively referred to as "update"). Or, realize a server function that manages these histories.

The voice recognition system 300 is a system capable of communicating with the dialogue device 100 and the server device 200. The voice recognition system 300 recognizes the user's voice based on the voice data (hereinafter, also simply referred to as “voice data”) indicating the user's voice received from the dialogue device 100 or the server device 200.

The local device 400a and the remote device 400b are devices whose operations are controlled by the dialogue device 100 in response to a user's voice instruction. The local device 400a is a device in the same LAN (second network N2) as the dialogue device 100. The remote device 400b is a device outside the LAN (second network N2) to which the dialogue device is connected. The local device 400a and the remote device 400b are collectively referred to as "device 400" when there is no particular need to distinguish them.

<2. System overview>
With reference to FIGS. 2 and 3, the outline of the dialogue system 1 will be described by dividing it into two situations: (A) when the dialogue device 100 is online, and (B) when the dialogue device 100 is switched from online to offline. do.

First, the scene (A) will be described with reference to FIG. The dialogue device 100 is in an online state, and is in a state in which communication with the server device 200 and the voice recognition system 300 is possible.

(1) As shown in FIG. 2, the voice acquisition unit 120 of the dialogue device 100 acquires the user's voice "start minutes". (2) Since the voice data of the voice acquired by the voice acquisition unit 120 of the dialogue device 100 is transmitted to the voice recognition system 300, the determination unit 111 determines whether or not communication with the voice recognition system 300 is possible.

When communication with the voice recognition system 300 is possible by the determination of the determination unit 111, the communication unit 130 of the dialogue device 100 transmits the voice data to the voice recognition system 300. (4) The communication unit 130 of the dialogue device 100 receives the first recognition information from the voice recognition system 300.

The "first recognition information" is information indicating the recognition result of the voice data by the voice recognition system 300. The first recognition information may be, for example, a text representation of the content of the voice (“start of minutes”). Since the first recognition information and the second recognition information are both information indicating the result of recognizing the user's voice, they are collectively referred to as "recognition information" unless there is a particular need to distinguish them.

(5) Since the first recognition information received by the communication unit 130 is transmitted to the server device 200, the determination unit 111 of the dialogue device 100 determines whether communication with the server device 200 is possible.

(6) When communication with the server device 200 is possible by the determination of the determination unit 111, the communication unit 130 of the dialogue device 100 transmits the first recognition information to the server device 200. (7) The communication unit 230 of the server device 200 receives the first recognition information from the dialogue device 100.

(8) The response generation unit 213 of the server device 200 generates the second response information based on the first recognition information received by the communication unit 230. Here, the "second response information" is information generated by the server device 200 for responding to the user's voice. The second response information may be, for example, a text representation of the content of the voice output from the dialogue device 100, "starting the minutes", or voice data for outputting this content. .. Since the first response information and the second response information are both information indicating the content of the response to the user's voice, they are collectively referred to as "response information" unless there is a particular need to distinguish them.

(9) The communication unit 230 of the server device 200 transmits the second response information to the dialogue device 100. (10) The communication unit 130 of the dialogue device 100 receives the second response information from the server device 200.

(11) The output unit 140 of the dialogue device 100 outputs a response to the voice based on the second response information received by the communication unit 130. Specifically, the output unit 140 outputs a voice saying "Start the minutes".

(12) The device control unit 215 of the server device 200 controls the operation of the own device based on the second response information. The device control unit 215 starts recording the contents of the voice indicated in the first recognition information as minutes based on the second response information. The device control unit 215 adjusts the content of this voice to match the format of the minutes and stores it in the storage unit 250.

Next, the scene of (B) above will be described with reference to FIG.

(1) As shown in FIG. 3, the voice acquisition unit 120 of the dialogue device 100 acquires the user's voice "start minutes". (2) Since the voice data of the voice acquired by the voice acquisition unit 120 of the dialogue device 100 is transmitted to the voice recognition system 300, the determination unit 111 determines whether or not communication with the voice recognition system 300 is possible.

(3) When communication with the voice recognition system 300 is impossible due to the determination of the determination unit 111, the voice recognition unit 112 of the dialogue device 100 recognizes the acquired voice and generates the second recognition information. Further, at this time, the voice recognition unit 112 may take over the voice data in the middle of transmission by referring to the first session information regarding the session at the time of voice data transmission between the communication unit 130 and the voice recognition system 300. Details of the first session information will be described later.

The "second recognition information" is information indicating the recognition result of the voice data by the voice recognition unit 112 of the dialogue device 100. The second recognition information may represent, for example, the content of the voice (“start of minutes”) in text.

(4) The response generation unit 113 of the dialogue device 100 generates the first response information based on the second recognition information. Here, the "first response information" is information for responding to the user's voice by the response generation unit 113. The first response information may, for example, represent the content of the voice output from the dialogue device 100 "start the minutes" in text, or may be a file of voice data for outputting this content. .. Further, at this time, the response generation unit 113 may take over the recognition information in the middle of transmission by referring to the second session information regarding the session at the time of transmitting the recognition information between the communication unit 130 and the server device 200. Details of the second session information will be described later.

(5) The output unit 140 of the dialogue device 100 outputs a response to the voice based on the first response information. Specifically, the output unit 140 outputs a voice saying "Start the minutes".

(6) The identification unit 114 of the dialogue device 100 identifies that the user's voice is a local instruction based on the second recognition information. In the case of a local instruction to the own device, the device control unit 116 of the dialogue device 100 controls the operation of the own device based on the first response information. The device control unit 116 starts recording the contents of the voice of the second recognition information as minutes based on the first response information. The device control unit 116 adjusts the second recognition information to match the predetermined minutes format and stores it in the storage unit 150 as the minutes.

The “local instruction” refers to an instruction to a device in a predetermined network to which the dialogue device 100 is connected in the user's voice. Here, the "predetermined network" is, for example, the second network N2. Further, this device includes a local 400a and a own device (dialogue device 100).

When communication between the dialogue device 100 and the server device 200 becomes possible, the stored minutes are added to the minutes stored in the storage unit 250 of the server device 200 until communication becomes impossible. You may send it.

According to the above configuration, the dialogue device 100 can recognize the user's voice by the voice recognition unit 112 in the dialogue device 100 even when communication with the voice recognition system is impossible. According to the above configuration, the dialogue device 100 can respond to the user's voice based on the result of voice recognition in the own device even when communication with the server device 200 is impossible. Therefore, according to the above configuration, the user can use the dialogue device 100 as usual even in an offline environment.

<3. Functional configuration>
The functional configuration of the dialogue device 100 according to the present embodiment will be described with reference to FIG. As shown in FIG. 4, the dialogue device 100 includes a communication unit 130, a control unit 110, a voice acquisition unit 120, an output unit 140, and a storage unit 150.

The control unit 110 includes a determination unit 111, a voice recognition unit 112, a response generation unit 113, and an identification unit 114. Further, the control unit 110 may include, for example, a specific unit 115 or a device control unit 116.

The determination unit 111 determines whether or not communication with the voice recognition system 300 is possible. The determination unit 111 may, for example, cyclically or event-drivenly monitor the network connection status (offline or online) and determine whether or not communication with the voice recognition system 300 is possible based on the monitoring result. ..

For example, the determination unit 111 may transmit a communication connection request to the server device 200 via the first network N1 by the communication unit 130, and may determine that communication is possible when the response of the communication connection request is received. On the other hand, the determination unit 111 transmits a communication connection request to the server device 200 via the first network N1 by the communication unit 130, for example, and communication is not performed when the response of the communication connection request is not received for a certain period of time. It may be determined that it is possible.

For example, the determination unit 111 inquires the relay / transfer device (not shown) of the first network N1 about the connection status of the server device 200 to the first network N1, and whether communication is possible or impossible by the response to this inquiry. You may judge.

The above-mentioned determination mode of the determination unit 111 can be applied not only to the communication with the voice recognition system 300 but also to the determination for the communication with the server device 200, the local device 400a, or the remote device 400b.

The determination unit 111 may determine, for example, whether or not communication with the server device 200 is possible.

For example, when the user's voice is a remote instruction, the determination unit 111 may determine whether communication with the remote device 400b is possible. For example, when the user's voice is a local instruction, the determination unit 111 may determine whether communication with the local device 400a is possible.

The voice recognition unit 112 recognizes the user's voice acquired by the voice acquisition unit 120. The voice recognition unit 112 generates second recognition information indicating the result of this recognition. The voice recognition unit 112 converts, for example, the voice data acquired by the voice acquisition unit 120 into text information by voice recognition technology. This converted text information corresponds to the second recognition information.

When the voice recognition unit 112 becomes unable to communicate with the voice recognition system 300 while the transmission unit 131 is transmitting voice data to the voice recognition system 300, the voice recognition unit 112 refers to the first session information and does not transmit the voice data. The second recognition information may be generated based on the voice of the voice data of.

The "first session information" is information about a session established when the transmission unit 131 transmits voice data to the voice recognition system 300. The first session information may indicate, for example, how far each packet or each segment of the transmitted voice data has been transmitted. The first session information may indicate, for example, the sequence number or ACK number of the TCP header of the last segment of all the segments of the voice data that has been transmitted, or the sequence number of the TCP header of the first untransmitted segment.

According to the above configuration, the voice recognition unit 112 can take over the voice data being transmitted to the voice recognition system 300 and generate the second recognition information. Therefore, according to the above configuration, the voice recognition unit 112 can smoothly and accurately recognize the user's voice even if the communication with the voice recognition system 300 is interrupted.

The response generation unit 113 generates the first response information based on the first recognition information or the second recognition information. The response generation unit 113 may analyze the recognition information by using, for example, natural language processing. The response generation unit 113 identifies the content of the response to the user's voice by this analysis. Specifically, as shown in FIGS. 2 to 3, the response generation unit 113 morphologically analyzes the content of the user's voice “start minutes” and extracts the words “minutes” and “start”. Next, the response generation unit 113 searches the dictionary information using these extracted words as a search key and identifies the content of the corresponding response. The contents of this response are (a) execution of processing to start recording recognition information in the storage unit 150 or storage unit 250 as minutes, and (b) a voice saying "start minutes" to the user. Execute the output process.

"Dictionary information" is information that associates a word or a combination of a plurality of words with the content of a response and a flag indicating whether it is a local instruction or a remote instruction. The dictionary information associates, for example, a combination of words "minutes" and "start" with the content of the response to be executed in the processes (a) and (b) above, and a flag indicating a remote instruction. It should be noted that this flag may be further divided according to whether or not the remote instruction can be replaced locally. That is, the flag information is one of three types of flags (for example, "1" to "3") of "local instruction", "remote instruction (local substitution possible)", and "remote instruction (local substitution not possible)". May be indicated.

When the transmission unit 131 cancels the transmission of the remote instruction, the response generation unit 113 may generate the first response information indicating that the remote instruction is canceled as a response to the voice of the remote instruction.

According to the above configuration, the response generation unit 113 can respond to the user that the remote instruction to the remote device 400b has been canceled. Therefore, according to the above configuration, the response generation unit 113 can make the user know that the remote instruction has been canceled.

The response generation unit 113 refers to the second session information when communication with the server device 200 becomes impossible while the transmission unit 131 is transmitting the first recognition information or the second recognition information to the server device 200. Then, the first response information may be generated based on the untransmitted first recognition information or the second recognition information.

The "second session information" is information about a session established when the transmission unit 131 transmits the first recognition information or the second recognition information to the server device 200. The second session information may indicate, for example, how far each packet or each segment of the transmitted recognition information has been transmitted. The second session information may indicate, for example, the sequence number or ACK number of the TCP header of the last segment of the recognition information that has been transmitted, or the sequence number of the TCP header of the first untransmitted segment.

According to the above configuration, the response generation unit 113 can generate the first response information by taking over the recognition information during transmission to the server device 200. Therefore, according to the above configuration, the response generation unit 113 can respond smoothly and accurately to the user's voice even if the communication with the server device 200 is interrupted.

Based on the first recognition information or the second recognition information, the identification unit 114 identifies a remote instruction to the remote device 400b outside the predetermined network to which the information processing device is connected in the user's voice.

Based on the first recognition information or the second recognition information, the identification unit 114 identifies the local device 400a in the predetermined network to which the information processing device is connected or the local instruction to the own device among the user's voices.

The identification unit 114 may extract words by performing analysis using natural language processing in the same manner as the response generation unit 113, for example. Based on the extracted word, the identification unit 114 may identify whether it is a remote instruction or a local instruction by searching / specifying dictionary information in the same manner as the response generation unit 113.

The identification unit 115 specifies the execution timing of the instruction in the remote instruction. The identification unit 115 specifies the execution timing of the remote instruction based on, for example, the time included in the remote instruction or the information representing the period until execution. The "information representing the time or the period until execution" is, for example, information indicating "7:00 am", "18:00", "5 minutes later", or the like.

The device control unit 116 controls the operation of the own device based on the first response information. Based on the first response information, when the response to the user's voice instruction is the start of the minutes, the device control unit 116 records the contents of the voice of the second recognition information in the storage unit 150 as the minutes.

The communication unit 130 transmits and receives various information and data to and from the server device 200, the voice recognition system 300, the device 400, and the like via the network N. The communication unit 130 includes a transmission unit 131 and a reception unit 132.

When communication with the voice recognition system 300 is possible, the transmission unit 131 transmits the voice data of the voice acquired by the voice acquisition unit 120 to the voice recognition system 300.

The transmission unit 131 may transmit the first recognition information or the second recognition information to the server device 200, for example, when communication with the server device 200 is possible.

The transmission unit 131 may transmit a remote instruction to the remote device 400b, for example, when communication with the remote device 400b is possible by the determination of the determination unit 111. On the other hand, the transmission unit 131 queues the remote instruction when communication with the remote device 400b is impossible, and then reads the queued remote instruction when communication with the remote device 400b becomes possible. It may be transmitted to the remote device 400b.

According to the above configuration, even if communication with the remote device 400b is temporarily impossible, the transmission unit 131 can transmit a remote instruction to the remote device 400b when communication becomes possible thereafter.

The transmission unit 131 may transmit a local instruction to the local device 400a, for example, when communication with the local device 400a is possible by the determination of the determination unit 111. On the other hand, the transmission unit 131 queues the local instruction when communication with the local device 400a is impossible, and then reads the queued local instruction when communication with the local device 400a becomes possible. It may be transmitted to the local device 400a.

For example, when the transmission unit 131 cannot communicate with the remote device 400b at the execution timing of the remote instruction specified by the specific unit 115, the transmission unit 131 may cancel the transmission of the remote instruction to the remote device 400b. For example, when the execution timing is "5 minutes after (from the time of voice acquisition)" and communication with the remote device 400b is impossible for more than 5 minutes from the time of acquisition, the transmission unit 131 sends to the remote device 400b. Stop sending remote instructions.

According to the above configuration, the transmission unit 131 can prevent the remote device 400b from transmitting the remote instruction that is out of time for execution. Therefore, according to the above configuration, the transmission unit 131 can suppress the transmission of unnecessary and redundant instructions to the remote device 400b.

For example, the transmission unit 131 refers to a storage unit that stores a batch processing time zone, and when the execution timing of the remote instruction is not a specific date and time and is not immediate, the batch is performed when communication with the remote device becomes possible thereafter. The remote instructions queued during the processing time may be read and collectively or sequentially transmitted to the remote device. This "batch processing time zone" is a time zone for performing communication processing with a relatively high load and communication processing with a low degree of urgency. As the batch processing time zone, for example, a time zone such as nighttime or holiday when online real-time processing is small may be set. Further, the "storage unit" referred to here may be a storage unit 150 of the own device or a storage unit of another device.

According to the above configuration, the transmission unit 131 can transmit to the remote device 400b at a specific date and time and a remote instruction that is not immediate during the batch processing time zone. Therefore, according to the above configuration, the transmission unit 131 does not transmit the entire amount of the queued remote instructions when the communication is restored, and limits the instructions to some of the instructions having high priority (importance / urgency). It can be sent and the rest can be passed during subsequent batch processing times.

The transmission unit 131 refers to, for example, a storage unit that stores the batch processing time zone, and when the voice data accumulated for a predetermined period can be communicated with the remote device thereafter, the server device in the batch processing time zone. It may be transmitted to 200.

According to the above configuration, the transmission unit 131 does not affect other priority processing or the response to the user when, for example, a large amount of minutes voice is acquired offline and transcribed in a batch on the server 200. Can be processed.

Here, with reference to FIG. 5, an example of the relationship between the local instruction to the own device / remote instruction to the remote device 400b and its execution / queuing will be described. As shown in FIG. 5, when the execution timing of the local instruction to the own device is immediate, the dialogue device 100 may immediately execute the local instruction without queuing. Further, when the execution timing of the local instruction is specified at a specific date and time or is not specified at all, the dialogue device 100 may queue the local instruction and read it from the specified date and time or sequential queuing and execute it.

When the execution timing of the remote instruction is immediate, the dialogue device 100 may transmit the remote instruction to the immediate remote device 400b without queuing. Further, when the execution timing of the remote instruction is specified at a specific date and time or no specification is made, the dialogue device 100 queues the remote instruction, reads it from the specified date and time or sequentially queuing, and transmits it to the remote device 400b. May be good.

The explanation will be continued by returning to FIG. The receiving unit 132 receives the first recognition information indicating the recognition result of the voice data transmitted by the transmitting unit 131 from the voice recognition system 300.

The receiving unit 132 may receive, for example, the second response information generated based on the first recognition information or the second recognition information from the server device 200.

The voice acquisition unit 120 acquires the user's voice.

The output unit 140 outputs a response to the voice based on the first response information or the second response information. The output mode of the output unit 140 may be any mode. The output mode of the output unit 140 may be, for example, voice output, screen output, file output, message output, or the like.

According to the above configuration, the output unit 140 can respond to the user's voice even when communication with the server device 200 or the voice recognition system 300 is impossible. Therefore, according to the above configuration, the user can use the dialogue device 100 as usual even in an offline environment. Further, according to the above configuration, the output unit 140 can use not only the first response information generated by the own device but also the second response information generated by the server device 200.

The storage unit 150 has voice data, first recognition information, second recognition information, first response information, second response information, first session information, second session information, minutes information or setting indicating recorded minutes. Memorize information etc. Here, the "setting information" is information indicating parameters set for operating the dialogue device 100. The setting information may include a batch processing time zone.

The storage unit 150 may use a database management system (DBMS) to store the above information, or may use a file system to store the above information. When using the DBMS, a table may be provided for each of the above information, and the tables may be associated with each other to manage the information.

The functional configuration of the server device 200 according to the present embodiment will be described with reference to FIG. As shown in FIG. 6, the server device 200 includes a control unit 210, a communication unit 230, and a storage unit 250. Since the functions of the communication unit 230 and the storage unit 250 are the same as those of the communication unit 130 and the storage unit 150 of the dialogue device 100, the description thereof will be omitted.

The control unit 210 includes a determination unit 211, a voice recognition unit 212, and a response generation unit 213. Further, the control unit 210 may include, for example, a device control unit 215. Since the functions of the functional units are the same as those of the determination unit 111 of the dialogue device 100, the voice recognition unit 112, the response generation unit 113, and the device control unit 116, the description thereof will be omitted.

<4. Operation example>
An operation example of the dialogue device 100 will be described with reference to FIG. 7. The order of processing of the operation example of FIG. 7 shown below is an example, and may be changed as appropriate.

As shown in FIG. 7, the voice acquisition unit 120 of the dialogue device 100 acquires the user's voice (S10). Next, the determination unit 111 determines whether or not communication with the voice recognition system 300 is possible (S11).

When communication with the voice recognition system 300 is possible by the determination of the determination unit 111 (Yes in S12), the voice data of the voice acquired by the voice acquisition unit 120 is transmitted to the voice recognition system (S13). The first recognition information indicating the recognition result of the voice data is received from the voice recognition system 300 (S14).

When communication with the voice recognition system 300 is impossible due to the determination of the determination unit 111 (No in S12), the voice recognition unit 112 recognizes the voice acquired by the voice acquisition unit 120 and shows the recognition result. Generate recognition information (S15).

The response generation unit 113 generates the first response information based on the first recognition information or the second recognition information (S16). The output unit 140 outputs a response to the user's voice based on the first response information (S17).

<5. Hardware configuration>
With reference to FIG. 8, an example of the hardware configuration in the case where the above-described dialogue device 100 and server device 200 are realized by the computer 800 will be described. The function of each device can be realized by dividing it into a plurality of devices.

As shown in FIG. 8, the computer 800 includes a processor 801 and a memory 803, a storage device 805, an input I / F unit 807, a data I / F unit 809, a communication I / F unit 811 and a display device 813. , Includes audio input device 817 and audio output device 819.

The processor 801 controls various processes in the computer 800 by executing a program stored in the memory 803. For example, each functional unit included in the control unit 110 of the dialogue device 100 and the control unit 210 of the server device 200 can be realized by the processor 801 executing a program temporarily stored in the memory 803.

The memory 803 is, for example, a storage medium such as a RAM (Random Access Memory). The memory 803 temporarily stores the program code of the program executed by the processor 801 and the data required when the program is executed.

The storage device 805 is a non-volatile storage medium such as a hard disk drive (HDD) or a flash memory. The storage device 805 stores an operating system and various programs for realizing each of the above configurations. In addition, the storage device 805 registers voice data, first recognition information, second recognition information, first response information, second response information, first session information, second session information, minutes information, setting information, and the like. It is also possible to store the table to be used and the DB that manages this table. Such programs and data are referred to by the processor 801 by being loaded into the memory 803 as needed.

The input I / F unit 807 is a device for receiving input from the user. Specific examples of the input I / F unit 807 include a keyboard, a mouse, a touch panel, various sensors, a wearable device, and the like. The input I / F unit 807 may be connected to the computer 800 via an interface such as USB (Universal Serial Bus).

The data I / F unit 809 is a device for inputting data from the outside of the computer 800. Specific examples of the data I / F unit 809 include a drive device for reading data stored in various storage media. It is also conceivable that the data I / F unit 809 is provided outside the computer 800. In that case, the data I / F unit 809 is connected to the computer 800 via an interface such as USB.

The communication I / F unit 811 is a device for performing data communication via the Internet N by wire or wirelessly with an external device of the computer 800. It is also conceivable that the communication I / F unit 811 is provided outside the computer 800. In that case, the communication I / F unit 811 is connected to the computer 800 via an interface such as USB.

The display device 813 is a device for displaying various information. Specific examples of the display device 813 include a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display device 813 may be provided outside the computer 800. In that case, the display device 813 is connected to the computer 800 via, for example, a display cable or the like. Further, when the touch panel is adopted as the input I / F unit 807, the display device 813 can be integrally configured with the input I / F unit 807.

The voice input device 817 is an input device for detecting voice such as a microphone. The voice input device includes, for example, a microphone that acquires an analog vibration signal including a voice signal, an amplifier that amplifies the analog vibration signal, and an A / D conversion unit that converts the analog vibration signal into a digital signal. The voice input device 817 detects, for example, the voice emitted by the user.

The audio output device 819 is an output device for outputting audio, and is, for example, a speaker or the like. Further, the audio output device 819 may be a device for outputting sound to headphones or earphones.

[Second Embodiment]
Next, a second embodiment of the present invention (hereinafter, referred to as "the present embodiment") will be described. In the present embodiment, the dialogue system 1a according to the present embodiment will be described with reference to (4) an example in which a voice conversation between a user and a remote partner (hereinafter referred to as “conversation partner”) is realized. It is not limited to this. Hereinafter, the points different from those of the first embodiment will be mainly described.

<1. System configuration and overview>
The dialogue device 100a according to the present embodiment is connected to the partner device via the first network N1.

The outline of the dialogue system 1a will be described with reference to FIGS. 9 to 10.

As shown in FIGS. 9 to 10, when the first network N1 is capable of communicating with the other device (hereinafter, also simply referred to as “online”), the dialogue device 100a transmits voice data to the other device in real time. .. That is, the user and the conversation partner can talk in real time via the dialogue device 100a and the other party device. At this time, the dialogue device 100a does not record the user's utterance data. Here, the "utterance data" is data obtained by removing at least a part of the silent section from the voice data of the user's voice.

Next, when it becomes impossible to communicate with the other device from online in the first network N1 (hereinafter, also simply referred to as “offline”), the dialogue device 100 records the user's utterance data. At this time, the other device naturally does not output the user's voice.

Next, when the first network N1 goes from offline to online again, the dialogue device 100a transmits the utterance data to the other device. At this time, the dialogue device 100a continues to record the user's utterance data. The dialogue device 100a stops recording the utterance data when the difference between the utterance timing of the user and the reproduction timing of the utterance data of the other device is resolved. The dialogue device 100a resumes the transmission of voice data to the other device after the transmission of the utterance data is completed. That is, the user and the conversation partner can talk again in real time via the dialogue device 100a and the other party device.

According to the above configuration, the dialogue system 1a can transmit the recorded voice data excluding all or a part of the silent section to the other device when going from online to offline. Therefore, according to the above configuration, by transmitting the voice data to the other party device by removing the silent section, it is possible to eliminate the difference between the user's utterance timing and the reproduction timing of the other party device due to being offline. Therefore, according to the above configuration, even if communication with the other party device becomes impossible during a remote conversation between the user and the conversation partner, the dialogue system 1a allows the user to talk to the conversation partner after the communication becomes possible. You can smoothly convey the spoken information.

<2. Functional configuration>
An example of the functional configuration of the dialogue device 100a will be described. The dialogue device 100a commonly includes a voice acquisition unit 120, a communication unit 130, an output unit 140, and a storage unit 150 of the dialogue device 100 according to the first embodiment, and the control unit 110 shares a determination unit 111. In addition to these functional units, a detection unit is provided.

The determination unit 111 determines whether or not communication with the other device is possible.

The detection unit detects the utterance section and the silent section from the voice data acquired by the voice acquisition unit 120. For example, the detection unit may record this voice data once and detect the utterance section and the silence section from the recorded voice data. This "silence section" is a section in which the voice level becomes zero in the voice data. Further, the detection unit may perform various acoustic processes such as an equalizer process and a time alignment process on the voice data as a pre-process for detection.

The recording unit records the utterance data when communication with the other device is impossible. For example, the recording unit may extract the data of the utterance section from the recorded voice data to generate the utterance data. The recording unit records the generated utterance data in the storage unit 150.

The recording unit may record the utterance data until, for example, at least a part of the difference between the utterance timing of the user and the reproduction timing of the user's utterance of the other device is eliminated.

In the recording unit, for example, from the start of the period during which communication with the other device becomes impossible (hereinafter referred to as "offline period") until the total length of the silent section exceeds the length of the offline period (for example,). The utterance data may be recorded (up to the time point P in FIG. 9). Further, as another example, the recording unit uses the utterance data until the total length of the silent section from the start of the offline period exceeds the total length of the utterance section in the offline period (for example, up to the time point P'in FIG. 10). May be recorded.

The transmission unit 131 transmits the voice data acquired by the voice acquisition unit 120 to the other device. The transmission unit 131 transmits the utterance data to the other device when communication with the other device becomes possible, and resumes the transmission of the voice data after the transmission of the utterance data is completed.

According to the above configuration, the dialogue system 1a can transmit the recorded voice data excluding all or a part of the silent section to the other device when going from online to offline. Therefore, according to the above configuration, by transmitting the voice data to the other device by removing the silent section, it is possible to eliminate the difference between the user's utterance timing and the reproduction timing of the other device due to being offline.

<3. Operation example>
An operation example of the dialogue device 100a will be described with reference to FIG. The order of processing of the operation example of FIG. 11 shown below is an example, and may be changed as appropriate.

As shown in FIG. 11, the voice acquisition unit 120 of the dialogue device 100 acquires the user's voice (S20). Next, the determination unit 111 determines whether or not communication with the other device is possible (S21).

When communication with the other device is impossible due to the determination of the determination unit 111 (No in S22), the recording unit records the utterance data (S23).

When communication with the other device is possible by the determination of the determination unit 111 (Yes in S22) and the transmission of the utterance data to the other device is not completed (No in S24), the transmission unit 131 transmits the utterance data. (S25). Further, when at least a part of the difference between the utterance timing of the user and the reproduction timing of the utterance of the user of the other device is not resolved (No in S26), the recording unit records the utterance data (S27).

When communication with the other device is possible by the determination of the determination unit 111 (Yes in S22) and when the transmission of the utterance data to the other device is completed (Yes in S24), the transmission unit 131 transmits the voice data (Yes). S28).

When the user and the conversation partner continue the conversation (Yes in S29), the process returns to the step S20.

It should be noted that the present embodiment is an example for explaining the present invention, and the present invention is not intended to be limited only to the embodiment. Further, the present invention can be modified in various ways as long as it does not deviate from the gist thereof. Further, those skilled in the art can adopt an embodiment in which each element described below is replaced with an equal one, and such an embodiment is also included in the scope of the present invention.

1, 1a ... Dialogue system, 100, 100a ... Dialogue device, 110 ... Control unit, 111 ... Judgment unit, 112 ... Voice recognition unit, 113 ... Response generation unit, 114 ... Identification unit, 115 ... Specific unit, 116 ... Device control Unit, 120 ... Voice acquisition unit, 130 ... Communication unit, 131 ... Transmission unit, 132 ... Reception unit, 140 ... Output unit, 150 ... Storage unit, 200 ... Server device, 210 ... Control unit, 211 ... Judgment unit, 212 ... Voice recognition unit, 213 ... Response generation unit, 214 ... Identification unit, 215 ... Device control unit, 230 ... Communication unit, 250 ... Storage unit, 300 ... Voice recognition system, 400 ... Device, 400a ... Local device, 400b ... Remote device , 800 ... Computer, 801 ... Processor, 803 ... Memory, 805 ... Storage device, 807 ... Input I / F section, 809 ... Data I / F section, 811 ... Communication I / F section, 813 ... Display device, 817 ... Voice Input device, 819 ... Audio output device.

Claims (10)

An information processing device that connects to a voice recognition system that recognizes voice via a network.
A voice acquisition unit that acquires the user's voice,
A determination unit that determines whether communication with the voice recognition system is possible, and
When communication with the voice recognition system is possible, a transmission unit that transmits the acquired voice data of the voice to the voice recognition system, and a transmission unit.
A receiving unit that receives the first recognition information indicating the recognition result of the voice data from the voice recognition system, and
When communication with the voice recognition system is impossible, a voice recognition unit that recognizes the acquired voice and generates second recognition information indicating the recognition result, and a voice recognition unit.
A response generation unit that generates first response information for responding to the voice based on the first recognition information or the second recognition information.
An output unit that outputs a response to the voice based on the first response information is provided.
Information processing device. The information processing device is connected to the server device that generates the second response information for responding to the voice based on the first recognition information or the second recognition information via the network.
The determination unit determines whether or not communication with the server device is possible, and determines whether or not communication with the server device is possible.
When communication with the server device is possible, the transmission unit transmits the first recognition information or the second recognition information to the server device.
The receiving unit receives the first recognition information or the second response information generated based on the second recognition information from the server device.
The output unit outputs a response to the voice based on the received second response information.
The information processing device according to claim 1. Based on the first recognition information or the second recognition information, the user's voice is further provided with an identification unit that identifies a remote instruction to a remote device outside a predetermined network to which the information processing device is connected.
When the voice is the remote instruction, the determination unit determines whether communication with the remote device is possible.
The transmitter
When communication with the remote device is possible, the remote instruction is transmitted to the remote device.
When communication with the remote device is not possible, the remote instruction is queued, and then when communication with the remote device becomes possible, the queued remote instruction is read and transmitted to the remote device.
The information processing device according to claim 1 or 2. Further provided with a specific unit for specifying the execution timing of the remote instruction,
When communication with the remote device is not possible at the specified execution timing, the transmitter cancels transmission of the remote instruction to the remote device.
The response generation unit generates the first response information indicating that the remote instruction has been canceled as a response to the voice of the remote instruction.
The information processing device according to claim 3. The transmitting unit refers to a storage unit that stores the batch processing time zone, and when the execution timing of the instruction is not a specific date and time and immediately, and then communication with the remote device becomes possible, the batch processing is performed. Read the remote instruction queued in the time zone and send it to the remote device.
The information processing device according to claim 4. When the transmission unit becomes unable to communicate with the voice recognition system while the transmission unit is transmitting the voice data to the voice recognition system, the voice recognition unit causes the transmission unit to send the voice recognition system to the voice recognition system. The second recognition information is generated based on the voice of the voice data that has not been transmitted by referring to the first session information regarding the session established when the voice data is transmitted.
The information processing device according to any one of claims 1 to 5. When the transmission unit becomes unable to communicate with the server device while the transmission unit is transmitting the first recognition information or the second recognition information to the server device, the response generation unit causes the transmission unit to perform. Based on the untransmitted first recognition information or the second recognition information with reference to the second session information regarding the session established when the first recognition information or the second recognition information is transmitted to the server device. Generate the first response information,
The information processing device according to claim 2. The information processing device is connected to the other device of the other party by voice conversation with the user via the network.
The determination unit determines whether or not communication with the other device is possible, and determines whether or not communication with the other device is possible.
The transmission unit transmits the acquired voice data to the other device, and then transmits the acquired voice data to the other device.
Information processing equipment
A detection unit that detects an utterance section and a silent section from the voice data,
When communication with the other device is impossible, a recording unit that records utterance data excluding at least a part of the silent section from the voice data, and a recording unit.
The transmission unit transmits the utterance data to the other device when communication with the other device becomes possible, and resumes the transmission of the voice data after the transmission of the utterance data is completed.
The information processing device according to any one of claims 1 to 7. For information processing devices that connect to a voice recognition system that recognizes voice via a network
With the voice acquisition function to acquire the user's voice,
A determination function for determining whether or not communication with the voice recognition system is possible, and
When communication with the voice recognition system is possible, a transmission function for transmitting the voice data of the acquired voice to the voice recognition system, and
A receiving function for receiving the first recognition information indicating the recognition result of the voice data from the voice recognition system, and
When communication with the voice recognition system is impossible, a voice recognition function that recognizes the acquired voice and generates a second recognition information indicating the recognition result, and a voice recognition function.
A response generation function that generates first response information for responding to the voice based on the first recognition information or the second recognition information.
An output function that outputs a response to the voice based on the first response information is realized.
program. An information processing device that connects to a voice recognition system that recognizes voice via a network
Get the user's voice and
It is determined whether communication with the voice recognition system is possible, and
When communication with the voice recognition system is possible, the voice data of the acquired voice is transmitted to the voice recognition system.
The first recognition information indicating the recognition result of the voice data is received from the voice recognition system, and the first recognition information is received.
When communication with the voice recognition system is impossible, the acquired voice is recognized and a second recognition information indicating the recognition result is generated.
Based on the first recognition information or the second recognition information, the first response information for responding to the voice is generated.
Outputs a response to the voice based on the first response information.
Information processing method.

Images