JP2020021040A - Information processing unit, sound output method, and sound output program - Google Patents

Abstract

To provide a sound output device that executes control in response to a sound instruction made by a user.SOLUTION: The sound output device includes an acceptance unit that accepts input of sound information indicating speech sound made by a user, a generation unit that generates robot speech data corresponding to a response to the speech sound on the basis of the sound information, and an output unit that outputs robot speech data. In a case where second sound information is accepted within a prescribed time after first sound information is accepted, and when a second word in the same category as a first word included in the first sound information is included in the second sound information, the generation unit generates the robot speech data at least on the basis of the second sound information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing device that performs control based on a voice instruction from a user, a voice output method, and a voice output program.

  Conventionally, the development of devices utilizing artificial intelligence has been remarkable. Among them, there are devices that execute the instructed content in accordance with a voice instruction from a user. For example, Patent Literature 1 discloses information processing for playing music, executing an alarm, calculating, and controlling another device (for example, a lighting device) in accordance with a voice instruction from a user. A device (smart speaker) is disclosed.

JP 2017-032895 A

  By the way, when giving an instruction, the user sometimes makes a mistake by saying a wrong word or changing his / her thought. However, in the case of the conventional smart speaker, there is a problem that since the user responds to the content spoken first by the user, it does not respond to restatement. Further, in such a case, there is a problem that the user is troublesome to give an instruction again, and cannot give a next instruction without waiting for the end of the utterance of a response to the first instruction.

  Therefore, the present invention has been made in view of the above problems, and provides an audio output device, an audio output method, and an audio output program that can appropriately respond even when a user rephrases. The purpose is to do.

  In order to solve the above-described problem, an information processing apparatus according to one aspect of the present invention includes a receiving unit that receives input of voice information indicating a user's uttered voice, and a robot corresponding to a reply to the uttered voice based on the voice information. A generation unit that generates utterance data; and an output unit that outputs robot utterance data, wherein the generation unit receives the first voice information and then receives the second voice information within a predetermined time. When the second voice information includes a second word of the same category as the first word included in the first voice information, at least the robot based on the second voice information Generate utterance data.

  In order to solve the above-mentioned problem, a voice output method according to one aspect of the present invention includes a receiving step of receiving an input of voice information indicating a speech voice by a user, and a robot corresponding to a reply to the speech voice based on the voice information. A generating step of generating utterance data; and an output step of outputting robot utterance data, wherein the generating step includes a step of receiving the first voice information and then receiving the second voice information within a predetermined time. When the second voice information includes a second word of the same category as the first word included in the first voice information, at least the robot based on the second voice information Generate utterance data.

  In order to solve the above problem, an audio output program according to one embodiment of the present invention provides a computer with a reception function for receiving input of audio information indicating an uttered voice by a user, and a reply to the uttered voice based on the audio information. A generation function of generating the corresponding robot utterance data and an output function of outputting the robot utterance data are realized, and the generation function converts the second voice information within a predetermined time after receiving the first voice information. In the case where the second voice information is received and the second voice information includes the second word in the same category as the first word included in the first voice information, at least the second voice information Based on this, robot utterance data is generated.

  In the information processing device, when the second voice information includes a word that denies the first voice information, the generation unit may generate the robot utterance data based only on the second voice information.

  In the information processing device, when the second voice information includes a word connected to the first voice information, the generation unit may generate the robot utterance data for both the first voice information and the second voice information. It may be generated.

  In the information processing apparatus, when the second audio information includes a second word of the same type as the first word included in the first audio information, the generation unit determines which one is correct. The robot utterance data to be queried may be generated.

  The information processing apparatus further includes a voice output unit that outputs a voice based on the robot utterance data, and a voice collection unit that collects a user's utterance voice, wherein the output unit outputs the robot utterance data to the voice output unit. The receiving unit receives the uttered voice collected by the voice collecting unit as voice information, and the output unit outputs the second voice within a predetermined time when outputting the robot utterance data based on the first voice information. When the information is received, the robot generated for the first voice information when the second voice information includes a second word of the same type as the first word included in the first voice information The output of the utterance data to the audio output unit may be stopped.

  In the information processing device, the output unit may output new robot utterance data based on the second audio information after stopping outputting the robot utterance data to the audio output unit.

  In the information processing device, the output unit outputs the robot utterance data to an external speaker, the receiving unit receives an input of the uttered voice collected by the external microphone as voice information, and the output unit outputs the external voice. When the speaker is outputting a voice based on the robot utterance data based on the first voice information, the speaker receives second voice information based on the uttered voice received from the user, and the first voice information includes the first voice information. When the second word of the same type as the first word included in the voice information is included, a stop instruction to stop outputting the voice based on the robot utterance data generated for the first voice information is output. It may be that.

  In the information processing device, the output unit may output new robot utterance data based on the second voice information after outputting the stop instruction.

  In the information processing device, the generation unit generates the robot utterance data for making an inquiry to the user, determines a category of a word desired as a response to the inquiry, and the generation unit determines When the first audio information and the second audio information are received, the words belonging to the category determined as the first audio information are included as the first words, and are determined as the second audio information. When a word belonging to the category is included as the second word, the robot utterance data may be generated based on at least the second voice information.

  The information processing device according to one embodiment of the present invention can appropriately respond even when the user makes a restatement.

FIG. 2 is a diagram illustrating a configuration example of a communication system. FIG. 3 is a block diagram illustrating a configuration example of a voice server. It is a block diagram which shows the example of a structure of a smart speaker. It is a data conceptual diagram which shows the example of a structure of a control model. FIG. 9 is a sequence diagram illustrating an example of an exchange between a voice server and a smart speaker when there is no restatement by a user. FIG. 11 is a sequence diagram illustrating an example of an exchange between a voice server and a smart speaker when a restatement by a user occurs. 5 is a flowchart illustrating an operation example of a voice server.

<Embodiment>
An embodiment of the present invention will be described with reference to the drawings.

  The voice output device 100, which can be an information processing device according to the present invention, is a device that outputs an uttered voice as an answer corresponding to the content of the instruction input, based on the instruction input by voice from the user 10. When a voice instruction (inquiry) is received from the user, the voice output device 100 generates and outputs robot utterance data as an answer corresponding to the voice instruction. At this time, the voice output device 100 determines whether the user has rephrased, and appropriately generates robot utterance data indicating an answer that is estimated to be appropriate. The audio output device 100 may be implemented in any manner, may be implemented as a server device or a computer system as shown in FIG. 1, or may be implemented in a speaker such as a smart speaker, a robot, or the like. It may be realized by. The audio output device 100 may be a control device for controlling a smart speaker, a robot, or an AI assistant.

  Hereinafter, such an audio output device 100 will be described.

(System configuration)
As shown in FIG. 1, the communication system 1 includes a smart speaker 200 as a device that receives a voice instruction (inquiry) from the user 10 and a voice output that generates robot utterance data indicating a response to the voice command from the user 10. Device 100.

  The smart speaker 200 has a built-in microphone, and sequentially collects surrounding sounds including a user's uttered voice, and transmits voice data obtained by collecting the voice to the voice output device 100. In addition, the smart speaker 200 outputs a sound based on the robot utterance data transmitted from the sound output device 100.

  The audio output device 100 receives the audio data, extracts an instruction of the user 10 from the received audio data, and generates robot utterance data indicating an answer according to the instruction of the user 10. Then, the generated robot utterance data is transmitted to the smart speaker 200.

  In the example of FIG. 1, after the user 10 makes an inquiry “Tell me the weather in Tokyo?”, And then rephrases “Oh, after all, tell me the weather in Shinagawa?” An example is shown in which the speaker 200 answers "Shinagawa's weather? Today's weather in Shinagawa ...". “Shinagawa” and “Shimbashi” are Japanese place names. As described above, the smart speaker 200 according to the present embodiment can respond in accordance with the restatement of the user 10 in response to the restatement of the user 10 under the instruction from the audio output device 100. it can.

  As shown in FIG. 1, the audio output device 100 is communicably connected to a smart speaker 200 via a network 300. Although not shown, an information processing device from which the audio output device 100 collects information may be communicably connected to the network 300.

  The network 300 is a network for mutually connecting the audio output device 100 and various devices, and is, for example, a wireless network or a wired network. Specifically, the network 300 includes a wireless LAN (wireless LAN: WLAN), a wide area network (WAN), ISDNs (integrated service digital networks), wireless LANs, LTE (long term evolution, and volume evolution). Fourth generation (4G), fifth generation (5G), CDMA (code division multiple access), WCDMA (registered trademark), Ethernet (registered trademark), and the like.

  The network 300 is not limited to these examples. For example, the public switched telephone network (PSTN), Bluetooth (Bluetooth (registered trademark)), Bluetooth Low Energy, optical line AD, optical line (Asymmetric Digital Subscriber Line) A line, a satellite communication network or the like may be used, and any network may be used. The network 300 may be called a home network when provided in the residence of the user 10.

  In addition, the network 300 may be, for example, NB-IoT (Narrow Band IoT) or eMTC (enhanced Machine Type Communication). Note that NB-IoT and eMTC are wireless communication systems for IoT, and are networks capable of long-distance communication with low cost and low power consumption.

  Further, the network 300 may be a combination of these. The network 300 may include a plurality of different networks obtained by combining these examples. For example, the network 300 may include a wireless network based on LTE and a wired network such as an intranet that is a closed network.

(Configuration example of audio output device)
FIG. 2 is a block diagram illustrating a configuration example of the audio output device 100. As shown in FIG. 2, the audio output device 100 includes, for example, a receiving unit 110, a storage unit 120, a control unit 130, and a transmitting unit 140. The voice output device 100 recognizes only the restatement of the content uttered by the user, specifies an answer requested by the user based on the recognition, and generates and outputs robot utterance data indicating the content of the answer. Things. That is, the voice output device 100 analyzes the first voice information based on the utterance of the user and the second voice information based on the subsequent utterance, and sets the second voice information to the same category as the word included in the first voice information. If there is a certain wording, it is recognized as restatement. Here, the same category refers to a type of information that the user wants to know at least in the first audio information and the second audio information. For example, when the user wants to know the weather at a certain place, the same category indicates the place. The information is a category. For example, when the information that the user wants to know is related to the store, and the type of the store (for example, men's clothing, accessories, furniture, etc.) is indicated by voice, the type of the store is the category. However, the category is not limited to these examples.

  The receiving unit 110 is a communication interface that receives audio data from the smart speaker 200 via the network 300. The receiving unit 110 receives voice data indicating an instruction input by voice from a user. When receiving the audio data, the receiving unit 110 transmits the audio data to the control unit 130. Further, the receiving unit 110 receives information transmitted from another information processing device connected to the network 300 (not shown).

  The storage unit 120 has a function of storing various programs and various data required for the operation of the audio output device 100. The storage unit 120 is realized by various storage media such as an HDD, an SSD, and a flash memory. Note that the sound output device 100 may store the program in the storage unit 120, execute the program, and cause the control unit 130 to execute processing as each functional unit included in the control unit 130. This program causes the audio output device 100 to realize the functions executed by the control unit 130.

  The storage unit 120 is used to generate a voice analysis program for performing voice analysis for estimating the instruction content from the user based on the received voice data received, and to generate robot utterance data indicating a robot voice based on the analysis result. Is stored. The storage unit 120 stores answer model information 121 for generating robot utterance data to be generated according to the situation with respect to the input voice. The details of the answer model information 121 will be described later.

  The control unit 130 controls each unit of the audio output device 100, and may be, for example, a central processing unit (CPU), a microprocessor, an ASIC, an FPGA, or the like. The control unit 130 is not limited to these examples, and may be any type.

  The control unit 130 includes a voice analysis unit 131 and a generation unit 132.

  The voice analysis unit 131 has a function of analyzing voice based on the received voice information, and transmits the analysis result to the generation unit 132. Analysis of the voice information may use a conventional voice recognition technology, and converts the input voice into text data and analyzes the context. For the analysis of the context, for example, a conventional morphological analysis can be used. The voice analysis unit 131 transmits the analysis result to the generation unit 132.

  The generation unit 132 has a function of generating robot utterance data based on the transmitted voice result. The generation unit 132 generates robot utterance data indicating an answer according to the analysis result of the received voice information as in the related art. In addition, in addition to the conventional function, when the analysis result is received from the voice analysis unit 131, the generation unit 132 generates the robot utterance data according to whether another analysis result has been received a predetermined time before that. I do. That is, when the analysis result of analyzing the second voice information is received within a predetermined time after receiving the first analysis result obtained by analyzing the first voice information, the flow of the utterance up to that time, A determination is made as to which of the first voice information and the second voice information is to be given priority or which is to be responded to, and then, robot utterance data which is an answer according to the situation is generated. . In addition, when the second voice information is received a predetermined time after the first voice information, the generation unit 132 may (or may not) generate the robot utterance data corresponding to the second voice information. . The generating unit 132 determines what kind of answer is given according to the situation with reference to the answer model information 121 stored in the storage unit 120, and generates robot utterance data. The generating unit 132 transmits the generated robot utterance data to the transmitting unit 140, and instructs the transmitting unit 140 to transmit the data to the smart speaker 200.

  The transmission unit 140 is a communication interface having a function of transmitting robot utterance data for causing the smart speaker 200 to utter the voice to the smart speaker 200 according to an instruction from the control unit 130 (the generation unit 132).

  The above is the configuration example of the audio output device 100.

(Example of smart speaker configuration)
FIG. 3 is a block diagram illustrating a configuration example of the smart speaker 200. As shown in FIG. 3, the smart speaker 200 includes a receiving unit 210, a storage unit 220, a speaker 230, a microphone 240, and a transmitting unit 250.

  The receiving unit 210 is a communication interface that receives a control signal (audio data) from the audio output device 100. Receiving section 210 transmits the received control signal (sound data) to speaker 230.

  The storage unit 220 has a function of storing various programs and data required for the operation of the smart speaker 200. The storage unit 220 is realized by various storage media such as an HDD, an SSD, and a flash memory. Note that the smart speaker 200 may store a program in the storage unit 220 and execute the program, so that a control unit (not shown) realizes a function to be realized as the smart speaker 200. The storage unit 220 stores, for example, audio data collected by the microphone 240.

  The speaker 230 has a function of reproducing a control signal (audio data) transmitted from the audio output device 100 and received.

  The microphone 240 has a function of collecting sound around the smart speaker 200. Microphone 240 may be configured with one microphone, or may be configured with a plurality of microphones. Further, the microphone may be a directional microphone having a limited sound collection direction. The microphone 240 stores audio data indicating the collected audio in the storage unit 220.

  The transmission unit 250 is a communication interface having a function of transmitting the audio data stored in the storage unit 220 to the audio output device 100. The transmission unit 250 may sequentially transmit the audio data stored in the storage unit 220 to the audio output device 100, or, when it is detected that an instruction input by a user has been made, Audio data for a predetermined length may be transmitted.

  The above is the configuration example of the smart speaker 200.

(Example of configuration of answer model information 121)
Next, an example of the answer model information 121 will be described with reference to FIG. FIG. 4 is a data conceptual diagram showing a data configuration example of the answer model information 121.

  As shown in FIG. 4, the answer model information 121 is information in which status information 410 and correspondence information 420 are associated with each other.

  The status information 410 is information indicating a status in which input of voice information is being received and a status indicated from an analysis result of the received voice information.

  The correspondence information 420 is information that defines the basis on which the voice output device 100 generates (or does not generate) the robot voice utterance data according to the corresponding status information 410.

  For example, when the status information 410 indicates that “the second voice information is acquired after a predetermined time after receiving the first voice information”, and that “there is no word corresponding to the question in the second voice information”, As shown in the corresponding correspondence information 420, the output device 100 either “does not generate robot utterance data corresponding to the second voice information” or determines what kind of voice the thrown voice was. Generate ".

  Further, for example, as the status information 410, “acquire the second audio information within a predetermined time after receiving the first audio information”, and “the second audio information includes words of the same category included in the first audio information” "Yes" and "the second voice information includes a word that denies the first voice information", the voice output device 100 responds to "only generate robot utterance data corresponding to the second voice information". .

  As described above, when the state set in the status information 410 is reached, the voice output device 100 generates the robot utterance data by performing the correspondence indicated by the corresponding correspondence information 420 (it may not be generated). ).

(Example of exchange of communication system 1)
FIG. 5 is a sequence diagram showing an exchange between the smart speaker 200 and the audio output device 100 in the case where no restatement by the user has occurred in the communication system 1.

  As shown in FIG. 5, the smart speaker 200 receives an input of a voice (hereinafter, a first voice) from a user (step S501). The smart speaker 200 transmits the first audio information obtained by converting the received audio to digital data, to the audio output device 100 (Step S502).

  Upon receiving the first audio information, the audio output device 100 analyzes the content (step S503). Then, the voice output device 100 generates robot utterance data indicating a response based on the analysis result, that is, the first voice information (step S504). The voice output device 100 transmits the generated robot utterance data to the smart speaker 200 (Step S505).

  The smart speaker 200 that has received the robot utterance data outputs a voice based on the robot utterance data (step S506), and responds to the instruction (inquiry) from the user received in step S501.

  The process shown in FIG. 5 is an operation that can be realized even in a conventional smart speaker.

  On the other hand, FIG. 6 is a sequence diagram showing an exchange between the smart speaker 200 and the audio output device 100 in a case where rephrasing by the user has occurred in the communication system 1. In the sequence diagram shown in FIG. 6, the processes related to steps S501 to S503 are the same as the processes of steps S501 to S503 in the process shown in FIG.

  After transmitting the first voice information, the smart speaker 200 further receives the input of the next voice (hereinafter, the second voice) from the user (step S601). Then, the smart speaker 200 transmits the second audio information obtained by converting the second audio to digital data to the audio output device 100 (step S602).

  Then, the audio output device 100 analyzes the second audio information (Step S603). As a result of this analysis, after receiving the first voice information, receiving the second voice information within a predetermined time (a time appropriate for receiving the rephrase, for example, 5 seconds); It is assumed that words in the same category as the words included in the instruction of the first sound information are included in the second sound information.

  In such a case, the voice output device 100 generates robot utterance data based on at least the second voice information (step S604). Here, to generate based on at least the second voice information means to generate based on only the second voice information, and to generate voice data based on both the first voice information and the second voice information. Including both cases.

  The voice output device 100 transmits the generated robot utterance data to the smart speaker 200 (Step S605).

  Upon receiving the robot utterance data, the smart speaker 200 outputs a voice based on the robot utterance data (step S606).

  As described above, when the user speaks one after another, the voice output device 100 determines whether or not it is a restatement within a predetermined time from the previous utterance, and is common to the utterance content. A determination can be made based on whether a word in the category is included, and a response can be appropriately made.

(Operation Example of Audio Output Device 100)
FIG. 7 is a flowchart illustrating the operation of the audio output device 100 when the device is controlled.

  As illustrated in FIG. 7, the receiving unit 110 of the audio output device 100 receives, from the smart speaker 200, first audio information indicating audio uttered by the user (step S701). The receiving unit 110 transmits the received first audio information to the control unit 130.

  The voice analysis unit 131 of the control unit 130 analyzes the transmitted first voice information (step S702), and specifies what instruction content (content of inquiry). The specification of the instruction can be performed by storing in advance a list of words that can be an inquiry content in the storage unit 120. The voice analysis unit 131 transmits the analysis result to the generation unit 132.

  Then, based on the analysis result, that is, the result of analyzing the first voice information, the generation unit 132 starts generating the robot utterance voice data that is a response to the instruction content (inquiry content) indicated by the first voice information. (Step S703).

  After starting the generation of the robot utterance voice data, the control unit 130 determines whether the second voice information, which is new voice information, has been received from the receiving unit 110 (step S704).

  If the second voice information has not been received (NO in step S704), the generation unit 132 generates the robot utterance data as a response to the first voice information as it is, and transmits the robot utterance data to the smart speaker 200 via the transmission unit 140. This is transmitted (step S705), and the process ends.

  On the other hand, when the second voice information has been received (YES in step S704), the voice analysis unit 131 determines whether or not it is within a predetermined time after receiving the first voice information (step S706). The determination can be made by taking a difference between the reception time of the first audio information and the reception time of the second audio information, and comparing the difference with a threshold value that is a predetermined time. At the timing when the second voice information is received, generation of the robot utterance data as a response to the first voice information and transmission to the smart speaker 200 may or may not be completed.

  When it is determined that the reception of the second audio information is within a predetermined time from the reception of the first audio information (YES in step S706), the audio analysis unit 131 converts the second audio information into the first audio information. It is determined whether or not there is a word in the same category as the included word (step S707). Here, whether or not there is a word in the same category is, for example, a word included in the first voice information, which is an attribute of a word serving as a target case of the inquiry and a word included in the second voice information. Thus, the determination can be made based on whether or not there is a match with the attribute of the word serving as the purpose of the same inquiry. As a specific example, when there is an inquiry “Tell me the weather in Tokyo” as the first audio information, “weather” is the content of the inquiry, and the purpose of the inquiry is “Tokyo”. At this time, “Tokyo” may have attributes such as a place name, a city name, and a place. Then, when there is an inquiry "Tell me the weather of Shinagawa" as the second audio information, "weather" is the content of the inquiry, and the purpose of the inquiry is "Shinagawa". At this time, since “Shinagawa” can have attributes of a place name, a city name, and a place, it is determined that the second voice information includes words in the same category as the words included in the first voice information. be able to.

  When it is determined that the second voice information includes a word in the same category as the word included in the first voice information (YES in step S707), the voice analysis unit 131 compares the first voice information with the second voice information. The result of the analysis of the second audio information, together with the analysis of the context between them, is transmitted to the generation unit 132. Then, the generation unit 132 generates, based on the transmitted analysis result, at least robot utterance data based on the second voice information, that is, robot utterance data serving as a response to the second voice information. Then, the generation unit 132 transmits the generated robot utterance data to the smart speaker 200 via the transmission unit 140 (step S709), and ends the processing. Here, at least the robot utterance data serving as a response to the second voice information includes a response to an inquiry included in the second voice information, and in some cases, includes a response to an inquiry included in the first voice information. . Further, at this time, when the generation and transmission of the robot utterance voice data based on the first voice information has not been completed, the generation unit 132 stops the generation and transmission, and then based on at least the second voice information. Generate and output robot utterance data. This is the case where the user 10 utters while the smart speaker 200 is outputting the robot utterance data based on the first audio information as audio, and the second audio information is obtained. When it is determined that the voice information and the first voice information are words in the same category, the voice output device 100 instructs the smart speaker 200 to stop the voice output based on the robot utterance data based on the first voice information. It may be. After the stop instruction, the generation unit 132 generates the robot utterance data based on the second voice information, and the voice assist device transmits the robot utterance data based on the second voice information to the smart speaker 200. Is also good.

  On the other hand, when the voice analysis unit 131 determines that the second voice information has not been received within a predetermined time after receiving the first voice information in step S706 (NO in step S706), or in step S707, the voice analysis If the unit 131 determines that the second voice information does not include a word in the same category as the word included in the first voice information (NO in step S707), the voice analysis unit 131 corresponds to the question in the second voice information. It is determined whether there is a word to be executed (step S708). Here, the predetermined time may be, for example, a time during which the smart speaker 200 outputs the robot utterance data based on the first voice information as voice.

  When the word corresponding to the question is included in the second voice information (YES in step S708), the generation unit 132 generates robot utterance data as an answer to the content of the question, and transmits the robot utterance data via the transmission unit 140. The data is transmitted to the smart speaker 200 (step S710), and the process ends.

  If there is no word corresponding to the question in the second voice information (NO in step S708), the process proceeds to step S705. In this case, instead of moving to the process of step S704, the system is configured to generate robot utterance data for requesting the user to re-inquire again and transmit the generated data to the smart speaker 200. May be done.

  The operation of the smart speaker 200 is such that the voice from the user is received by the microphone 240, the voice information is transmitted from the transmission unit 250 to the voice output device 100, and the robot utterance data output from the voice output device 100 is transmitted. Since the signal is only received by the receiving unit 210 and output (notified) from the speaker 230, detailed description is omitted.

(Example of response)
Hereinafter, a process executed by the audio output device 100 based on audio data collected by the smart speaker 200 will be specifically described. The following specific example will be described using an example in which the user inquires about the weather.

  (Example 1) When the user says, "Tell me the weather in Tokyo? Ah, after all, tell me the weather in Shinagawa?"

  Example 1 corresponds to a case where the status in the column 412 is satisfied as the status information 410 in the answer model information 121 shown in FIG. In this case, the correspondence information 420 corresponds to the correspondence shown in the column 422. Specifically, in this case, first, the information “Tell me the weather in Tokyo?” Is transmitted to the audio output device 100 as the first audio information, and the analysis is performed. Then, the weather forecast server is accessed from the word “weather” to obtain the weather information of “Tokyo”. Then, robot utterance data based on the weather information is generated. Here, to the audio output device 100, information of "Oh, after all, tell me the weather in Shinagawa?" Is transmitted as the second audio information.

  Then, the sound output device 100 determines whether the second sound information is received within a predetermined time (for example, 5 seconds) after receiving the first sound information, and determines that the second sound information is received within the predetermined time. And Then, the generation unit 132 determines that there is a word specifying the application “weather” included in the first voice information and “weather” included in the second voice information, and “Tokyo” and “Shinagawa”. It detects that both words include the same category of word "place name".

  In this case, the generation unit 132 determines that the second voice information is a rephrase, stops generating the robot voice data for the first voice information, and generates the robot voice data indicating the answer to the second voice information. For example, the generation unit 132 accesses the weather forecast server, acquires the weather information of Shinagawa, and generates robot utterance data (for example, the weather in Shinagawa is fine and the temperature is XX degrees) according to the content.

  Then, in this case, the smart speaker 200 makes an announcement only about the weather in Shinagawa, so that the user can obtain information that he / she really wants to know without re-instruction.

  At this time, the generation unit 132 may further analyze the context of the user's utterance to improve the accuracy of the determination that the latter is a restatement. In the above example, since there is a context that denies the former “after all”, the accuracy of the determination that the latter is a restatement can be improved.

  (Example 2) When the user says "Tell me the weather in Tokyo? Oh, and also Osaka"

  Example 2 corresponds to a case where the status of the column 413 is satisfied as the status information 410 in the answer model information 121 shown in FIG. In this case, the correspondence information 420 corresponds to the correspondence shown in the column 423. Specifically, in this case, first, the information “Tell me the weather in Tokyo?” Is transmitted to the audio output device 100 as the first audio information, and the analysis is performed. Then, the weather forecast server is accessed from the word “weather” to obtain the weather information of “Tokyo”. Then, robot utterance data based on the weather information is generated. Here, the information “Oh, then Osaka” is transmitted to the audio output device 100 as the second audio information.

  Then, the sound output device 100 determines whether the second sound information is received within a predetermined time (for example, 5 seconds) after receiving the first sound information, and determines that the second sound information is received within the predetermined time. And Then, the generating unit 132 detects that both words “Tokyo” included in the first voice information and “Osaka” included in the second voice information in the same category of “place name” are included.

  On the other hand, in this case, the voice analysis unit 131 analyzes the context of the second voice information, and interprets that a word having an additional meaning such as “then” or “also” is included. In such a case, the generation unit 132 determines that the second voice information is not a rephrase, but a request for additional information, generates and transmits robot utterance data for the first voice information, and Robot utterance data indicating an answer to the voice information is generated and transmitted. Therefore, the information on the weather in both Tokyo and Osaka is transmitted to the smart speaker 200, and the smart speaker 200 notifies the information on the weather in both.

  (Example 3) When the user says "Tell me the weather in Tokyo?", And after a predetermined time elapses, he says "And also Osaka"

  Example 3 corresponds to a case where the status in the column 411 is satisfied as the status information 410 in the answer model information 121 shown in FIG. In this case, the correspondence information 420 corresponds to the correspondence shown in the column 421. Specifically, in this case, first, the information “Tell me the weather in Tokyo?” Is transmitted to the audio output device 100 as the first audio information, and the analysis is performed. Then, the audio output device 100 generates robot utterance data including information on the weather in Tokyo from the contents. After the lapse of the predetermined time, if the information "Oh, Osaka" is transmitted as the second voice information to the voice output device 100, the voice output device 100 is Is determined to be unrelated.

  In such a case, the voice output device 100 generates only the robot utterance data in response to the first voice information, or generates and transmits the robot utterance data in response to the first voice information. Please ask a question. "The robot utterance data for requesting may be generated and transmitted. At this time, it may be configured to generate and transmit only the robot utterance data requesting "Please ask a question again" without generating the robot utterance data in response to the first voice information. .

  (Example 4) When the smart speaker 200 inquires "Where do you want to know the weather?" And the user says "Shimbashi ..., Shinagawa"

In this case, the sound output device 100 receives the sound "Shimbashi" as the first sound information. Then, the voice "Shinagawa" is received as the second voice information. The user's remark is an answer to the question asked by the smart speaker 200, and the voice analysis unit 131 can analyze that both "Shinbashi" and "Shinagawa" are words of the same category of "place name". In such a case, the audio output device 100 determines that this is a paraphrase as described in (Example 1) above, and determines that the latter “Shinagawa” is a weather location that the user wants to know. Then, robot utterance data indicating the weather of "Shinagawa" may be generated and transmitted, but if the distance between the two place names is within a predetermined distance, the weather of the area including both of the place names is calculated. You may make it acquire and notify the information. In the case of this example, the audio output device 100 generates robot utterance data indicating weather information of an area of “Tokyo” including “Shimbashi” and “Shinagawa”, and transmits the generated data to the smart speaker 200. It may be. The answer model information 121 for the case of this example 4 is not shown in FIG. 4, but as the status information 410, for example,
"-Acquire the second audio information within a predetermined time after receiving the first audio information.
・ There is a word corresponding to the question in the second voice information "
As the correspondence information 420,
This means that “only the robot utterance data corresponding to the second voice information is generated”.

  As described above, the voice output device 100 can generate robot utterance data for making a natural response to various inquiries from the user. Further, according to Example 4, the voice output device 100 recognizes the restatement by the user even if there is no word denying the first voice information in the user's voice, that is, even if there is no wording having the specific vocabulary. Then, the robot utterance data for making a natural response can be generated.

  (Example 5) When the smart speaker 200 inquires, "Where do you want to know the weather?" And the user says "Shimbashi, ..., Shinagawa! Shinagawa! Shinagawa!"

  In this case, the sound output device 100 receives the sound "Shimbashi" as the first sound information. Then, the voice "Shinagawa" is received as the second voice information. The user's remark is an answer to the question asked by the smart speaker 200, and the voice analysis unit 131 can analyze that both "Shinbashi" and "Shinagawa" are words of the same category of "place name". On the other hand, in the case of Example 5, unlike the example 4, the voice output device 100 can recognize by analysis that the user has uttered the same phrase a plurality of times. In such a case, when a plurality of texts of the same category are included within a predetermined time as an answer to the question, the voice output device 100 may use the text that the user has said most among the plurality of texts. May be specified as an answer to the question, and the robot utterance data may be generated. Further, in particular, when it can be analyzed that the user repeatedly repeats the same text, the repeated text may be specified as an answer to the question, and the robot utterance data may be generated.

Although the answer model information 121 for the case of this example 5 is not shown in FIG. 4, as the status information 410, for example,
"-Acquire the second audio information within a predetermined time after receiving the first audio information.
・ The first voice information and the second voice information have a plurality of answers, "
As the correspondence information 420,
It is also possible to say that “only the robot voice data corresponding to the word having the highest number of appearances among the words to be answered is generated by the first voice information and the second voice information”.
Alternatively, as the status information 410, for example,
"-Acquire the second audio information within a predetermined time after receiving the first audio information.
・ There is a wording that is repeated multiple times in the second audio information "
As the correspondence information 420,
"・ Generate robot utterance data based on words repeated a plurality of times in the second voice information".

  Since the answer emphasized a plurality of times in this way can be presumed to be the answer that the user wants to know, the audio output device 100 specifies such a word as the answer of the information that the user wants to know. Then, the robot utterance data for making a natural response can be generated. When the robot utterance data is generated based on the answer emphasized by the user, it may be based on the volume of the user's voice instead of the number of answers. That is, the answer with the higher volume of the voice of the user may be specified as the answer to the item that the user wants to know.

(Summary)
As described above, when the user makes a restatement, the voice output device 100 determines that the restatement is performed within a predetermined time from the timing at which the word before the restatement was issued. It can be determined whether or not the second voice information is only a restatement, based on whether or not the word includes the same category as the word in the word before the restatement. Therefore, the voice output device 100 needs to generate the robot voice data for both the first voice information and the second voice information, or the robot voice data indicating only the answer to the second voice information. Can be determined. Then, when it can be determined that it is a rephrase, the voice output device 100 responds by determining that the second voice information is the content that the user actually wants to inquire, so that a more natural conversation is performed. Can be provided.

(Supplement)
It goes without saying that the device according to the above embodiment is not limited to the above embodiment, and may be realized by another method. Hereinafter, various modifications will be described.

  (1) In the above embodiment, an example is shown in which an inquiry about the weather is made. However, it goes without saying that the audio output device 100 can deal with cases other than the weather. The present invention can be used not only for inquiries about the weather from the user but also, for example, for home appliance operation, music playback, list management of shopping, and the like. In the case of home appliance operation, as an example, it is assumed that an instruction to “turn on cooling at 24 degrees,..., 26 degrees” is received. At this time, if the time between the utterance of 24 degrees and the utterance of 26 degrees is within a predetermined time, the audio output device 100 sets the category of “24 degrees” and “26 degrees” as the same temperature. Interprets that "26 degrees" is the information specified by the user, so that the audio output device 100 controls the air conditioner to be turned on at a setting of 26 degrees as an information processing device for operating home appliances. It can be carried out. Further, in the case of music reproduction, it is assumed that the user has uttered "playing A. ... is B better?" In this case, the audio output device 100 determines that “A” and “B” are equal to each other if the time between the statement “apply A” and the statement “B is better” is within a predetermined time. Since both belong to the category of music (music), the smart speaker is instructed to reproduce the music of "B". If the user manages a shopping list, the voice output device 100 is used as an information processing device that manages a shopping list when the user makes a statement such as "sweet potato instead of carrot, eggplant, potato, ...". If the time from the statement "potato" to the statement "sweet potato" is within a predetermined time, only the sweet potato is added to the shopping list. As described above, the audio output device 100 can perform a process estimated to be desirable for the user in response to the user's restatement in various situations other than the inquiry about the weather. As another example, the same can be said for a place name in route guidance.

  (2) In the above embodiment, a part of the functions held by the audio output device 100 may be held by another device, and the other device may be caused to bear a part of the processing executed by the audio output device 100. . For example, another information processing device having a voice analysis function first analyzes the user's voice received by the smart speaker 200, and transfers the analysis result to the voice output device 100. Then, the sound output device 100 may be configured to generate robot utterance data based on the transferred analysis result.

  (3) In the above embodiment, a plurality of audio output devices 100 may exist depending on the processing to be executed. For example, various devices such as a device for notifying information about weather, a device for notifying information about cooking, and a device for operating home appliances can be considered. At this time, the communication system 1 may further include an information processing device that first analyzes the voice received by the smart speaker 200 and determines which device should make an inquiry based on the voice. In such a configuration, the information processing apparatus may be useful for determining the restatement described in the above embodiment and for specifying a device that transmits sound. For example, when the user rephrases the type of inquiry that he / she wants to make, such as "tell me the departure time of the train, not the weather ...", the device that responds is replaced by a device that manages weather information. Can not be executed at the same time as to whether the device should be used to manage the time of the train, so if one of them is determined to be a restatement, the device corresponding to the latter will be asked by voice. May be configured to be transmitted.

  (4) In the above embodiment, the smart speaker 200 and the audio output device 100 are described as separate devices, but the smart speaker 200 and the audio output device 100 may be formed integrally. That is, the smart speaker 200 may include some or all of the functions of the audio output device 100.

  (5) In the above-described embodiment, it is determined whether or not words in the same category are included in the first audio information and the second audio information as words that are desirable as an answer to the context and the question in the dialogue with the user. Had been determined. However, when asking the user any question from the audio output device 100 side (smart speaker 200 side), when generating the robot utterance data for the question, the generation unit 132 responds to the question. Also, a category assumed to be appropriate is determined, and the determined category is stored in the storage unit 120. Then, the control unit 130 of the audio output device 100, when the second audio information is obtained within a predetermined time from the first audio information and the first audio information as a response from the user to the inquiry, It is determined whether or not one piece of audio information includes a word belonging to the determined (stored) category. Similarly, it is determined whether or not the second voice information includes a word belonging to the determined (stored) category. Then, when both voice information include a word belonging to the determined category, the generation unit 132 may generate at least robot utterance data based on the second voice information. By doing so, it becomes easy to narrow down when verifying whether or not words of the same category are included in the first audio information and the second audio information, and the processing time can be reduced.

  (6) The program according to each embodiment of the present disclosure may be provided in a state stored in a computer-readable storage medium. The storage medium is capable of storing the program on a “temporary tangible medium”. The storage medium may include any suitable storage medium such as an HDD or an SDD, or a suitable combination of two or more thereof. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile. The storage medium is not limited to these examples, and may be any device or medium as long as it can store a program.

  Note that the audio output device 100 can realize the functions of the plurality of functional units described in each embodiment, for example, by reading a program stored in a storage medium and executing the read program. Further, the program may be provided to the audio output device 100 via an arbitrary transmission medium (such as a communication network or a broadcast wave). The sound output device 100 realizes the functions of the plurality of functional units described in each embodiment by executing a program downloaded via the Internet or the like, for example.

  The program can be implemented using, for example, a script language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5.

  At least part of the processing in the audio output device 100 may be realized by cloud computing including one or more computers. Further, each functional unit of the audio output device 100 may be realized by one or a plurality of circuits that realize the functions described in the above-described embodiments, and the function of the plurality of functional units may be realized by one circuit. Is also good.

  (7) Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various changes and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present disclosure. For example, the functions and the like included in each means, each step, and the like can be rearranged so as not to be logically inconsistent, and a plurality of means, steps, and the like can be combined into one or divided. . Further, the configurations shown in the embodiments may be appropriately combined.

Reference Signs List 100 audio output device 110 reception unit 120 storage unit 130 control unit 131 audio analysis unit 132 generation unit 140 transmission unit

Claims (11)

A receiving unit that receives input of voice information indicating an uttered voice by the user;
A generation unit that generates robot utterance data corresponding to a reply to the utterance voice based on the voice information;
An output unit that outputs the robot utterance data,
The generation unit may include a case where the second audio information is received within a predetermined time after receiving the first audio information, wherein the second audio information includes a second audio information included in the first audio information. An information processing device that generates robot utterance data based on at least the second voice information when a second word of the same category as one word is included. The said generation part, when the word which denies the said 1st audio | voice information is contained in the said 2nd audio | voice information, produces | generates the robot utterance data based only on the said 2nd audio | voice information. 2. The information processing device according to 1. The generation unit generates robot utterance data for both the first voice information and the second voice information when the second voice information includes a word connected to the first voice information. The information processing apparatus according to claim 1 or 2, wherein When the second voice information includes a second word of the same type as the first word included in the first voice information, a robot utterance inquiring which is correct The information processing apparatus according to claim 1, wherein the information processing apparatus generates data. An audio output unit that outputs an audio based on the robot utterance data,
A voice collection unit that collects the voice of the user,
The output unit outputs the robot utterance data to the voice output unit,
The receiving unit receives the uttered voice collected by the voice collecting unit as the voice information,
The output unit outputs the first voice information to the second voice information when receiving the second voice information within a predetermined time while outputting the robot utterance data based on the first voice information. When the second word of the same type as the first word included in the first voice information is included, the output of the robot utterance data generated for the first voice information to the voice output unit is stopped. The information processing apparatus according to claim 1. The information processing according to claim 5, wherein the output unit outputs new robot utterance data based on the second voice information after stopping outputting the robot utterance data to the voice output unit. apparatus. The output unit outputs the robot utterance data to an external speaker,
The receiving unit is configured to receive an uttered voice collected by an external microphone as the voice information,
The output unit receives the second voice information based on the uttered voice received from the user when the external speaker is outputting voice based on the robot utterance data based on the first voice information, and When the second voice information includes a second word of the same type as the first word included in the first voice information, the second voice information is based on the robot utterance data generated for the first voice information. The information processing apparatus according to claim 1, wherein the information processing apparatus outputs a stop instruction to stop outputting the voice. The information processing device according to claim 7, wherein the output unit outputs new robot utterance data based on the second voice information after outputting the stop instruction. The generation unit, while generating robot utterance data for making an inquiry to the user, determines a category of words desired as a response to the inquiry,
The generating unit, when receiving the first voice information and the second voice information as an answer to the inquiry, when the word belonging to the determined category in the first voice information is the first voice information. When the second voice information includes a word belonging to the determined category as the second word, the robot utterance data is based on at least the second voice information. The information processing apparatus according to any one of claims 1 to 8, wherein the information processing apparatus generates: A receiving step of receiving an input of voice information indicating an uttered voice by the user;
A generation step of generating robot utterance data corresponding to a reply to the utterance voice based on the voice information;
An output step of outputting the robot utterance data,
The generation step is a case where the second audio information is received within a predetermined time after the first audio information is received, and the second audio information includes a second audio information included in the first audio information. An audio output method executed by a computer for generating robot utterance data based on at least the second audio information when a second word of the same category as one word is included. On the computer,
A reception function for receiving an input of voice information indicating an uttered voice by the user;
A generation function of generating robot utterance data corresponding to a reply to the utterance voice based on the voice information;
Output function of outputting the robot utterance data,
The generation function is a case where the second audio information is received within a predetermined time after the first audio information is received, and the second audio information includes a second audio information included in the first audio information. A voice output program for generating robot utterance data based on at least the second voice information when a second word of the same category as the first word is included.

Images