JP2008249893A - Speech response device and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech response device capable of performing response corresponding to user's action. <P>SOLUTION: The speech response device 10 comprises an input acquiring section 101, a response content generation section 102, a response output section 103, a peripheral sound acquiring section 104 and a listening status judging section 105. User's speech, response speech and environment sound included in peripheral sound information are analyzed. (1) When the user answers the speech response, new speech response to the answer is generated. (2) When the user does not hear the response speech, the speech response is repeated. (3) When the user cannot answer the speech response, the speech response is suspended. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a voice response apparatus and method suitable for home appliances such as a television having a voice response function.

  In recent years, many products for households equipped with a voice interface have been commercialized. In particular, the usage guidance by voice makes it easier to use home appliances that require complicated operations, and various functions can be used effectively. However, this is not necessarily the situation where the user listens to a system response such as guidance and makes an appropriate input, and a technique for performing a response process according to the situation of the user is required.

  Therefore, in Patent Document 1, a processing method is proposed in which the user's workload is estimated from the complexity of user input and the time until input, and the response content of the system is changed.

  Patent Document 2 proposes a function of determining a user's situation from vehicle information and changing a response process in a car navigation system having a voice interaction interface.

  Further, Patent Document 3 proposes a function of changing the response content according to information such as “laughter” or “applause” of the listener.

  Patent Document 4 proposes a method of estimating the noise level from the input voice of the user and changing the response content.

Further, in Patent Document 5, when the user does not reply or execute an operation for a certain period of time with respect to the sound output of the apparatus, the user performs a response process for outputting a response prompting the user again, thereby performing A method to support the answer is proposed.
JP 2003-108191 A JP 2004-233676 A JP 2004-24867 A US Published Patent 2002-95295 Japanese Patent Laid-Open No. 2005-22065

  For home appliances with voice response function and voice interaction function as in the background art above, it is assumed that the user can listen to the response and execute the necessary answer or operation when outputting the voice response It has become.

  However, considering the usage situation in the home, various noises in the home cause the situation where the user cannot hear the response voice of the device, or the user is performing actions other than the exchange with the device and the response to the device response You must consider situations where you cannot perform answers or operations.

  For example, when considering an operation of a television having a voice response function, when a voice response is output, there may be a case where there is an operation sound of other home appliances around the user and the voice of the voice response device cannot be heard. In addition, there may be a case where an answer or operation cannot be executed with respect to the response content of the apparatus, such as when the user starts talking on the mobile phone while the response output is being performed.

  As described above, when the response output cannot be heard and the user cannot input, if the voice response device proceeds without interrupting the process, an operation unintended for the user may be executed. Further, if the response output is repeated when an action not related to the voice response device is performed, an unnecessary response output is required for the user.

  In view of the above problems, the present invention provides a voice response device capable of reacting to a user's action and a method thereof.

  The present invention provides a voice response device for voice response to a user, an ambient voice acquisition unit that acquires ambient sound information related to sounds around the user when the voice response is made, and the ambient sound information (1) a state in which the user answers the response voice, (2) a state in which the user does not hear the response voice, or ( 3) a listening state determination unit that determines whether the user cannot answer the voice response; and (1) if the user answers the voice response, a new voice response is generated for the answer. (2) If the user does not hear the response voice, repeat the voice response. (3) If the user cannot answer the voice response, generate a response content that interrupts the voice response. And comprising It is a voice response unit.

  According to the present invention, even when it is difficult for the user to answer the voice response device by determining the listening state of the user, it is possible to control the response flexibly according to the situation of the user.

  A voice response device according to an embodiment of the present invention will be described below with reference to the drawings.

(First embodiment)
A voice response device 10 according to a first embodiment of the present invention will be described with reference to FIGS.

(1) Configuration of Voice Response Device FIG. 1 is a configuration example of a voice response device 10 according to the present embodiment. The voice response device 10 includes an input acquisition unit 101, a response content generation unit 102, a response output unit 103, an ambient sound acquisition unit 104, and a listening state determination unit 105.

(1-1) Input acquisition unit 101
The input acquisition unit 101 acquires input from the user. For example, a voice signal emitted by a user is captured by a microphone, etc., voice determination processing is performed, converted into text information, and output. Alternatively, it receives an infrared signal from a remote controller and outputs the received content.

(1-2) Response content generation unit 102
The response content generation unit 102 determines the response content based on the user input information acquired by the input acquisition unit 101.

  The response content generation unit 102 outputs input information to the application. When the application is a database system, the response content generation unit 102 acquires a search result corresponding to the input search request, and generates a response content based on the search result.

  In addition, the response content generation unit 102 can generate and output a response content when a response voice is required when the situation changes in the set time or application regardless of whether or not the user has input. .

(1-3) Response output unit 103
The response output unit 103 acquires the response content generated by the response content generation unit 2 as, for example, text, converts it into synthesized sound, and outputs it. It should be noted that the output sound may be simply reproduction of a previously recorded sound.

(1-4) Ambient sound acquisition unit 104
The ambient sound acquisition unit 104 may be a microphone near the user, a microphone attached to a device such as a remote controller near the user, a microphone attached to a robot near the user, or a wall or ceiling. Ambient sound information input to the microphone provided in the is acquired.

(1-5) Listening state determination unit 105
The listening state determination unit 105 analyzes the ambient sound information that is the output of the ambient sound acquisition unit 104, determines the listening state of the user based on a predetermined determination criterion of the listening state using the analysis result, and The determination result is output as a listening state determination result.

(2) Listening state determination method The listening state is determined by analyzing the ambient sounds of the user and determining the listening determination content based on the determined rules. Specific examples will be described below.

(2-1) First Example FIG. 2 is a diagram illustrating a first example of the listening state determination method.

  The listening state determination unit 105 acquires the utterance start time and utterance end time information of the response sound output from the response output unit 103, and ambient sound information from the utterance start time of the acquired response sound to the utterance end time of the response sound To obtain average volume information. When the average volume exceeds the reference value, the listening state is “sounding”, and when the average volume is below a certain value, the listening state is “not heard”, and this listening state determination result is output. . This reference value may be a predetermined value, or based on the average value of the volume of the device response voice in the ambient sound information at the time when the user's response to the response voice was made in the past use of the device. It may be calculated.

  In the present embodiment, the average sound volume is used. However, when the noise level is measured from the ambient sound information and exceeds a predetermined value, “not audible” may be determined, and the listening state determination result may be used. For example, the ratio between the audio signal and the noise level may be calculated, and if the value is 0 dB or more, it may be determined that “not audible”.

(2-2) Second Example FIG. 3 is a diagram showing a second example of the listening state determination method.

  The listening state determination unit 105 acquires the utterance section and the utterer of the utterance section from the ambient sound information. That is, the utterance time and utterance end time information of the response sound output from the response output unit 103 is acquired, and the user's own utterance and utterances other than the user are included within the utterance time of the response sound. If it is confirmed, a determination is made that “the user is speaking to another speaker and cannot answer”, and the result is output as a listening state determination result.

  Also, if the percentage of the user voice interval is larger than a certain value, for example, 50% within the response voice utterance time, it is judged that the user is speaking with another speaker and cannot answer. May be.

(2-3) Third Example FIG. 4 is a diagram showing a third example of the listening state determination method.

  The listening state determination unit 105 acquires the utterance section and the utterer of the utterance section from the ambient sound information. That is, the utterance time and utterance end time information of the response voice output from the response output unit 103 is acquired, and the utterance start time of the response voice or the utterance end time is included in the utterance section of the user acquired from the ambient sound information. If it is included, it is determined that “the user is talking to another speaker and cannot answer”, and the determination result is output.

(2-4) Fourth Example FIG. 5 is a diagram showing a fourth example of the listening state determination method.

The listening state determination unit 105 determines the presence / absence of sounds other than human utterances that are not related to the use of the device from ambient sound information. Sounds that determine presence / absence are sounds that occur in the device usage environment, such as sounds other than human speech, such as the sound of an object colliding, and sounds that may have occurred in the device usage environment, such as telephone and intercom ringing sounds. The sound generated when no response is made to the response device when the device is used in the past.

  When the ambient sound information includes such acoustic features other than human speech, it is determined that there is a specific sound that is not related to the use of the device.

  That is, the utterance time and utterance end time information of the response voice output from the response output unit 103 is acquired, and within the utterance time of the response voice or within a certain interval such as from 10 seconds before the utterance start time to the utterance end time. When a specific sound that is not related to the use of the device is detected, the user determines that “other things cannot be answered” and the determination result is output.

(3) Generation of Response Content Based on the listening state determination result acquired by the listening state determination unit 105, the response content generation unit 102 generates response content.

  If it is determined that the user is “sounding” and a response is received from the user, a response to the response is generated.

  On the other hand, if it is determined that the listening state is “not heard”, the response content with the same content is output at an increased volume, or the sound quality (for example, a male voice to a female voice) is easily heard. Change the output.

  If it is determined that the listening state is “cannot answer because the user is talking to another speaker” or “cannot answer because he / she is doing other things”, the response content indicating interruption of the voice response device 10 is displayed. While generating and outputting, the voice response device 10 interrupts its operation.

  In addition, you may set so that the response content based on a listening state judgment result may be produced | generated after progress for a fixed time from response voice end time.

  Even when the operation of the voice response device 10 is interrupted, the user's listening status is determined for each predetermined section, and “other things cannot be answered” and “the user is speaking with another speaker and responding” When the determination “cannot be made” is no longer made, the response content generation unit 102 generates and outputs the response content indicating that the interrupted state is finished again, and restarts the operation of the voice response device 10.

  The generation of the response content based on the listening state determination result as described above can be realized by prescribing a response generation method corresponding to the listening state determination.

(4) Processing Procedure Hereinafter, the processing procedure A of the voice response device 10 will be described in detail. FIG. 6 is a flowchart for explaining the processing contents of processing procedure A.

  In step A1, ambient sound extraction is started.

  In step A2, the voice response device 10 outputs a response voice.

  In step A3, the listening state is determined after the response voice is output. If it is determined that the listening state determination result is “listening”, the process proceeds to step A4, and the listening state determination result is “not heard” or “other”. If it is determined that the device response voice such as “I cannot answer by talking to a person” or “I cannot answer by doing other things” is not transmitted, the process proceeds to Step A5.

  In step A4, if there is a user input within a predetermined time, processing for the user input is executed. If there is no user input within the predetermined time, the process proceeds to step A2 in order to output the response contents again.

  If it is determined in step A5 that the listening state determination result is “not audible”, the process proceeds to step A2 to output a response again. In this case, the response output volume may be increased or decreased. Also, if the listening state judgment result is a listening state in which it is difficult to input the device response, such as "I can't answer because I'm talking to another person" or "I can't answer because I'm doing other things" The operation of the response device is interrupted.

(5) Specific Example The above process will be described in detail with a specific example of a television incorporating the voice response device 10.

  FIG. 7 is an example of user interaction with a television with a built-in voice response device 10, in which baseball results are managed as a database, answers to inquiries from users, and related video playback is possible. A television is assumed.

  The dialogue example shown in FIG. 7 assumes a situation in which the user cannot speak and answer with another person on the mobile phone while the device response 2 is being output. The process will be described.

  The voice response device 10 asks the user “Do you want to play the home run scene?” After the device response 2 finishes playing, the listening state determination unit 105 receives the output of the surrounding voice acquisition unit 103 and outputs the listening state determination result To do.

  Here, in order to determine the listening state, a listening state determination rule is used. FIG. 8 is an example of the listening state determination rule.

  For example, when talking on a mobile phone and the user utterance time during response voice playback exceeds a reference value, the listening state determination unit 105 determines that “Speaking with another person and answering is not possible” based on the listening state determination rule. Is output as a determination result.

  The response content generation unit 102 that has acquired the listening state determination result “speaking with another person and cannot answer” makes it difficult to input the device response 2, and the voice response device 10 is interrupted by interrupting the response process. Unnecessary response output can be interrupted. Further, at this time, as the device response 3, “response is interrupted” may be replied.

  Although the response processing operation is interrupted, ambient sound information is acquired at regular intervals, and if the user's utterance is no longer detected, the interrupted response processing is resumed and “Team A's home run scene can be reproduced. The user can continue to interact with the television by performing a response output such as “Masu”.

  It should be noted that the processing for such a listening state determination result is prescribed in advance, and it is only necessary to operate based on the processing method, and even when the listening state determination result is `` speaking with another person and cannot answer '', Depending on the importance of the response, processing may be performed so that the response is repeated again.

(6) Effect As described above, according to the present embodiment, it is possible to control the voice response according to the listening state of the user, and the voice response device even when the user cannot answer the voice response device. 10 does not operate unintentionally and does not cause an extra burden on the user.

  In addition, when the user is performing an action unrelated to the voice response device 10, the voice response device 10 does not output a response that hinders the user's action, and the user is not burdened.

(Second Embodiment)
Next, a voice response device 20 according to a second embodiment of the present invention will be described with reference to FIG.

(1) Configuration of Voice Response Device FIG. 9 is a configuration example of the voice response device 20 according to the present embodiment.

  The voice response device 20 includes an input acquisition unit 201, a response content generation unit 202, and a response output unit 203. The listening state determination device 21 includes an ambient sound acquisition unit 211 and a listening state determination unit 212. The voice response device 20 and the listening state determination device 21 are connected via a network such as a wireless LAN so that necessary information can be exchanged freely.

(1-1) Input acquisition unit 201
For example, the input acquisition unit 201 captures a voice signal emitted by a user with a microphone or the like, performs voice recognition processing, converts it into text information, and outputs the text information. Alternatively, it receives an infrared signal from a remote controller and outputs the received content.

(1-2) Response content generation unit 202
The response content generation unit 202 determines the response content based on the user input information acquired by the input acquisition unit 201.

  For example, the response content generation unit 202 outputs input information to the application. For example, when the application is a database system, the response content generation unit 202 acquires a search result corresponding to the input search request, and generates a response content based on the search result. To do.

  The response content generation unit 202 can generate and output the response content when a response voice is required when the situation changes at the set time or application regardless of whether or not the user has input. .

(1-3) Response output unit 203
The response output unit 203 acquires the response content generated by the response content generation unit 202 as, for example, text, converts it into synthesized sound, and outputs it. It should be noted that the output sound may be simply reproduction of a previously recorded sound.

(1-4) Ambient sound acquisition unit 204
The ambient sound acquisition unit 204 acquires ambient sound information input to a microphone near the user, a microphone attached to the robot near the user, or a microphone provided on the wall or ceiling.

(1-5) Listening state determination unit 205
The listening state determination unit 205 analyzes the ambient sound information that is the output of the ambient sound acquisition unit 104, determines the listening state of the user based on a predetermined determination criterion of the listening state using the analysis result, and listens The state determination result is output to the voice response device 20.

  When the listening state determination unit 205 needs the response utterance start time of the voice response device 20 to determine the listening state, it can be acquired from the voice response device 20.

  Even when the voice start time of the voice response device 20 cannot be acquired, it is possible to determine the listening state by analyzing the ambient sound acquisition information and specifying the response voice section from the voice response device 20.

(2) Response content The response content generation unit 202 outputs a response process, inquires the listening state determination unit 205 about the listening state determination result, and determines the response processing content according to the listening state determination result. Note that the response content generation unit 202 may acquire the listening state determination result when there is no input from the user for a predetermined time.

(3) Effect With the configuration of the present embodiment as described above, the response content generation control based on the listening state determination result is possible even if the voice response device 20 does not include a microphone for acquiring ambient sound information. It becomes possible.

  In addition, even when the processing amount of the listening state determination unit 105 is enormous, since the voice response device 20 is not loaded, the response content generation can be controlled without affecting the processing of the voice response device 20.

(Example of change)
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist thereof.

  Although the above embodiment has been described as a television having a voice response function, for example, the present invention can be applied to a system that accepts a voice response or a device having a voice response function such as car navigation.

  Moreover, although the said embodiment demonstrated the input from a user as a voice input, it can be applied also to the voice response apparatus on the assumption that a user input is a touch panel or remote control input.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is a block diagram of a voice response device according to a first embodiment of the present invention. It is the figure which showed the 1st example of the listening state judgment method. It is the figure which showed the 2nd example of the listening state judgment method. It is the figure which showed the 3rd example of the listening state judgment method. It is the figure which showed the 4th example of the listening state judgment method. It is a flowchart explaining the processing content of the process procedure A which concerns on 1st Embodiment. It is an example of a user's interaction with the television incorporating the voice response device according to the first embodiment. It is an example of the listening state judgment rule which concerns on 1st Embodiment. It is a block diagram of the voice response device of the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Voice response apparatus 101 Input acquisition part 102 Response content generation part 103 Response output part 104 Ambient sound acquisition part 105 Listening state judgment part

Claims (11)

In a voice response device for voice response to a user,
An ambient sound acquisition unit that acquires ambient sound information related to sounds around the user when performing the audio response;
Analyzing user voice, response voice, and environmental sound included in the ambient sound information, (1) the user is answering the response voice, (2) the user is hearing the response voice Or (3) a listening state determination unit that determines whether the user cannot answer the voice response;
(1) When the user answers the voice response, a new voice response is generated for the answer. (2) When the user does not hear the response voice, the voice response is generated. (3) If the user cannot answer the voice response, a response content generation unit that interrupts the voice response;
A voice response device comprising: Comprising a voice recognition unit for recognizing voice uttered by the user;
The response content generation unit generates the voice response based on the recognized user voice.
The voice response device according to claim 1. The listening state determination unit determines that the voice response is not heard by the user when the volume of the response voice included in the ambient sound information does not reach a reference value.
The voice response device according to claim 1. When the listening state determination unit determines that the response voice is not heard,
The response content generation unit repeats with the volume of the response voice increased.
The voice response device according to claim 1. The listening state determination unit determines whether or not the user cannot answer the voice response based on a sound other than the response voice included in the ambient sound information within the utterance time of the response voice. To
The voice response device according to claim 1. The listening state determination unit is in a state where the user cannot answer the voice response when the ambient sound information within the utterance time of the response voice includes the user voice and voices other than the user. To judge,
The voice response device according to claim 5. When the utterance start time or the utterance end time of the response voice is included in the utterance section of the user acquired from the ambient sound information, the listening state determination unit determines that the user Judge that it is not possible to answer
The voice response device according to claim 5. The listening state determination unit determines that the user is unable to answer the voice response when any physical sound is included in the ambient sound information within the utterance time of the response voice.
The voice response device according to claim 5. Based on ambient sound information acquired after the interruption, comprising a response restart processing unit for restarting the voice response;
The voice response device according to claim 1. In the voice response method for voice response to the user,
Ambient voice acquisition step of acquiring ambient sound information related to a user's ambient sound when performing the voice response;
Analyzing user voice, response voice, and environmental sound included in the ambient sound information, (1) the user is answering the response voice, (2) the user is hearing the response voice Or (3) a listening state determination step for determining whether the user cannot answer the voice response;
(1) When the user answers the voice response, a new voice response is generated for the answer. (2) When the user does not hear the response voice, the voice response is generated. Repetitively, (3) when the user cannot answer the voice response, a response content generation step for interrupting the voice response;
A voice response method comprising: In a voice response program realized by a computer for voice response to a user,
Ambient voice acquisition function for acquiring ambient sound information related to the surrounding sounds of the user when performing the voice response;
Analyzing user voice, response voice, and environmental sound included in the ambient sound information, (1) the user is answering the response voice, (2) the user is hearing the response voice Or (3) a listening state determination function for determining whether the user cannot answer the voice response;
(1) When the user answers the voice response, a new voice response is generated for the answer. (2) When the user does not hear the response voice, the voice response is generated. (3) If the user cannot answer the voice response, a response content generation function for interrupting the voice response;
Voice response program that realizes.

Images