JP2015004928A - Response target voice determination device, response target voice determination method, and response target voice determination program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a response target voice determination device, a response target voice determination method, and a response target voice determination program that can determine, with high accuracy and without loosing usability, a voice vocalized by a user to a robot or a device with the intention of inputting.SOLUTION: Provided is a response target vocalization determining unit 11 which: detects that a silent interval continuance length which is the length of a silent interval continuing after vocalization by a user, is longer than a prescribed threshold length of silent interval continuance, in order to determine that the vocalization is vocalization conducted by the user to a system; and detects that the silent interval continuance length is shorter than the threshold length of silent interval continuance, in order to determine that the vocalization is vocalization conducted by the user to a target other than the system.

Description

  The present invention relates to a response target speech determination device, a response target speech determination method, and a response target speech determination program for detecting whether or not a speech made to the system is detected in a speech dialogue system.

  In a system that realizes human-robot conversation using voice recognition technology, or a system that recognizes voice commands uttered by the user and provides information according to the contents, malfunction due to sound that is not the response target becomes a problem . Hereinafter, these systems are simply referred to as systems. In addition, the voice that the user utters with the intention of input to the robot or device is referred to as response target voice.

  Japanese Patent Application Laid-Open No. 2004-151867 discloses a speech validity determination device (apparatus) that performs face recognition, determines a direction in which a user's face is facing, and responds only to a utterance made when the face is facing the system. and method for determining relevance of input speech).

  Non-Patent Document 1 describes a technique (voice spotter) that allows voice commands to be input to the system during human conversation. By using the voice spotter, the user explicitly notifies the system of the processing target voice by performing a special utterance that is not a normal utterance, such as deliberately uttering with a high voice after speaking in a voiced pause. can do.

US Patent Application Publication No. 2012/0259638

Masataka Goto, Koji Kitayama, Katsunobu Ito, Tetsunori Kobayashi, “Voice Spotter: A voice input interface that can use voice recognition during human conversation”, Journal of Information Processing Society of Japan, Mar 2007, Vol. 48, no. 3, pp. 1274

  The technique described in Patent Document 1 has a problem that accurate determination cannot always be performed only by the face direction when humans talk while gazing at the system. In particular, when it is assumed that the system displays some information on a display or the like and the user discusses the content, the voice validity determination based on the face direction is not reliable. Further, the technique described in Non-Patent Document 1 has a problem that it is necessary to perform a special utterance when the user notifies the system of processing target speech, and usability is impaired.

  Accordingly, the present invention provides a response target speech determination device, a response target speech determination method, and a response that can accurately determine speech uttered by a user with an intention to input to a robot or device without impairing usability. An object is to provide a target speech determination program.

  The response target speech determination device according to the present invention detects that the silence interval duration, which is the length of the silence interval following the utterance by the user, is longer than a predetermined silence interval duration threshold, and the user performs the utterance by the system. A response that determines that the utterance was made to the user, detects that the silence duration duration is shorter than the silence duration duration threshold, and determines that the utterance is made by a user other than the system A target utterance determination unit is provided.

  The response target speech determination method according to the present invention detects that the silence interval duration, which is the length of the silence interval following the utterance by the user, is longer than a predetermined silence interval duration threshold, and the user performs the utterance in the system It is determined that the utterance has been made to the system, and it is detected that the silence duration duration is shorter than the silence duration duration threshold, and the utterance is determined to be utterance made by the user to a system other than the system. It is characterized by.

  The response target voice determination program according to the present invention detects that the silence interval duration, which is the length of the silence interval following the utterance by the user, is longer than a predetermined silence interval duration threshold value, and sends the utterance to the computer. It is determined that the utterance is made by the user to the system, it is detected that the silence interval duration is shorter than the silence interval duration threshold, and the utterance is made by the user other than the system. The determination process is executed.

  ADVANTAGE OF THE INVENTION According to this invention, the audio | voice which the user uttered with the intention of input with respect to a robot or an apparatus can be determined with sufficient accuracy and without impairing usability.

It is a block diagram which shows the structure of 1st Embodiment of the response object audio | voice determination apparatus by this invention. It is a flowchart which shows operation | movement of 1st Embodiment of a response object audio | voice determination apparatus. It is explanatory drawing which shows the frequency distribution of the silence interval continuation length in the conversation between people. It is a block diagram which shows the structure of 2nd Embodiment of the response object audio | voice determination apparatus by this invention. It is a flowchart which shows operation | movement of 2nd Embodiment of a response object audio | voice determination apparatus. It is explanatory drawing which shows the frequency distribution of the audio | voice feature-value extracted from the utterance made with respect to the system, and the audio | voice feature-value extracted from the utterance made with respect to the outside of a system. It is a block diagram which shows the structure of 3rd Embodiment of the response object audio | voice determination apparatus by this invention. It is a block diagram which shows the structure of 4th Embodiment of the response object audio | voice determination apparatus by this invention. It is a block diagram which shows the structure of 5th Embodiment of the response object audio | voice determination apparatus by this invention. It is a block diagram which shows the structure of 6th Embodiment of the response object audio | voice determination apparatus by this invention. It is a block diagram which shows the minimum structure of the response object audio | voice determination apparatus by this invention. It is a block diagram which shows the other minimum structure of the response object audio | voice determination apparatus by this invention.

Embodiment 1. FIG.
A first embodiment of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a first embodiment of a response target speech determination device according to the present invention.

  As illustrated in FIG. 1, the response target speech determination device includes an input sound signal acquisition unit 101, an input sound signal cutout unit 102, a speech segment determination threshold storage unit 103, a speech segment determination unit 104, and a speech segment sound signal. A storage unit 105, a silence interval duration threshold storage unit 106, and a response target utterance determination unit 107 are provided.

  The input sound signal acquisition unit 101 acquires a time series of input sound signals. The input sound signal acquisition unit 101 acquires a time series of input sound signals using a sound input device, for example, a microphone.

  The input sound signal cutout unit 102 receives an input sound signal and outputs a sound signal cut out for each frame.

  The voice segment determination threshold storage unit 103 stores a predetermined threshold for voice segment determination (pre-determined threshold for voice activity detection).

  The voice segment determination unit 104 receives a sound signal cut out for each frame and a threshold related to the voice segment determination, and the frame is included in the voice segment (active voice segment) or a silent segment (silence where no voice exists) segment).

  The voice segment sound signal storage unit 105 stores the sound signal determined as the voice segment by the voice segment determination unit 104.

  The silence interval duration threshold storage unit 106 stores a predetermined silence interval duration threshold (pre-determined threshold of duration of silence segment).

  The response target utterance determination unit 107 compares the duration of the section determined to be a silence period by the voice segment determination unit 104 with the silence period duration threshold, and responds with the voice period before the silence period as a response target, Determine whether to hold the response.

  The input sound signal acquisition unit 101, the input sound signal cutout unit 102, the speech segment determination unit 104, and the response target utterance determination unit 107 are realized by, for example, a computer that operates according to a response target speech determination program. In this case, the CPU reads the response target voice determination program, and operates as the input sound signal acquisition unit 101, the input sound signal cutout unit 102, the voice segment determination unit 104, and the response target utterance determination unit 107 according to the program. Further, the input sound signal acquisition unit 101, the input sound signal cutout unit 102, the voice segment determination unit 104, and the response target utterance determination unit 107 may be realized by separate hardware.

  In addition, the voice segment determination threshold storage unit 103, the voice segment sound signal storage unit 105, and the silence segment duration threshold storage unit 106 are specifically an optical disk device, a magnetic disk device, a memory, or the like included in the response target voice determination device. Realized by a storage device.

  Next, the operation of this embodiment will be described.

  FIG. 2 is a flowchart showing the operation of the first embodiment of the response target speech determination device. FIG. 3 is an explanatory diagram showing the frequency distribution of the duration of silence intervals in a conversation between humans.

  As shown in FIG. 2, first, the input sound signal cutout unit 102 inputs a time series of input sound acquired by the input sound signal acquisition unit 101. Then, the input sound signal cutout unit 102 cuts out waveform data for a unit time frame from the time series of the input sound (step S101).

  For example, the input sound signal cutout unit 102 acquires analog data acquired by a microphone or the like as digital data with a sampling frequency of 8000 Hz, a quantization bit of 16 bits, and Linear-PCM, and a waveform for 256 points every 10 milliseconds. Cut out the data. Note that the input sound signal cutout unit 102 may cut out waveform data based on other sampling frequencies, quantization bits, cutout intervals, cutout points, and the like.

  The response target speech determination device performs the processes of steps S102 to S104 shown below for each frame.

  In step S102, the speech segment determination unit 104 performs speech segment determination on the input sound cut out for each frame in step S101. As a method for determining a speech section, for example, a method of obtaining amplitude power or the like and comparing the threshold value stored in the speech section determination threshold storage unit 103 with a magnitude is conceivable. The amplitude power Pt is calculated by the following equation (1).

  N is the number of sample points included in one frame. xt is the value of the input sound data (waveform data) at time t. The voice section determination unit 104 determines that the voice state is present when the amplitude power is greater than the threshold value, and determines that the voice state is silent when the amplitude power is smaller than the threshold value. Although the amplitude power is used here, other feature quantities such as the number of zero crossings, the likelihood ratio between the speech model and the non-speech model, the pitch frequency, and the SN ratio may be used. A section in which the same state continues among voice states determined every unit time is defined as a voice section. In addition, an interval in which the same state continues among silence states determined every unit time is defined as a silence interval.

  The beginning of the speech section is the point in time when the continuous silence state is interrupted and changes to a speech state. This time is the end of the silence period at the same time. The end of the voice interval is when the continuous voice state changes to a discontinuous silence state. This is the beginning of the silence period at the same time. The voice segment and the silence segment are determined in this way when the continuous state is interrupted.

  Here, in order to prevent a short voice interval from occurring, a hangover such that after changing from a silence state to a voice state, it is not recognized as the beginning of the voice interval (end of the silence interval) unless it is determined that the voice state is of a certain length. Processing may be performed. In addition, a hangover process such that the end of the voice interval (the start of the silence interval) is not recognized unless it is determined to be a silence state of a certain length after changing from the voice state to the silence state so that a short silence interval does not occur. May be performed.

  Subsequent processing branches depending on whether the frame of interest, that is, the frame to be processed is included in the speech interval or the silence interval.

  When the speech segment determination unit 104 determines that the frame of interest is included in the speech segment, the speech segment determination unit 104 stores the sound signal of the frame in the speech segment sound signal storage unit 105 (step S103). Thereafter, the processing for the next frame is performed from the processing of step S101.

  When the speech segment determination unit 104 determines that the frame of interest is included in the silence segment, the response target utterance determination unit 107 determines the duration of the silence segment that continues from the previous speech segment to the frame, and the silence segment duration threshold. And compare. If the duration of the silence interval is greater than the silence interval duration threshold, the response target utterance determination unit 107 determines that the immediately preceding voice interval is a response target. In other cases, the response target utterance determination unit 107 determines that the response is suspended (step S104).

  When the response target utterance determination unit 107 determines that the immediately preceding voice segment is the response target speech, the response target utterance determination unit 107 acquires and outputs the sound signal of the corresponding voice segment from the voice segment sound signal storage unit 105 (step S105). That is, the response target utterance determination unit 107 outputs response target speech.

  When the response target utterance determination unit 107 determines that the response is on hold, processing for the next frame is performed from the processing of step S101.

  Whether the response target is determined to be a response hold or not is determined by using the following formulas (2) and (3) when the length of the silence to be continued is L and the threshold for the silence interval duration is Th: Determined.

L> Th The voice section immediately before is determined as the voice to be responded, and the response (2)
L ≦ Th Hold the response ・ ・ ・ Formula (3)

  The response target utterance determination unit 107 considers the previous speech segment as a non-response subject utterance and rejects it when the speech segment starts while the response hold is continued. In addition, since there is no immediately preceding voice section in the first silence period, even if Expression (1) is satisfied, the response target voice is not determined.

  It is conceivable that the silence duration duration threshold value is obtained in advance by an experiment by a user or the like. For example, as shown in FIG. 3, the user rarely determines that the conversation between humans is a response target voice erroneously based on the utterance to the outside of the system, for example, the duration of the silence interval in the conversation between humans and the frequency thereof. The silence duration duration threshold is determined so that

  The system performs processing on the response target voice output by the response target voice determination device. For example, it is conceivable that the system performs speech recognition on the response target speech and returns a response corresponding to the text output from the speech recognition to the user. In addition, the system performs voice recognition for the voice section that is put on hold, and holds it as a temporary voice recognition output text, and holds it when the response target voice determination device determines that it is a response target voice. It is also conceivable to validate the provisional speech recognition output text.

  As described above, in the present embodiment, it is determined whether or not the voice is a response target voice by using the duration of the silence period after the utterance. Therefore, the user only has to wait for a system response after speaking. Therefore, according to the present embodiment, it is possible to provide a voice interaction user interface with high usability.

Embodiment 2. FIG.
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings.

  FIG. 4 is a block diagram showing a configuration of the second embodiment of the response target speech determination device according to the present invention.

  As shown in FIG. 4, in addition to the configuration of the first embodiment, the response target speech determination device according to the second embodiment includes a speech feature amount calculation unit 201, a speech feature amount threshold / weight storage unit 202, And a second silence interval duration threshold value calculation unit 203.

  The voice feature quantity calculation unit 201 receives the sound signal determined as the voice section by the voice section determination unit 104, calculates the voice feature quantity, and outputs it.

  The voice feature amount threshold / weight storage unit 202 stores a predetermined threshold value and weight for a voice feature amount (pre-determined threshold and weight for prosody features).

  The second silence interval duration threshold value calculation unit 203 receives the audio feature value, the threshold value and weight of the audio feature value, and the silence interval duration threshold value, and calculates and outputs the second silence interval duration length.

  Note that the audio feature quantity calculation unit 201 and the second silence interval duration threshold calculation unit 203 are realized by, for example, a computer that operates according to a response target audio determination program. In this case, the CPU reads the response target sound determination program, and operates as the sound feature amount calculation unit 201 and the second silence interval duration threshold value calculation unit 203 according to the program. Further, the audio feature amount calculation unit 201 and the second silence interval duration threshold value calculation unit 203 may be realized by separate hardware.

  The voice feature value threshold / weight storage unit 202 is specifically realized by a storage device such as an optical disk device, a magnetic disk device, or a memory included in the response target voice determination device.

  Next, the operation of this embodiment will be described.

  FIG. 5 is a flowchart showing the operation of the second embodiment of the response target speech determination device. FIG. 6 is an explanatory diagram showing the frequency distribution of speech feature values extracted from utterances made to the system and speech feature values extracted from utterances made outside the system.

  The processing in steps S201 to S203 is the same as the processing in steps S101 to S103 in the first embodiment.

  After the speech segment determination unit 104 determines that the frame of interest is included in the speech segment and executes the process of step S203, the speech feature amount calculation unit 201 calculates the speech feature amount from the sound signal of the frame of interest. (Step S204). The voice feature amount is a feature amount extracted from the sound signal. The speech feature amount is a feature amount widely used in speech recognition such as amplitude power in a speech section, average value, variance value, maximum value, minimum value, formant frequency, and cepstrum. It is also conceivable to use the duration of the speech segment as the speech feature amount.

  Next, the second silence interval duration threshold value calculation unit 203 calculates the audio feature value calculated in step S204, the threshold value and weight of the audio feature value stored in the audio feature value threshold / weight storage unit 202, A second silence interval duration threshold is calculated from the silence interval duration threshold stored in the silence interval duration threshold storage unit 106 (step S205). Specifically, when the silence interval duration threshold is Th1, the speech feature amount is F, the speech feature amount threshold is ThF, and the speech feature amount weight is wF, the second silence interval duration threshold calculation unit 203 is as follows. The second silence interval continuation length threshold Th2 is calculated by the equation (4) shown.

Th2 = Th1−sgn × wF × (F−ThF) (4)

  Here, sgn takes a value of +1 or -1. Sgn is set to +1 when using a feature value that increases with respect to the utterance to the system, and sgn is set to -1 when using a feature value that decreases with respect to the utterance to the system. For example, when the user speaks to the system, it is considered that the voice tends to become louder. Therefore, when the loudness of the voice is used as the feature amount, sgn becomes +1. In addition, since it is considered that the user tends to speak slowly with respect to the system, sgn is -1 when the speaking speed is used as the feature amount.

  Note that the response target speech determination apparatus may perform the processing from step S203 to step S205 for each frame, or may be performed collectively at the end of the speech interval (the start end of the silence interval).

  After the second silence interval duration threshold calculation unit 203 executes the process of step S205, the process for the next frame is performed from the process of step S101.

  When the speech segment determination unit 104 determines in step S202 that the frame of interest is included in the silence segment, the response target utterance determination unit 107 does not compare the silence segment duration length with the silence segment duration threshold value. And the second silence interval duration threshold are compared, and it is determined whether to respond with the immediately preceding speech interval as a response target or to suspend the response (step S206).

  When the response target utterance determination unit 107 determines that the immediately preceding voice segment is the response target speech, the response target utterance determination unit 107 acquires and outputs the sound signal of the corresponding voice segment from the voice segment sound signal storage unit 105 (step S207). That is, the response target utterance determination unit 107 outputs response target speech.

  When the response target utterance determination unit 107 determines that the response is on hold, processing for the next frame is performed from the processing of step S201.

  It is conceivable that the voice feature amount threshold is obtained in advance by an experiment by a user or the like. For example, as shown in FIG. 6, the user can calculate from the frequency of voice feature values extracted from utterances made to the system and the frequency of voice feature values extracted from utterances made to systems other than the system. The voice feature amount threshold is determined so as to separate them as much as possible.

  As described above, in the present embodiment, when it is determined that there is a high possibility that the user has made an utterance to the system based on the comparison between the voice feature quantity and the voice feature quantity threshold, the second silence is set. The section duration threshold is made shorter than the silence section duration threshold. Thereby, it is possible to determine the response target voice with little delay. On the other hand, if it is determined by the comparison between the voice feature quantity and the voice feature quantity threshold value that the user is unlikely to utter the system, the second silence period duration threshold value is set as the silence period duration threshold value. Make it longer than the threshold. As a result, the response hold time becomes longer, and the possibility that the response is rejected increases when the user makes the next utterance while holding the response hold.

  As described above, in this embodiment, it is determined whether or not the voice is a response target voice based on the second silence interval duration threshold value and the duration of the silence interval after the utterance. Therefore, according to the present embodiment, it is possible to determine the response target voice in consideration of the volume of the voice when the user speaks to the system, the speaking speed, etc., and to provide a voice interaction user interface with higher usability it can.

Embodiment 3. FIG.
Hereinafter, a third embodiment of the present invention will be described with reference to the drawings.

  FIG. 7 is a block diagram showing the configuration of the third embodiment of the response target speech determination apparatus according to the present invention.

  As shown in FIG. 7, in addition to the configuration of the first embodiment, the response target audio determination device according to the third embodiment includes a video signal acquisition unit 301, a video feature amount calculation unit 302, and a video feature amount threshold value. A weight storage unit 303 and a second silence interval duration threshold calculation unit 304 are provided.

  The video signal acquisition unit 301 acquires a video signal using a camera or the like.

  The video feature amount calculation unit 302 receives the video signal and information about the section determined by the audio section determination unit 104 (hereinafter referred to as section information), and the video of the audio section, the silence section, or both sections. Calculate and output feature values. The section information includes a voice section continuation length and the like.

  The video feature amount threshold / weight storage unit 303 stores a predetermined threshold and weight of the video feature amount.

  The second silence interval duration threshold calculation unit 304 receives the image feature amount, the silence interval duration threshold value, the image feature amount threshold value, and the image feature amount weight, and calculates and outputs the second silence interval duration threshold value. .

  Note that the video signal acquisition unit 301, the video feature amount calculation unit 302, and the second silence interval duration threshold calculation unit 304 are realized by, for example, a computer that operates according to a response target audio determination program. In this case, the CPU reads the response target audio determination program, and operates as the video signal acquisition unit 301, the video feature amount calculation unit 302, and the second silence interval duration threshold calculation unit 304 according to the program. In addition, the video signal acquisition unit 301, the video feature amount calculation unit 302, and the second silence interval duration threshold value calculation unit 304 may be realized by separate hardware.

  The video feature amount threshold / weight storage unit 303 is specifically realized by a storage device such as an optical disk device, a magnetic disk device, or a memory included in the response target sound determination device.

  Next, the operation of this embodiment will be described.

  The video feature amount calculation unit 302 calculates a video feature amount from the video signal obtained by the video signal acquisition unit 301. As the video feature amount, a face, a line of sight, a body orientation, and the like can be considered. Note that the video feature amount calculation unit 302 may obtain an average of the video feature amount in the audio section. In addition, the video feature quantity calculation unit 302 obtains the time during which the user is facing the system or the ratio between the time when the user is facing the system and the duration of the audio section based on the video feature quantity. May be.

  The second silence interval duration threshold calculation unit 304 calculates a second silence interval duration threshold from the silence interval duration threshold and the video feature amount threshold. Assuming that the silence interval duration threshold is Th0, the video feature amount is F, the video feature amount threshold is ThF, and the video feature amount weight is wF, the second silence interval duration threshold Th2 is calculated by the following equation (5). Is done.

Th2 = Th1−wF × (F−ThF) (5)

  The response target utterance determination unit 107 uses the calculated second silence interval duration threshold Th2 as a response target or responds to the previous speech interval by the same method as in the first embodiment. To determine.

  The video feature amount can be calculated even in the silent section. When the video feature quantity calculation unit 302 calculates the feature quantity separately for the voice section and the silence section, Expression (5) is transformed into Expression (6).

Th2 = Th1−wFv × (Fv−ThFv) −wFs × (Fs−ThFs) (6)

  Here, Fv indicates the video feature amount of the audio section. ThFv indicates a video feature amount threshold value in an audio section. wFv indicates the video feature amount weight of the audio section. Fs indicates the video feature amount in the silent section. ThFs indicates a video feature amount threshold value in the silent section. wFs indicates the weight of the video feature amount in the silent section.

  Note that the audio section determination unit 104 may determine the audio section and the silence section using video in addition to determining the audio section and the silence section from only the sound signal. For example, it is conceivable to analyze the size and movement of the mouth using an image, and when the mouth is small or when the mouth is not moving, it is determined that it is a silence interval.

  As described above, in this embodiment, using the second silence interval duration threshold value calculated based on the video feature amount, whether to respond with the immediately preceding audio interval as a response target or whether to hold the response Determine. Accordingly, it is possible to determine the response target sound in consideration of the user's face, line of sight, body orientation, and the like. Thereby, when the user wants to respond to the system, the user only has to keep an eye on the system direction during and after speaking and be silent for a while. Therefore, according to this embodiment, a voice interaction user interface with higher usability can be provided.

Embodiment 4 FIG.
Hereinafter, a fourth embodiment of the present invention will be described with reference to the drawings.

  FIG. 8 is a block diagram showing a configuration of the fourth embodiment of the response target speech determination device according to the present invention.

  As shown in FIG. 8, in addition to the configuration of the first embodiment, the response target speech determination apparatus according to the fourth embodiment includes a dialog activity level calculation unit 401, a dialog activity level threshold / weight storage unit 402, And a second silence interval duration threshold value calculation unit 403.

  The conversation activity level calculation unit 401 calculates a conversation activity (conversation activity) from the temporal relationship between a plurality of voice intervals and silence intervals determined by the voice interval determination unit 104. In the present embodiment, the dialogue activity level calculation unit 401 uses the frequency of switching between the voice period and the silence period as the temporal relationship between the plurality of voice periods and the silence period. The range of the speech interval and the silence interval used for calculation by the conversation activity level calculation unit 401 is, for example, the speech interval and the silence interval existing in a certain fixed time, going back from the frame of interest.

  The dialogue activity level threshold / weight storage unit 402 stores a threshold and a weight of a predetermined dialogue activity level.

  The second silence interval duration threshold calculation unit 403 receives the silence interval duration threshold stored in the silence interval duration threshold storage unit 106, the dialogue activity, the dialogue activity threshold and the weight, and Calculate and output the second silence interval duration threshold.

  Note that the dialogue activity level calculation unit 401 and the second silence interval duration threshold value calculation unit 403 are realized by, for example, a computer that operates according to a response target voice determination program. In this case, the CPU reads the response target voice determination program and operates as the dialogue activity calculation unit 401 and the second silence interval duration threshold calculation unit 403 according to the program. Moreover, the dialogue activity level calculation unit 401 and the second silence interval duration threshold value calculation unit 403 may be realized by separate hardware.

  The interactive activity threshold / weight storage unit 402 is specifically realized by a storage device such as an optical disk device, a magnetic disk device, or a memory included in the response target voice determination device.

Embodiment 5. FIG.
Hereinafter, a fifth embodiment of the present invention will be described with reference to the drawings.

  FIG. 9 is a block diagram showing the configuration of the fifth exemplary embodiment of the response target speech determination device according to the present invention.

  As shown in FIG. 9, in addition to the configuration of the first embodiment, the response target speech determination device according to the fifth embodiment includes a multiple sound signal acquisition unit 501, an input sound signal cutout unit 502, and a speech segment determination. Unit 503, dialogue activity level calculation unit 504, dialogue activity level threshold / weight storage unit 505, and second silence interval duration threshold value calculation unit 506.

  The multi-sound signal acquisition unit 501 acquires multi-channel input sound signals for each speaker and direction using a plurality of sound input devices, for example, microphones.

  The input sound signal cutout unit 502 receives input sound signals of a plurality of channels and outputs sound signals cut out for each frame.

  The voice segment determination unit 503 includes a plurality of voice segment detection units (VAD (voice activity detection) 1 to VADN). The voice section determination unit 503 receives the sound signals of a plurality of channels cut out for each frame using VAD1 to VADN and the threshold value related to the voice section determination stored in the voice section determination threshold storage unit 103, and It is determined for each channel whether the frame is included in a voice segment (active voice segment) or a silence segment (silence segment) in which no voice exists.

  The conversation activity level calculation unit 504 calculates the conversation activity (conversation activity) from the temporal relationship between the voice period and the silence period obtained by the voice period determination unit 503.

  The interactive activity threshold / weight storage unit 505 stores a predetermined threshold and weight of interactive activity.

  The second silence interval duration threshold value calculation unit 506 receives the silence interval duration threshold value stored in the silence interval duration threshold storage unit 106, the dialogue activity level, the dialogue activity level threshold / weight, Calculate and output the second silence interval duration threshold.

  In the present embodiment, the dialogue activity level calculation unit 504 calculates the dialogue activity level using temporal relationships between a plurality of voice intervals and silence intervals obtained for a plurality of channels. As the temporal relationship used by the conversation activity level calculation unit 504 for calculation, for example, the frequency of switching between a speech segment and a silence segment, the frequency of speech segment overlap of multiple channels, the speaker entropy calculated from the occupancy rate of the speech, etc. It is done.

  The occupancy rate of the utterance is the ratio of the length of the voice section in which a specific speaker is speaking out of the sections determined to be voice sections regardless of the speaker. The speaker entropy S is calculated using the following equation (7).

S = ― Σ_i P_i log P_i (7)

  Here, P_i represents the occupancy rate of the utterance of the i-th speaker. For example, if there are three speakers, the first speaker (speaker A) has a voice duration of 5 seconds, the second speaker (speaker B) has a voice duration of 2 seconds, and the third When the duration of the speech section of the speaker (speaker C) is 1 second, P_1, P_2, and P_3 are values shown in equations (8) to (10). Further, the speaker entropy S is calculated by the equation (11).

P_1 = 5/8 ・ ・ ・ Formula (8)
P_2 = 2/8 ... Formula (9)
P_3 = 1/8 ... Formula (10)
S =-P_1 log (P_1)-P_2 log (P_2)-P_3 log (P_3) ... Formula (11)

  The multiple sound signal acquisition unit 501, the input sound signal cutout unit 502, the voice segment determination unit 503, the dialogue activity calculation unit 504, and the second silence segment duration threshold calculation unit 506 are, for example, according to a response target voice determination program. Realized by an operating computer. In this case, the CPU reads the response target voice determination program, and according to the program, the multiple sound signal acquisition unit 501, the input sound signal cutout unit 502, the voice segment determination unit 503, the dialogue activity calculation unit 504, and the second silence interval continuation It operates as the long threshold value calculation unit 506. Further, the multiple sound signal acquisition unit 501, the input sound signal cutout unit 502, the voice segment determination unit 503, the dialogue activity calculation unit 504, and the second silence segment duration threshold calculation unit 506 are realized by separate hardware. Also good.

  The interactive activity threshold / weight storage unit 505 is specifically realized by a storage device such as an optical disk device, a magnetic disk device, or a memory included in the response target voice determination device.

Embodiment 6. FIG.
The sixth embodiment of the present invention will be described below with reference to the drawings.

  FIG. 10 is a block diagram showing the configuration of the sixth embodiment of the response target speech determination apparatus according to the present invention.

  As illustrated in FIG. 10, the response target speech determination device according to the sixth exemplary embodiment includes a maximum delay time storage unit 601 in addition to the configuration of the second exemplary embodiment.

  Note that the maximum delay time storage unit 601 is specifically realized by a storage device such as an optical disk device, a magnetic disk device, or a memory included in the response target sound determination device.

  In this embodiment, the response target utterance determination unit 107 compares the second silence interval duration threshold calculated by the second silence interval duration threshold calculation unit 203 with the maximum delay time. In this embodiment, the maximum delay time is the maximum value of the delay time until the system responds to the user. Then, the response target utterance determination unit 107 rejects the immediately preceding voice segment as a non-response target when the second silence interval duration threshold is longer than the maximum delay time. Specifically, when the length of the continuous silence is L, the second silence interval duration threshold is Th, and the maximum delay time is D, the response target utterance determination unit 107 uses the following equation (12): From Equation (14), it is determined whether or not the immediately preceding speech segment is a response target speech.

D <Th It is determined that the voice segment immediately before is outside the response target voice and is rejected. Expression (12)
D ≧ Th and L ≦ Th Response hold ... Formula (13)
D ≧ Th and L> Th The speech segment immediately before is determined as the response target speech, and the response (14)

  When the response target utterance determination unit 107 determines that the immediately preceding voice section is outside the response target voice and rejects it, the response target voice determination device or system may notify the user. For example, it may be possible to notify a message that “the utterance could not be heard”.

  Note that a plurality of the above embodiments can be used in combination.

  Next, the outline of the present invention will be described. FIG. 11 is a block diagram showing the minimum configuration of the response target speech determination device according to the present invention. FIG. 12 is a block diagram showing another minimum configuration of the response target speech determination device according to the present invention.

  As shown in FIG. 11, the response target speech determination device according to the present invention detects that the silence interval duration, which is the length of the silence interval following the utterance by the user, is longer than a predetermined silence interval duration threshold. The utterance is determined to be the utterance made by the user to the system, the silence interval duration is detected to be shorter than the silence interval continuation length threshold, and the utterance is made by the user other than the system. A response target utterance determination unit 11 (corresponding to the response target utterance determination unit 107 shown in FIG. 1) that is determined to be uttered is provided.

  According to such a configuration, since it is determined whether or not the voice is a response target voice using the duration of the silence period after the utterance, the user only has to wait for a system response after the utterance. Therefore, it is possible to provide a voice interaction user interface with high usability.

  Also, a voice segment determination unit 12 (corresponding to the voice segment determination unit 104 shown in FIG. 4 or FIG. 7) that determines a voice segment and a silence segment with respect to a time series of sound signals collected by the voice input device, and a voice. A feature amount calculation unit 13 that extracts feature amounts corresponding to a section, a silence section, or both sections (corresponding to the audio feature amount calculation unit 201 shown in FIG. 4 or the video feature amount calculation unit 302 shown in FIG. 7). And the second silence interval duration threshold value from the feature amount, a predetermined feature amount threshold value and weight, and a predetermined first silence interval duration threshold value (corresponding to the silence interval duration threshold value). The second silence interval duration threshold calculation unit 14 (corresponding to the second silence interval duration threshold calculation unit 203 shown in FIG. 4 or the second silence interval duration threshold calculation unit 304 shown in FIG. 7). And the response target utterance determination unit 11 It determined using the silence interval duration threshold may be performed. According to such a configuration, it is possible to determine the response target voice with less delay without degrading the accuracy of the determination of the response target voice, and it is possible to provide a voice interaction user interface with higher usability.

  Further, the feature quantity calculation unit 13 may extract one or more voice feature quantities from the sound signal corresponding to the voice section, and the second silence section duration threshold calculation unit 14 may use the voice feature quantity. According to such a configuration, it is possible to determine the response target voice in consideration of the loudness and speaking speed when the user speaks to the system, and it is possible to provide a voice interaction user interface with higher usability.

  Also, the feature amount calculation unit 13 extracts a video feature amount from the video corresponding to the audio section, extracts a video feature amount from the video corresponding to the silence section, or extracts a video feature amount from the video corresponding to both sections. And the second silence interval duration threshold value calculation unit 14 may use one or more video feature quantities. According to such a configuration, it is possible to determine the response target speech in consideration of the user's face, line of sight, body orientation, and the like. Thereby, when the user wants to respond to the system, the user only has to keep an eye on the system direction during and after speaking and be silent for a while. Therefore, a voice interaction user interface with higher usability can be provided.

  Also, as shown in FIG. 12, the dialogue activity level calculation unit 15 (corresponding to the dialogue activity level calculation unit 504 shown in FIG. 9) is provided, and the voice segment determination unit 12 (corresponding to the voice segment determination unit 503 shown in FIG. 9). .) Determines a voice interval and a silence interval for each of the time series of the sound signals of a plurality of channels collected by a plurality of sound input devices, and the dialogue activity calculation unit 15 determines the sound intervals and the silence intervals of the plurality of channels. And the second silence interval duration threshold value calculation unit 14 (corresponding to the second silence interval duration threshold value calculation unit 506 shown in FIG. 9) calculates the dialogue activity level. Alternatively, the second silence interval duration threshold value may be calculated from a predetermined dialogue activity threshold value and weight and a predetermined first silence interval duration threshold value. According to such a configuration, it is possible to determine the response target speech in consideration of the dialogue activity calculated from the temporal relationship between the plurality of speech intervals and the silence intervals.

  The system also includes a maximum delay time storage unit 16 (corresponding to the maximum delay time storage unit 601 shown in FIG. 10) that stores in advance a maximum delay time that is the maximum delay time until the system responds to the user. The utterance determination unit 11 may detect that the second silence interval continuation length threshold is longer than the maximum delay time, and may reject the utterance by the user as a non-response subject utterance. According to such a configuration, for example, when it is difficult to hear the user's voice, that is, when the voice section cannot be recognized correctly, the user can be notified of that.

DESCRIPTION OF SYMBOLS 11 Response object utterance determination part 12 Voice area determination part 13 Feature-value calculation part 14 2nd silence period duration threshold value calculation part 15 Dialogue activity calculation part 16 Maximum delay time storage part 101 Input sound signal acquisition part 102 Input sound signal extraction Unit 103 speech segment determination threshold storage unit 104 speech segment determination unit 105 speech segment sound signal storage unit 106 silence segment duration threshold storage unit 107 response target utterance determination unit 201 speech feature amount calculation unit 202 speech feature amount threshold / weight storage unit 203 Second silence interval duration threshold calculation unit 301 Video signal acquisition unit 302 Video feature amount calculation unit 303 Video feature amount threshold / weight storage unit 304 Second silence interval duration threshold calculation unit 401 Dialog activity calculation unit 402 Dialog activity Threshold value / weight storage unit 403 Second silence interval duration threshold value calculation unit 501 Multiple sound signal acquisition unit 502 Input sound signal Number cutout section 503 Voice section determination section 504 Dialogue activity calculation section 505 Dialogue activity threshold value / weight storage section 506 Second silence section duration threshold calculation section 601 Maximum delay time storage section

Claims (10)

Detects that the duration of the silence interval, which is the length of the silence interval following the utterance by the user, is longer than a predetermined silence interval duration threshold, and determines that the utterance is the utterance made by the user to the system A response target utterance determination unit that detects that the silence interval duration is shorter than the silence interval duration threshold and determines that the utterance is made by a user other than the system. A characteristic response target speech determination device. A voice segment determination unit that determines a voice segment and a silence segment with respect to a time series of sound signals collected by the voice input device;
A feature amount calculation unit that extracts feature amounts corresponding to the speech section, the silence section, or both sections;
Second silence interval duration threshold value calculation for obtaining a second silence interval duration threshold value from the feature amount, a predetermined feature amount threshold value and weight, and a predetermined first silence interval duration threshold value With
The response target speech determination apparatus according to claim 1, wherein the response target utterance determination unit performs determination using the second silence interval duration threshold. The feature quantity calculation unit extracts one or more voice feature quantities from the sound signal corresponding to the voice section,
The response target sound determination apparatus according to claim 2, wherein the second silence interval duration threshold value calculation unit uses the sound feature amount. The feature quantity calculation unit extracts video feature quantities from the video corresponding to the audio section, extracts video feature quantities from the video corresponding to the silent section, or extracts video feature quantities from the video corresponding to both sections. ,
The response target sound determination device according to claim 2, wherein the second silence interval duration threshold value calculation unit uses one or more of the video feature amounts. With a dialogue activity calculator
The voice section determination unit determines a voice section and a silence section for each of the time series of sound signals of a plurality of channels collected by a plurality of voice input devices,
The interaction activity calculation unit calculates interaction activity from the temporal relationship between the voice interval and the silence interval of the plurality of channels,
A second silence interval duration threshold value calculation unit calculates a second silence from the dialogue activity level, a predetermined dialogue activity threshold value and weight, and a predetermined first silence interval duration threshold value. The response target speech determination apparatus according to claim 2, wherein a section duration threshold is calculated. A maximum delay time storage unit that stores in advance a maximum delay time that is the maximum delay time until the system responds to the user;
The response target utterance determination unit detects that the second silence interval duration threshold is longer than the maximum delay time, and rejects the utterance by the user as a non-response target utterance. The response target voice determination device according to any one of the preceding claims. Detects that the duration of the silence interval, which is the length of the silence interval following the utterance by the user, is longer than a predetermined silence interval duration threshold, and determines that the utterance is the utterance made by the user to the system And detecting that the silence interval duration is shorter than the silence interval duration threshold, and determining that the utterance is an utterance made by a user other than the system. Method. For the time series of the sound signal collected by the voice input device, determine the voice interval and silence interval,
Extracting feature quantities corresponding to the speech section, the silence section, or both sections;
A second silence interval duration threshold is determined from the feature amount, a predetermined feature amount threshold and weight, and a predetermined first silence interval duration threshold.
The response target speech determination method according to claim 7, wherein the utterance determination process by the user is performed using the second silence interval duration threshold value. On the computer,
Detects that the duration of the silence interval, which is the length of the silence interval following the utterance by the user, is longer than a predetermined silence interval duration threshold, and determines that the utterance is the utterance made by the user to the system A response target voice for detecting that the silence duration duration is shorter than the silence duration duration threshold value and executing a process of determining that the speech is made by a user other than the system. Judgment program. On the computer,
A process for determining a voice section and a silence section for a time series of sound signals collected by the voice input device;
A process of extracting feature amounts corresponding to the speech section, the silence section, or both sections;
A process for obtaining a second silence interval duration threshold from the feature amount, a predetermined feature amount threshold and weight, and a predetermined first silence interval duration threshold;
The response target speech determination program according to claim 9, wherein a process for determining a speech by a user is executed using the second silence interval duration threshold.

Images