JP2006039108A - Prescribed speaker speech output device and prescribed speaker determination program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a prescribed speaker speech output device and a prescribed speaker determination program that can precisely extract only speech data of a prescribed speaker from speech data containing a crosstalk component with a small arithmetic quantity. <P>SOLUTION: A crosstalk speech recognizing device 1 is equipped with a speech data input means 2 of inputting speech data (x) and (y) from microphones Mx and My provided for speakers X and Y respectively, a frame extracting means 3 of extracting frames from the speech data (x) and (y), speech data power calculation parts 4a and 4b which calculate power levels of the frames, a cross-correlation coefficient calculating means 5 of calculating cross-correlation coefficients of the frames of the two speech data (x) and (y), a speaker speech determination means 7 of determining whether the frames are the speech data of the speakers X and Y based upon the frame power levels and cross-correlation coefficients, and an attenuator 8 which outputs a frame determined as the speech data of the speaker X or Y. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  In the present invention, when a plurality of speakers utter sound toward each microphone, the sound data of the speaker corresponding to the microphone and the sound data of other speakers output from each microphone, The present invention relates to a technique for outputting only voice data of a speaker corresponding to a microphone from voice data including

  Conventionally, voice recognition for the purpose of automatically subtitled broadcast programs has been put into practical use (for example, see Non-Patent Document 1). In this technology, based on a news program manuscript (electronic manuscript) created in advance with text data, the voice of the announcer who read out the manuscript with a part of the electronic manuscript read out is recognized and corrected. By doing so, subtitles are generated. Then, for example, by performing speech recognition using an acoustic model that depends on speakers such as men and women (modeled phoneme features), the recognition rate of speech recognition can be improved.

  In addition, when a plurality of speakers such as conversations alternately speak, a microphone provided in the vicinity of the speaker corresponding to each speaker is a speaker near the microphone (hereinafter referred to as a specific speaker). The voice data output from the microphone includes the voices of a plurality of speakers because the voice (crosstalk component) of other speakers is also collected. There is a technique for extracting only the voice of a specific speaker from voice data including the voices of a plurality of speakers (see, for example, Non-Patent Document 2). In this technology, when the input power (power) of the voice data input from the microphone is small, it is determined as a crosstalk component, and when the power is high, it is determined as voice data of a specific speaker, By attenuating the crosstalk component, only the target speaker's voice can be extracted.

Further, as another method for extracting only the voice of a specific speaker, a technique for calculating a crosstalk component using a transfer characteristic estimated from a cross-correlation coefficient and canceling the crosstalk component is disclosed. (Refer nonpatent literature 3).
Satoshi Imai, 3 others, “Speech recognition system for automatic subtitle conversion of news program”, Spoken Language Information Processing Technical Report, October 17, 1998, 23-11, p. 59-64 DPR-522: BSS Audio Manual, p. 18-25 Masaaki Masaya and two others, “Anti-Crosstalk Noise Canceller Based on Nonlinear Sequential Least Squares”, IEICE Transactions, February 2002, A Vol. J85-A, no. 2, p. 162-169

  However, in order to use a language model tailored to the speaker at the time of speech recognition, it must be the speech data of only this speaker, and it is recognized when speech data of other speakers are included. There was a problem that the rate would decrease. In addition, when a microphone is installed corresponding to each of a plurality of speakers and each voice data is recognized as voice, other speakers' voices are also recognized in addition to the voices of the specific speakers corresponding to the microphones. There is a problem that duplicate recognition results are output.

  Further, in the method of extracting only the voice data of a specific speaker based on the power level of the voice data, if there is a difference in the relative voice volume of each speaker, the voice volume difference of the speaker is eliminated. Therefore, there is a difference in the amplification factor of each audio data in the amplifier provided in each microphone. For this reason, the difference in amplification factor reverses the power ratio between the voice data of the specific speaker and the crosstalk component, and the power of the crosstalk component becomes larger than the power of the specific speaker, resulting in false detection. was there. In addition, the method for estimating the transfer characteristic requires a relatively large amount of calculation.

  The present invention has been made to solve the above-described problems of the prior art, and is a specific story that can extract only the voice data of a specific speaker from voice data including a crosstalk component with a small amount of calculation and high accuracy. It is an object to provide a speaker voice output device and a specific speaker determination program.

  In order to solve the problem, the specific speaker voice output device according to claim 1 inputs voice data from a microphone provided for each speaker, and outputs the voice data from at least one voice data. A specific speaker voice output device for outputting voice data of a speaker corresponding to a microphone, wherein voice data input means, frame extraction means, power calculation means, cross-correlation coefficient calculation means, and speaker voice determination And a voice data output means.

  According to such a configuration, the specific speaker voice output device inputs the voice data obtained by converting the voice of the speaker from the microphone provided for each speaker by the voice data input unit, and the voice is input by the frame extraction unit. A frame having a predetermined data length is extracted from each audio data input from the data input means. Then, the specific speaker voice output device calculates the power level of the frame output from the frame extraction means by the power calculation means, and uses one of the plurality of voice data by the cross-correlation coefficient calculation means. Reciprocal relationship indicating correlation between frames in which the time axis of the other audio data frame is shifted by a predetermined time width with respect to the time axis of the target frame which is the frame of the audio data. Calculate the number.

  Here, a microphone is provided for each speaker, and a voice uttered by a certain speaker is first input to a microphone provided for the speaker that is closest to the speaker. Then, the other microphones are delayed and input with a time difference corresponding to the difference in distance from the speaker to each microphone.

  Therefore, when the target frame is a frame including the voice data of the speaker input from the microphone corresponding to the speaker, the cross-correlation coefficient between the target frame and other frames is A large value is obtained when the time axis of another frame is advanced by the time difference with respect to the time axis. If the target frame is not a frame of audio data input from the microphone corresponding to the speaker, the cross-correlation coefficient with the frame of audio data input from the microphone corresponding to the speaker is The value is large when the time axis of the frame of the audio data input from the microphone corresponding to the speaker is delayed by the time difference with respect to the time axis of the target frame.

  Then, the specific speaker voice output device uses the speaker voice determination unit to calculate the power level of each voice data frame calculated by the power calculation unit and the cross correlation coefficient calculated by the cross correlation coefficient calculation unit. Among these, the lead cross-correlation coefficient that is a cross-correlation coefficient shifted in the direction of advancing the time axis of the other audio data frame by a predetermined time width with respect to the time axis of the target frame, and the other Based on the delayed cross-correlation coefficient, which is a cross-correlation coefficient shifted in the direction of delaying the time axis of the frame of the audio data for each predetermined time width, the voice corresponding to the audio data of the target frame is It is determined whether it is audio data.

  Note that, when the target frame power is greater than the power of the other audio data frame, the speaker voice determination means determines that the speaker's voice input to the microphone that has output the target frame has a different level. Since it is larger than the voice of the speaker input to the microphone, it can be determined that the voice of the target frame is the voice of the specific speaker. Further, the speaker voice determination means determines whether the input is made before or after the other frame based on the lead cross-correlation coefficient and the delay cross-correlation coefficient. It can be determined whether the target frame is voice data of a specific speaker who is a speaker corresponding to the microphone that has output the voice data of the target frame. Then, the target frame determined to be the voice data of the specific speaker by the speaker voice determination unit is output by the voice data output unit.

  Thus, the specific speaker voice output device extracts a frame from each of the voice data input from the plurality of microphones, and determines whether each of the frames of at least one voice data is the voice data of the specific speaker. Thus, it is possible to output voice data only for a specific speaker.

  Further, the specific speaker voice output device according to claim 2 is the specific speaker voice output device according to claim 1, wherein the speaker voice determination means performs the advance mutual processing for each of the other voice data. When the difference between the sum of the correlation coefficients and the sum of the delayed cross-correlation coefficients exceeds a threshold, the target frame is determined to be the voice data of the specific speaker.

  Thus, the specific speaker voice output device, when the difference between the sum of the lead cross-correlation coefficients and the sum of the delay cross-correlation coefficients exceeds a threshold, the voice of the speaker corresponding to the voice data of the target frame However, it can be determined that the microphone corresponding to the speaker is input before the microphone that has output other audio data, and the target frame can be determined to be the audio data of the specific speaker.

  Furthermore, the specific speaker determination program according to claim 3 inputs voice data from a microphone provided for each speaker, and a speaker corresponding to a microphone that outputs the voice data from at least one of the voice data. The computer is caused to function as voice data input means, frame extraction means, power calculation means, cross-correlation coefficient calculation means, speaker voice determination means, and voice data output means.

  According to this configuration, the specific speaker determination program inputs the voice data from the microphone provided for each speaker by the voice data input unit, and the voice data input by the voice data input unit by the frame extraction unit. A frame having a predetermined data length is extracted from each of the above. Then, the power calculation means calculates the magnitude of the power of the frame output from the frame extraction means, and the cross correlation coefficient calculation means calculates the target frame which is a frame of one audio data among the plurality of audio data. For each of the other audio data, a cross-correlation coefficient indicating a correlation between frames obtained by shifting the time axis of the other audio data frame by a predetermined time width is calculated.

  Further, the time axis of the target frame among the magnitude of the power of each voice data frame calculated by the power calculation unit and the cross correlation coefficient calculated by the cross correlation coefficient calculation unit by the speaker voice determination unit. On the other hand, a lead cross-correlation coefficient that is a cross-correlation coefficient shifted in a direction to advance the time axis of the frame of the other audio data every predetermined time width, and a time axis of the frame of the other audio data Based on a delayed cross-correlation coefficient that is a cross-correlation coefficient shifted in the direction of delay for each predetermined time width, the target frame is identified as a speaker corresponding to the microphone that output the audio data of the target frame It is determined whether the data is the voice data of the speaker. The target frame determined by the voice data output means to be the voice data of the specific speaker is output by the speaker voice determination means.

  Accordingly, the specific speaker determination program extracts a frame from each of the audio data input from the plurality of microphones, and determines whether each of the frames of at least one audio data is the audio data of the specific speaker. Thus, voice data of only a specific speaker can be output.

  The specific speaker voice output device and the specific speaker determination program according to the present invention have the following excellent effects.

  According to the first or third aspect of the invention, only the voice data of the specific speaker can be output from the voice data including the crosstalk component. Therefore, for example, when recognizing the sound of a talk program or the like to generate subtitles, the same speaker's sound is input to a plurality of microphones, thereby preventing the same sound from being recognized as a plurality of sounds. be able to. In addition, voice recognition can be performed with a high recognition rate by voice recognition of voice data based on an acoustic model corresponding to a specific speaker.

  Further, since it is determined whether the target frame is the voice data of the specific speaker based on the magnitude of the frame power and the cross-correlation coefficient, the determination can be made with high accuracy. Then, instead of calculating the crosstalk component included in the audio data, it is determined whether it is a crosstalk component for each frame or only the voice of a specific speaker, and the crosstalk component is removed. Therefore, it is not necessary to perform a complicated calculation for calculating the crosstalk component, and the processing amount can be reduced and the processing speed can be improved.

  According to the second aspect of the present invention, based on the difference between the sum of the lead cross-correlation functions and the sum of the delay cross-correlation functions, the voice of a certain speaker can be compared between the microphone corresponding to the target frame and another microphone. Since it is determined which is input first, it is possible to easily determine whether the target frame is voice data of a specific speaker.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Here, the present invention is applied to the case of recognizing the sound of a program or the like in which a plurality of speakers, such as a conversation, alternately utter sound, and is configured as a crosstalk sound recognition device.

[Configuration of Crosstalk Speech Recognition Device (Specific Speaker Speech Output Device)]
With reference to FIG. 1, the structure of the crosstalk speech recognition apparatus 1 which is embodiment of this invention is demonstrated. FIG. 1 is a block diagram showing the configuration of a crosstalk speech recognition apparatus according to the present invention. The crosstalk voice recognition device 1 converts voice data x (x (t (t)) obtained by converting voices of the speaker X and the speaker Y input to the microphones Mx and My provided to the speaker X and the speaker Y, respectively. )) And y (y (t)) are input from the microphones Mx and My, and the voice recognition result of the voice of the speaker X only and the voice recognition result of the voice of the speaker Y only are output. Here, t indicates the time from the predetermined time on the time axis (hereinafter referred to as the time axis) of the time when the audio corresponding to the audio data is input to the microphones Mx and My. Here, the crosstalk speech recognition apparatus 1 includes speech data input means 2, frame extraction means 3, frame power calculation means 4, cross-correlation coefficient calculation means 5, smoothing processing means 6, speaker voice determination. Means 7, an attenuator 8, storage means 9, speech recognition means 10, and speech recognition result output means 11 are provided.

  Here, the crosstalk voice recognition apparatus 1 includes microphones Mx and My that convert voices of the speakers X and Y into voice data x and y, and voice data x and y input from the microphones Mx and My. Are attenuated at a desired attenuation rate by the operation of a speaker or the like, and fader units FUx and FUy that are output, and audio data x and y input from the fader units FUx and FUy are amplified at a desired amplification rate and amplified. Amplifiers Ax and Ay that output the audio data x and y to the crosstalk speech recognition apparatus 1 are connected to the outside, and are further A / D converted by an A / D (Analog to Digital) converter (not shown). In addition, time-series audio data x and y are input.

  The microphones Mx and My correspond to the speakers X and Y, respectively, the microphone Mx is installed at a position closer to the microphone My when viewed from the speaker X, and the microphone My is more than the microphone Mx when viewed from the speaker Y. It is installed in a close position. Then, when the speaker X and the speaker Y emit voices alternately, the voice H (XX) of the speaker X and the voice H (YX) of the speaker Y are alternately input to the microphone Mx. In addition, the voice H (XY) of the speaker X and the voice H (YY) of the speaker Y are alternately input to the microphone My.

  The voice data input means 2 is for inputting a plurality of voice data from the outside. Here, the audio data input means 2 inputs the audio data x and y from the amplifiers Ax and Ay. The voice data input means 2 includes a voice data input unit 2a and a voice data input unit 2b.

  The audio data input unit 2a inputs audio data x from the amplifier Ax. The voice data x includes voice data of the speaker X voice H (XX) (speaker X voice data) and voice data of the speaker Y voice H (YX) (crosstalk component). Yes. The audio data x input here is output to the audio data frame extraction unit 3a of the frame extraction means 3.

  The audio data input unit 2b inputs the audio data y from the amplifier Ay. The voice data y includes voice data (crosstalk component) of the voice H (XY) of the speaker X and voice data (voice data of the speaker Y) of the voice H (YY) of the speaker Y. Yes. The audio data y input here is output to the audio data frame extraction unit 3b of the frame extraction means 3.

  The frame extraction unit 3 extracts a frame having a predetermined data length from each of the audio data x and y input from the audio data input unit 2. Here, the frame extraction means 3 includes an audio data frame extraction unit 3a and an audio data frame extraction unit 3b. The data length of the frame can be set to an arbitrary length in which the product of the time width of the frame on the time axis and the sound speed is larger than the distance between the speaker X and the speaker Y. Here, a frame having a data length of 400 points of sampling data is extracted at 16 kHz sampling.

  The audio data frame extraction unit 3a extracts a frame having a predetermined data length from the audio data x input from the audio data input unit 2a. The extracted frame is output to the audio data power calculation unit 4a, the cross-correlation coefficient calculation unit 5 and the attenuator 8a of the frame power calculation unit 4.

  The audio data frame extraction unit 3b extracts a frame having a predetermined data length from the audio data y input from the audio data input unit 2b. The extracted frame is output to the audio data power calculation unit 4b of the frame power calculation unit 4, the cross correlation coefficient calculation unit 5, and the attenuator 8b.

  The frame power calculation means 4 calculates the magnitude (frame power) of the frame input from the frame extraction means 3, and based on this frame power, the fader units FUx and FUy are input from the microphones Mx and My. It is determined whether audio data has been output to the amplifiers Ax and Ay. The frame power calculation unit 4 includes an audio data power calculation unit 4a, an audio data power calculation unit 4b, and an FU state determination unit 4c.

  The audio data power calculation unit (power calculation means) 4a calculates the frame power of the frame extracted from the audio data x by the audio data frame extraction unit 3a. The voice data power calculation unit (power calculation means) 4b calculates the frame power of the frame extracted from the voice data y by the voice data frame extraction unit 3b. The frame power calculated here is output to the FU state determination unit 4c and the smoothing processing means 6.

  Here, the audio data power calculation units 4a and 4b of the frame power calculation unit 4 calculate the sum of squares of the amplitudes of the respective points of the frame as the frame power of the frame. Here, the frame power P (l, x) of the audio data x calculated by the audio data power calculation unit 4a and the frame power P (l, y) of the audio data y calculated by the audio data power calculation unit 4b are: Is represented by the following formula (1). Here, the frame extraction means 3 extracts frames with a time width N on the time axis from the audio data x (t) and y (t) every shift width M. Also, l is a frame number assigned to the frames in time series for each of the audio data x and y.

  The FU state determination unit 4c is based on the frame powers P (l, x) and P (l, y) of the audio data x and y input from the audio data power calculation units 4a and 4b. 8a, 8b) is set, or a command for calculating the cross-correlation coefficient of the frame is output to the cross-correlation coefficient calculation means 5 described later. Here, the FU state determination unit 4c determines whether the fader units FUx and FUy are ON or OFF based on the frame powers P (l, x) and P (l, y), that is, The fader units FUx and FUy determine whether the audio data x and y input from the microphones Mx and My are directly output to the crosstalk speech recognition apparatus 1 via the amplifiers Ax and Ay or are attenuated. .

Fader unit Fux, compared Fuy is background noise level P _sil when is ON (the speaker X, Y is the level of sound when not speaking), the frame power when the fader units Fux, Fuy is OFF P _FU-OFF is sufficiently small. Therefore, when the frame power P (l, x), P (l, y) is smaller than the threshold Th _{FU that} satisfies P _FU-OFF <Th _FU <P _sil , the FU state determination unit 4c performs the fader unit FUx, When FUy is OFF and the frame power P (l, x) and P (l, y) are larger than the threshold Th _FU, it can be determined that the fader units FUx and FUy are ON. .

  When either one of the fader units FUx and FUy is OFF, only the speaker (X or Y) corresponding to the fader unit FUx or FUy that is ON emits sound. The x and y frames do not contain a crosstalk component. Further, when both of the fader units FUx and FUy are OFF, neither of the speakers X and Y emits speech, so that the frames of the speech data x and y do not include a crosstalk component. Therefore, the cross-correlation coefficient calculation means 5, smoothing processing means 6, and speaker voice determination means 7 to be described later are processed for the frame to determine whether the frame includes a crosstalk component. There is no need.

  Therefore, when it is determined that one or both of the fader units FUx and FUy are OFF, the FU state determination unit 4c sets the attenuation rate of the attenuator 8 (8a, 8b) to zero, and the mutual phase A command for calculating the cross-correlation coefficient of the frame is not output to the relation number calculation means 5. As a result, the crosstalk speech recognition apparatus 1 does not perform the processing on the frame by the cross-correlation coefficient calculation unit 5, the smoothing processing unit 6, and the speaker speech determination unit 7, thereby reducing the amount of calculation. And the processing speed can be improved.

  Further, when it is determined that both of the fader units FUx and FUy are ON, the FU state determination unit 4c outputs a command for processing the frame to the cross-correlation coefficient calculation unit 5.

  The cross-correlation coefficient calculation means 5 is based on a command input from the FU state determination section 4c, and the mutual phase of the frames of the audio data x and y input from the audio data frame extraction sections 3a and 3b of the frame extraction means 3 The number of relationships is calculated. The cross-correlation coefficient calculated here is output to the smoothing processing means 6.

  The cross-correlation coefficient is obtained by shifting one time axis of two time-series functions by a predetermined time width and multiplying the two functions. When the correlation between the two functions is high, When the correlation is small, the value is relatively small. Here, the cross-correlation coefficient calculation means 5 uses the time axis of the frame of the audio data y (t) for a predetermined time for each frame of the audio data x (t) as shown in the following equation (2). The cross-correlation coefficient C (τ, l) shifted by the width τ was calculated. Note that σx (t) and σy (t) are standard deviations of the audio data x and y in the corresponding frame, and each frame is obtained from the audio data x (t) and y (t) by the frame extraction means 3. It is assumed that a time width N is extracted every shift width M on the time axis.

The smoothing means 6 receives the frame powers P (l, x) and P (l, y) and the cross-correlation coefficient C (τ, l) input from the audio data power calculation units 4a and 4b of the frame power calculation means 4. Smoothing is performed. Here, the smoothing processing means 6, as shown in equations (3) and (4), for each frame of the audio data x and y, a predetermined number (n _p ) of frames centered on each frame. Average values P ′ (l, x) and P ′ (l, y) of powers P (l, x) and P (l, y) and a predetermined number (n _c ) of frames centered on each frame Smoothing is performed by calculating an average value C ′ (τ, l) of the cross-correlation coefficient C (τ, l). As a result, it is possible to prevent unnecessary determination results from being switched in the speaker voice determination means 7 to be described later, which are caused by short pause intervals in speech such as breath breaths and noise such as paper noise. The average values P ′ (l, x) and P ′ (l, y) of the frame power calculated here and the average value C ′ (τ, l) of the cross-correlation coefficient are sent to the speaker voice judging means 7. Is output.

  The speaker voice determination means 7 determines each of the average values P ′ (l, x) and P ′ (l, y) of the frame power and the average value C ′ (τ, l) of the cross correlation coefficient. The speaker corresponding to this frame is determined, and the attenuation rate of the attenuator 8 to be described later is set.

  The voice of the speaker (X or Y) that has produced the voice is input to the microphone (Mx or My) closest to the speaker at a relatively high volume, and another microphone (My or Mx). Is input at a low volume. Therefore, the speaker voice determination means 7 outputs a frame (Mx) that outputs a frame having a larger average power P ′ (l, x), P ′ (l, y) of frames of voice data converted at the same time. Alternatively, it can be determined that the speaker corresponding to My) is the speaker who has output the sound of the frame.

  In addition, the voice of the speaker (X or Y) who emitted the voice is input relatively early to the microphone (Mx or My) closest to the speaker (X or Y), and another microphone (My or Mx) is input with a delay. Therefore, the speaker voice determination unit 7 determines that the average value C ′ (τ, l) of the cross-correlation coefficient of the frame of the voice data (x or y) is different from the voice data (y or x) for the frame to be determined. ), The speaker (X or Y) corresponding to the microphone (Mx or My) that output the frame outputs the voice of the frame when the time axis of the frame becomes larger when the time axis is shifted forward. It can be determined that the person is a person.

Therefore, here, the speaker voice determination means 7 uses the logarithm of the average value (P ′ (l, x) or P ′ (l, y)) of the frame power of the frame to be determined to calculate other voice data (y or A value (log power ratio) R (l) obtained by subtracting the logarithm of the average value (P ′ (l, y) or P ′ (l, x)) of the frame power of the frame x) is the threshold Th _R (0 <Th _R ), or the average of the leading cross-correlation coefficients that are cross-correlation coefficients when the time axis of the frame of other audio data (y or x) is shifted in the direction of advancing with respect to the frame to be judged From the sum of the values, the sum of the average values of the delayed cross-correlation coefficients that are cross-correlation coefficients when the time axis of the frame of the other audio data (y or x) is shifted in the direction of delaying with respect to the frame to be determined The subtracted value (cross-correlation difference) D (l) is the threshold Th _D (0 <Th _D ) If the speaker becomes larger than the speaker (X or Y) corresponding to the microphone (Mx or My) that output the frame, the speaker (X or Y) that outputs the sound of the frame is determined. did. The logarithmic power ratio R _x (l) and the cross-correlation difference D _x (l) for determining the frame of the audio data x, and the logarithmic power ratio R _y (l) for determining the frame of the audio data y and The cross-correlation difference D _y (l) is expressed by the following equations (5) and (6).

Then, the speaker voice determination means 7, when R _x (l) ≧ Th _R or D _x (l) ≧ Th _D , the frame of the voice data x is the voice data of the speaker X, and the voice data y Is determined to be a crosstalk component. Then, the speaker voice determination means 7 sets the attenuation rate of the attenuator 8a, which will be described later, to be sufficiently small and sets the attenuation rate of the attenuator 8b to be sufficiently large.

Further, the speaker voice determination means 7 indicates that when R _y (l) ≧ Th _R or D _y (l) ≧ Th _D , the frame of the voice data y is the voice data of the speaker Y, and the voice data x Is determined to be a crosstalk component. Then, the speaker voice determination means 7 sets the attenuation rate of the attenuator 8b, which will be described later, to be sufficiently small (for example, zero) and sets the attenuation rate of the attenuator 8a to be sufficiently large.

Note that here, the speaker voice determination means 7 determines that the determination results based on the logarithmic power ratio and the cross-correlation difference contradict each other, that is, the logarithmic power ratio R _x (l) and the cross-correlation difference D _y (l). If both exceed the threshold Th _R , Th _D , or if both the log power ratio R _y (l) and the cross-correlation difference D _x (l) exceed the threshold Th _R , Th _D , The decision result immediately before the frame is adopted. When log power ratio R _x (l), log power ratio R _y (l), cross-correlation difference D _x (l) and cross-correlation difference D _y (l) do not exceed thresholds Th _R and Th _D In addition, the determination result immediately before the frame is adopted. As a result, the speaker voice determination means 7 can prevent frequent switching of the speaker and obtain a stable detection result.

  Further, here, the speaker voice determination means 7 sets the minimum number of continuous frames that is the minimum number of frames determined to be the same speaker's voice data continuously, and after the determination result changes, By maintaining the same determination result for at least the frames having the minimum number of frames, it is possible to prevent frequent switching of speakers and obtain stable detection results.

Here, referring to FIG. 2, the voice data of the speaker or the crosstalk based on the logarithmic power ratio R _x (l) and the cross-correlation difference D _x (l) by the speaker voice judging means 7. A method for determining whether it is a component will be described. FIG. 2 is an explanatory diagram for explaining a method of determining a speaker by the speaker voice determining means, (a) is a graph showing a speaker's utterance section and a logarithmic power ratio with time, (b) Is a graph showing the change in cross-correlation over time, and (c) is a diagram showing a speaker determination result by the speaker voice determination means.

Here, when the male speaker (speaker X) and the female speaker (speaker Y) speak alternately, the volume of the male speaker is large, and the volume of the female speaker is small, Since the amplifier Ay amplifies the voice input from the microphone My more greatly, as shown in FIG. 2A, the logarithmic power ratio R _x (l) in the utterance section of the male speaker is the female story. The value may be relatively smaller than the logarithmic power ratio R _y (l) (R _y (l) = − R _x (l)) of the person's utterance interval. At this time, if the speaker voice determination means 7 determines the speaker only with the logarithmic power ratio R (l), the logarithmic power in the utterance section of the male speaker (for example, between 5 seconds and 12 seconds). The ratio R _x (l) does not exceed the threshold Th _R and erroneous determination occurs.

Here, as shown in FIG. 2B, the cross-correlation difference D _x (l) is a threshold Th in the utterance interval of a male speaker whose logarithmic power ratio R _y (l) is insufficient. _The cross-correlation difference D _y (l) (D _y (l) = − D _x (l)) exceeded the threshold Th _D in the utterance interval of the female speaker. . Thus, by performing speaker determination based on not only the logarithmic power ratio R (l) but also the cross-correlation difference D (l), as shown in FIG. It is possible to obtain a speaker determination result that is close to the speaker's utterance section.

  Returning to FIG. 1, the description will be continued. The attenuator (voice data output means) 8 is set by the FU state determination unit 4c of the frame power calculation means 4 or the speaker voice determination means 7 for the frames of the voice data x and y input from the frame extraction means 3. It is attenuated by the attenuation rate. The voice data attenuated here is output to the voice recognition means 10. Here, the attenuator 8a attenuates the frame of the audio data x input from the audio data frame extraction unit 3a and outputs it to the audio recognition means 10a, and the attenuator 8b inputs from the audio data frame extraction unit 3b. The frame of the voice data y that has been received is attenuated and output to the voice recognition means 10b. Thereby, the attenuator 8a can output only the voice data of the speaker X to the voice recognition means 10a, and the attenuator 8b can output only the voice data of the speaker Y to the voice recognition means 10b.

  The storage unit 9 stores an acoustic model necessary for speech recognition by the speech recognition unit 10 described later, and is a general storage unit such as a semiconductor memory or a hard disk. Here, the storage unit 9a stores an X acoustic model that is an acoustic model corresponding to the speaker X, and the storage unit 9b stores a Y acoustic model that is an acoustic model corresponding to the speaker Y.

  The voice recognition unit 10 recognizes the voice data input from the attenuator 8 based on the X acoustic model or the Y acoustic model stored in the storage unit 9. Here, the voice recognition means 10a recognizes the voice data input from the attenuator 8a based on the X acoustic model stored in the storage means 9a, and the voice recognition means 10b is stored in the storage means 9b. Based on the Y acoustic model, the speech data input from the attenuator 8b is recognized as speech. Then, the speaker X speech recognition result speech-recognized by the speech recognition unit 10a is output to the speech recognition result output unit 11a, and the speaker Y speech recognition result speech-recognized by the speech recognition unit 10b is the speech recognition result output unit 11b. Is output.

  As described above, the voice recognition unit 10a recognizes the voice data determined as the voice data of the speaker X by the speaker voice determination unit 7 based on the X acoustic model corresponding to the speaker X. The voice recognition means 10b recognizes the voice data determined as the voice data of the speaker Y by the speaker voice determination means 7 based on the Y acoustic model corresponding to the speaker Y. Compared to voice recognition based on an acoustic model that does not depend on the person, or voice recognition based on an acoustic model (X acoustic model or Y acoustic model) corresponding to a specific speaker for voice data including a crosstalk component High speech recognition rate.

  The voice recognition result output means 11 outputs the voice recognition result input from the voice recognition means 10. Here, the speech recognition result output unit 11 includes a speech recognition result output unit 11a and a speech recognition result output unit 11b.

  The speech recognition result output unit 11a outputs the speaker X speech recognition result input from the speech recognition means 10a to the outside. The speech recognition result output unit 11b outputs the speaker Y speech recognition result input from the speech recognition means 10b to the outside.

  By configuring the crosstalk speech recognition apparatus 1 as described above, the crosstalk speech recognition apparatus 1 is configured so that the speech data x and y input from the microphones Mx and My provided to the plurality of speakers X and Y, respectively. Is attenuated, and only the voice data of the speaker X is extracted from the voice data x input from the microphone Mx, and only the voice data of the speaker Y is extracted from the voice data y input from the microphone My. Can be extracted. And voice recognition can be performed at a high recognition rate by recognizing each voice data based on an acoustic model corresponding to each speaker.

  Further, the crosstalk speech recognition apparatus 1 of the present invention calculates a crosstalk component and determines whether it is a crosstalk component for each frame, instead of removing the crosstalk component from the input speech data. The crosstalk component is removed by attenuating the frame determined to be the crosstalk component. Therefore, the crosstalk speech recognition apparatus 1 of the present invention does not need to perform a complicated calculation for calculating the crosstalk component, and can reduce the calculation amount and improve the processing speed.

  Note that the crosstalk speech recognition apparatus 1 can also realize each unit as a function program in a computer, and can also operate the unit as a specific speaker determination program by combining the function programs.

  Further, here, two voice data x and y are inputted from the microphones Mx and My corresponding to the two speakers X and Y, the voice data of the speaker X is inputted from the attenuator 8a, and the voice data is inputted from the attenuator 8b. The voice data of the person Y is output to the voice recognition means 10a and 10b. However, the crosstalk voice recognition apparatus 1 of the present invention attenuates the crosstalk component from either one of the voice data (x or y). The voice data of only one speaker may be output.

Furthermore, the crosstalk speech recognition apparatus 1 of the present invention may input three or more speech data from microphones corresponding to each of three or more speakers. At this time, the cross-correlation coefficient calculation means 5 calculates a cross-correlation coefficient between the voice data input from the microphone corresponding to the specific speaker and each other voice data, and the speaker voice determination means 7 Each cross-correlation difference is calculated from the cross-correlation coefficient with other voice data, and when all the cross-correlation differences exceed the threshold Th _D , the frame is determined as the voice data of the specific speaker. it can.

  Here, the frame determined to be a crosstalk component by the speaker voice determination means 7 is attenuated by the attenuator 8. However, for example, the crosstalk voice recognition apparatus 1 replaces the attenuator 8 with a frame. There is provided switch means (not shown) for switching the output to either one of the frames of the voice data x and y inputted from the extraction means 3, and this switch means is set by the speaker voice judgment means 7 to the speaker (X or Y ) May be switched so as to output a frame determined to be audio data.

[Operation of crosstalk speech recognition device]
Next, referring to FIG. 3 and FIG. 4 (refer to FIG. 1 as appropriate), the crosstalk speech recognition apparatus 1 according to the present invention inputs speech data converted by the microphones Mx and My, and crosstalk from the speech data. The operation of recognizing the voice data of the speakers X and Y by removing the components will be described. FIG. 3 is a flowchart showing the operation of the crosstalk speech recognition apparatus according to the present invention. FIG. 4 shows whether the crosstalk speech recognition apparatus according to the present invention determines the speech data of the speakers X and Y for each frame or the crosstalk component, and sets the attenuation rate for attenuating the crosstalk component. 6 is a flowchart showing an operation (speaker determination / attenuation rate setting operation).

  The crosstalk speech recognition apparatus 1 receives the speech data x converted by the microphone Mx by the speech data input unit 2a of the speech data input means 2, and the speech data y converted by the microphone My by the speech data input unit 2b. (Step S11; voice data input step). The crosstalk speech recognition apparatus 1 then extracts a frame from each of the speech data x and y input in step S11 by the frame extraction means 3 (step S12; frame extraction step).

  Further, the crosstalk speech recognition apparatus 1 performs the step of speaker determination / attenuation rate setting operation described later by the frame power calculation means 4, the cross-correlation coefficient calculation means 5, the smoothing processing means 6 and the speaker voice determination means 7. For each frame extracted in S12, it is determined whether it is voice data of speakers X and Y corresponding to the microphones Mx and My that output the frame or a crosstalk component, and the crosstalk component is attenuated. The attenuation rate of the attenuators 8a and 8b is set so as to cause (step S13).

  Then, the crosstalk speech recognition apparatus 1 attenuates each frame of the speech data x with the attenuation rate set in step S13 by the attenuator 8a, and outputs the speech data of the speaker X to the speech recognition means 10a. The attenuator 8b attenuates each frame of the voice data y with the attenuation rate set in step S13, and outputs the voice data of the speaker Y to the voice recognition means 10b (step S14; voice data output step).

  Furthermore, the crosstalk speech recognition apparatus 1 uses the X acoustic model and the Y acoustic model stored in the storage units 9a and 9b, respectively, for the speech data in which the crosstalk component is attenuated in step S14 by the speech recognition units 10a and 10b. Based on the voice recognition (step S15). Then, the crosstalk speech recognition apparatus 1 outputs the speaker X speech recognition result speech-recognized by the speech recognition unit 10a in step S15 by the speech recognition result output unit 11a of the speech recognition result output unit 11, and step The speaker Y speech recognition result speech-recognized by the speech recognition means 10a in S15 is output by the speech recognition result output unit 11b (step S16), and the operation ends.

(Speaker determination / attenuation rate setting operation)
Next, referring to FIG. 4 (refer to FIG. 1 as appropriate), the crosstalk speech recognition apparatus 1 is the speech data of speakers X and Y for each frame of speech data x and y, or a crosstalk component. A speaker determination / attenuation rate setting operation (step S13 in FIG. 3) for determining whether or not there is and setting the attenuation rate of the attenuators 8a and 8b so as to attenuate the crosstalk component will be described. Here, the operation for a set of frames of audio data x and y in the same section on the time axis of audio data x and y will be described.

  First, in the crosstalk speech recognition apparatus 1, the frame power of the frames (frame number l) of the speech data x and y extracted in step S12 of FIG. P (l, x) and P (l, y) are calculated (step S31; power calculation step).

In the crosstalk speech recognition apparatus 1, the FU state determination unit 4c determines whether the fader unit FUx is ON based on the frame power P (l, x) calculated in Step S31 (Step S32). Here, FU state determination unit 4c, when frame power P (l, x) is greater than the threshold value Th _FU, it is determined that the fader unit FUx is is ON.

If the fader unit FUx is ON (Yes in step S32), the crosstalk speech recognition apparatus 1 determines whether the fader unit FUy is ON by the FU state determination unit 4c (step S33). Here, FU state determination unit 4c, frame power P (l, y) calculated in step S31 if is greater than the threshold Th _FU determines the fader unit FUy is is ON.

  When the fader unit FUy is also ON (Yes in step S33), the crosstalk speech recognition apparatus 1 uses the cross-correlation coefficient calculation means 5 to extract the speech data x, extracted in step S12 in FIG. A cross-correlation coefficient C (τ, l) is calculated by shifting one time axis of the frame of y by a predetermined time width τ (step S34; cross-correlation coefficient calculation step).

Then, the crosstalk speech recognition apparatus 1 uses the smoothing means 6 to calculate the frame powers P (l, x) and P (l, y) calculated in step S31 and the cross-correlation coefficient C calculated in step S34. (Τ, l) is smoothed (step S35). In this case, the smoothing processing means 6 uses the average values P ′ (l, x) and P ′ (l, y) of the frame powers P (l, x) and P (l, y) of the predetermined number n _p frames. ) by calculating a frame power P (l, x), P (l, y) of the carried smoothing, the cross-correlation coefficients of the frame of a predetermined number n _{c C} (τ, l) of the average value C ' (Τ, l) is calculated and the cross-correlation coefficient C (τ, l) is smoothed.

Further, the crosstalk speech recognition apparatus 1 uses the speaker speech determination means 7 to calculate the average frame powers P ′ (l, x) and P ′ (l, y) of the speech data x and y calculated in step S35. The difference between the log power ratios R _x (l) and R _y (l), which are logarithmic differences, and the sum of the average values of the lead cross-correlation coefficients and the sum of the average values of the delay cross-correlation coefficients Certain cross-correlation differences D _x (l) and D _y (l) are calculated (step S36).

Then, the crosstalk speech recognition apparatus 1 determines that the logarithmic power ratio R _x (l) calculated in step S36 by the speaker speech determination means 7 is greater than or equal to the threshold Th _R or the cross-correlation difference D _x (l ) Is greater than or equal to the threshold Th _D (step S37). If the log power ratio R _x (l) is equal to or greater than the threshold Th _R or the cross-correlation difference D _x (l) is equal to or greater than the threshold Th _D (Yes in step S37), the crosstalk speech recognition device. 1 is that the logarithmic power ratio R _y (l) calculated in step S36 by the speaker voice determination means 7 is greater than or equal to the threshold Th _R , or the cross-correlation difference D _y (l) is greater than or equal to the threshold Th _D. It is determined whether or not there is (step S38).

If the log power ratio R _y (l) is greater than or equal to the threshold Th _R or the cross-correlation difference D _y (l) is greater than or equal to the threshold Th _D (Yes in Step S38), the process proceeds to Step S46 as it is. . If the log power ratio R _y (l) is less than the threshold Th _R and the cross-correlation difference D _y (l) is less than the threshold Th _D (No in step S38), the crosstalk speech recognition device. 1, the speaker voice determination unit 7 determines that the frame of the voice data x is the voice data of the speaker X, and determines whether the determination result is the same as the determination result of the speaker of the immediately preceding frame. (Step S39). If they are not the same (No in step S39), it is determined whether the frames with the same determination result have continued beyond the minimum number of frames until the immediately preceding frame (step S40).

  If the minimum number of sustained frames is not exceeded (No in step S40), the process proceeds to step S46. Further, when the determination result of the speaker in step S38 is the same as the determination result of the immediately preceding frame (Yes in step S39), or when the same determination result continues beyond the minimum number of continuous frames (step In S40, the crosstalk speech recognition apparatus 1 sets the attenuation factor of the attenuator 8a, that is, the attenuation factor of the voice data x to zero by the speaker voice determination means 7, and the attenuation factor of the attenuator 8b. That is, the attenuation rate of the audio data y is set sufficiently large (step S41), and the operation is terminated.

When the log power ratio R _x (l) is less than the threshold Th _R and the cross-correlation difference D _x (l) is less than the threshold Th _D (No in step S37), the crosstalk speech recognition apparatus. 1 is that the logarithmic power ratio R _y (l) calculated in step S36 by the speaker voice determination means 7 is greater than or equal to the threshold Th _R , or the cross-correlation difference D _y (l) is greater than or equal to the threshold Th _D. It is determined whether or not there is (step S42).

When the log power ratio R _y (l) is equal to or greater than the threshold Th _R or the cross-correlation difference D _y (l) is equal to or greater than the threshold Th _D (Yes in step S42), the crosstalk speech recognition device. 1, the speaker voice determination means 7 determines that the frame of the voice data y is the voice data of the speaker Y, and determines whether this determination result is the same as the determination result of the speaker of the immediately preceding frame. (Step S43). If they are not the same (No in step S43), it is determined whether the frames with the same determination result have continued beyond the minimum number of frames until the immediately preceding frame (step S44).

  If the minimum number of sustained frames is not exceeded (No in step S44), the process proceeds to step S46. Further, when the determination result of the speaker in step S42 is the same as the determination result of the immediately preceding frame (Yes in step S43), or when the same determination result continues beyond the minimum number of continuous frames (step In S44, the crosstalk speech recognition apparatus 1 sets the attenuation rate of the attenuator 8a, that is, the attenuation rate of the audio data x to be sufficiently large by the speaker audio determination means 7, and the attenuation of the attenuator 8b. The rate, that is, the attenuation rate of the audio data y is set to zero (step S45), and the operation ends.

If the log power ratio R _y (l) is less than the threshold Th _R and the cross-correlation difference D _y (l) is less than the threshold Th _D (No in step S42), the crosstalk speech recognition apparatus. 1 shows that the speaker voice determination means 7 makes the attenuation rate of the attenuators 8a and 8b, that is, the attenuation rate of the voice data x and y the same as the previous frame, based on the determination result of the speaker of the previous frame. (Step S46), and the operation is terminated.

  On the other hand, when the FU state determination unit 4c determines that the fader unit FUx is not ON based on the frame powers P (l, x) and P (l, y) calculated in step S31 (No in step S32). Alternatively, when it is determined that the fader unit FUy is not ON (No in step S33), the crosstalk speech recognition apparatus 1 uses the FU state determination unit 4c to determine the attenuation rate of the attenuators 8a and 8b, that is, the audio data. The attenuation rate of both x and y is set to zero (step S47), and the operation ends.

  With the above operation, the crosstalk speech recognition apparatus 1 determines whether each frame of the speech data is a crosstalk component. If the frame is a crosstalk component, the attenuator (8a) for outputting the frame is used. Alternatively, if the attenuation factor of 8b) is set sufficiently large and is not a crosstalk component, the attenuation factor of the attenuator (8a or 8b) when outputting the frame can be set to zero.

It is the block diagram which showed the structure of the crosstalk audio | voice recognition apparatus in this invention. Explanatory drawing for demonstrating the method of determining a speaker by a speaker audio | voice determination means, (a) is a graph which showed the speaker's speech area and the logarithmic power ratio change with time, (b) is a cross correlation A graph showing the change over time of the ratio, (c) is a diagram showing the speaker determination result by the speaker voice determination means. It is the flowchart which showed operation | movement of the crosstalk audio | voice recognition apparatus in this invention. The crosstalk speech recognition apparatus according to the present invention determines whether each frame is speech data of speakers X and Y or a crosstalk component, and sets an attenuation rate for attenuating the crosstalk component (speaker) 6 is a flowchart showing a determination / attenuation rate setting operation.

Explanation of symbols

1 Crosstalk voice recognition device (specific speaker voice output device)
2 voice data input means 3 frame extraction means 4 power calculation means 4a voice data power calculation section (power calculation means)
4b Audio data power calculation unit (power calculation means)
5 Cross-correlation coefficient calculation means 6 Smoothing processing means 7 Speaker voice determination means 8a, 8b Attenuator (voice data output means)
9a, 9b Storage means 10a, 10b Speech recognition means 11 Speech recognition result output means Mx, My microphone FUx, FUy fader unit Ax, Ay amplifier

Claims (3)

A specific speaker voice output device that inputs voice data from a microphone provided for each speaker and outputs voice data of a speaker corresponding to a microphone that outputs the voice data from at least one of the voice data. ,
Voice data input means for inputting the voice data from the microphone;
Frame extraction means for extracting a frame having a predetermined data length from each of the audio data input from the audio data input means;
Power calculating means for calculating the magnitude of the power of the frame output from the frame extracting means;
With respect to the time axis of the target frame, which is a frame of one of the plurality of pieces of sound data, extracted by the frame extraction unit, for each piece of other sound data, the frame of the other sound data Cross-correlation coefficient calculating means for calculating a cross-correlation coefficient indicating a correlation between frames with the time axis shifted by a predetermined time width;
Of the magnitude of the power of each audio data frame calculated by the power calculation means and the cross-correlation coefficient calculated by the cross-correlation coefficient calculation means, the other relative to the time axis of the target frame Leading cross-correlation coefficient, which is a cross-correlation coefficient shifted in the direction of advancing the time axis of the audio data frame every predetermined time width, and the time axis of the other audio data frame every predetermined time width Based on a delayed cross-correlation coefficient that is a cross-correlation coefficient shifted in the delay direction, the target frame is voice data of a specific speaker that is a speaker corresponding to the microphone that output the voice data of the target frame. Speaker voice determination means for determining whether there is,
A specific speaker voice output device comprising: voice data output means for outputting a target frame determined to be voice data of the specific speaker by the speaker voice determination means.   If the difference between the sum of the lead cross-correlation coefficients and the sum of the delay cross-correlation coefficients exceeds a threshold for each of the other voice data, the speaker voice determination means determines that the target frame is The specific speaker voice output device according to claim 1, wherein the specific speaker voice data is determined to be voice data of the specific speaker. In order to output voice data of a speaker corresponding to a microphone that has input voice data from a microphone provided for each speaker and output the voice data from at least one of the voice data,
Voice data input means for inputting the voice data from the microphone;
Frame extraction means for extracting a frame having a predetermined data length from each of the audio data input from the audio data input means;
Power calculating means for calculating the magnitude of the power of the frame output from the frame extracting means;
With respect to the time axis of the target frame, which is a frame of one of the plurality of pieces of sound data, extracted by the frame extraction unit, for each piece of other sound data, the frame of the other sound data A cross-correlation coefficient calculating means for calculating a cross-correlation coefficient indicating a correlation between frames with the time axis shifted by a predetermined time width;
Of the magnitude of the power of each audio data frame calculated by the power calculation means and the cross-correlation coefficient calculated by the cross-correlation coefficient calculation means, the other relative to the time axis of the target frame Leading cross-correlation coefficient, which is a cross-correlation coefficient shifted in the direction of advancing the time axis of the audio data frame every predetermined time width, and the time axis of the other audio data frame every predetermined time width Based on a delayed cross-correlation coefficient that is a cross-correlation coefficient shifted in the delay direction, the target frame is voice data of a specific speaker that is a speaker corresponding to the microphone that output the voice data of the target frame. Speaker voice judgment means for judging whether or not there is,
A specific speaker determination program that functions as a voice data output unit that outputs a target frame determined to be voice data of the specific speaker by the speaker voice determination unit.

Images