JP2011172081A - Amplifying conversation method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an amplifying conversation device, method and program, in which a sound volume can be automatically controlled for each microphone not degrading follow-up performance, without increasing a calculation amount of an adapted filter or a memory amount. <P>SOLUTION: The amplifying conversation device includes a main channel estimating section 430 for estimating one or more microphones to be a main channel, an addition section 440 for adding a collected sound as a sound signal, an echo cancel section 450 for canceling echo of the added sound signal, a sound detecting section 461 for detecting sound from the echo-canceled sound signal, a superposition gain calculating section 462 for setting an initial value based on the main channel when sound is detected and calculating a superposition gain using the initial value and the echo-canceled sound signal, a gain superposing section 463 for superposing the superposition gain and the echo-canceled sound signal, and a superposition gain storing section 464 for respective channels for storing the superposition gain corresponding to the main channel. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a voice call method, a voice call device, and a voice call program for automatically controlling a voice level output from a speaker in accordance with a voice level inputted to a microphone in remote communication using a sound signal. .

As a function of the voice communication device used in the remote communication conference, there is an echo canceller that prevents the generation of acoustic echo using an adaptive filter as shown in FIG. Some echo cancellers do not require initial learning for an adaptive filter by using a voice switch as shown in FIG.
For example, in the echo canceller shown in FIG. 1, the adaptive filter 914 estimates a transmission path corresponding to the acoustic echo path 913 from the speaker 911 to the microphone 912. The adaptive filter 914 synthesizes the pseudo echo signal based on the estimated transmission path, and deletes the echo signal by subtracting the pseudo echo signal from the echo signal.
For example, in the echo canceller shown in FIG. 2, the BG adaptive filter 924 performs an adaptive process, and the echo cancellation filter 925 estimates a pseudo echo with a coefficient transferred from the BG adaptive filter 924. The voice switch control circuit 926 mainly inserts a loss when the power is turned on, and reduces the amount of loss to be inserted as the adaptive filter converges (see Non-Patent Document 1).

Moreover, there is an automatic volume control device as shown in FIG.
In the automatic volume control apparatus of FIG. 3, first, the input signal is cut out by the cutout window and stored in the buffer 931. Next, the audio signal identification circuit 932 identifies whether or not the input signal is an audio signal. The volume amplification circuit 933 determines the amplification factor of the input signal based on the identification result of the audio signal identification circuit 932. The waveform superimposing circuit 934 superimposes a cosine window based on the amplification factor determined by the volume amplifier circuit 933 on the input signal, and uses the superimposition result as an output signal. The algorithm used for the audio signal identification circuit 932 is a hidden Markov model, vector quantization, a neural network, or the like (see Patent Document 1).

JP-A-8-250944

Edited by Nobuhiko Kitawaki, `` Future Netto Technology Series Digital Voice / Audio Technology '', published by Ohmsha, pp218-255.

  For example, in a telecommunication conference, when there are multiple speakers and a microphone is installed for each speaker, the distance between the speaker and the microphone adjacent to the speaker, and the speaker's voice volume, Due to the difference, there is a problem that the output sound level varies from microphone to microphone, making it difficult to hear the output sound.

In order to solve this problem, it is necessary to perform automatic volume control individually for each of the plurality of microphones.
However, while the adaptive filter operates by estimating the acoustic echo path as a linear system, the automatic volume control as described above performs non-linear processing, and thus the control cannot be arranged between the adaptive filter and the acoustic echo path.
Therefore, automatic volume control needs to be performed on the transmission network side rather than the adaptive filter. For this reason, when performing automatic volume control individually for each of a plurality of microphones, the same number of sound signals as the microphones must be input to the adaptive filter, and the amount of computation and memory of the adaptive filter are The problem of increasing in proportion to the number of microphones arises.

In order to solve this new problem, it is necessary to add input sound signals from a plurality of microphones to one and then input to the adaptive filter, thereby reducing the amount of computation and memory of the adaptive filter.
However, since the automatic volume control as described above takes time to control, the follow-up performance deteriorates when the microphone (main channel, the same applies hereinafter) that collects sound frequently changes due to the change of the speaker. A further problem arises.

  The present invention has been made to solve these problems. Each of the plurality of microphones does not deteriorate the following performance even if the main channel is frequently changed without increasing the calculation amount and memory amount of the adaptive filter. It is an object to provide a loudspeaking method, a loudspeaker device, and a loudspeaker program that enable individual automatic volume control.

The voice call method of the present invention includes main channel estimation processing, addition processing, echo cancellation processing, voice detection processing, superposition gain calculation processing, gain superposition processing, and channel-specific superposition gain storage processing.
In the main channel estimation process, one or more of N (where N is an integer of 2 or more) microphones are estimated as the main channel. In the addition process, the sound collected by the microphone is added as a sound signal. In the echo cancellation process, the echo of the added sound signal is canceled. In the sound detection process, a sound is detected from the sound signal subjected to the echo cancellation process. In the superposition gain calculation process, when speech is detected, an initial value is set based on the estimated main channel, and a superposition gain is calculated using the initial value and the sound signal subjected to the echo cancellation process. . In the gain superimposing process, the superimposing gain calculated by the superimposing gain calculating process and the sound signal subjected to the echo canceling process are superimposed. In the channel-specific superimposing gain storage process, the superimposing gain calculated by the superimposing gain calculation process is stored in the channel-specific superimposing gain storage unit in association with the estimated main channel.
Here, as the initial value set in the superposition gain calculation process, a past superposition gain stored for each main channel in the superposition gain storage unit for each channel can also be used during the initial value setting process.
Further, the microphones are divided into M groups (where M is an integer of 2 or more) so that each group has a plurality of microphones, and the sound collected by the microphones of each group is described for each group. Processing can also be performed.

  According to the present invention, input sound signals from a plurality of microphones are added by addition processing, and echo cancellation processing is performed on the added sound signals, so that the calculation amount and memory amount of the adaptive filter are not increased. Furthermore, one or more of the N microphones are estimated as main channels, and a superposition gain is calculated using the initial value set based on the estimated main channel and the sound signal subjected to echo cancellation, and superimposition is performed. Therefore, even if the main channel is frequently changed, the following performance is not deteriorated. Therefore, the loudspeaking method of the present invention enables individual automatic volume control for each of a plurality of microphones without increasing the computation amount of the adaptive filter and the amount of memory, and the follow-up performance does not deteriorate due to frequent changes in the main channel. To do.

The figure explaining a prior art example. The figure explaining a prior art example. The figure explaining a prior art example. The block diagram which shows the structure of the loudspeaker apparatus which concerns on Example 1 and the modification 1. FIG. 3 is a flowchart showing the operation of the loudspeaker device according to the first embodiment. 9 is a flowchart showing the operation of a loudspeaker device according to Modification 1. FIG. 4 is a block diagram illustrating a configuration of a loudspeaker device according to a second embodiment. 9 is a flowchart showing the operation of the loudspeaker device according to the second embodiment.

  Hereinafter, embodiments of the present invention will be described in detail.

  With reference to FIGS. 4 and 5, the loudspeaker device and the loudspeaker method according to Embodiment 1 of the present invention will be described. FIG. 4 is a block diagram illustrating the configuration of the loudspeaker apparatus 400 according to the first embodiment. FIG. 5 is a flowchart illustrating the operation of the loudspeaker apparatus 400 according to the first embodiment. The loudspeaker apparatus 400 includes a speaker 911, N microphones (where N is an integer of 2 or more, and the same applies hereinafter), a main channel estimation unit 430, an addition unit 440, and an echo cancellation unit 450. And an automatic volume controller 460. The automatic volume control unit 460 includes a voice detection unit 461, a superimposition gain calculation unit 462, a gain superposition unit 463, and a channel-specific superimposition gain storage unit 464.

A sound signal input from the network 470 to the loudspeaker apparatus 400 is loudened from the speaker 911 via the echo canceling unit 450. Microphones 912-1 to 912 -N convert the collected sound into sound signals and input the sound signals to the adding unit 440 and the main channel estimating unit 430, respectively.
The main channel estimation unit 430 estimates one or more of the N microphones 912-1 to 912 -N as the main channel (S 530). As a main channel estimation method, there is a method in which the time signal powers of sound signals output from the microphones 912-1 to 912-N are compared, and the microphone having the maximum power is used as the main channel. However, it is not necessary to limit to the estimation method, and any other method may be used as long as it can be estimated with good followability with respect to the change of speakers. As long as the estimated main channel is 1 or more and less than N, any number may be used. When the main channel is estimated by comparing the time signal power, the time channel power is determined in descending order. Several microphones may be used as main channels, or all microphones that output time signal power exceeding a predetermined threshold may be used as main channels. Also in this case, as described above, other methods may be used as long as they can be estimated with good followability with respect to the change of speakers.
The adding unit 440 adds the sound collected by the microphone as a sound signal (S540). The echo cancel unit 450 cancels the echo of the added sound signal (S550). The echo canceling unit 450 may be a general type echo canceller of FIG. 1, or may be of a type using a voice switch as shown in FIG.

  The voice detection unit 461 detects whether or not there is a voice from the sound signal from which the echo is canceled (S561). The sound detection method may be a method of determining that sound is detected when a sound signal level exceeding a threshold is input. The threshold used for detection is determined in advance from the ambient noise level. However, it is not necessary to limit to the detection method, and other methods can be used as long as they can be detected with good follow-up with respect to the change of the speaker, similarly to the method in the main channel estimation unit 430.

The superposition gain calculation unit 462 sets an initial value based on the estimated main channel when speech is detected (S562a), and superimposes using the initial value and the sound signal subjected to echo cancellation processing. The gain is calculated (S562b). If no sound is detected, the output sound level is not adjusted. Details of the calculation method of the superposition gain will be described below. When the main channel is estimated to be Cn at time t, the calculated superposition gain is G _{Cn [t]} , the input sound signal level from the main channel Cn is V _{Cn [t]} , and the sound signal output to the network 470 is Assuming that the target output sound signal level is V _d , the initial value is G _i , and the time constant is α, the superposition gain G _{Cn [t]} is obtained by adding the forgetting factor,
G _{Cn [t]} = (1−α) × V _d / V _{Cn [t]} + α × G _i
In the calculation.
In the above equation, the ratio V _d / V _{Cn [t]} between the target output sound signal level and the input sound signal level is used, but other methods may be used.

Here, as the initial value G _i , the superposition gain G _{Cn [t−k]} of the corresponding main channel in the past k times can be used, where k is a positive integer. Therefore, when the initial value G _i is a superposition gain at k times in the past, the superposition gain at time t is
G _{Cn [t]} = (1−α) × V _d / V _{Cn [t]} + α × G _{Cn [t−k]}
In the calculation.
Further, in order to calculate the initial value G _i at time t, it is also possible to use a plurality of superimposed gain from time t-k to a time t-1. In this case, when the main channel is Cn at time t, the initial value G _i and the superposition gain G _{Cn [t]} are time constants α _t , α _t−1 , α _{t−2 ,.} And is calculated by the following formula.

G _{Cn [t]} = α _t × V _d / V _{Cn [t]} + G _i
However,

  The calculation method of the superposition gain is not limited to the method expressed by the above formula, and a method of smoothing and calculating the superposition gain change with time using the superposition gain of the corresponding main channel in the past is used. Also good.

When the main channel estimation unit 430 uses a plurality of main channels to be estimated, the estimated main channels are n of C1, C2, C3,..., Cn, and the input sound signal level from each main channel is V _{When C1 [t]} , V _{C2 [t]} , V _{C3 [t]} ,..., V _{Cn [t]} , the superposition gains G _{C1, C2, C3,.}

Can be calculated. Here, a (n) is a correction coefficient depending on the number of main channels n, and takes a value larger than 1.

The gain superimposing unit 463 superimposes the superimposing gain calculated by the superimposing gain calculating unit 462 and the sound signal subjected to the echo canceling process (S563). The channel-specific superposition gain storage unit 464 stores the superposition gain calculated by the superposition gain calculation processing (S562a, S562b) in association with the estimated main channel (S564). For example, when the channel is the main channel Cn at time t, the channel-specific superposition gain storage unit 464 stores the superposition gain G _{Cn [t]} in the storage area corresponding to the channel Cn. At this time, for the channel not estimated as the main channel, the superposition gain stored in the channel-specific superposition gain storage unit 464 is held as it is at time t, and is read out at any time during calculation after time t. Shall be used. Further, in order to calculate the initial value G _i, in the case of using a plurality of superimposed gain from time t-k to a time t-1, for example, in the storage area corresponding to the channel Cn, time t-k is The superposition gain G _{Cn [t−k]} is stored. Accordingly, the number of superposition gains to be stored is N for each channel, k for each time, and the total number stored is N × k.

Further, when there are N microphones and n main channels to be estimated (n is an integer of 2 or more), the superposition gain is a combination of selecting n main channels from the N microphones, _N C _n ways are calculated. Accordingly, the channel-by-channel superposition gain storage unit 464 stores a total of _N C _n superposition gains in a storage area corresponding to each channel combination.
In addition, when a plurality of superposition gains from time t-k to time t-1 are used to calculate the initial value G _i , the total number of superposition gains to be stored is _N C _n × k. .
The superposition gain stored in the channel-specific superposition gain storage unit 464 in this manner is read and used as needed by the superposition gain calculation unit 462 when calculating an initial value or a new superposition gain. Loudspeaker 400 sets a value 1 in advance in each storage area of channel-specific superposition gain storage section 464 when power is turned on. Each time the superposition gain is calculated, the superposition gain in the storage area corresponding to the main channel is updated or stored in the storage area corresponding to each time.

  According to the present embodiment, the adding unit 440 adds a plurality of input sound signals (S540), and the echo canceling unit 450 performs echo cancellation (S550), so that the calculation amount and memory amount of the adaptive filter are reduced. There is no increase. Further, the main channel estimation unit 430 estimates one or more of the N microphones as main channels (S530), and the superposition gain calculation unit 462 sets an initial value based on the estimated main channel (S562a), The superposition gain is calculated using the initial value and the sound signal subjected to echo cancellation (S562b), and the gain superposition unit 463 superimposes the superposition gain on the sound signal (S563). However, the following performance is not deteriorated. Therefore, the loudspeaker device and the loudspeaker method according to the present embodiment do not increase the amount of calculation of the adaptive filter and the amount of memory, and the automatic performance for each of the plurality of microphones does not deteriorate due to frequent changes of the main channel. Allows volume control.

  Further, in the present embodiment, the main channel estimation unit 430 estimates a plurality of microphones as the main channel, so that automatic volume control can be appropriately performed even when the speaker voice is divided and input to the plurality of microphones. it can. Furthermore, in the present embodiment, the superposition gain calculation unit 462 uses the superposition gain calculated in the past k times as an initial value when calculating the current superposition gain, so the superposition gain calculated in the past is reflected. Automatic volume control can be performed. Furthermore, since the superposition gain calculation unit 462 multiplies the initial value by a forgetting factor, it is possible to arbitrarily set the degree of reflecting the superposition gain calculated in the past.

[Modification 1]
A first modification of the first embodiment will be described with reference to FIGS.
In the first modification, the weight gain calculation unit 462 ′ sets the superposition gain stored in the channel-specific superposition gain storage unit 464 at the time of initial value calculation as an initial value (S662a). Specifically, the initial value G _i is set to the superposition gain G _{Cn [t−1]} stored in the channel-specific superposition gain storage unit 464 at time t−1. Specifically, the superposition gain G _{Cn [t]} at the time t and the main channel Cn is calculated by the following equation using the time constant α and the target output sound signal level V _d .
G _{Cn [t]} = (1−α) × V _d / V _{Cn [t]} + α × G _{Cn [t−1]}
The calculated superposition gain G _{Cn [t]} is stored in the channel-specific superposition gain storage unit 464 corresponding to the main channel, and when the main channel is estimated to be n again after time t, the initial value G _i Is used for a new superposition gain calculation.
Since the present modification has such a configuration, the same effects as those of the first embodiment can be obtained.

With reference to FIGS. 7 and 8, a loudspeaker device and a loudspeaker method according to Embodiment 2 of the present invention will be described. FIG. 7 is a block diagram illustrating the configuration of the loudspeaker apparatus 700 according to the second embodiment. FIG. 8 is a flowchart illustrating the operation of the loudspeaker apparatus 700 according to the second embodiment. In the second embodiment, the microphones are divided into M groups (where M is an integer of 2 or more, and the same shall apply hereinafter) so that each group has a plurality of microphones. The processing described in Example 1 is performed for each group.
The loudspeaker apparatus 700 uses N _m (where m is a positive integer, N _m is an integer greater than or equal to 2 and the same applies below) microphones 912-m-1 to N _m in the speaker 911 and the m th microphone group. A group of M microphones, a main channel estimation unit 430-1 to M for each microphone group, an addition unit 440-1 to M for each microphone group, an echo cancellation unit 750, and an automatic for each microphone group Volume adjustment units 460-1 to 460-M are provided.
The automatic volume control units 460-1 to 460 -M of the loudspeaker communication device 700 are, for each microphone group, voice detection units 461-1 to M, superimposition gain calculation units 462-1 to M, and gain superposition units 463-1 to M. And channel-specific superposition gain storage units 464-1 to 464-1M.

A sound signal input from the network 470 to the loudspeaker apparatus 700 is loudened from the speaker 911 via the echo canceling unit 750. The microphones 912-1-1 to 912 -M-N _M convert the collected sound into sound signals and input the sound signals to the addition units 440-1 to 440 -M and the main channel estimation units 430-1 to 430 -M, respectively.
The main channel estimation units 430-1 to 430-1 to M estimate one or more microphones of each group as main channels (S830-1 to M). The adders 440-1 to M add the sounds collected by the microphones as sound signals for each group (S540-1 to M). The echo cancel unit 750 cancels the echo of the sound signal added for each group (S550-1 to M). The sound detection units 461-1 to M detect whether or not there is sound from the sound signal whose echo is canceled for each group (S561-1 to M). Superimposition gain calculators 462-1 to 462-1 set initial values based on the estimated main channel when speech is detected (S562a-1 to M562), and performs echo cancellation processing with these initial values. The superposition gain is calculated using the obtained sound signal (S562b-1 to M). The gain superimposing units 463-1 to 463 -M superimpose the superimposing gain calculated by the superimposing gain calculating units 462-1 to M and the sound signal subjected to the echo cancellation processing for each group (S <b> 563-1 to M). The channel-specific superposition gain storage units 464-1 to 464-M store the superposition gains calculated by the superposition gain calculation processing (S562a-1 to M, S562b-1 to M) in association with the estimated main channel ( S564-1 to M).

  Since the present embodiment has such a configuration, the same effects as those of the first embodiment can be obtained. Furthermore, in the present embodiment, since the processing is performed for each of the M groups, individual automatic volume control for each of the plurality of microphones is performed without degrading the followability with respect to input sounds from more microphones. It can be carried out. Furthermore, since a sound signal is output for each microphone group, output from the speaker corresponding to each microphone group is possible, and the number of speakers on the output side can be expanded.

Claims (5)

A main channel estimation process for estimating one or more of N (where N is an integer of 2 or more) microphones as a main channel;
An addition process of adding the sound collected by the microphone as a sound signal;
Echo cancellation processing for canceling echo of the sound signal added in the addition processing;
A sound detection process for detecting a sound from the sound signal subjected to the echo cancellation process;
When the voice is detected by the voice detection process, an initial value is set based on the estimated main channel, and a superposition gain is set using the initial value and the sound signal subjected to the echo cancellation process. Superimposing gain calculation processing for calculating
Gain superimposition processing for superimposing the superposition gain calculated by the superimposition gain calculation processing and the sound signal subjected to the echo cancellation processing;
Channel-by-channel superposition gain storage processing for storing the superposition gain calculated by the superposition gain calculation processing in a channel-specific superposition gain storage unit in association with the estimated main channel;
Having
A voice call method characterized by the above. A voice call method according to claim 1, wherein
The initial value of the superposition gain calculation process is the superposition gain corresponding to the estimated main channel stored in the channel-specific superposition gain storage unit during the initial value setting process;
A voice call method characterized by the above. 3. The process according to claim 1, wherein the microphones are divided into M groups (M is an integer of 2 or more) so that each group has a plurality of microphones, and the sound collected by the microphones of each group is processed. To do the
A voice call method characterized by the above. A main channel estimation unit that estimates one or more of N (where N is an integer of 2 or more) microphones as a main channel;
An adder for adding the sound collected by the microphone as a sound signal;
An echo cancellation unit for canceling echo of the sound signal added in the addition process;
A sound detection unit that detects sound from the sound signal subjected to echo cancellation by the echo cancellation unit;
When the voice is detected by the voice detection unit, an initial value is set based on the estimated main channel, and a superposition gain is set using the initial value and the sound signal subjected to the echo cancellation processing. A superposition gain calculation unit for calculating
A gain superimposing unit that superimposes the superimposing gain calculated by the superimposing gain calculating process and the sound signal subjected to the echo canceling process;
A channel-specific superimposition gain storage unit that stores the superposition gain calculated by the superimposition gain calculation unit in a channel-specific superimposition gain storage unit in association with the estimated main channel;
Having
A voice communication device characterized by the above.   A loudspeaker program for giving a command to the computer to execute the loudspeaker method according to any one of claims 1 to 3.

Images