JP2009094707A - Sound signal processor and sound signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To set the sound volume balance between channels automatically and appropriately for a self-amplified sound in a sound signal processor for composing a stereo-capable sound amplification communication system having a self-amplified sound output function. <P>SOLUTION: While the sound of the speaker of one's own side is input into one microphone, the ratio of each distance to two speakers from the microphone is obtained as the ratio of the level of output signals of two adaptive filters for performing echo cancellation to the sound signal collected by the microphone. Based on the obtained level ratio, the level balance between an L channel that is used for outputting to each of the L and R channels and is used as the sound of the speaker of one's own side, and the sound signal of the R channel is determined. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an audio signal processing apparatus having an audio signal processing function called so-called echo cancellation, and a method thereof.

In addition to hands-free phone calls, voice communication systems in voice conference systems and video conference systems can be used to make calls and conversations between speakers at remote locations and locations. Such a sound system is also called a loudspeaking call system, and has already been put into practical use and has become widespread.
In the above voice communication system, for example, communication terminal devices that can communicate with each other according to a predetermined communication method are arranged at a plurality of different locations. In addition, the sound picked up by the microphone on one communication terminal device side is transmitted to the other communication terminal device, and is emitted as a sound from the speaker on the other communication terminal device side that has received the sound. . As a result, conversations between speakers at remote locations are possible.

  However, in the loudspeaker communication system, the sound from the other communication terminal device side emitted from the speaker on one communication terminal device side is picked up again by the microphone on the one communication terminal device side and is collected on the other side. It is emitted as sound from the speaker of the communication terminal device. Such an operation is repeated so as to circulate (loop). As a result, for example, a so-called echo phenomenon occurs in which the voice spoken by the person other than the voice spoken by the other party is heard from the speaker in a mixed manner. If the echo sound becomes louder, the above loop is repeated infinitely and a phenomenon called howling occurs. In this way, in the voice communication system, the collected voice of the microphone is circulated by communication between the communication terminal devices, so that the voice quality is deteriorated due to echo and howling, or the telephone system becomes difficult to use. You will have a problem.

Therefore, it is known to provide an audio signal processing system for echo cancellation for the loudspeaker system.
As such signal processing for echo cancellation, one using an adaptive filter system is known.
This adaptive filter system obtains the characteristics of the impulse response for the sound (echo path) transmitted between the speaker and the microphone, and then convolves the input signal with the impulse response described above using the sound to be emitted from the speaker as an input signal. Thus, a pseudo echo signal component is generated as an output. Then, the signal component of this echo sound is subtracted from the audio signal to be collected by the microphone and transmitted to the communication terminal device on the other side. In a state where the operation of the adaptive filter system has converged, a voice with the echo sound canceled is transmitted to the communication terminal device on the other side. Therefore, the echo sound is emitted from the sound emitted from the speaker. Is no longer audible (cancelled).

In addition, there is a system that supports a stereo channel (2-channel) voice call using L (left) and R (right) as a loudspeaker call system. A configuration of an echo canceller corresponding to such a stereo-compatible loudspeaker communication system is also known.
In a stereo-compatible loudspeaker communication system, as an echo path, a path (path) from an L channel speaker to an L channel microphone, a path from an L channel speaker to an R channel microphone, an R channel speaker There are four paths: a path from the R channel to the L channel microphone and a path from the R channel speaker to the R channel microphone. Corresponding to this, the echo canceller is configured to prepare these four adaptive filter systems, and to execute echo cancellation processing for each of the four echo paths by each of these adaptive filter systems. .

  Furthermore, when using a conference system consisting of a loudspeaker system, if the place (venue) where the communication terminal device is installed is very wide, the voice of a speaker can be heard from within the same venue. It is conceivable that a conference participant who is in a remote location may be too far away to listen. In response to such a situation, as described in, for example, Patent Document 1, the communication terminal device outputs and outputs the sound collected by the same near-end microphone from the speaker. It is known to add a PA system function (self-sound output function). As a result, the near-end speaker's voice input using the microphone is amplified and output by the same near-end speaker, and the conference participants can hear the speakers in the same conference hall. It becomes possible to listen to the sound clearly with a loud sound.

  Against the background of the above technology, it is possible to consider a loudspeaker communication system that is stereo-compatible and is provided with a self-speaking sound output function. In such a loudspeaker communication system, the voice of the near-end speaker picked up by the L-channel microphone and the voice of the near-end talker picked up by the R-channel microphone can be provided on the same near-end side. The sound is emitted from the L channel and R channel speakers.

JP-A-9-307626

  As described above, when the stereo-compatible loudspeaker communication system is configured to be provided with the self-speech sound output function, how to set the left and right volume balance settings of the self-speech sound is 1 Comes up as one problem. As the simplest volume balance setting, for example, the sound collected by the L channel microphone and the sound collected by the R channel microphone are respectively transmitted from the L channel speaker and the R channel speaker. It is possible to consider a method of sorting by outputting.

However, since the self-speaking sound output function is usually required due to the reasonably large space such as a conference hall, the distance between the left and right microphones or the distance between the left and right microphones is actually increased. If the left and right volume balance of the self-sounding sound is simply distributed as described above, the original purpose of the self-sounding sound output function may not be sufficiently achieved.
For example, a voice uttered by a speaker in a wide conference hall becomes harder to hear as the distance from the speaker increases. From this, for example, the voice uttered by the speaker toward the L-channel microphone is installed at a position farther than the L-channel speaker installed at a position close to the speaker. It is preferable that loud sound is output from the channel speaker. However, if the simple distribution method described above is performed, the speaker's voice is output only from the speaker of the L channel opposite to the R channel, which is not good.
Then, the voice collected by the L channel microphone is output from the R channel speaker, and the voice collected by the R channel microphone is output from the L channel speaker. This means that the input channel and the output channel need only be switched and distributed. However, in such distribution, for example, the voice uttered toward the L channel microphone is not output from the L channel speaker at all. As mentioned above, the self-speaking sound output function assumes that the place of use is wide, so even if it is close to the speaker, no self-speaking sound will be output from the speaker. After all, it is difficult for participants in the conference nearby to hear the speaker's voice.

  From this, the left and right volume balance of the self-amplifying sound should be adjusted so that the sound is output from both the L channel and R channel speakers and at an appropriate volume. become. For such adjustment of the left / right volume balance, it is common to adopt a simple method in which the user operates the balance volume while actually listening to the self-sounding sound, by providing a manual balance volume. .

However, manual adjustment of the left / right volume balance is cumbersome for the user.
Therefore, as an invention of the present application, in an audio signal processing apparatus that is considered to constitute a multi-channel (stereo channel) -compatible loudspeaker communication system provided with a self-speaking sound output function, The purpose is to enable automatic and appropriate setting.

In view of the above-described problems, the present invention is configured as an audio signal processing apparatus as follows.
In other words, each collected sound signal obtained by collecting sound by a plurality of microphones provided corresponding to each channel constituting a predetermined multi-channel configuration is used as a desired signal, and the sound signal transmitted from the communication partner side A plurality of speaker output audio signals corresponding to each channel, which is an audio signal that has undergone a predetermined processing stage until it is emitted as a sound from a plurality of speakers corresponding to each channel of the multi-channel configuration. As a result, adaptive processing is performed so that the output signal obtained by subtracting the cancellation signal generated based on the reference signal from the desired signal is minimized, and the output signal for each channel corresponding to the multi-channel configuration Is an audio signal to be transmitted and output to the other communication device A balance setting means for inputting the output signal for each channel obtained by the adaptive processing means, changing and setting the balance of the signal level between the channels, and the output from the balance setting means It is obtained by collecting sound by a synthesizing means for synthesizing so that an output signal for each channel is included as a component of the audio signal for speaker output of the corresponding channel, and a microphone corresponding to a certain channel among a plurality of microphones. Depending on the distance between the microphone corresponding to a certain channel and the plurality of speakers corresponding to each channel constituting the multi-channel configuration when the collected sound signal has contents corresponding to the self-speaker speech input state. , Balance the signal level between channels that should be set by the balance setting means It was decided and a balance determining unit configured to constant.

  In the above description, the “self-speaker voice input state” is obtained by speaking into the corresponding microphone in the content of the collected voice signal (a level above a certain level considered valid) The voice of the speaker (self-speaker voice) is included.

In the audio signal processing apparatus having the above configuration, first, adaptive processing means corresponding to the multi-channel configuration is provided, so that in the multi-channel voice communication system, the sound collected by the microphone is circulated by communication with the communication partner side. The function of canceling the echo (echo sound of the other party's speaker) generated by performing the adaptive processing is given. In addition, since the balance setting means and the synthesizing means are provided, the sound transmitted from the communication partner side can be amplified and output from the speaker of each channel, and the sound signal processing apparatus side can complete the sound. Thus, a self-sounding sound output function corresponding to multi-channels is provided, in which sound picked up by the microphones of each channel is amplified and output from a speaker according to the volume balance between the required channels.
In addition, when the content of the collected voice signal of a certain microphone is set to the self-side speaker voice input state and the self-side speaker voice is output from the speaker, the collected sound content of the self-side speaker voice input state is collected. The balance setting means determines the balance of the signal level between the channels to be set according to the distance between the microphone that has obtained the audio signal and the speaker of each channel, that is, the self side that is output from the speaker of each channel. The volume of the speaker voice (self-speaking sound) is determined.

  With the configuration as described above, depending on the present invention, the self-speech sound output from the speaker of each channel corresponding to the distance between the microphone to which the self-speaker voice is input and the speaker of each channel, for example, the location Therefore, it is possible to automatically obtain an appropriate volume balance that makes it easy for the user in the room to listen to the self-speaker's voice. Depending on the invention of the present application, even if the distance between the installed speaker and the microphone changes due to changes in the usage environment, location, etc. of the loudspeaker communication system, an appropriate volume balance can be adapted accordingly. Will be set. Thus, for example, the user is not forced to perform troublesome work as in the case of manually adjusting the volume balance, and a more convenient loudspeaking call system can be provided.

As the best mode for carrying out the present invention (hereinafter referred to as an embodiment), the present invention is applied to an audio transmission / reception system in a video conference system (video conference system).
In general, a video conference system installs a communication terminal device for each conference hall at different locations, and transmits an image captured by a camera device and sound collected by a microphone to another communication terminal device from the communication terminal device. And an image and a sound transmitted from another communication device are received and output from the display device and the speaker, respectively. That is, the video conference system includes a video transmission / reception system that transmits / receives images to / from each other and an audio transmission / reception system that transmits / receives audio to / from each other. The present embodiment is a communication terminal device (voice communication terminal device) provided so as to transmit and receive voice as the voice transmission / reception system and to enable loudspeaking output from a speaker or the like. In addition, the audio transmission / reception system of the present embodiment has a configuration corresponding to a stereo channel of L (left) and R (right) as a multi-channel.

FIG. 1 shows a system configuration example of an audio transmission / reception system in a video conference system, which is the basis of the present embodiment.
In this case, two places A and B that are separated from each other are used as conference halls, and in each of these places A and B, the voice communication terminal apparatuses 1-1 and 1-2 that form a voice transmission / reception system are provided. Installed. These voice communication terminal apparatuses 1-1 are connected by a communication line corresponding to a predetermined communication method so that mutual communication is possible. In this case, the locations A and B are, for example, those who are separated from each other even in the same room, such that it is difficult to clearly hear the content of the other party's utterance with a voice volume of normal conversation level. It is assumed that it is quite large.
In addition, first, microphones 2-1 (L) and 2-1 (R) corresponding to the L and R channels are installed at the location A. The microphones 2-1 (L) and 2-1 (R) are for collecting the voices of conference participants in the location A, and correspond to the L channel and the R channel in the location A. Provided for each appropriate position.
Speakers 3-1 (L) and 3-1 (R) are for listening to the voices of conference participants in other places (place B) on the far end side, with place A as the near end side. Are also provided at appropriate locations in the location A corresponding to the L channel and the R channel.
Similarly, at location B, microphones 2-2 (L) and 2-2 (R), and speakers 3-2 (L) and 3-2 (R) corresponding to the L and R channels are arranged. To be done.
In the following description, when it is not necessary to distinguish between the voice communication terminal device, the microphone, and the speaker, particularly the same one in a remote place, the voice communication terminal device 1, the microphone 2 (2 (L) 3 (L)), speaker 3 (3 (L) · 3 (R)), etc.

Here, according to FIG. 2, the distances according to the arrangement of the microphones 2-1 (L), 2-1 (R), the speakers 3-1 (L), and 3-1 (R) in the place A are determined. Let's show the relationship.
First, the speakers 3-1 (L) and 3-1 (R) are arranged with respect to predetermined positions in the respective areas as the left side and the right side in the location A, and thereby the speakers 3-1 (L). And the speaker 3-1 (R) are located at a distance K0.
In addition, it is assumed that the microphone 2-1 (L) corresponding to the L channel is placed in the area on the left side at the location A together with the speaker 3-1 (L) corresponding to the same L channel. Similarly, the microphone 2-1 (R) corresponding to the R channel is placed in the area on the right side at the location A together with the speaker 3-1 (R) corresponding to the same R channel.
Thereby, the distance Kll between the microphone 2-1 (L) and the speaker 3-1 (L) is shorter than the distance Krl between the microphone 2-1 (L) and the speaker 3-1 (R). Become. Similarly, the distance Krr between the microphone 2-1 (R) and the speaker 3-1 (R) is larger than the distance Klr between the microphone 2-1 (R) and the speaker 3-1 (L). Shorter.
Note that the microphones 2-2 (L), 2-2 (R), the speakers 3-2 (L), and 3-2 (R) at the location B are also the microphones 2-1 (L), 2 at the location A. -1 (R), speakers 3-1 (L), and 3-1 (R) are arranged so as to obtain the same relationship.

Returning to FIG.
First, at the location A, the audio signals (stereo audio signals) of the L channel and the R channel obtained by the microphones 2-1 (L) and 2-1 (R) are collected by the audio communication terminal device 1. -1. The audio communication terminal apparatus 1-1 performs volume balance adjustment between the L and R channels on the input stereo audio signal as will be described later, and then performs an audio communication terminal apparatus 1- via a communication line. 2 is transmitted. The audio communication terminal device 1-2 receives the stereo audio signal transmitted as described above, and based on the received stereo audio signal, the L-3 audio signal is output from the speaker 3-2 (L). The R channel audio signal is output from the speaker 3-2 (R). Thereby, the meeting participant in the place B can listen to the voice of the meeting participant in the place A.
Similarly, stereo sound obtained by picking up the microphones 2-2 (L) and 2-2 (R) in the place B is obtained by the voice communication terminal device 1-2. 1 is transmitted. The voice communication terminal device 1-1 outputs the L channel and R channel sounds from the speakers 3L-1 and 3R-2 based on the received stereo audio signal.
In this way, the audio transmission / reception system of the video conference system performs two-way audio communication, whereby, for example, a conference participant in one certain place (near end side) and another place ( It becomes possible to have a conversation with a conference participant on the far end. In addition, in the case of this video conference system, it is assumed that there are a plurality of conference participants at each location. For this reason, all the conference participants at each location are considered to be conference participants at other locations. The speaker 3 is provided so that the voice can be heard. A system that performs two-way audio exchange using a speaker in this manner is also called a loudspeaker call system.

In addition, the voice communication terminal device 1 according to the present embodiment also uses the speakers 3 (L) and 3 (R) as loud sounds for the voice signals obtained by collecting the sounds using the microphones 2 (L) and 2 (R). It is possible to output. That is, the voice communication terminal device 1 in this case can output from the speaker 3 not only the voice transmitted from the other voice communication terminal device but also the voice collected by the microphone connected to the voice communication terminal device 1. , Has a self-speaking sound output function.
As described above, it is assumed that the locations A and B, which are the conference venues in this case, are considerably wide. However, such a self-speaking sound output function is used in a certain conference location. It is prepared for the other conference participants to listen to the sound when the participant (self-speaker) speaks (self-speaker speech) with a sufficiently loud sound.

FIG. 3 shows a configuration example of the voice communication terminal device 1. For confirmation, the voice communication terminal apparatuses 1-1 and 1-2 shown in FIG. 1 have the configuration shown in FIG. 3 in common.
The voice communication terminal device 1 includes a voice signal processing unit 11, a codec unit 12, and a communication unit 15, for example, as shown in FIG.

For the audio signal processing unit 11, the collected sound signals of the L channel and the R channel obtained by collecting the sound with the microphones 2 (L) and 2 (R), and a decoder 14 in the codec unit 12 to be described later. The output L channel (ch) and R channel (ch) audio signals are input. The audio signal processing unit 11 outputs an L / R channel audio signal after echo cancellation processing, which will be described later, to the encoder 13 in the codec unit 12 and also outputs an L / R channel audio signal to be output as a loud sound. Are output to the speakers 3 (L) and 3 (R), respectively.
Actually, it is an A / D converter that converts an analog audio signal obtained by collecting sound by the microphone 2 into a digital signal, or digital that is output from the audio signal processing unit 11. A configuration such as a D / A converter and an amplifier circuit for converting an audio signal into an analog signal, amplifying it, and outputting it to the speakers 3 (L) and 3 (R) should be provided. These parts are not shown for the sake of simplicity. In addition, each of these parts may be provided in the voice communication terminal device 1 or may be provided in a device that is external to the voice communication terminal device 1.

As described above, if the loudspeaker communication system is used as it is, phenomena such as echo and howling occur. That is, as shown in FIG. 3, the sound emitted from the speakers 3 (L) and 3 (R) into the space actually passes directly through the spatial propagation paths (echo paths) Sll, Srl, Srr, Slr. Each of the microphones 2L and 2R is reached in a state where the sound and the indirect sound are mixed. That is, the voice of the other party's speaker transmitted from the voice communication terminal device on the other side of the communication and emitted from the speaker 3 (3L · 3R) is picked up by the microphone 2 (2L · 2R). It is transmitted to the voice communication terminal device. Further, on the communication partner side, sound emitted from the speaker is further picked up by the microphone and transmitted to the voice communication terminal device here. That is, in the loudspeaker communication system, the sound once released into the space is transmitted / received in a circulating manner between the voice communication terminal devices. As a result, the sound emitted from the speaker includes a sound in which the voice he / she is currently speaking can be heard with a certain delay time. This is an echo, and howling occurs when the loop is repeated to some extent.
In view of this, in the loudspeaker communication system, an echo canceling system that eliminates and suppresses such an echo phenomenon is provided. The audio signal processing unit 11 is configured to have a signal processing function as the echo cancellation system.
Also, the audio signal processing unit 11 corresponds to the above-described self-speaking sound output function, and the audio signal obtained by collecting the sound with the microphones 2 (L) and 2 (R) is output from the speakers 3 (L) and 3 ( A signal system for self-sounding sound for output from R) is also provided.
The audio signal processing unit 11 is actually configured as a DSP (Digital Signal Processor), for example. The configuration for echo cancellation in the audio signal processing unit 11 and the signal system for self-sounding sound will be described later.

  The L channel and R channel audio signals subjected to echo cancellation processing by the audio signal processing unit 11 are input to the encoder 13 in the codec unit 12. The encoder 13 performs signal processing, such as audio compression encoding according to a predetermined method, on the input L channel and R channel audio signals, for example, and a predetermined L channel / R channel audio signal is multiplexed. The compressed encoded audio signal data in the stereo format is generated and output to the communication unit 15 as an audio signal for transmission. The communication unit 15 outputs the input audio signal for transmission to another audio communication terminal device via a communication line according to a predetermined communication method.

The communication unit 15 receives a transmission audio signal transmitted from another audio communication terminal device, restores the audio signal in a predetermined compression encoding format, and outputs the audio signal to the decoder 14 of the codec unit 12.
The decoder 14 performs a demodulation process on the compression encoding of the input audio signal, converts it into a digital audio signal in a predetermined PCM format, for example, and outputs it to the audio signal processing unit 11. In this case, the compression-encoded playback audio signal has a stereo format similar to that of the compression-encoded audio signal data generated by the encoder 13, for example. The audio signals of the L channel and the R channel in the PCM format can be obtained. The components of the L channel and R channel audio signals that have passed through the audio signal processing unit 11 are finally the other party's speaker audio output from the speakers 3 (L) and 3 (R).

  FIG. 4 shows an internal configuration example of the audio signal processing unit 11 which is an echo cancellation system in the present embodiment. In this figure, the microphone 2, the speaker 3, and the codec unit 12 (encoder 13 and decoder 14) are shown together with the audio signal processing unit 11.

As shown also in FIG. 3, first, in a stereo channel loudspeaker communication system such as the voice communication terminal device 1 of the present embodiment, there are four spatial propagation paths. That is, a spatial propagation path Sll reaching the L channel microphone 2 (L) from the L channel speaker 3 (L), and a spatial propagation path reaching the L channel microphone 2 (L) from the R channel speaker 3 (R). Srl, spatial propagation path Srr from the L channel speaker 3 (L) to the R channel microphone 2 (R), spatial propagation path from the R channel speaker 3 (R) to the R channel microphone 2 (R) Srr exists.
As will be understood from the following description, the audio signal processing unit 11 which is an echo cancellation system of the present embodiment is configured to execute echo cancellation processing by the adaptive filter system. In order to perform echo cancellation corresponding to each of the propagation paths Sll, Srl, Slr, and Srr, the adaptive filter system is configured as follows.

  As shown in the figure, the adaptive filter system in this case can be regarded as consisting of two adaptive filter systems 20A and 20B. As will be understood from the following description, the adaptive filter system 20A propagates the spatial propagation paths Slr and Srr out of the four spatial propagation paths described above and is picked up by the R channel microphone 2 (R). The adaptive filter system 20B propagates through the remaining two spatial propagation paths Sll and Srl and is picked up by the L-channel microphone 2 (L). Is for canceling.

First, a configuration corresponding to the adaptive filter system 20A side will be described.
This adaptive filter system 20A includes, for example, adaptive filters (ADF: Adaptive Digital Filter) 21RR, 21LR, an adder 22 (R), and a subtractor 23 (R) as shown in the figure.
First, the adaptive filter 21RR is provided for canceling echo sound generated by being picked up by the R channel microphone 2 (R) through the spatial propagation path Srr. Is to input an audio signal to be output to the R channel speaker 3 (R) as a reference signal. This audio signal is used as an output signal of the adder 33 as shown in the figure. The adder 33 responds to the voice signal of the R-channel counterpart speaker voice (Xd1) output from the decoder 14 and the voice of the R-channel self-speaker voice output from the self-amplifying volume 31. Synthesize the signal.
One adaptive filter 21LR is provided to cancel the echo sound generated by being picked up by the R channel microphone 2 (R) through the spatial propagation path Slr. Is to input an audio signal to be output to the L-channel speaker 3 (L) as a reference signal. This audio signal is an output signal of the adder 34, and the adder 34 is a self-amplifying volume 31 for the audio signal (Xd 2) of the other party's speaker voice on the L channel output from the decoder 14. Synthesizes the L-channel self-speaker speech signal that is output from

The output signals (cancellation signals) Yrr and Ylr of the adaptive filters 21RR and 21LR are added and synthesized by the adder 22 (R) and input to the subtractor 23 (R). The subtracter 23 (R) subtracts the output signal (Yrr + Ylr) of the adder 22 (R) from the collected sound signal obtained by the microphone 2 (R). The output of the subtracter 23 (R) is output as the output signal Ye1 of the adaptive filter system 20A.
The collected sound signal obtained by the microphone 2 (R) input to the subtracter 23 (R) is a desired signal common to the adaptive filters 21RR and 21LR. That is, the subtracter 23 (R) subtracts a signal obtained by synthesizing the output signals (cancellation signals) of the adaptive filters 21RR and 21LR from the collected sound signal obtained by the microphone 2 (R) as a desired signal. It is what.
The signal Ye1 which is the output of the subtractor 23 (R) is transmitted to the other voice communication terminal device on the communication partner side (far end side) via the transmission volume 32 as an R channel audio signal. Input to the encoder 13. The signal Ye1 is fed back to the adaptive filters 21RR and 21LR as a common error signal (residual signal) corresponding to the adaptive filters 21RR and 21LR. Furthermore, in the present embodiment, the signal Ye1 is also input to the self-speaking sound volume 31 as an R-channel self-speaker voice signal.

Next, a configuration corresponding to the adaptive filter system 20B side will be described.
The adaptive filter system 20B includes adaptive filters 21RL and 21LL, an adder 22 (L), and a subtractor 23 (L) as shown in the figure, for example.
The adaptive filter 21RL is provided to cancel an echo sound generated by being picked up by the L channel microphone 2 (L) through the spatial propagation path Srl, and is output from the adder 33 as a reference signal. Then, a signal to be inputted to the R channel speaker 3 (R) is inputted.
The adaptive filter 21LL is provided for canceling echo sound generated by being picked up by the L-channel microphone 2 (L) through the spatial propagation path Sll. The adder 34 is used as a reference signal. The signal to be input to the L channel speaker 3 (L) is input.

The output signals (cancellation signals) Yrl and Yll of the adaptive filters 21RL and 21LL are added and synthesized by the adder 22 (L) and input to the subtractor 23 (L). The subtractor 23 (L) subtracts the output signal (Yrl + Yll) of the adder 22 (L) from the collected sound signal obtained by the microphone 2 (L). The output of the subtracter 23 (L) becomes the output signal Ye2 of the adaptive filter system 20B.
Here, the collected sound signal obtained by the microphone 2 (L) input to the subtracter 23 (L) is a desired signal common to the adaptive filters 21RL and 21LL, and the subtracter 23 (L) A signal obtained by synthesizing output signals (cancellation signals) of the adaptive filters RL and 21LL is subtracted from the signal.
The signal Ye2 is input to the encoder 13 via the transmission volume 32 as an L channel audio signal to be transmitted to another audio communication terminal device on the communication partner side (far end side). The signal Ye2 is fed back to the adaptive filters 21RL and 21LL as a common error signal (residual signal) corresponding to the adaptive filters 21RL and 21LL. Further, the signal Ye2 is also input to the self-sound volume 31 as an L-channel self-speaker voice signal.

  The self-sound volume 31 inputs the signals Ye1 and Ye2 as the R-side and L-channel self-speaker voices as described above, and the volume balance between the L-channel sound and the R-channel sound as described later. After the adjustment, an audio signal Xsr that becomes the R-channel self-speaker voice is output to the adder 33, and an audio signal Xsl that becomes the L-channel self-speaker voice is output to the adder 34. Output.

  The transmission sound volume 32 is configured to be capable of outputting an input R-channel and L-channel audio signal by applying level gain attenuation according to a set attenuation factor, and will be described below. Thus, the adaptive filter systems 20A and 20B are provided to reinforce the cancellation effect on the echo sound of the other party's speaker voice. That is, the adaptive filters 21 (RR), 21 (LR), 21 (RL), and 21 (LL) provided in the adaptive filter systems 20A and 20B have converged, and a sufficiently effective echo cancellation effect is obtained. Even in such a state, in reality, some echo components may remain. In the transmission sound volume 32, the adaptive filter converges, and the content of the collected sound signal does not include the self-speaker speech, and possibly includes only the echo sound of the other-speaker speech. When the so-called counterparty single talk state is detected, the input signal is not output by giving an almost 100% attenuation rate, or is operated to give an attenuation rate of a certain level or more. . As a result, it is possible to make the above-mentioned echo residual component inaudible or difficult to hear in the counterpart communication terminal device.

The operation of the adaptive filter systems 20A and 20B in the audio signal processing unit 11 having the above configuration will be described first by taking the adaptive filter system 20A as an example.
The description of the adaptive filters 21 (RR) and 21 (LR) in the adaptive filter system 20A is omitted, but, for example, an FIR (Finite Impulse Response) type based on a required order through which the desired signal passes is used. And a coefficient setting circuit for variably setting the coefficient (filter coefficient) of the digital filter according to a predetermined adaptive algorithm. The output of the above digital filter is the output signal of the adaptive filters 21 (RR) and 21 (LR) and becomes a pseudo echo signal (cancellation signal).

In the adaptive filters 21 (RR) and 21 (LR), an output signal (cancellation signal) that minimizes the residual amount indicated by the error signal (Ye1) is always obtained. The setting circuit is configured to change and set the filter coefficient of the required order stage coefficient unit.
As a result, the coefficient vector of the adaptive filter 21 (RR) (corresponding to an array of coefficients corresponding to the order level) is transferred from the R channel speaker 3 (R) to the microphone 2 (R) via the spatial propagation path Srr. Thus, an impulse response representing a pseudo transfer function of the sound collected is formed. Similarly, the coefficient vector of the adaptive filter 21 (LR) is a pseudo transfer function of sound collected by the microphone 2 (R) from the L-channel speaker 3 (L) via the spatial propagation path Slr. Is formed. In other words, the signal components of each sound obtained via the spatial propagation paths Srr and Slr are adaptively canceled according to the state of the signal to be processed (reference signal, desired signal) at that time. It can be said that this is
The sound picked up by the microphone 2 (R) via the echo path as the spatial propagation path Srr is an echo sound based on the sound signal to be reproduced by the R channel speaker 3 (R). The sound collected by the microphone 2 (R) via the echo path as the spatial propagation path Slr is an echo sound based on the sound signal to be reproduced by the L channel speaker 3 (L). It is an ingredient. Accordingly, the output signals Yrr and Ylr (cancellation signals) of the adaptive filters 21 (RR) and 21 (LR) are pseudo echoes for the R channel and L channel speaker output sounds that have arrived at the microphone 2 (R), respectively. It will be captured as sound. Then, in the adaptive filter system 20A, the subtracter 23 (R) synthesizes cancellation signals from the adaptive filters 21 (RR) and 21 (LR) from the collected sound signal from the R channel microphone 2 (R). That is, the above pseudo echo sound is subtracted. In this way, the adaptive filter system 20A performs an operation of adaptively removing the component of the echo sound from the sound collected by the R channel microphone 2 (R).

In the adaptive filter system 20B, sound is collected by the adaptive filter 21 (RL) from the R channel speaker 3 (R) to the L channel microphone 2 (L) via an echo path as the spatial propagation path Srl. An audio signal (cancellation signal) Yrl of a pseudo echo sound corresponding to the echo sound is generated and output, and the adaptive filter 21 (LL) passes through the echo path as the spatial propagation path Sll and passes through the L channel speaker 3 (L ) To generate and output a pseudo echo sound signal (cancellation signal) Yll corresponding to the echo sound picked up by the L-channel microphone 2 (L). And these audio | voice signals are synthesize | combined by the synthesizer 22 (L), and are output to the subtractor 23 (L).
In the subtractor 23 (L), the audio signal picked up by the L-channel microphone 2 (L) is used as a desired signal, and the combined cancellation signal output from the subtractor 23 (L) is output from the desired signal. Subtract (Yrl + Yll) and output as signal Ye2. According to the above description of the adaptive filter system 20A, this signal Ye2 is obtained from the sound signal collected by the L-channel microphone 2 (L) via the spatial propagation paths Srl and Sll and the speaker 3 (R ), And the echo sound component propagated by each of 3 (L) is canceled.

In this way, the audio signal processing unit 11 according to the present embodiment basically has an echo sound component generated by sound propagating through the spatial propagation paths Sll, Srl, Slr, Srr corresponding to the stereo channel. Each is configured to cancel each of the above.
The audio signals Ye1 and Ye2 obtained by canceling the echo sound as described above are transmitted from the transmission sound volume 32 via the encoder 13 as transmission audio signals for the R channel and the L channel, respectively. It is transmitted to other voice communication terminal devices. Thereby, the echo sound is also removed from the sound that is heard by releasing the sound signal received by the other sound communication terminal device from the speaker. In this way, an echo canceling effect is produced.

The audio signals Ye1 and Ye2 are self-speaking volumes 31 as signals based on the audio signals Xsr and Xsl as self-speaker voices to be output from the speakers 3 (R) and 3 (L). Is entered against.
For example, if a self-speaker voice (sound collected by a microphone at a level above a certain level considered valid) is input to the microphone 2 (R), the voice signal component is the adaptive filter system 20A. The signal Ye1 is input to the self-sound volume 31 via the subtracter 23 (R). Similarly, if a self-speaker voice (sound collected by the microphone at a level above a certain level considered valid) is input to the microphone 2 (L), the voice signal component is an adaptive filter. The signal Ye2 is input to the self-sound volume 31 via the subtractor 23 (L) of the system 20B.
As described later, the self-sound volume 31 performs attenuation rate setting and balance adjustment using the input signals Ye1 and Ye2, and outputs audio signals Xsr and Xsl. The voice signals Xsr and Xsl become the self-speaker voice collected by the microphone 2 (R) or the microphone 2 (L). The audio signals Xsr and Xsl are output to the speakers 3 (R) and 3 (L) via the adders 33 and 34, respectively. Thereby, the self-speaker voice collected by the microphones 2 (R) and 2 (L) is emitted as sound from the speakers 3 (R) and 3 (L) in the same place. Become. That is, a self-speaking sound output function is realized.
The self-speaker speech emitted from the speakers 3 (R) and 3 (L) as described above also propagates through the spatial propagation paths Sll, Srl, Slr, and Srr to the microphones 2 (L), 2 ( R), an echo sound of the self-speaker voice is generated. As described above, the adaptive filter systems 20A and 20B are connected to the spatial propagation paths (echo paths) Sll, Srl and Slr. The reference signal input to the adaptive filters 21 (RR) and 21 (RL) is used as the output signal of the synthesizer 33 when operating so as to cancel the echo sound propagating via Srr. That is, the audio signal Xsr as the R-side self-speaker sound output from the self-sound volume 31 is included, and similarly, the adaptive filters 21 (RR) and 21 (RL) are included in the adaptive filters 21 (RR) and 21 (RL). Since the input reference signal is an output signal of the synthesizer 34, an audio signal Xsl as an L-channel self-speaker voice is included. Therefore, the echo sound of the self-speaker voice is also canceled.
Incidentally, the configuration in which the signals Ye1 and Ye2 are input to the adders 33 and 34 as the signals Xsr and Xsl through the self-sound volume 31 does not have a self-sound output function. The configuration is equivalent to a normal stereo-compatible echo cancellation system.

Next, the left / right (L, R) volume balance setting for the self-amplifying sound, which is performed by the audio signal processing unit 11, will be described.
FIG. 5 shows an example of the internal configuration of the self-sounding volume 31 shown in FIG.
As described with reference to FIG. 4, the self-speaking sound volume 31 includes the signal Ye1 based on the collected sound signal of the R channel microphone 2 (R) and the L channel microphone 2 (L). A signal Ye2 based on the collected sound signal is input.
The signal Ye1 is further branched into a signal Ye1 (R) corresponding to the R channel and a signal Ye1 (L) corresponding to the L channel, and then the signal Ye1 (R) is input to the volume varying unit 41A and the signal Ye1 (L ) Is input to the volume changing unit 41B.
Similarly, the signal Ye2 is branched into a signal Ye2 (R) corresponding to the R channel and a signal Ye2 (L) corresponding to the L channel, and then the signal Ye2 (R) is input to the volume varying unit 41C, Ye2 (L) is input to the sound volume variable unit 41D.

The sound volume variable sections 41A, 41B, 41C, 41D are respectively for changing the level (volume) of the input signals (signals Ye1 (R), Ye1 (L), Ye2 (R), Ye2 (L)). Signal processing can be performed independently.
In addition, the volume changing units 41A and 41B can set the “balance” of the level (volume) between the signals Ye1 (R) and Ye1 (L) by changing the volume in cooperation with each other. It is said that. Similarly, the volume changing units 41C and 41D can set the level (volume) balance between the signals Ye2 (R) and Ye2 (L) by changing the volume in cooperation with each other. It is said that.

The signal Ye1 (R) after the volume adjustment by the volume variable unit 41A is output to the adder 42A as the signal Yf1 (R), and the signal Ye1 (L) after the volume adjustment by the volume variable unit 41B is performed. Is output to the adder 42B as a signal Yf1 (L). The signal Ye2 (R) after the volume adjustment by the volume variable unit 41C is output to the adder 42A as the signal Yf2 (R), and the signal Ye2 (L after the volume adjustment by the volume variable unit 41D is performed. ) Is output to the adder 42B as a signal Yf2 (L).
The adder 42A inputs the signals Yf1 (R) and Yf2 (R) input as described above to the adder 33 (FIG. 4) as the signal Xsr. The adder 42B inputs the signals Yf1 (L) and Yf2 (L) input as described above to the adder 34 (FIG. 4) as the signal Xsl.

  Since the self-sound volume 31 is configured as described above, the self-speaker sound collected by the microphone 2 (R) input as the signal Ye1 is the volume between the L channel and the R channel. After the balance is set, it is possible to output sound from the speaker 3 (R) and the speaker 3 (L). Similarly, the volume balance between the L channel and the R channel is set for the self-speaker voice collected by the microphone 2 (L) input as the signal Ye2, and then the speaker 3 (R ) And the speaker 3 (L).

  In addition, the audio signal processing unit 11 of the present embodiment including the self-sounding volume 31 having the above-described configuration is configured to execute the processing procedure shown in the flowchart of FIG. 6 during the operation. When the audio signal processing unit 11 is configured by a DSP, the processing shown in this figure is realized by a program called an instruction given to the DSP.

  First, in step S101, the state of the collected voice signal (Rch collected voice signal) obtained by collecting sound with the microphone 2 (R) corresponding to the R channel is set to a state other than the self-side speaker voice input state. To wait for a transition to the self-speaker voice input state. The self-speaker voice input state here is obtained by speaking to the microphone 2 (R) in the contents of the collected voice signal (acquired by the microphone at a level above a certain level considered to be valid). It is a state in which self-speaker speech is included.

There are several possible methods for determining the transition to the self-speaker voice input state, but one is input as a desired signal to the subtracter 23 (R) in the adaptive filter system 20A. A discrimination result is obtained by comparing the collected sound signal from the microphone 2 (R) with the signal Ye1 output from the same subtractor 23 (R).
In order to make the explanation easy to understand, the adaptive filter in the adaptive filter system 20A converges in a state of canceling the echo sound of the other party's speaker voice before and after the transition to the self-speaker voice input state. Consider when you are in a state. At this time, as a state other than the self-speaker voice input state, for example, a sound arriving from at least one of the speakers 3 (L) and 3 (R) via the spatial transmission path (Srr, Slr), that is, It is assumed that only the echo sound is picked up by the microphone 2 (R) and the echo sound is input.
Corresponding to the above-described echo sound input state, the desired signal from the microphone 2 (R) has a level larger than a certain level with respect to the signal Ye1. In the single talk state, as the desired signal, sounds arriving at the microphone 2 (R) from the speakers 3 (R) and 3 (L) via the spatial propagation paths Srr and Slr, that is, echo sounds are collected. The echo sound is canceled and the signal Ye1 which is an output signal of the adaptive filter system 20A is in a very low level while being obtained as a sound and sound signal.
On the other hand, in response to the self-speaker voice input state, the desired signal from the microphone 2 (R) and the signal Ye1 have levels that are approximated to the extent that they may be regarded as substantially the same level. At this time, the component corresponding to the self-speaker voice is dominant in the desired signal, which is the collected sound signal of the microphone 2 (R). Depending on 20A, it becomes a component which is not canceled. For this reason, the component of the self-speaker voice remains in the signal Ye1 without being canceled.
If it is determined in step S101 that the state has been changed to the self-speaker voice input state, the procedure proceeds to step S102 and subsequent steps.

In response to the collected sound signal of microphone 2 (R) transitioning to the self-speaker speech input state as described above, the sound-collected sound signal in which the content of the self-speaker speech becomes dominant Is input to the self-sound volume 31 as the signal Ye1. As shown in FIG. 5, the input signal Ye1 is branched into signals Ye1 (R) and Ye1 (L) and input to the balance adjuster 41A.
In step S102, first, the volume control units 41A and 41B are linked to operate the level balance of the signals Yf1 (R) and Yf1 (L) at the same level (Yf1 (R): Yf1 (L) = The initial setting is performed so that 1: 1). By this initial setting, the self-speaker speech picked up by the microphone 2 (R) is output at the same volume from each of the speakers 3 (R) and 3 (L).

With the initial setting in step S102 described above, the voice of the self-side speaker picked up by the microphone 2 (R) is output in the same volume from the speakers 3 (R) and 3 (L). The signal processing unit 11 executes the procedure of step S103.
Step S103 is the same adaptation as the level Vrr of the cancellation signal Yrr output from the adaptive filter 21 (RR) in the adaptive filter system 20A on the side that performs echo cancellation using the collected sound signal of the microphone 2 (R) as a desired signal. The level Vlr of the cancellation signal Ylr output from the adaptive filter 21 (LR) in the filter system 20A is detected. In this detection, for example, the levels of the cancellation signals Yrr and Ylr may be actually measured. Alternatively, it can be obtained by estimation from the coefficient vectors of the adaptive filters 21 (RR) and 21 (LR).

Here, the ratio of the level Vrr of the cancellation signal Yrr to the level Vlr of the cancellation signal Ylr is the self-speaker voice that reaches the microphone 2 (R) from the speaker 3 (R) via the spatial propagation path Srr. And the volume ratio of the self-speaker voice reaching the microphone 2 (R) via the spatial propagation path Slr from the speaker 3 (L).
For example, in order to simplify the description, it is assumed that the sound that reaches the microphone 2 (R) from the speakers 3 (R) and 3 (L) is only direct sound, and the microphone 3 (R) from the speaker 3 (R). ) And the distance Klr from the speaker 3 (L) to the microphone 2 (L) is Krr: Klr = 1: 2. If it is assumed that the sound intensity (volume) simply decreases in inverse proportion to the square of the distance, the initial setting in step S102 causes the speakers 3 (R) and 3 (L) to self-same at the same volume. Under the state where the side speaker voice is emitted, the volume (Arr) of the self side speaker voice reaching the microphone 2 (R) via the spatial propagation path Srr from the speaker 3 (R), and the speaker The ratio of the volume (Alr) of the self-speaker speech that reaches the microphone 2 (R) from 3 (L) via the spatial propagation path Slr is Arr: Alr = 4: 1.
The collected sound signal from the microphone 2 (R) includes the component of the speaker's own speech that has reached the microphone 2 (R) via the spatial propagation path Srr from the speaker 3 (R) and the space from the speaker 3 (L). The self-speaker speech component that reaches the microphone 2 (R) via the propagation path Slr is included, but the ratio of the former level to the latter level is 4: 1. As a result of executing the adaptive processing using this signal as a desired signal, the level Vrr of the cancellation signal Yrr output from the adaptive filter 21 (RR) corresponding to the spatial propagation path Srr and the adaptive filter 21 corresponding to the spatial propagation path Slr. The ratio of the level Vlr of the cancellation signal Ylr output by (LR) is also Vrr: Vlr = 4: 1. The ratio of the levels Vrr and Vlr detected in this way can be understood from the above description. Originally, the R-channel microphone 2 (R) to which the self-speaker voice is input is used. And the ratio of the distances to the speakers 3 (R) and 3 (L). In other words, by detecting the levels Vrr and Vlr in step S103, it is possible to determine the distance between the microphone 2 (R) and the speakers 3 (R) and 3 (L). I can say.

In the next step S104, the volume balance units 41A and 41B are used to balance the levels of the signals Yf1 (R) and Yf1 (L).
Yf1 (R): Yf1 (L) = (Vlr) ^ 2: (Vrr) ^ 2 (^ 2 indicates that it should be squared) ... (Formula 1)
Then, change from the initial setting state.
For example, as described above, when the environment is such that Krr: Klr = 1: 2, the relationship between the levels Vrr and Vlr detected in step 103 becomes Vrr: Vlr = 4: 1. if there is,
Yf1 (R): Yf1 (L) = 1: 16
This means that the signal level balance is set (determined). As a result, if the self-speaker speech picked up by the microphone 2 (R) is output at a volume that is one time from the speaker 3 (R), the speaker 3 (L) has 16 It will be output with double the volume.

As described above, in the case of a conference room that is considerably wide, when a self-sounding sound is output in stereo, the sound collected by the microphone corresponding to one channel is not transmitted to the channel on the same side. It is preferable to output the sound from the speaker at a certain level or higher and to output the sound at a higher volume from the speaker of the other channel, so that the balance is appropriately adjusted. With such a balance setting, conference participants in the vicinity of the speaker on the channel opposite to the microphone to which the self-speaker voice is input can clearly hear the self-speaker voice. This is because a conference participant who is in the vicinity of the speaker of the same channel as the microphone to which the side speaker voice is input can also hear the self side speaker voice clearly with a certain amount of self-amplifying volume.
And if it is the balance setting by the said step S104, the self-side speaker audio | voice picked up by the microphone 2 (R) corresponding to R channel will use the speakers 3 (R), 3 (L ) And the sound volume output from the same R channel speaker 3 (R) as the microphone 2 (R), the distance between the microphone and the speaker from the L channel speaker 3 (L). The sound is output at a predetermined volume corresponding to the ratio (16 times if Krr: Klr = 1: 2). At this time, a conference participant in the vicinity of the speaker 3 (L) located far away from the speaker who uses the microphone 2 (R) clearly hears his / her side talk by the sound output from the speaker 3 (L). Can hear the voice of a person. In addition, a conference participant who is near the speaker who uses the microphone 2 (R) can also hear the speaker's live voice from the speaker 3 (R). The speaker's voice can be heard clearly. That is, an appropriate volume balance suitable for a conference hall that is considerably wide is obtained, and such volume balance setting is automatically executed.
Further, according to the signal level balance setting in step S104, the ratio of the distance Klr from the speaker 3 (L) to the microphone 2 (R) to the distance Krr from the speaker 3 (R) to the microphone 2 (R) is further increased. If the magnification is increased, the volume of the speaker 3 (L) is further increased, and the balance ratio with the volume of the speaker 3 (R) is also increased. Conversely, if the ratio of the distance (Klr) from the speaker 3 (L) to the microphone 2 (R) to the distance (Krr) from the speaker 3 (R) to the microphone 2 (R) decreases, the speaker 3 (L ) Is reduced, and the volume of the speaker 3 (R) is increased, so that the balance ratio is reduced. That is, depending on step S104, for example, a microphone is handed over between conference participants during a conference, the distance relationship between the speaker and the microphone changes, or the conference location is changed, so that the previous conference location is Even if the distance between the different speakers and the microphones becomes different, it is possible to always obtain an appropriate volume balance by adapting to these changes. For example, when balance adjustment is performed by manual operation, the user needs to perform an operation for balance adjustment whenever the above-described change occurs. This is not necessary, and very high convenience is obtained in this respect.

  Here, the balance of the signal level set in step S104 is continued until the single talk state is entered. When transitioning to the single talk state, for example, the above signal level balance setting is canceled, and the setting change to a predetermined level balance adapted to the single talk state is performed.

  Further, the procedure of FIG. 6 described so far is based on the signal Ye2 when the content of the collected voice signal by the microphone 2 (L) corresponding to the L channel is changed to the self-side speaker voice input state. For the signals Yf2 (R) and Yf2 (L) obtained in this way, the same is executed on the volume variable sections 41B and 41C.

  In step S104, the signals Yf1 (R) and Yf1 (L) (or the signals Yf2 (R) and Yf2 (L) are based on the values obtained by squaring the levels Vrr and Vlr detected in step 103. However, this is merely an example. In practice, if a more optimal volume balance can be obtained, the balance may be determined according to other rules. It is also possible to prepare a plurality of balance determination algorithms in step S104 so that the user can select them by a pre-operation or the like.

7 is a process executed by the audio signal processing unit 11 according to the present embodiment together with the balance adjustment of the self-sounding sound according to the procedure shown in FIG. 6, and is performed by the adaptive filter 21 (RR). The process for controlling the operations of the adaptive filter 21 (RR) and the volume changing unit 41A in the self-sound volume 31 according to the input state of the reference signal and the desired signal is shown.
Note that the processing similar to FIG. 7 is executed for each of the remaining three adaptive filters 21 (LR), 21 (RL), and 21 (LL) as will be described later.
The processing shown in the figure is also realized by a program called an instruction given to the DSP when the audio signal processing unit 11 is a DSP.
In addition, in starting the execution of the processing shown in this figure for the first time, the adaptive processing of the adaptive filter systems 20A and 20B (adaptive filters 21 (RR), 21 (LR), 21 (RL), and 21 (LL)) will be described. Is also started in the running state. For confirmation, in a state where the adaptive filter systems 20A and 20B are executing adaptive processing, the audio signals output from the adders 33 and 34 are input as reference signals and the microphone 2 (R ) The collected sound signal from 2 (L) is used as a desired signal, and the error signal (Ye1, Ye2), which is the output of the subtractors 23 (R), 23 (L), is minimized so that the adaptive filter 21 Each of (RR), 21 (LR), 21 (RL), and 21 (LL) varies the coefficient vector of the FIR filter.

In the description of FIG. 7, the input state of the reference signal and the desired signal in the adaptive filter 21 is 4 in the “other party single talk state”, “self side single talk state”, “double talk state”, and “non-talk state”. Assume that it is in one of the two states.
In the “other party single talk state”, the reference signal includes a voice signal of the other party's speaker voice of a certain level or higher that is considered to be valid, and is obtained from the decoder 14, while the desired signal is obtained. Depending on the microphone 2 of the channel, the self-speaker voice of a certain level or more that is valid is not picked up.
For example, when the adaptive filter 21 (RR) is in the partner-side single talk state, the speech signal Xd1 (reference signal) of the R channel output from the decoder 14 has a certain level or more of the other-party speech that is considered valid. However, the R-channel microphone 2 (R) (which is the microphone of the channel that obtains the desired signal) has a certain level of self-speaker speech that is effective above a certain level. The sound is not picked up.

In the “self-side single talk state”, the reference signal does not include the voice signal of the other party's speaker voice regarded as valid from the decoder 14, but the microphone 2 of the channel on the side where the desired signal is obtained is This refers to the state in which the self-speaker speech above a certain level that is considered valid is being collected.
In the “double talk state”, the audio signal of the other party's voice that is regarded as valid from the decoder 14 is included in the reference signal, and the microphone 2 of the channel that obtains the desired signal is regarded as valid. The self-speaker voice above a certain level is also being picked up.
In the “non-talk state”, the reference signal does not include the voice signal of the other party's speaker voice that is regarded as valid from the decoder 14, and is also regarded as valid by the microphone 2 of the channel that obtains the desired signal. This is a state where the self-speaker voice above a certain level is not picked up.

First, in step S201, the level (value) of the error signal (speech signal) Ye1 corresponding to the adaptive filter 21 (RR), which is an output from the subtractor 23 (R), is sent from the decoder 14 to the adder 33. It is determined whether or not the level (value) of the input R channel audio signal Xd1 is equal to or less than a certain rate (Ye1 ≦ Xd1 * m (m is a predetermined positive number less than 1)). Yes.
Here, the state in which the level of the voice signal Ye1 is equal to or lower than a certain rate with respect to the voice signal Xd1 means that the L channel of the other party's speaker voice above a certain level that is considered valid from the R channel of the decoder 14. While the voice signal is being input, the microphone 2 (R) is in a state where the self-speaker voice of a level higher than a certain level that is valid is not picked up. It will be.
That is, if an audio signal of the other party's speaker voice that is considered to be valid is input from the R channel output of the decoder 14, the audio signal Xd1 that is the audio signal has a large level (amplitude) greater than a certain level. ) Value. On the other hand, for the speech signal Ye1, on the assumption that the adaptive filter 21 (RR) converges in a state where the echo sound of the other party's speaker speech is cancelled, the spatial propagation path Srr from the speaker 3 at this time Since the echo sound of the other party's speaker voice that reaches the microphone 2 (R) via the route is canceled appropriately, the state becomes a very low level.

Note that, in a state other than the other party single talk state (self side single talk state, double talk state, non-talk state), the level of the audio signal Ye1 exceeds the above-mentioned fixed rate with respect to the level of the audio signal Xd1. become.
That is, first, in the self-side single talk state, the signal of the self-side speaker voice picked up by the microphone 2 (R) passes without being canceled by the adaptive filter 21 (RR). Ye1 is a correspondingly large level corresponding to this self-speaker voice. On the other hand, the audio signal Xd1 has a very small level because there is no audio signal output regarded as valid from the L channel output of the decoder 14. Therefore, the error signal Ye1 is larger than the audio signal Xd1, and exceeds the above-mentioned fixed rate.
In the double talk state, although there is a certain difference, the other party's speaker from the L channel output of the decoder 14 and the signal of the own speaker's voice obtained by picking up the microphone 2 (R). Since all of the audio signals are above a certain level that is considered valid, the level difference between the audio signal Ye1 and the audio signal Xd1 is smaller than that in the other party's single talk state, and therefore exceeds the above-mentioned fixed rate. It becomes.
Further, in the non-talk state, both the self-speaker speech signal obtained by collecting sound by the microphone 2 (R) and the other-speaker speech signal from the L channel output of the decoder 14 are valid. In this case, the level difference between the audio signal Ye1 and the audio signal Xd1 is smaller than that in the other party's single talk state, and thus exceeds the above-mentioned fixed rate. It will be.

If an affirmative determination result is obtained in step S201 because the counterparty single talk state has occurred, the process proceeds to step S202.
In step S202, the volume variable unit 41A in the self-sounding volume 31 sets an attenuation factor of a certain level or higher, thereby blocking the input signal in the volume variable unit 41A and the reference signal of the adaptive filter 21 (RR). In other words, the sound signal Yf1 (R), that is, the self-speech sound component is not included. At this time, first, the volume is changed in accordance with the above-described attenuation rate with reference to the volume set by the volume balance adjustment described with reference to FIG. 6 (for example, treated as the maximum level (minimum attenuation rate)). It is supposed to be. The same applies to the attenuation rate setting for the sound volume variable unit 41A in steps S207 and S208 described later.

  In step S203 subsequent to step S202, it is determined whether or not the adaptive filter 21 (RR) is in a state where it has been sufficiently converged. For example, if it is assumed that the coefficient vector in the FIR filter of the adaptive filter 21 (RR) has reached a predetermined state that is regarded as being sufficiently converged, a positive determination result is obtained here. Alternatively, for example, the adaptive filter 21 may be configured to output the evaluation value of the convergence state of itself, for example, as the evaluation value as the degree of convergence. Can be realized.

In step S203, when a negative determination result is obtained that the adaptive filter 21 (RR) has not converged, the process proceeds to step S204, and the adaptive filter 21 (RR) is subjected to adaptive processing. Control to execute (set to an active tendency state). For example, if the adaptive process as the adaptive filter 21 (RR) has been executed up to the time point of reaching step S204, the conventional adaptive process is continued in step S204. On the other hand, if the adaptive process as the adaptive filter 21 (RR) has been stopped, the execution of the adaptive process is started in step S204.
For confirmation, since the volume varying unit 41A is set to the signal cutoff state in step S202, the adaptive processing executed in step S204 is appropriate and good as described above. Operation is obtained.

On the other hand, if a positive determination result is obtained in step S203 that the adaptive filter 21 (RR) has converged, the process proceeds to step S205, and the adaptive processing by the adaptive filter 21 (RR) is stopped. (A state of tendency to stop) Also in this case, if the adaptive process of the adaptive filter 21 (RR) has been executed up to step S205, the adaptive process is changed to a state in which the adaptive process is stopped in step S205. . Further, if the adaptive process has been stopped, this state is continued.
Here, for example, when the adaptive processing is changed from the state in which the adaptive processing has been executed to the stopped state in step S205, the coefficient vector of the FIR filter in the adaptive filter 21 (RR) is maintained with the setting state immediately before the stop being fixed. Will be. That is, the reference signal (output of the adder 33) input to the adaptive filter 21 (RR) is input from the adder 22 (R) to the subtracter 23 (R) in a state where the coefficient vector is fixed in this way. Then, it is subtracted from the output signal (cancellation signal) Ylr of the adaptive filter 21 (RR), and this component is included in the audio signal Ye1.
In the case of the other party single talk state, there is no particular problem even if the adaptive processing is continued while the adaptive filter system is converged. However, if the adaptive process is stopped as in step S205, for example, it is not necessary to perform an operation required for the adaptive process during that period, so that the processing load and resources can be reduced.
If the steps S204 and S205 are executed, the process returns to, for example, step S201.

If a negative determination result is obtained in step S201, that is, if any of the self-side single talk state, the double talk state, and the non-talk state, the process proceeds to step S206.
In step S206, as in the previous step S203, it is determined whether or not the adaptive filter 21 (RR) has converged. However, regarding the degree of convergence, it is determined that the adaptive filter 21 (RR) is determined to be in a converged state, corresponding to the other party single talk state and the other talk state. Thus, different conditions may be set in step S203 and step S206. Further, as the actual step S206, the determination process may be performed after setting the convergence condition adapted to each of the self-side single talk state, the double talk state, and the non-talk state.

  If an affirmative determination result is obtained in step S206, the process proceeds to step S207, which is equivalent to passing an input signal through the sound volume variable unit 41A by setting an attenuation rate below a certain value for the sound volume variable unit 41A. State. On the other hand, if a positive determination result is obtained in step S206, a predetermined attenuation rate corresponding to a certain level or more (not necessarily the same attenuation rate as step S202) is set in step S208. The sound volume variable unit 41A is equivalent to a state where the input signal is blocked and not output.

  After executing the procedures of steps S207 and S208, the adaptive processing of the adaptive filter 21 (RR) is stopped in step S209 in the same manner as in the previous step S205, and the process returns to step S201. For confirmation, if the adaptive processing that has been executed so far is stopped by this step S209, the adaptive filter of the adaptive filter 21 (RR) is the same as in step S205. The coefficient vector of the FIR filter at 21 is maintained in a fixed state immediately before stopping.

Depending on the processing of FIG. 7 described so far, depending on the state of the reference signal and the desired signal input to the adaptive filter 21 (RR), the adaptive filter 21 (RR) performs adaptive processing and the sound volume variable unit 41A. Is controlled as follows.
First, in the other party single talk state, the process of either step S204 or step S205 is performed from step S202 to step S203. Thereby, first, about the volume variable part 41A, the state which interrupts | blocks and outputs the audio | voice signal Yf1 (R) by step S202 is set.

As described above, the signal cut-off state is set for the volume varying unit 41A in correspondence with the counterparty single talk state for the following reason.
First, in the other party single talk state, the state of the desired signal Mr is not in a state in which the microphone 2 (R) is picking up the self-side speaker voice regarded as valid, that is, the microphone 2 (R The voice signal that requires self-speaking is not obtained on the) side. Therefore, there is no problem even if the signal cut-off state is set for the volume varying unit 41A. The echo canceling system corresponding to the adaptive filter 21 (RR) (the system for canceling the echo sound via the spatial propagation path Srr) when the volume varying unit 41A is in the signal cutoff state in this way is self It can be said that a circuit configuration equivalent to a normal echo cancellation system in which the loud sound output function is omitted is formed.
Also, if the volume variable unit 41A remains in the signal passing state, the space included in the audio signal Ye1 is included in the audio signal Ye1 when the actual adaptive filter 21 (RR) is not sufficiently converged in the counterpart single talk state. The residual component of the echo sound that has passed through the propagation path Srr passes through the system for self-sounding sound output (self-sounding sound volume 31, synthesizer 33), adaptive filter 21 (RR), and speaker 3 (R). Will be input again. The reference signal necessary for the adaptive filter 21 (RR) is only the component of the audio signal Xd1, which is the R channel output from the decoder 14. For this reason, if the audio signal Yf1 (R) via the volume variable unit 41A is input to the adaptive filter 21 (RR) as a reference signal, there is a possibility that proper adaptive processing of the adaptive filter 21 (RR) may be hindered. Come out. Further, in reality, even if the adaptive filter 21 (RR) is sufficiently converged, there is a possibility that a certain residual component of the echo sound appears in the error signal Ye1.
Therefore, the normal and good adaptive processing of the adaptive filter 21 (RR) is ensured by setting the signal cutoff state for the sound volume varying unit 41A in step S202.

  In the other party single talk state, for example, it may be considered that there is a situation in which, for example, the self-speaker voice is temporarily picked up by the microphone 2 (R) and transitions to the double talk state. However, in the other party single talk state, the conference participant mainly listens to the other party's speaker voice output from the decoder 14, and at that time, for example, temporarily joins a conference in the same place. Even if a person utters a voice, the conference participants do not feel uncomfortable that this cannot be heard from the speaker. Therefore, even if the state transition as described above occurs, there is no particular problem with setting the signal cutoff state for the sound volume variable unit 41A.

  Further, as described above, the transmission sound volume 32 is used for suppressing the residual component of the echo sound in the audio signal Ye1 output when the adaptive filter 21 (RR) converges. In this respect, the adjustment of the attenuation factor in the transmission sound volume 32 is correspondingly delicate, and the control is somewhat advanced. For example, if an extreme attenuation rate is set, there is a high possibility that the sound heard on the other party's voice communication terminal device side becomes unnatural. On the other hand, in the other party single talk state, as described above, even if the volume variable unit 41A is set to 100% or a strong attenuation factor close to this to cut off the signal output, for example. There is no hindrance.

Further, in the same counterparty single talk state, when the adaptive filter 21 (RR) is not converged as a result of determination in step S203, the adaptive filter 21 (RR) is in a state in which adaptive processing is executed (step S203). , S204) When the state is converged, the adaptive processing of the adaptive filter 21 (RR) is stopped (steps S203, S205).
First, the other party single talk state is a state in which an audio signal component effective as the other party's speaker voice to be canceled is input to the near end side. This means that if the adaptive filter 21 (RR) is not converged, the adaptive filter 21 (RR) converges in a state where the echo sound of the other party's speaker voice that passes through the spatial propagation path Srr is canceled. It can be said that it is a time when the adaptive processing should be actively executed for (RR).
Therefore, when the adaptive filter 21 (RR) is not converged, the adaptive process is executed. In the present embodiment, as described above, the reference signal input to the adaptive filter 21 (RR) is obtained when the volume variable unit 41A is turned off by the processing in step S202. , Only the component of the audio signal Xd1 from the decoder 14 is obtained. For this reason, the adaptation process executed in response to step S204 is an appropriate operation for canceling the original cancel target sound.

  Further, according to the processing of FIG. 7 described above, in the self-side single talk state, double talk state, or non-talk state corresponding to the case where a negative determination result is obtained in step S201, the volume variable unit For 41A, a signal passing state is set corresponding to the state where the adaptive filter 21 (RR) has converged (S206, S207), and a signal blocking state is set corresponding to the state where the adaptive filter 21 (RR) has not converged (S206). , S208). Further, the adaptive filter 21 (RR) is in a state where the adaptive processing is stopped in common (S209). The reason for setting the state of the audio signal processing unit 11 will be described in correspondence with each of the three states.

  First, consider the correspondence between the double talk state and the self-side single talk state. In the double talk state, the other party's speaker voice regarded as valid is obtained as the voice signal Xd1, and the valid self-speaker voice is picked up by the microphone 2 (R). This is a state obtained as a component of the desired signal Mr. On the other hand, the self-side single talk state is a state in which a valid self-speaker voice is obtained as a desired signal, but a counterpart-speaker voice deemed valid is not obtained as the signal Xd1. That is, the double talk state and the self-side single talk state are common in that an audio signal of the self-side speaker voice is obtained.

In this way, at least in the state where the voice signal of the self-side speaker voice is obtained by the microphone 2 (R), the voice signal of the self-side talker voice is obtained as long as it has the self-speech enhancement function. Should be reproduced and output (self-speaking) from the speaker 3 (R) according to the volume balance set by the procedure of FIG. 6 as much as possible. From this, it is only necessary to set a signal passing state for the sound volume variable unit 41A.
However, the adaptive filter 21 (RR) originally inputs only the speech signal component corresponding to the other party's speaker speech from the R channel output of the decoder 14 as the reference signal Xd1, and also as the desired signal Mr, By inputting only the voice signal component of the voice that has reached the microphone 2 (R) from the speaker 3 (R) via the spatial transmission path Srr, the echo sound of the other party's speaker voice is canceled and converged. Is something that can be done.
If the adaptive filter 21 (RR) is not converged and the sound volume variable unit 41A is set in a signal passing state, the microphone 2 (in the double talk state, the microphone 2 ( A considerable amount of the speech signal component of the self-speaker speech collected by R) is included. In the self-side single talk state, the reference signal is dominated by the audio signal component of the self-side speaker voice. In addition, the desired signal Mr of the adaptive filter 21 (RR) also includes a considerable amount of the component of the self-speaker speech obtained by speaking toward the microphone 2 (R). In the self-side single talk state, the desired signal Mr is dominated by the audio signal component of the self-side speaker voice.
If the adaptive process of the adaptive filter 21 (RR) is executed in the above state, the echo sound of the other party's speaker voice that passes through the spatial propagation path Srr, which is the original purpose of the adaptive filter 21 (RR), can be canceled. In other words, the state cannot be converged to a state, and a coefficient vector far from the convergence is set. Then, in this double talk state, many echo sounds of the other party's speaker voice remain, and the sound heard from the speaker 3 becomes very hard to hear. Further, after that, for example, when the state transitions to the counterparty single talk state, the time until convergence is increased accordingly.

Based on this, when the adaptive filter 21 (RR) has converged in the double talk state or the self-side single talk state, the volume variable unit 41A is set in the signal passing state, and then the adaptive filter 21 is set. For (RR), the adaptive processing is stopped.
Thereby, first, the voice signal of the self-speaker voice picked up by the microphone 2 (R) passes from the adaptive filter 21 (RR) through the volume varying unit 41A, and further via the adder 33 to the speaker 3. (R) will be output as sound. That is, it is output as a self-speaking sound. However, at this time, the adaptive processing of the adaptive filter 21 (RR) is stopped in a state where the converged state (coefficient vector) so far is fixed. For this reason, the adaptive filter 21 (RR) does not change such that the reference signal in which the speech signal of the self-speaker speech is dominant is input and the convergence filter is moved away from the convergence state.
Further, at this time, the transmitted sound transmitted from the speaker 3 (R) to the microphone 2 (R) via the spatial propagation path Srr appropriately includes a dominant component of the speaker voice on the side. This is generated as an echo sound. However, the echo sound of the self-speaker voice is also transmitted from the speaker 3 (R) to the microphone 2 (R) via the spatial propagation path Srr. Accordingly, since the adaptive filter 21 (RR) is fixed in a converged state, the echo sound of the self-side speaker voice is properly canceled together with the echo sound of the counterpart-side speaker voice.

Further, when the adaptive filter 21 (RR) has not converged, the signal cutoff state is set for the volume variable unit 41A. Assuming that the component of the self-side speaker sound collected by the microphone 2 (R) is output from the speaker 3 (R), the adaptive filter 21 (RR) has not converged. As a result, a lot of echo sounds remain, making it very difficult to hear, and the possibility of howling increases. Therefore, in this case, priority is given to suppressing and canceling the echo sound and howling as much as possible, so that the self-speaker voice is not output from the speaker 3. In the double talk state, the echo sound of the other party's voice that remains will be heard according to the degree of convergence. A state in which sound is suppressed is obtained.
At this time, the adaptive processing of the adaptive filter 21 (RR) is stopped, but depending on this, the component of the self-speaker voice collected by the microphone 2 (R) is included as a desired signal (or is dominant). Despite this, the adaptive filter 21 (RR) will no longer change in a direction away from convergence.

Further, the non-talk state is a state in which neither the other party's speaker voice nor the self-side speaker's voice signal is obtained, and therefore, an effective reference signal composed of the other party's speaker's voice signal, No effective desired signal composed of the echo signal of the other party's speaker voice is obtained. In this case, even if the adaptive processing is executed by the adaptive filter 21 (RR), the operation of convergence cannot be obtained. Therefore, the adaptation process of the adaptive filter 21 (RR) is stopped, so that the adaptive filter 21 (RR) is prevented from making a transition to a state far from convergence. When transitioning to the single talk state, the adaptive processing can be started from a state that is closest to convergence within a possible range.
In addition, when the adaptive filter 21 (RR) is converged, by setting a signal passing state for the volume variable unit 41, for example, from a non-talk state to a self-side single talk state or a double talk state. When a transition is made to a state in which the voice signal of the self-speaker voice is obtained as a desired signal, for example, the self-speaker voice can be quickly output from the speaker 3 without the beginning portion being interrupted. .
Further, if the volume variable unit 41A is set in a signal stop state in response to a state where the adaptive filter 21 (RR) has not converged, the self-side single talk state or the double talk state (self side) is changed from the non-talk state. When the state of the adaptive filter 21 (RR) has not converged in the self-side single-talk state and the double-talk state described above, The state of the corresponding audio signal processing unit 11 is obtained.

  Thus, by adopting the configuration as the audio signal processing device 11 according to the present embodiment, the system that cancels the echo sound that passes through the spatial propagation path Srr has reached the state where the adaptive filter 21 (RR) has converged. Thus, the echo sound of the other party's speaker voice can be canceled not only in the other party's single talk state but also in the double talk state. Further, in the double talk state, the echo sound of the self-speaker voice is also canceled. Even in the self-side single talk state, the echo sound of the self-side speaker voice is canceled. That is, in the system for canceling the echo sound passing through the spatial propagation path Srr, it is possible to appropriately cancel both the echo sound of the partner speaker voice and the echo sound of the self speaker voice.

Then, the processing equivalent to that in FIG. 7 is executed for each of the other three adaptive filters 21 (LR), 21 (RL), and 21 (LL).
In other words, the adaptive filter 21 (LR) performs and stops adaptive processing and the volume is variable according to the input state (signal input state) of the reference signal (Xd2) and the desired signal (Mr) of the adaptive filter 21 (LR). Control about the passage / blocking of the unit 41C is executed.
Further, depending on the input state (signal input state) of the reference signal (Xd1) and the desired signal (Ml) of the adaptive filter 21 (RL), execution / stop of the adaptive processing of the adaptive filter 21 (RL), and volume adjustment Control about the passage / blocking of the unit 41B is executed.
Also, depending on the input state (signal input state) of the reference signal (Xd2) and the desired signal (Ml) of the adaptive filter 21 (LL), execution / stop of the adaptive processing of the adaptive filter 21 (LL) and the volume change Control about the passage / blocking of the unit 41D is executed.
With such a configuration, for the system that cancels the echo sound that passes through the spatial propagation paths Srr and SrlSll, together with the spatial propagation path Srr, the echo sound of the other party's speaker voice and the echo sound of the self-side speaker voice Both can be canceled appropriately.

  In executing the processing shown in FIG. 7, the configuration of the audio signal processing unit 11 of the present embodiment is essentially configured with an adaptive filter for canceling the echo sound of the other party's speaker audio. It is also used to cancel the echo sound of the self-speaker voice. That is, it is not configured to newly provide an adaptive filter for canceling the echo sound of the self-speaker speech, and the amount of computation and resources can be reduced accordingly.

By the way, in steps S202, S207, and S208 in the process of FIG. 7, the sound volume variable unit 41A (41B, 41C, and 41D) will be described as setting attenuation factors corresponding to two states of a signal blocking state and a signal passing state. However, in practice, continuous change control of the attenuation rate (or a control value corresponding thereto) may be performed.
For example, a control value λ indicating the degree of signal passage in the sound volume variable unit 41A (41B, 41C, 41D) is defined. This control value λ is assumed to be λ = 1 when the signal is completely passed, and λ = 0 when the signal is completely cut off.
In addition, in setting the attenuation factor, in the case of the volume variable unit 41A, for example, λ = (max (1, Ye1 / Xd1) * w (max (1, Ye1 / Xd1) is 1 and the level of the audio signal Y and the level of the audio signal Xd means that the larger value is selected, and the coefficient w represents an operation expressed by the adaptive filter 21 (RR). In accordance with the control value obtained in this way, the attenuation rate of the volume variable section can be set more flexibly.
Similarly, with regard to the adaptive filter 21 as well, in steps S204, S205, and S209 in FIG. 7, the adaptive processing is set to either the execution state or the stop state, but this is also continuous. Control can be performed. In other words, the adaptive process is continuously transitioned between a state of activation tendency (active tendency state) and a state of tendency to stop or stop (stop tendency state). can do.
For this purpose, for example, one of the parameters of the adaptive filter 21, and the step size parameter μ for setting the coefficient update amount of the FIR filter, μ = (1-λ) * (max (1, Y / By performing the calculation represented by Xd)), the response speed of the adaptive processing of the adaptive filter 21 can be changed.
If such continuous control is performed, for example, it depends on the intermediate state between the partner-side single talk state, the self-side single talk state, the double talk state, and the non-talk state described above. An adapted signal processing operation can be obtained. For example, even in the double talk state, if the adaptive filter 21 does not converge in a talk state in which the self-side speaker voice is low and is close to the counterpart single talk state, the attenuation of the corresponding volume variable unit 41 is attenuated. It is possible to increase the rate to some extent so as to suppress the self-speaking sound and to activate the adaptive processing to some extent so as to make it converge.

  Moreover, although the embodiment described so far shows a configuration in the case of supporting a stereo channel as a multi-channel, it is possible to configure a loudspeaker communication system that transmits and receives multi-channel audio of 3 channels or more. Even in this case, it is possible to apply the configuration based on the present invention and perform the volume balance setting between three or more channels.

In addition, adaptive algorithms used for the adaptive filters 21 (RR), 21 (LR), 21 (RL), 21 (LL) in the adaptive filter systems 20A and 20B shown in FIG. What is necessary is just to select an appropriate thing from the future technology proposed from now on, in addition to what has been proposed. Further, each of the adaptive filter systems 20A and 20B shown in FIG. 4 first synthesizes the cancellation signals output from the two adaptive filters, and uses the collected sound of the microphone of the corresponding channel as a desired signal. The output of the adaptive filter system (Ye1, Ye2) is obtained by subtracting the combined canceling signal of, but according to a more basic format, for example, every two adaptive filters After subtracting the cancel signal from the collected sound of the microphone of the channel to obtain the echo cancel output corresponding to each spatial propagation path, the echo cancel output is added and synthesized as signals Ye1 and Ye2. It is good also as such a structure. In any case, the configuration of the adaptive filter systems 20A and 20B shown in FIG. 4 is merely one of the basic configurations for convenience of explanation. In fact, more advanced and improved configurations may be adopted.
Further, in the explanation of the embodiments so far, for the sake of convenience of explanation, the audio signal processing unit 11 is configured to execute the audio signal processing corresponding to the entire audible band. As an example, in practice, for example, the collected sound signal obtained by collecting sound by the microphone 2 and the sound signal received by the decoder 14 are divided into predetermined frequency bands and divided. Alternatively, a so-called filter bank configuration may be employed in which the configuration of the audio signal processing unit 11 illustrated in FIG. 4 is assigned to each frequency band.

In the above embodiments, the audio signal processing unit 11 as an echo canceller has been described as executing digital signal processing. However, for example, at least a part of the echo cancellation operation executed by the audio signal processing unit 11 is described. The present invention can also be applied when the above is realized by an analog circuit.
In the description of the embodiments so far, it is assumed that the two audio communication terminal devices 1-1 and 1-2 communicate in a one-to-one relationship in the video conference system. This is because the simplest form of the video conference system is taken as an example in consideration of the simplicity. Therefore, in practice, it is conceivable to construct a video conference system with three or more voice communication terminal devices to perform one-to-many communication. However, even in such a system configuration, a configuration based on the present invention is also possible. Can be applied to individual voice communication terminal devices.
Further, the processing of the transmission audio signal and the reproduction audio signal in the audio communication terminal apparatus 1 is mainly based on the digital signal processing, but the transmission audio signal and the reproduction audio signal when the digital signal processing is performed are performed. The format is not particularly limited. For example, in the case of outputting a playback audio signal, a configuration in which a ΔΣ-modulated bit stream format audio signal is reproduced by class D amplification may be considered in some cases.
Moreover, although the voice communication terminal device provided for voice transmission / reception in the video conference system is taken as an example as an embodiment, other than this, for example, a voice conference system or a hands-free call in a telephone device It can be applied to all devices that can be regarded as a so-called loudspeaker communication system, including functions.

It is a block diagram which shows the structural example of the audio | voice transmission / reception system in the video conference system corresponding to embodiment of this invention. It is a figure which shows the example of a relationship regarding the distance by the arrangement position of a microphone / speaker in the place where the audio | voice transmission / reception system shown by FIG. 1 is provided. It is a block diagram which shows the internal structural example of the audio | voice communication terminal device shown by FIG. It is a figure which shows the structural example of the audio | voice signal processing part as embodiment. It is a figure which shows the structural example of the volume for self-speech sounds provided in the audio | voice signal processing part as embodiment. It is a flowchart which shows the example of a procedure which the audio | voice signal processing part of embodiment performs. It is a flowchart which shows the example of a procedure which the audio | voice signal processing part of embodiment performs.

Explanation of symbols

  1 (1-1 · 1-2) Voice communication terminal device, 2 (R) · 2 (L) (2-1 (R) · 2-1 (L) · 2-2 (R) · 2-2 ( L)) Microphone, 3 (R), 3 (L) (3-1 (R), 3-1 (L), 3-2 (R), 3-2 (L)) Speaker, 11 Audio signal processing unit , 12 codec section, 13 encoder, 14 decoder, 15 communication section, 20A / 20B adaptive filter system, 21 (RR) / 21 (LR) / 21 (RL) / 21 (LL) adaptive filter, 22 (R) / 22 (L) Adder, 23 (R) / 23 (L) Subtractor, 31 Self-sound volume, 32 Transmitted sound volume, 33/34 Adder, 41A / 41B / 41C / 41D Volume variable section, 42A / 42B Adder

Claims (3)

Each picked-up sound signal obtained by picking up sound with a plurality of microphones provided corresponding to each channel constituting a predetermined multi-channel configuration is used as a desired signal, and a sound signal transmitted from the communication partner side is received. And a plurality of speaker output audio signals corresponding to each channel, which are audio signals that have undergone a predetermined processing stage until sound is emitted from a plurality of speakers corresponding to each channel of the multi-channel configuration. And performing adaptive processing so that the output signal obtained by subtracting the cancellation signal generated based on the reference signal from the desired signal is minimized, for each channel corresponding to the multi-channel configuration. So that the output signal is the audio signal to be transmitted to the other communication device. And adaptive processing means,
Balance setting means for inputting the output signal for each channel obtained by the adaptive processing means, changing and setting the balance of the signal level between the channels, and
Synthesis means for synthesizing the output signal for each channel output from the balance setting means so as to be included as a component of the audio signal for speaker output of the corresponding channel;
Corresponding to a certain channel when a collected voice signal obtained by collecting a microphone corresponding to a certain channel among the plurality of microphones has a content corresponding to the voice input state of the speaker on the own side. Balance determining means for determining a balance of signal levels between channels to be set by the balance setting means in accordance with each distance between the microphone and a plurality of speakers corresponding to each of the channels constituting the multi-channel configuration. When,
An audio signal processing device comprising: The balance determining means is
Based on the volume ratio of the arrival sound from each speaker to the microphone corresponding to the certain channel, which is obtained when the sound of the same volume is emitted from a plurality of speakers corresponding to each channel constituting the multi-channel configuration. The balance of the signal level between the channels to be set by the balance setting means is determined.
The audio signal processing apparatus according to claim 1. Each picked-up sound signal obtained by picking up sound with a plurality of microphones provided corresponding to each channel constituting a predetermined multi-channel configuration is used as a desired signal, and a sound signal transmitted from the communication partner side is received. And a plurality of speaker output audio signals corresponding to each channel, which are audio signals that have undergone a predetermined processing stage until sound is emitted from a plurality of speakers corresponding to each channel of the multi-channel configuration. And performing adaptive processing so that the output signal obtained by subtracting the cancellation signal generated based on the reference signal from the desired signal is minimized, for each channel corresponding to the multi-channel configuration. So that the output signal is the audio signal to be transmitted to the other communication device. And adaptive processing procedure,
A balance setting procedure for inputting the output signal for each channel obtained by the adaptive processing procedure, changing and setting the balance of the signal level between the channels, and outputting,
A synthesis procedure for synthesizing the output signal for each channel output by the balance setting procedure so as to be included as a component of the audio signal for speaker output of the corresponding channel;
A microphone corresponding to a certain channel when a collected voice signal obtained by collecting a microphone corresponding to a certain channel among the plurality of microphones has a content corresponding to a self-speaker voice input state. And a balance determination procedure for determining the balance of the signal level between the channels to be set by the balance setting procedure according to each distance from a plurality of speakers corresponding to each channel constituting the multi-channel configuration,
The audio signal processing method characterized by performing.

Images