JP2009124386A - Voice signal processor, and voice signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To constantly obtain successful reproduced sound as much as possible corresponding to uncertainty of a speaker which is used in an acoustic system of sound reinforcement communication system. <P>SOLUTION: An echo canceler which performs echo cancellation by adaptive processing is constituted so that frequency characteristics of a voice signal for reproduction sound source are changed based on frequency characteristics of the voice signal for reproduced sound source input as the one which becomes the basis of speaker reproduced sound and frequency characteristics of a collected voice signal collected from a microphone, and the voice signal after change is output to the speaker side. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an audio signal processing apparatus having an audio signal processing function called so-called echo cancellation and a method thereof.

In addition to hands-free phone calls, voice communication systems such as audio conferencing systems and video conferencing systems can be used to make calls and conversations between speakers at remote locations and locations. The constructed acoustic system is also called a loudspeaker call system, and has already been put into practical use and has become widespread.
In the above voice communication system, for example, communication terminal devices that can communicate with each other according to some communication method are arranged at a plurality of different locations. In addition, the sound picked up by the microphone on one communication terminal apparatus side is transmitted from the one communication terminal apparatus to the other communication terminal apparatus, and is received from the speaker on the other communication terminal apparatus side that has received the sound. It is made to emit as sound. As a result, conversations between speakers at remote locations are possible.
For example, Patent Document 1 describes a loudspeaker-based sound system configured to perform echo cancellation by providing an adaptive filter.

JP 2007-235770 A

By the way, when actually using the sound system of the loudspeaking call system, the speaker is often connected using a common one. For example, in the case of a video conference system, a monitor for displaying the state of the other conference hall, a speaker attached to the television receiver, and the like are often used. This means that when a sound system of a loudspeaking call system is used, it is not possible to specify what kind of sound reproduction characteristic speaker is used, that is, there is uncertainty about the speaker used. I mean.
However, the communication terminal device that is the core of the sound system of the loudspeaking call system is generally configured only to output a sound signal for speaker output with a certain frequency characteristic setting.
For this reason, when the reproduction acoustic characteristic of the speaker used is not good, this characteristic is reflected as it is in the reproduced sound, and it becomes impossible to obtain good reproduction sound quality.
SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to make it possible to always obtain the best possible reproduced sound in response to the uncertainties of the speakers used in a loudspeaker-based sound system.

In order to solve the above-described problems, the present invention is configured as follows as an audio signal processing apparatus.
In other words, the sound collection sound signal obtained based on the sound collected by the microphone is obtained by using the sound signal obtained based on the sound signal for reproduction sound source input by the sound signal processing apparatus as the sound source to be emitted from the speaker. Echo cancellation processing means for executing adaptive signal processing so that the audio signal component of the sound that is included in the desired signal and is emitted from the speaker and reaches the microphone is minimized as the desired signal Obtained by changing the frequency characteristic of the sound signal for reproduction sound source based on the frequency characteristic of the sound signal for reproduction sound source and the frequency characteristic of the sound coming from the speaker that has been emitted from the speaker and reached the microphone. Frequency characteristic changing means for outputting the characteristic-changed audio signal to the speaker side.

  The audio signal processing apparatus having the above configuration is provided with an echo cancellation function of canceling a component of sound emitted from the speaker picked up by the microphone. Then, the frequency characteristic of the reproduction sound source audio signal is changed based on the frequency characteristic of the reproduction sound source audio signal and the frequency characteristic of the collected sound signal. Then, an audio signal in which the frequency characteristic of the reproduction sound source audio signal is changed is output so as to be emitted as sound from the speaker.

  As a result, the present invention can always obtain a good reproduced sound regardless of the uncertainty of the speaker used in the sound system of the loudspeaking call system.

As the best mode for carrying out the present invention (hereinafter referred to as an embodiment), for example, the present invention is applied to an audio transmission / reception system in a video conference system (video conference system).
The video conference system installs a communication terminal device for each of a plurality of conference halls at different locations, and causes the communication terminal device to transmit an image captured by a camera device and sound collected by a microphone to other communication terminal devices. At the same time, it is configured to receive an image and a sound transmitted from another communication device and output them from a display device and a speaker, respectively. That is, the video conference system includes a video transmission / reception system that transmits / receives images to / from each other and an audio transmission / reception system that transmits / receives audio to / from each other. And as this Embodiment, it is set as the communication terminal device (voice communication terminal device) provided in order to transmit / receive an audio | voice as said audio | voice transmission / reception system.

FIG. 1 shows a system configuration example of an audio transmission / reception system in a video conference system.
In this case, two places A and B that are separated from each other are used as conference halls, and in each of these places A and B, the voice communication terminal apparatuses 1-1 and 1-2 that form a voice transmission / reception system are provided. Installed. These voice communication terminal apparatuses 1-1 are connected by a communication line corresponding to a predetermined communication method so that mutual communication is possible. In addition, microphones 2-1 and 2-2 and speakers 3-1 and 3-2 are installed at locations A and B, respectively. The microphones 2-1 and 2-2 are for collecting voices of the conference participants in the locations A and B (self-speaker speech), and are provided at appropriate positions in the locations. . The speakers 3-1 and 3-2 are for listening to the voice of the conference participant in the other place (the other party's voice), and are also provided at appropriate positions in each place. In the following description, the voice communication terminal device 1, the microphone 2, the speaker 3, etc., unless it is necessary to distinguish between the voice communication terminal device, the microphone, and the speaker, particularly those in the remote place. The notation is as follows.

First, at a location A, a sound signal (sound collected sound signal) obtained by collecting sound by the microphone 2-1 is input to the sound communication terminal device 1-1. The voice communication terminal device 1-1 transmits the input voice signal to the voice communication terminal device 1-2 via the communication line. The voice communication terminal device 1-2 receives the voice signal transmitted as described above and outputs it from the speaker 3-2. Thereby, the meeting participant in the place B can listen to the voice of the meeting participant in the place A.
Similarly, the sound obtained by picking up the sound from the microphone 2-2 in the location B is transmitted to the sound communication terminal apparatus 1-1 by the sound communication terminal apparatus 1-2. The voice communication terminal apparatus 1-1 outputs the received voice signal from the speaker 3-1.
In this way, the audio transmission / reception system of the video conference system performs two-way audio communication. For example, a conference participant in one location and a conference participant in another location can It is possible to have a conversation between. In addition, in the case of this video conference system, it is assumed that there are a plurality of conference participants at each location. For this reason, all the conference participants at each location are considered to be conference participants at other locations. The speaker 3 is provided so that the voice can be heard. A system that performs two-way audio exchange using a speaker in this manner is also called a loudspeaker call system.

FIG. 2 shows a configuration example of the voice communication terminal device 1. For confirmation, the voice communication terminal apparatuses 1-1 and 1-2 shown in FIG. 1 have the configuration shown in FIG. 2 in common.
The voice communication terminal device 1 includes an A / D converter (ADC) 11, a D / A converter (DAC) 12, an audio signal processing unit 13, a codec unit 14, and a communication unit 17, for example, as shown in FIG. Become.

  The A / D converter 11 converts an analog audio signal (collected audio signal) obtained by collecting sound with the microphone 2 into a digital format and outputs the digital signal to the audio signal processing unit 13.

As described above, if the loudspeaker communication system is used as it is, phenomena such as echo and howling occur. That is, as shown in FIG. 2, the sound emitted from the speaker 3 to the space reaches the microphone 2 via a spatial propagation path (echo path) S as a direct sound and an indirect sound. That is, the voice of the communication partner transmitted from the voice communication terminal device on the communication partner side and emitted from the speaker 3 is picked up by the microphone 2 and transmitted again to the voice communication terminal device on the communication partner side. Further, on the communication partner side, sound emitted from the speaker is further picked up by the microphone and transmitted to the voice communication terminal device here. That is, in the loudspeaker communication system, the sound once released into the space is transmitted / received in a circulating manner between the voice communication terminal devices. As a result, the sound emitted from the speaker includes a sound in which the voice he / she is currently speaking can be heard with a certain delay time. This is an echo, and howling occurs when the loop gain exceeds 1.
In view of this, in the loudspeaker communication system, an echo canceling system that eliminates and suppresses such an echo phenomenon is provided. The audio signal processing unit 13 is configured to have a signal processing function as the echo cancellation system. The audio signal processing unit 13 is actually configured as a DSP (Digital Signal Processor), for example. A configuration for echo cancellation by the audio signal processing unit 13 will be described later.

  The audio signal that has been subjected to echo cancellation processing by the audio signal processing unit 13 is input to the encoder 15 in the codec unit 14. The encoder 15 performs signal processing such as voice compression encoding according to a predetermined method on the input voice signal and outputs the signal to the communication unit 17. The communication unit 17 outputs the audio signal input from the encoder 15 to another audio communication terminal device via a communication line according to a predetermined communication method.

In addition, the communication unit 17 receives an audio signal transmitted from another audio communication terminal device, restores the audio signal in a predetermined compression encoding format, and outputs the audio signal to the decoder 16 of the codec unit 14.
The decoder 16 performs a demodulation process on the compression encoding of the input audio signal, converts it into a predetermined PCM (Pulse Code Modulation) format digital audio signal, and outputs it to the audio signal processing unit 13. The audio signal that has passed through the audio signal processing unit 13 is converted into an analog signal by the D / A converter 12 and then output. The output audio signal is output from the speaker 3.
Actually, in order to emit the audio signal converted into analog by the D / A converter 12 as sound by the speaker 3, it is necessary to amplify the audio signal. Is not shown.

  FIG. 3 shows an example of the internal configuration of the audio signal processing unit 13 that is the basis of the present embodiment. In this figure, the A / D converter 11, the D / A converter 12, and the codec unit 14 (encoder 15 and decoder 16) are shown together with the audio signal processing unit 13.

The audio signal processing unit 13 shown in this figure is configured as an adaptive filter system including an adaptive filter (ADF: Adaptive Digital Filter) 21 and a subtractor 22.
The adaptive filter 21 inputs the audio signal output from the decoder 16 and input to the D / A converter 12 as a reference signal. The adaptive filter 21 generates and outputs a pseudo echo signal (cancellation signal) from the input signal by an adaptive process according to a predetermined adaptive algorithm using the reference signal and an error signal described later, and a subtractor 21.
The subtractor 22 inputs a sound signal output from the A / D converter 11, that is, a sound-collected sound signal obtained by collecting sound by the microphone 2 as a desired signal. Then, the output signal of the adaptive filter 21 is subtracted from the desired signal and output. The output of the subtracter 22 is output to the encoder 14. The output of the subtractor 22 input to the adaptive filter 21 is called an error signal or residual signal.

Although the description of the inside of the adaptive filter 21 is omitted, an FIR (Finite Impulse Response) type digital filter of the required order through which the above reference signal passes, and a coefficient (filter coefficient) of this digital filter ) Is variably set according to a predetermined adaptive algorithm. The output of the digital filter is the output signal of the adaptive filter 21 and becomes a pseudo echo signal (cancellation signal).
Then, the adaptive filter 21 always obtains an output signal (cancellation signal) that minimizes the residual amount indicated by the error signal, so that the coefficient setting circuit performs a filter of the coefficient unit at the required order stage. Change and set the coefficient.
As a result, the coefficient vector of the adaptive filter 21 (corresponding to an array of coefficients corresponding to the order stage) is output from the speaker 3 as an audio signal (reference signal) at the stage input to the D / A converter 12. A transmission path (hereinafter referred to as canceling sound transmission) from the A / D converter 11 to the subtracter 22 as a desired signal after being picked up by the microphone 2 via the spatial propagation path S (see FIG. 2). An impulse response representing a pseudo transfer function (also called a path) is formed. That is, this operation is an operation for adaptively canceling the signal component of the sound obtained via the canceling sound transmission path according to the state of the signal to be processed at that time.
The sound that passes through the canceling sound transmission path is output from the decoder 16 and finally supplied to the speaker 3 as can be seen from the spatial propagation path S that is an echo path. This is the component of the echo sound based on it. Therefore, the output signal (cancellation signal) of the adaptive filter 21 is regarded as a pseudo echo for the audio signal to be reproduced as sound from the speaker 3. In the audio signal processing unit 13 as the adaptive filter system, the pseudo echo sound is subtracted by the subtracter 22 from the audio signal to be supplied to the encoder 15 for transmission to the communication partner side. In this way, the audio signal processing unit 13 performs an operation of adaptively removing the component of the echo sound from the audio signal to be transmitted to the communication partner side.
Then, the voice communication terminal device 1 transmits the voice signal from which the echo sound component has been removed to the voice communication terminal device on the communication partner side. As a result, the echo sound is also removed from the sound that is heard by releasing the sound signal received by the voice communication terminal device on the communication partner side from the speaker. In this way, an echo canceling effect is produced.

By the way, in the audio transmission / reception system in the video conference system described so far, an appropriate speaker 3 may be introduced in advance in the system. Connecting is often done. Then, in the latter case, there arises uncertainty that the speaker to be used by being connected to the voice communication terminal device 1 cannot be specified.
When uncertainty is generated for the speaker used, this means that uncertainty is also generated in the reproduction frequency characteristic of the speaker used (here, the frequency characteristic of the speaker itself). This also means that there is a possibility that an unsound sound that is not suitable for a video conference system may be used as the speaker 3. As a specific example of this, in a video conference system, a speaker provided in a television receiver is often used. However, with such a speaker, there is a low frequency in the frequency characteristics and it is difficult to hear a muffled feeling. It may become.
In such a case, for example, in the case of the basic configuration of the voice communication terminal device 1 as shown in FIG. 3, a voice signal having a certain fixed frequency characteristic is transmitted from the voice communication terminal device 1 to the speaker side. Can only output. For this reason, the sound emitted from the speaker 3 is directly affected by the reproduction performance (reproduction frequency characteristics) of the speaker 3 itself, and only deteriorated sound is produced.

  Therefore, in order to eliminate such inconvenience, the present embodiment is configured such that the frequency characteristic of the sound signal of the sound to be output from the speaker can be automatically changed according to the reproduction frequency characteristic of the speaker, for example. . As a result, it is possible to emit sound that is as good as possible and easy to hear from the speaker in conformity with the uncertainty of the reproduction frequency characteristic of the speaker.

First, FIG. 4 shows a configuration example of the audio signal processing unit 13A corresponding to the first embodiment. In this figure, the same parts as those in FIG.
In the audio signal processing unit 13A shown in this figure, a frequency characteristic correction unit 23 is provided together with the adaptive filter system 20.
The frequency characteristic correction unit 23 receives the audio signal S1 output from the decoder 16 and a signal (desired signal) at a stage output from the A / D converter 11 and input to the subtractor 22 of the adaptive filter system 20. Thus, as described later, the frequency characteristic of the audio signal S1 can be dynamically corrected (changed) and output. The audio signal S1 whose frequency characteristic is corrected by the frequency characteristic correcting unit 23 is output as the audio signal S2. The audio signal S2 is first input to the D / A converter 12 as an audio signal of a sound to be output from the speaker 3. In this case, the audio signal S2 is also input as a reference signal to the adaptive filter 21 of the adaptive filter system 20.
2 and FIG. 3, for example, when viewed from the audio signal processing unit 13A, the audio signal S1 at the stage of output from the decoder 16 is the audio to be output from the speaker 3 (the other party side). It can be viewed as a voice signal (sound signal for reproduction sound source) input as a source of speaker voice.

With reference to FIGS. 5 and 6, the configuration of the frequency characteristic correction unit 23 and an operation example thereof will be described.
FIG. 5 illustrates an internal configuration example of the frequency characteristic correction unit 23. First, the audio signal S1 output from the decoder 16 is input to the frequency characteristic analysis unit 30a and branches to the frequency characteristic variable unit 32 as a change process target signal whose frequency characteristic is to be changed. Entered.
The audio signal S3 output from the A / D converter 11 is input to the frequency characteristic analysis unit 30b.
The frequency characteristic analysis unit 30a obtains a frequency characteristic for the audio signal S1 by executing, for example, a fast Fourier transform process on the input audio signal S1, that is, for the audio signal S1, from the time domain signal to the frequency domain. Convert to signal. Similarly, the frequency characteristic analysis unit 30b obtains a frequency characteristic for the audio signal S3.
Note that the frequency characteristic of the collected sound signal, which is the sound signal S3 obtained in this way, is the frequency characteristic of the sound that has passed through the cancellation sound transmission path, that is, the sound signal emitted from the speaker 3 and has reached the microphone 2. It can be grasped as a frequency characteristic of a supposed sound (speaker incoming sound).
The frequency characteristic analysis unit 30a causes the correction amount acquisition unit 31 to input a frequency characteristic signal Sa, which is a signal having frequency characteristic information obtained by itself, as comparison reference data. Similarly, the frequency characteristic analysis unit 30b causes the correction amount acquisition unit 31 to input a frequency characteristic signal Sb, which is a signal having frequency characteristic information obtained by itself, as comparison target data.

  The correction amount acquisition unit 31 obtains a correction amount (frequency characteristic correction amount) for the frequency characteristic of the audio signal S1 based on the frequency characteristic indicated by the input frequency characteristic signals Sa and Sb. Then, a correction amount signal Se, which is a signal having information on the frequency characteristic correction amount obtained in this way, is output to the frequency characteristic variable unit 32.

  Based on the frequency characteristic correction amount indicated by the correction amount signal Se, the frequency characteristic variable unit 32 executes a process for changing the frequency characteristic of the audio signal S1 input as a change processing target, and the changed audio signal Is output as an audio signal S2 (characteristic change audio signal).

FIG. 6 shows an example of specific operations of the correction amount acquisition unit 31 and the frequency characteristic variable unit 32 in the frequency characteristic correction unit 23.
Assume that the frequency characteristic indicated by the frequency characteristic signal Sa is as shown in FIG. 6A, whereas the frequency characteristic indicated by the frequency characteristic signal Sb is as shown in FIG. 6B. Note that the frequency characteristic here represents the relationship of the level (amplitude) to the frequency.
The frequency characteristics shown in FIG. 6 are simplified in a simplified manner, but the frequency characteristics indicated by the frequency characteristic signal Sb are higher than a certain frequency compared to the frequency characteristics indicated by the frequency characteristic signal Sa. The amplitude in the band is small.

Here, in order to make the explanation simple and easy to understand, it is assumed that the microphone 2 and the speaker 3 are close to each other and the frequency characteristics of the microphone 2 and the A / D converter 11 are flat.
In this case, the frequency characteristic indicated by the frequency characteristic signal Sb indicates the frequency characteristic of the sound emitted from the speaker 3 as it is. Then, the change in the frequency characteristic indicated by the frequency characteristic signal Sb (comparison target data) with respect to the reference frequency characteristic indicated by the frequency characteristic signal Sa (comparison reference data) corresponds to the reproduction frequency characteristic of the speaker 3. . Therefore, the correction amount acquisition unit 31 in the frequency characteristic correction unit 23 first estimates the reproduction frequency characteristic of the speaker 3 based on comparing the frequency characteristics indicated by the frequency characteristic signal Sa and the frequency characteristic signal Sb. For example, based on the frequency characteristics shown in FIGS. 6A and 6B, the reproduction frequency characteristics of the speaker 3 shown in FIG. 6C are estimated. In this figure, the frequency characteristic is shown in a simplified manner, but the reproduction frequency characteristic shown in FIG. 6C is a characteristic in which the level of reproduced sound decreases in a band above a specific frequency. It shows that there is. This actually represents an example in which the high frequency falls corresponding to the area indicated by the broken line in FIG.

Next, in the correction amount acquisition unit 31, the frequency characteristic of the sound emitted from the speaker 3 is based on the reproduction frequency characteristic of the speaker 3 estimated as described above, and the frequency of the audio signal S1 that is the reproduction sound source audio signal. In order to be as close as possible to the characteristic, a frequency characteristic correction amount to be given to the audio signal S1 is obtained.
For example, the frequency characteristic correction amount determined in correspondence with the case where the reproduction frequency characteristic of the speaker 3 shown in FIG. 6C is estimated is as shown in FIG. In this example, a frequency characteristic correction amount is obtained by giving a level (correction level) that just compensates for the reduced frequency band of the estimated reproduction frequency characteristic of the speaker 3 as it is. Note that the correction amount in this case has a frequency band to be corrected and information on a level to be corrected in this frequency band as physical quantity information, as can be understood from the above description. The frequency band and correction amount acquisition unit 31 outputs a correction amount signal Se, which is a signal of information representing the frequency characteristic correction amount obtained in this way, to the frequency characteristic variable unit 32.

  Based on the correction amount signal Se input as described above, the frequency characteristic variable unit 32 executes a process for changing the frequency characteristic of the audio signal S1 input as the change process target signal. Is output as an audio signal S2. In the case of FIG. 6, the frequency characteristic of the audio signal S2 obtained by changing the frequency characteristic of the audio signal S1 is as shown in FIG. The frequency characteristic shown in FIG. 6E is obtained by adding the correction amount shown in FIG. 6D to the frequency characteristic of the audio signal S1 (FIG. 6A).

When the audio signal S2 having the frequency characteristic shown in FIG. 6E is supplied to the speaker 3 via the D / A converter 12, the frequency characteristic of the sound emitted from the speaker 3 is as shown in FIG. In the frequency characteristic shown in e), the portion of the frequency band whose level is raised corresponding to the addition of the correction amount is canceled by the reproduction frequency characteristic of the speaker 3 shown in FIG. In this way, the level is suppressed. That is, the frequency characteristic shown in FIG. 6A is obtained. This means that the sound having the same frequency characteristic as that of the audio signal S1 (reproduction sound source audio signal) is emitted from the speaker 3 by correcting the change in the frequency characteristic caused by the reproduction frequency characteristic of the speaker 3. .
In this way, by correcting the frequency characteristics of the sound emitted from the speaker 3, the sound that is as faithful as possible to the original sound (reproduction sound source audio signal) without being affected by the reproduction frequency characteristic of the speaker 3. Can be emitted by the speaker 3. Further, in the configuration described with reference to FIGS. 5 and 6, the frequency characteristic correction amount is performed based on the result of estimating the reproduction frequency characteristic of the speaker 3, so that any speaker is connected. Even if it is done, it can respond. In other words, even when the speaker connected to the voice communication terminal device 1 is indefinite, the correction for the sound emitted from the speaker is automatically performed so as to conform to the reproduction frequency characteristics of the actually connected speaker. You can get results done in.
Further, in the configuration described with reference to FIGS. 5 and 6, correction can be performed dynamically. Therefore, even in an environment in which the reproduction frequency characteristic of the speaker 3 can change with the passage of time, a sound that is always optimized is output from the speaker 3 so as to respond to the change. Correction will be performed. For example, when used together with an acoustic system that can switch a speaker to be used in the middle, correction that is adapted to the reproduction frequency characteristics of the switched speaker is automatically performed.

For confirmation, FIG. 6 shows an example in which a result obtained by obtaining a frequency characteristic correction amount with a higher level in a specific frequency band is obtained. However, depending on the algorithm for obtaining the frequency characteristic correction amount described with reference to FIG. 6, the frequency characteristic correction amount for lowering the level of the specific frequency band is obtained according to the relationship between the frequency characteristics of the comparison reference data and the comparison target data. It may become. In some cases, a frequency characteristic correction amount may be obtained in which a level is increased in a certain frequency band and a level is decreased in another certain frequency band.
In addition, the example shown in FIG. 6 results in the frequency characteristic indicated by the comparison reference data (frequency characteristic signal Sa) shown in FIG. 6A and the comparison target data (frequency shown in FIG. 6B). The difference (frequency difference characteristic) regarding the frequency characteristic indicated by the characteristic signal Sb) is obtained as the frequency characteristic correction amount. In this case, the actual processing executed by the correction amount acquisition unit 31 includes the frequency characteristic of the comparison reference data (frequency characteristic signal Sa) and the frequency of the comparison target data (frequency characteristic signal Sb) shown in FIG. By calculating the difference from the characteristic, the frequency characteristic correction amount shown in FIG. 6D can be obtained. That is, conceptually, it can be regarded as estimating the reproduction frequency characteristic of the speaker, but in actual processing, the actual frequency characteristic for estimating the reproduction frequency characteristic of the speaker shown in FIG. Thus, the algorithm can be efficiently constructed by omitting the above procedure.

FIG. 7 is a flowchart showing a signal processing procedure in the voice communication terminal apparatus 1 having the configuration of the voice signal processing unit 13A corresponding to the first embodiment shown in FIG.
First, in step S101, the communication unit 17 receives a voice signal transmitted from the voice communication terminal device on the communication partner side.
As described above, the audio signal received in step S101 is in a format subjected to audio compression encoding. Therefore, in step S102 of the procedure following step S101, the decoder 16 in the codec unit 14 performs demodulation (decoding) processing for decoding the audio compression coding on the audio signal obtained in step S101. For example, an audio signal S1 in a predetermined PCM format is obtained.

Further, the voice communication terminal device 1 executes the procedures of steps S103 and S104 in parallel with the procedures of steps S101 and S102. In step S103, an analog collected sound signal obtained by collecting the sound from the microphone 2 is input. In subsequent step S104, the A / D converter 11 converts the collected sound signal into a digital signal (A / D conversion).
The digital collected sound signal (sound signal S3) obtained in step S104 is input to the subtracter 22 of the adaptive filter system 20 as a desired signal for the process of step S106 described later. In addition, the frequency characteristic correction unit 23 is caused to input for the process of step S105 described later.

In step S105, based on the audio signal S1 input according to the process of step S102 and the audio signal S3 input according to the process of step S105 by the frequency characteristic correction unit 23, for example, FIG. As described with reference to FIG. 6, the frequency characteristic of the audio signal S1 is changed (corrected) and output as the audio signal S2.
The audio signal S2 is first input to the D / A converter 12 as an audio signal of sound to be emitted from the speaker 3. In step S109, the D / A converter 12 converts the input audio signal S2 into an analog signal (D / A conversion) and outputs the analog signal to the speaker 3. As a result, the speaker 3 reproduces and outputs the audio signal S2 as sound.
The audio signal S2 is also output as a reference signal to the adaptive filter 21 in the adaptive filter system 20. In step S106, the adaptive filter system 20 uses the audio signal S2 input corresponding to the process of step S105 as a reference signal and the audio signal S3 input corresponding to the process of step S105 as a desired signal. The echo cancellation process described in (1) is executed. Then, the audio signal (the output signal of the subtractor 22) from which the echo sound component has been removed by this echo cancellation processing is output to the encoder 14 for the next step S107.

  The encoder 14 performs voice compression coding on the input voice signal according to the procedure of step S107 and outputs the result to the communication unit 17. As the procedure of step S108, the communication unit 17 executes a process for transmitting the audio signal input from the encoder 14 to the audio communication terminal device 1 on the communication partner side according to a predetermined communication format.

  By the way, in such a loudspeaker communication system, there are a single talk state and a double talk state as call states. The single talk state is a state in which the other party's speaker voice is emitted from the speaker, but the own speaker is not speaking into the microphone. In correspondence with the present embodiment, the voice signal of the other party's speaker voice having an effective level is output from the decoder 16, but the self speaker's voice having an effective level is collected by the microphone 2. There is no state. The double talk state is a state in which the other party's speaker voice is emitted from the speaker and the own speaker is also speaking into the microphone. In correspondence with this embodiment, the decoder 14 is effective. The sound signal of the other party's speaker sound that is regarded as being output, and the self-speaker sound of a certain level or higher that is regarded as valid is also being collected by the microphone 2.

In the correction of the speaker sound according to the present embodiment, the frequency characteristic correction amount is obtained based on the estimation of the reproduction frequency characteristic of the speaker 3. Therefore, what is necessary as the comparison target data is to transmit the cancellation sound transmission path. This is information on the frequency characteristics of the sound, that is, the speaker arrival sound that is supposed to reach the microphone 2 from the speaker 3 via the spatial propagation path S.
In the present embodiment, the voice content of the collected voice signal that is the basis of the comparison target data, but in the single talk state, it can be regarded as being almost the sound coming from the speaker. In this case, there is no particular problem in the correction processing procedure described so far with reference to FIGS. However, in the double talk state, the self-speaker speech component is included in addition to the above-mentioned speaker incoming sound. At this time, the frequency characteristics of the collected sound signal include not only the characteristics of the sound coming from the speaker but also the characteristics of the self-speaker voice. In this state, the reproduction frequency characteristics of the speaker are It is difficult to estimate with high accuracy, and as a result, it may be difficult to obtain a good frequency characteristic correction amount.

  Therefore, as a countermeasure for this, for example, in the double talk state, the frequency characteristic variable unit 32 is output from the correction amount acquisition unit 31 at the timing in the single talk state immediately before the transition to the current double talk state. It can be considered that the frequency characteristic is changed by using a latched (held) correction amount signal Se. The correction amount signal Se latched in this way can be treated as being based on the correct estimation result of the reproduction frequency characteristic of the speaker in the single talk state. Therefore, if the latched correction amount signal Se is fixedly used in the double talk state, it can be sufficiently expected that the sound emitted from the speaker 3 is corrected appropriately.

When the correction amount signal Se in the single talk state latched in the double talk state is used, the frequency characteristic correction amount acquisition process (update of the frequency characteristic correction amount) by the correction amount acquisition unit 31 may be stopped. Absent. For example, when the operation as the correction amount acquisition unit 31 is realized by a DSP, the processing load can be reduced by stopping the operation of the correction amount acquisition unit 31.
Further, an example of a method for determining whether or not the telephone conversation state is the double talk state will be described in the second embodiment below.

FIG. 8 shows an internal configuration example of the audio signal processing unit 13B corresponding to the second embodiment. In this figure, the same parts as those in FIG.
The audio signal processing unit 13B shown in this figure has a configuration in which coefficient information Dk is input from the adaptive filter 21 to the frequency characteristic correction unit 23 with respect to the configuration of the audio signal processing unit 13A shown in FIG. Has been.

The adaptive process executed by the adaptive filter system 20 corresponds to learning the impulse response of the desired signal. As a result of learning the impulse response, the coefficient vector in the FIR filter forming the adaptive filter 21 is changed, and therefore the coefficient vector represents the impulse response described above. The coefficient information Dk is information indicating the coefficient vector, and therefore represents the learned impulse response. The impulse response can be converted into frequency characteristic information by performing frequency analysis (Fourier transform or the like). The impulse response obtained in this case is for the sound that passes through the canceling sound transmission path.
That is, in the second embodiment, in order to obtain the frequency characteristic (comparison target data) of the incoming sound of the speaker that is emitted from the speaker 3 and reaches the microphone 2, the collected sound signal itself is used. In addition to a certain sound signal S1, information on coefficient vectors set in the adaptive filter 21, that is, information on impulse responses for sound passing through the canceling sound transmission path can be acquired.

FIG. 9 shows an internal configuration example of the frequency characteristic correction unit 23 corresponding to the second embodiment. In this figure, the same parts as those in FIG.
As shown in this figure, in the second embodiment, a frequency analysis processing unit 30c corresponding to the coefficient information Dk is provided together with frequency analysis processing units 30a and 30b that perform frequency analysis processing on the audio signals S1 and S2. It is done.
For example, the frequency analysis processing unit 30c converts the input coefficient information Dk into information of an impulse response waveform, and performs frequency analysis processing on the impulse response waveform to obtain frequency characteristics. Then, the frequency characteristic signal Sc is output to the correction amount acquisition unit 31 as a signal having the frequency characteristic information obtained in this way. With this configuration, the correction amount acquisition unit 31 uses the frequency characteristic obtained based on the coefficient vector in the adaptive filter 21 together with the frequency characteristic signal Sb corresponding to the collected sound signal itself as the frequency characteristic information to be compared. The signal Sc can be captured.

FIG. 10 is a flowchart showing a signal processing procedure in the voice communication terminal device 1 having the configuration of the voice signal processing unit 13B corresponding to the second embodiment.
The flow of the procedure from step S201 to step S209 shown in this figure and the basic contents of the processing in each step are the same as steps S101 to S109 in FIG. 7 corresponding to the first embodiment. . However, in the second embodiment, the echo cancellation process by the adaptive filter system 20 is executed in step S206, and the coefficient vector in the adaptive filter 21 set at that time is shown. The coefficient information Dk, which is information, is generated and can be output to the frequency characteristic correction unit 23. At the same time, the processing of the frequency characteristic correction unit 23 shown as step S205 is executed as shown in the flowchart of FIG.

First, in FIG. 11, it is determined in step S301 whether or not it is in a double talk state.
Whether or not it is in the double talk state is determined by, for example, the level of the audio signal S1 or the audio signal S2 in the path inputted from the decoder 16 to the D / A converter 12 and the output of the subtractor 22, that is, the adaptive filter 21. It can be determined by comparing the error signal levels. In order to make the explanation easy to understand, it is assumed that the adaptive filter system 20 is in a converged state, and first consider a single talk state. In this case, an effective level of a certain level or more is obtained as the audio signal S1 (or S2), whereas the error signal is a very small level because the echo component is canceled. It has become. On the other hand, in the above-described double talk state, the component of the self-speaker speech that is included in the desired signal (collected speech signal) is not canceled by the adaptive filter system 20. Therefore, an effective level of a certain level or more is obtained in the voice signal S1 (or S2), and a corresponding level corresponding to the self-speaker voice appears in the error signal. Based on the occurrence of such a level difference, it is possible to determine whether or not the state is a double talk state by comparing both signals.

  When it is determined in step S301 that the double talk state is not obtained, the frequency characteristic correction unit 23 captures the collected sound signal itself, that is, the audio signal S3 as comparison target data in step S302. On the other hand, if it is determined in step S301 that the state is a double talk state, coefficient information Dk is captured as comparison target data in step S303.

  After the comparison target data is fetched in step S302 or S303, frequency characteristic analysis processing is performed on the audio signal S1 that is comparison reference data, the audio signal S3 that is comparison target data, or the coefficient information Dk in step S304. Execute. The frequency characteristic analysis processing here is performed by the frequency characteristic analysis unit 30a for the audio signal S1. Further, the frequency characteristic analyzing unit 30b performs the sound signal S3, and the frequency characteristic analyzing unit 30c performs the coefficient information Dk.

In the subsequent step S305, the correction amount acquisition unit 31 obtains (acquires) the frequency characteristic correction amount and outputs it to the frequency characteristic variable unit 32 as the correction amount signal Se. At this time, if the collected sound signal itself is taken in as comparison target data in step S302, the same processing as in the first embodiment described with reference to FIGS. 5 and 6 is executed. On the other hand, when the coefficient information Dk is captured as comparison target data in step S303, the correction amount acquisition unit 31 captures from the frequency characteristic analysis unit 30c instead of the frequency characteristic signal Sb in FIG. After using the frequency characteristic signal Sc, the reproduction frequency characteristic of the speaker 3 is estimated, the frequency characteristic correction amount is acquired, and the correction amount signal Se is output.
Subsequently, in step S306, the frequency characteristics variable unit 32 changes the frequency characteristics of the audio signal S1 (change processing target signal) based on the correction amount signal Se input in step S305, and the audio characteristics are changed. Output as signal S2.

In such a configuration as the second embodiment, in the double talk state, an impulse response of the collected sound signal obtained as a result of the adaptive processing of the adaptive filter system 20 instead of the collected sound signal itself. The frequency characteristic obtained based on (that is, the coefficient vector of the adaptive filter 21) is used as comparison target data. Since the impulse response in this case is for the above-mentioned speaker incoming sound as a sound component included in the collected sound signal, it is possible to appropriately obtain the frequency characteristic correction amount (correction amount signal Se). . That is, even when the call state is the double talk state, the sound emitted from the speaker 3 can be appropriately corrected.
Further, when the adaptive filter system 20 is properly echo canceled and converged, the impulse response indicated by the coefficient vector in the adaptive filter 21 corresponds to the sound coming from the speaker. Therefore, in the configuration of the second embodiment, when the adaptive filter system 20 is converged, the coefficient information Dk is used as the comparison target data even in a state other than the double talk state (including the single talk state). It can be considered that the configuration is used.

Further, as described above, if it is possible to adopt a configuration in which the coefficient information Dk is used as comparison target data even in a state other than the double talk state, only the coefficient information Dk is used as the comparison target data. A configuration in which the sound signal S3 that is the collected sound signal itself is not used can also be considered as the present embodiment. Such a configuration of the audio signal processing unit 13C is shown in FIG. 12 as a third embodiment. In FIG. 12, the same parts as those in FIGS. 4 and 8 are denoted by the same reference numerals, and description thereof is omitted.
As can be seen from FIG. 12, the comparison target data input to the frequency characteristic correction unit 23 is only coefficient information Dk, and no audio signal S3 (sound-collected audio signal) is input.

FIG. 13 shows an internal configuration example of the frequency characteristic correction unit 23 corresponding to the third embodiment. Note that the same parts as those in FIGS. 5 and 9 corresponding to the second embodiment are denoted by the same reference numerals and description thereof is omitted.
The frequency characteristic correction unit 23 shown in FIG. 13 is for inputting the audio signal S3 and performing frequency analysis processing from the configuration of the frequency characteristic correction unit 23 corresponding to the second embodiment shown in FIG. The frequency characteristic analysis unit 30b is omitted.

  Further, if the signal processing procedure executed by the voice communication terminal device 1 of the third embodiment is described based on the flowchart shown in FIG. 10 as the second embodiment, the digital signal obtained in step S204 is obtained. The collected sound signal (sound signal S3) is output to the adaptive filter system 20 (subtractor 22) of the sound signal processing unit 13C for the process of step S206, and the frequency characteristic correction is performed for the process of step S205. The data is not output to the unit 23. In addition, as a process for changing the frequency characteristics in step S205, the procedure of steps S303 to S306 is regularly executed.

Incidentally, in the embodiment described so far, the adaptive filter system 20 (adaptive filter 21) uses the reference signal as the audio signal S1 after the frequency characteristic change output from the frequency characteristic correction unit 23, that is, the audio signal S2. Have been entered as. In other words, the frequency characteristic correcting unit 23 finally supplies the audio signal input by the audio signal processing unit 13 (13A, 13B, 13C) to the speaker 3 as a sound base to be emitted from the speaker 3. Is inserted before the point where the adaptive filter system 20 inputs the reference signal.
As one modification, the frequency characteristic correction unit 23 is inserted in the subsequent stage of the reference signal. That is, for the frequency characteristic correction unit 23, the reference signal input to the adaptive filter system 20 is processed as an audio signal (comparison reference data). It is also possible to adopt a configuration in which it is inserted as if.
However, the impulse response learned by the adaptive processing by the adaptive filter system 20 for performing echo cancellation is strictly speaking that the reference signal input to the adaptive filter system 20 is transmitted from the speaker 3 via the spatial transmission path S to the microphone 2. , And further corresponds to the sound transmitted through the path (cancellation sound transmission path) input to the adaptive filter system 20 as a desired signal. This means that the reference signal input to the adaptive filter system 20 and the audio signal supplied to the input stage of the speaker 3 should be the same. For this purpose, when the frequency characteristic correction unit 23 is inserted after the reference signal to dynamically change the frequency characteristic, the reference signal input to the adaptive filter system 20 and the input stage of the speaker 3 are supplied. The audio signal will be different. This is equivalent to a mistake in the solution learned by the adaptive process, and therefore it is difficult to execute an appropriate adaptive process. Therefore, in the present embodiment, considering that the frequency characteristic change and correction by the frequency characteristic correction unit 23 are dynamically performed in real time, the frequency characteristic correction unit 23 is connected to the reference signal input point. Is inserted in the previous stage.
However, when the frequency characteristic correction amount is fixed and the frequency characteristic is changed without dynamically changing the frequency characteristic by the frequency characteristic correction unit 23, the frequency characteristic may be inserted after the reference signal. There is no particular problem. As a configuration for changing the frequency characteristic while fixing the frequency characteristic correction amount, for example, the frequency characteristic correction amount is obtained at the initial startup of the audio signal processing unit 13, and thereafter, the frequency characteristic correction amount is fixed and the frequency characteristic correction amount is fixed. You can think of what you want to make changes.

  In the description with reference to FIG. 6, in obtaining the frequency characteristic correction amount, the difference between the frequency characteristic as the comparison reference data and the frequency characteristic as the comparison target data is simply obtained, but this is one example. As long as it is based on the frequency characteristic as the comparison reference data and the frequency characteristic as the comparison target data, it can be considered variously. For example, strictly speaking, a speaker incoming sound that reaches the microphone 2 from the speaker 3 via the spatial transmission path S is not changed until it reaches the microphone 2 depending on the actual distance of the spatial transmission path S or the like. Sound quality (frequency characteristics) can change. The actual sound coming from the speaker is a combination of the direct sound and the reflected sound. For example, the frequency characteristics of the reflected sound can change depending on the material of the reflected wall or the like. In consideration of this, it is conceivable to construct an algorithm and a processing procedure for obtaining a frequency characteristic correction amount.

Further, as an adaptive algorithm employed as the adaptive filter system 20, for example, an appropriate algorithm may be selected from techniques known so far.
The processing of the transmission audio signal and the reproduction sound source audio signal in the audio communication terminal apparatus 1 is mainly performed by the digital signal processing. However, the transmission audio signal and the reproduction sound source audio when performing the digital signal processing are used. The format of the signal is not particularly limited. For example, in the case of outputting a reproduction sound source audio signal, a configuration in which a ΔΣ modulated bit stream format audio signal is reproduced by class D amplification may be considered in some cases.
Moreover, although the voice communication terminal device provided for voice transmission / reception in the video conference system is taken as an example as an embodiment, other than this, for example, a voice conference system or a hands-free call in a telephone device It can be applied to all devices that can be regarded as a so-called loudspeaker communication system, including functions.

  In addition, the configuration for correcting the speaker output sound described so far is established even if the echo cancellation processing function is excluded. Therefore, for example, in an audio system, the speaker output sound has a desired frequency characteristic. The present invention can also be applied to devices, circuits, and the like that perform corrections.

  In addition to the DSP, the audio signal processing operation according to the embodiments described so far can be realized by causing a computer system (CPU) to execute a program. For example, such a program may be stored in a DSP, a computer system, or the like after being stored in, for example, a removable storage medium and then installed (including update) from the storage medium. It is also conceivable that the program can be installed through control from another host device, such as via a predetermined data interface. Furthermore, after storing it in a storage device such as a server on the network, the device having the audio signal processing function corresponding to this embodiment is provided with the network function, downloaded from the server and acquired. It can also be configured to be installable.

It is a block diagram which shows the structural example of the audio | voice transmission / reception system in the video conference system corresponding to embodiment of this invention. It is a block diagram which shows the example of an internal structure of the voice communication terminal device of embodiment. It is a figure which shows the basic structural example about the audio | voice signal processing part with which an audio | voice communication terminal device is provided. It is a figure which shows the internal structural example of the audio | voice signal processing part corresponding to 1st Embodiment. It is a figure which shows the internal structural example of the frequency characteristic correction | amendment part corresponding to 1st Embodiment. It is a figure which shows typically the operation example of the correction amount acquisition part in the frequency characteristic correction part of this Embodiment, and a frequency characteristic variable part. It is a flowchart which shows the example of an audio | voice signal processing procedure which the audio | voice communication terminal device corresponding to 1st Embodiment performs. It is a figure which shows the internal structural example of the audio | voice signal processing part corresponding to 2nd Embodiment. It is a figure which shows the example of an internal structure of the frequency characteristic correction | amendment part corresponding to 2nd Embodiment. It is a flowchart which shows the example of an audio | voice signal processing procedure which the audio | voice communication terminal device corresponding to 2nd Embodiment performs. It is a flowchart which shows the example of a procedure in the step for the frequency characteristic change shown by FIG. It is a figure which shows the internal structural example of the audio | voice signal processing part corresponding to 3rd Embodiment. It is a figure which shows the example of an internal structure of the frequency characteristic correction | amendment part corresponding to 3rd Embodiment.

Explanation of symbols

  1 (1-1, 1-2) Voice communication terminal device, 2 (2-1, 2-2) Microphone, 3 (3-1, 3-2) Speaker, 11 A / D converter, 12 D / A converter , 13 Audio signal processing unit, 14 Codec unit, 15 Encoder, 16 Decoder, 17 Communication unit, 20 Adaptive filter system, 21 Adaptive filter, 22 Subtractor, 23 Frequency characteristic correction unit, 30a / 30b / 30c Frequency characteristic analysis unit, 31 correction amount acquisition unit, 32 frequency characteristic variable unit

Claims (6)

As a sound source to be emitted from the speaker, the sound signal obtained based on the sound signal for reproduction sound source input by the sound signal processing device is used as a reference signal, and the sound collection sound signal obtained based on the sound collected by the microphone is a desired signal. Echo cancellation processing means for performing adaptive signal processing so that the audio signal component of the sound that is emitted from the speaker and reaches the microphone included in the desired signal is minimized,
The frequency characteristic of the reproduction sound source audio signal is changed based on the frequency characteristic of the reproduction sound source sound signal and the frequency characteristic of the speaker arrival sound that is emitted from the speaker and reaches the microphone. Frequency characteristic changing means for outputting the characteristic-changed audio signal obtained by the above to the speaker side;
An audio signal processing device comprising: The frequency characteristic changing means is
It is possible to obtain the frequency characteristics of the collected sound signal and the frequency characteristics of the impulse response obtained as a result of the adaptive signal processing in the echo cancellation processing means,
At least in the double talk state, the frequency characteristic of the impulse response obtained by the adaptive signal processing in the echo cancellation processing means is the frequency characteristic of the speaker incoming sound,
The audio signal processing apparatus according to claim 1. The echo cancellation processing means is
As the reference signal, configured to input the characteristic change audio signal output by the frequency characteristic change means,
The audio signal processing apparatus according to claim 1. The frequency characteristic changing means is
As the frequency characteristic of the sound coming from the speaker, the frequency characteristic of the collected sound signal is obtained.
The audio signal processing apparatus according to claim 1. The frequency characteristic changing means is
As the frequency characteristic of the speaker arrival sound, the frequency characteristic of the impulse response obtained by the adaptive signal processing in the echo cancellation processing means is obtained.
The audio signal processing apparatus according to claim 1. As a sound source to be emitted from the speaker, the sound signal obtained based on the sound signal for reproduction sound source input by the sound signal processing device is used as a reference signal, and the sound collection sound signal obtained based on the sound collected by the microphone is a desired signal. An echo cancellation processing procedure for performing adaptive signal processing so that the sound signal component of the sound that is emitted from the speaker and reaches the microphone included in the desired signal is minimized,
The frequency characteristic of the reproduction sound source audio signal is changed based on the frequency characteristic of the reproduction sound source sound signal and the frequency characteristic of the speaker arrival sound that is emitted from the speaker and reaches the microphone. Frequency characteristic changing procedure for outputting the characteristic-changed audio signal obtained by the above-mentioned to the speaker side,
The audio signal processing method characterized by performing.

Images