JP2007151038A - Sound processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound processing apparatus in which computational complexity required for echo removal processing is decreased, effective echo processing can be performed and echo can be removed from system fluctuation to settlement. <P>SOLUTION: A band selecting means 3 holds a signal component of a frequency band where a sound discharging signal band-divided by a sound discharging signal dividing means 1 is selected, and removes a signal component of a frequency band where the sound discharging signal is not selected. Regarding a sound collecting signal band-divided by a sound collecting signal dividing means 2, on the other hand, the signal component of a frequency band where a sound discharging signal is selected, is removed and a signal component for which the sound discharging signal is not selected is held. Sound discharging signals from which the signal component of a predetermined frequency band is removed, are synthesized by a sound discharging signal synthesizing means 4, the signal component of a frequency band from which a sound discharging signal is removed, is held and sound collecting signals from which the signal component of a frequency band wherein a sound discharging signal is held are synthesized by a sound collecting signal composing means 5. Thus, frequency components of a sound discharging signal and a sound collecting signal are not overlapped. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to an audio processing device, and more particularly to an audio processing device that executes audio processing when a full-duplex call is performed in a loudspeaker communication system including a speaker and a microphone (hereinafter referred to as a microphone).

  In a voice call system where each device has a speaker and microphone, such as hands-free telephones and video conference systems, the sound collected by the far-end device's microphone is sent to the near-end device and emitted from the near-end device's speaker. Is done. On the other hand, the voice of the near-end speaker picked up from the microphone provided in the near-end device is also sent to the far-end device and emitted from the speaker of the far-end device. For this reason, the voice of the other party emitted from the speaker at each of the far end and the near end is applied to the microphone. If no processing is performed, this voice is sent again to the other device, causing a phenomenon called “echo” that the user's utterance can be heard from the speaker with a slight delay as if it is a piece. When the echo becomes large, it is applied to the microphone again to loop the system and cause howling.

  Conventionally, in such a loudspeaker communication system, an echo canceller is incorporated as a voice processing device for preventing echo and howling. A general echo canceller measures the impulse response between a speaker and a microphone using an adaptive filter, generates a pseudo echo obtained by convolving the impulse response with a reference signal emitted from the speaker, and applies it to the microphone. The echo component is removed by subtracting the pseudo echo from the sound of the speaker.

However, since the impulse response between the speaker and the microphone changes due to temperature changes in the room and movements of people, it is difficult to eliminate the echo sufficiently if the all-band echo canceller is configured with a fixed coefficient type. . Therefore, a band division type echo canceller that divides an audio signal into a plurality of bands and performs echo cancellation processing for each band has been proposed (for example, see Patent Document 1).
JP 59-64932 A

  However, it is difficult to generate an accurate pseudo echo in a speech processing apparatus that removes an echo using a conventional adaptive filter. Therefore, there is a problem that the amount of calculation becomes enormous in order to sufficiently cancel the echo. It was.

  As described above, the impulse response between the speaker and the microphone changes only by changing the relationship of voice reflection, such as a person moving in the room, but there is some degree that the adaptive filter converges following the change. take time. In addition, due to the principle of the adaptive filter, since it is not possible to adapt to frequency components not included in the sound emitted from the speaker, convergence is fast in the case of sound including all frequencies such as white noise. When a human voice is emitted from a speaker as in a video conference, it is known that a certain amount of time is required for convergence. As described above, since it is not possible to generate an accurate pseudo echo during the time from the change of the system to the convergence of the adaptive filter, there is a problem that echo remains or howling is caused. The band division type also has a similar problem because it uses an adaptive filter.

  In general, the amount of calculation of the adaptive filter is larger than that of Fast Fourier Transform (FFT) or a filter bank, and there is a problem that it becomes a burden when used in a low-cost system. In particular, when applied to audio signal processing in a large place such as a gymnasium, the distance from the speaker to the microphone increases and the remaining time increases, so a long tap length is required for the adaptive filter. It is known. In this case, the calculation amount further increases and the burden becomes heavy.

  The present invention has been made in view of these points, and can reduce the amount of calculation required for echo removal processing, enable effective echo processing, and remove time echoes from system fluctuations to convergence. An object is to provide a sound processing device.

  In the present invention, in order to solve the above-mentioned problem, in a voice processing device that executes voice processing in a full-duplex call in a loudspeaker call system including a speaker and a microphone, An audio processing apparatus including a band selection unit, a sound emission signal synthesis unit, and a sound collection signal synthesis unit is provided. The sound emission signal dividing means divides the sound emission signal obtained from another device and output from the speaker into a plurality of frequency bands. On the other hand, the sound collection signal dividing means divides the sound collection signal input from the microphone into the same frequency band. The band selection means divides a predetermined frequency band range including the entire range of a plurality of frequency bands into a frequency band for selecting a sound emission signal and a frequency band for selecting a sound collection signal, and is not selected for each frequency band. The signal component of the sound emission signal or sound collection signal in the selected frequency band is removed. The sound emission signal synthesizing unit synthesizes the sound emission signal obtained by dividing the band and removing the signal component of the frequency band not selected by the band selection unit. Similarly, the collected sound signal synthesizing unit synthesizes the collected sound signal from which the signal components of the frequency band that has been divided into bands and not selected by the band selecting unit are removed.

  According to such an audio processing device, the band selecting means converts the predetermined frequency band range including all areas of the plurality of frequency bands into a frequency band for selecting the sound emission signal and a frequency band for selecting the sound collection signal. In the frequency band belonging to the predetermined frequency band range, either the sound emission signal or the sound collection signal is selected. In this frequency band, one signal component of the selected sound emission signal or sound collection signal is retained, and the other is removed. The sound emission signal dividing means divides the sound emission signal acquired from the other device and output from the speaker into a plurality of frequency bands, and outputs it to the band selection means. The band selection means holds the signal component of the selected frequency band for the sound emission signal divided into a plurality of frequency bands, removes the signal component of the frequency band not selected, and outputs it to the sound emission signal synthesis means To do. Then, the sound emission signal synthesizing unit synthesizes the sound emission signal from which the signal components in the frequency band not selected are removed, and outputs the synthesized sound signal to the speaker. As a result, a sound signal (sound emission signal) from which the signal component of the frequency band that has not been selected is removed is emitted from the speaker. On the other hand, the collected sound signal dividing unit divides the collected sound signal input from the microphone into a plurality of frequency bands that are the same as the sound output signal, and outputs the divided frequency band to the band selecting unit. The band selection means holds the signal component of the frequency band in which the sound collection signal is selected and the sound emission signal is not selected for the sound collection signal divided into a plurality of frequency bands, and the sound emission signal is selected and collected. The signal component in the frequency band where the signal is not selected is removed and output to the collected sound signal synthesizing means. The sound emission signal synthesizing unit synthesizes the collected sound signal from which the signal component of the frequency band that has not been selected is removed. Thereby, the collected sound signal from which the signal component of the frequency band in which the sound output of the speaker input from the microphone is superimposed is removed is synthesized and sent to another device.

  In the sound processing apparatus of the present invention, the sound emitted from the speaker and the sound picked up by the microphone are divided into a plurality of frequency regions, and the region in which one signal component is valid is removed by removing the other signal component. The frequency components of the sound signal to be emitted and the sound signal to be collected can be prevented from overlapping. Therefore, the sound collected by the microphone is superimposed with the sound emitted from the speaker, but the superimposed component is included only in the region where the signal component is removed from the frequency region of the collected sound signal. . As a result, it is possible to realize a two-way simultaneous call in a loudspeaker call system without causing echo or howling.

  In addition, it takes a certain amount of time for convergence when the system fluctuates, and does not require a convergence time and requires a smaller amount of calculation compared to an adaptive filter that does not require a small amount of calculation. Thereby, even when a microphone or a speaker is moved or a speaker moves, the effect can be exhibited.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, the concept of the invention applied to the embodiment will be described, and then the specific contents of the embodiment will be described.
FIG. 1 is a conceptual diagram of the invention applied to the embodiment.

  The sound processing apparatus according to the present invention includes a sound emission signal dividing means 1 for dividing a sound emission signal into a plurality of frequency bands, a sound collection signal dividing means 2 for dividing the sound collection signal into a plurality of frequency bands, and a plurality of frequency bands. Band selection means 3 for selecting a band to be used so that frequency bands including the signal components of the divided sound emission signal and the collected sound signal do not overlap each other, and sound emission signal synthesis for synthesizing the sound emission signals subjected to the band selection processing Means 4 and sound collection signal synthesis means 5 for synthesizing the sound collection signals subjected to band selection processing are provided.

  The sound emission signal dividing means 1 obtains a sound emission signal acquired from another device and output from a speaker, that is, a sound signal collected by the other device with a microphone, and then changes from the time domain to the frequency domain such as Fourier transform. A voice signal (sound emission signal) is divided into a plurality of frequency bands by a conversion method or multi-rate signal processing using a filter bank. The sound emission signal that has been divided into bands is output to the band selecting means 3.

  When the sound collection signal dividing means 2 receives the sound collection signal inputted from the microphone, that is, the sound signal superimposed with the sound emission signal outputted from the speaker, the sound signal is divided by the same method as the sound emission signal division means 1. The (sound pickup signal) is divided into a plurality of frequency bands. The collected sound signal divided into the bands is outputted to the band selecting means 3.

  The band selection unit 3 inputs the sound emission signal and the sound collection signal divided into a plurality of frequency bands, and a frequency band for selecting the sound emission signal and a frequency for selecting the sound collection signal for a predetermined frequency band range. Determine the bandwidth. The predetermined frequency band range can be the entire frequency band range or a part of the entire range. In the frequency band for selecting the sound emission signal, the signal component of the sound emission signal in that frequency band is not removed, but the signal component of the sound collection signal in that frequency band is removed. On the other hand, in the frequency band for selecting the sound collection signal, the signal component of the sound emission signal in that frequency band is removed, and the signal component of the sound collection signal in that frequency band is retained. Thereby, the frequency band including the signal component of the audio signal (sound emission signal) output to the speaker and the audio signal (sound collection signal) input from the microphone does not overlap. For this reason, even if the sound emission signal is superimposed on the sound collection signal input from the microphone, it can be removed. Further, the signal component of the audio signal in the removed frequency band may be interpolated with the signal component in the frequency band that has not been removed. Details of the frequency band range and frequency band selection will be described in detail in the embodiment.

  The sound emission signal synthesizing unit 4 synthesizes the sound emission signal which has been band-divided by the sound emission signal dividing unit 1 and from which the signal component of a predetermined frequency band has been removed by the band selection unit 3 and outputs the synthesized sound signal.

  The collected sound signal synthesizing means 5 holds the signal component of the frequency band that is divided by the collected sound signal dividing means 2 and from which the sound emission signal is removed by the band selection means 3, and has the frequency band in which the sound emission signal is held. The collected sound signal from which the signal component is removed is synthesized. The synthesized sound collection signal is sent to the other device as a sound emission signal output from the speaker of the other device.

  In the audio processing apparatus having such a configuration, when an audio signal picked up by a partner apparatus that performs full-duplex communication is input, audio processing is performed for emitting the sound signal to a speaker. The collected sound signal of the other party acquired from the other party device is used as a sound emission signal, and the sound emission signal dividing means 1 performs band division into a plurality of frequency bands. The sound signal that has been subjected to the band division is synthesized by the sound emission signal synthesizing unit 4 after the signal component of a predetermined frequency band is removed by the band selecting unit 3 and emitted from the speaker. Therefore, the signal component of a predetermined frequency band is removed from the audio signal emitted from the speaker.

  The audio signal emitted from the speaker is input to the audio processing device via the microphone together with the voice of the speaker. The sound signal input from the microphone is superimposed with the sound signal emitted from the speaker on the sound signal of the speaker. The collected sound signal dividing means 2 divides the collected sound signal into a plurality of frequency bands in the same manner as the emitted sound signal dividing means 1, and outputs it to the band selecting means 3. In the sound processing of the band selecting means 3, the signal component of the collected sound signal in the frequency band in which the sound emission signal is selected is removed, and the sound emission signal is not selected, that is, the sound collection signal is collected in the selected frequency band. Does not remove sound signal components. As a result, the signal component of the frequency band on which the sound emission signal is superimposed is removed from the band-divided sound collection signal. The collected sound signal synthesizing means 5 synthesizes the collected sound signal and transmits it to the counterpart device. The signal component of the sound emission signal from the speaker, that is, the echo component is removed from the synthesized sound collection signal, and howling is prevented by outputting the sound collection signal from which the echo component has been removed to another device. can do.

  As described above, according to the voice processing device of the present invention, it is possible to realize two-way simultaneous calling while suppressing echo and howling. In addition, since the frequency components of the audio signal (sound emission signal) output from the speaker and the audio signal (sound collection signal) input from the microphone do not overlap, echo and howling are suppressed, so compared to the adaptive filter There is also an advantage that the amount of calculation is small, it is possible to cope with the case where the system fluctuates, and the time until convergence is not required.

Hereinafter, an embodiment will be described in detail with reference to the drawings, taking as an example a case where the embodiment is applied to an audio processing unit of a video conference system.
FIG. 2 is a diagram showing the configuration of the video conference system according to the embodiment of the present invention. In the figure, a processing unit relating to an image that is not related to the description of the present invention is omitted.

  In the video conference system of the present embodiment, a conference terminal 10a connecting a speaker 21a and a microphone 22a and a conference terminal 10b connecting a speaker 21b and a microphone 22b are connected by a communication line 23. Hereinafter, the conference terminal 10a arranged near an arbitrary speaker is connected to the near-end device, the near-end device 10a via the communication line 23, and the conference terminal 10b located far from the speaker is connected to the far-end device. 10b. The near-end device 10a and the far-end device 10b have the same configuration, and an internal block diagram of the far-end device 10b is omitted in the figure. The communication line 23 is a general digital communication line such as Ethernet (registered trademark).

  The speaker 21a connected to the near-end device 10a processes the sound collected by the microphone 22b connected to the far-end device 10b with the near-end device 10a and emits the sound. The microphone 22a connected to the near-end device 10a picks up the speech voice of the video conference attendee of the near-end device 10a. At this time, the sound emitted from the speaker 21a input through the space is superimposed and collected. The same applies to the far-end device 10b.

  Hereinafter, the internal configuration of the near-end device 10a and the far-end device 10b will be described in the case of the near-end device 10a. The near-end device 10a includes a D / A converter 11 connected to a speaker, an A / D converter 12 connected to a microphone 22b, a signal processing unit 13 that processes an audio signal, and an audio that performs encoding / decoding processing of the audio signal. A communication unit 15 connected to the codec 14 and the communication line 23 is provided.

  The D / A converter 11 converts the digital audio data processed by the signal processing unit 13 into analog. The analog audio signal is amplified by an amplifier (not shown) and then emitted from the speaker 21a. The A / D converter 12 converts an analog voice signal obtained by amplifying a voice collected by the microphone 22a by an amplifier (not shown) into digital voice data. The signal processing unit 13 is configured by a digital signal processor (DSP), and performs processing for converting input and output sound data into desired data, and also overlaps the frequency components of the collected sound and the sound to be emitted. Audio processing is performed so that it does not occur. Details of this sound processing will be described later. The audio codec 14 converts the audio data based on the input of the microphone 22 a sent from the signal processing unit 13 into a code that is standardly defined in the communication of the video conference system, and is sent from the communication unit 15. The audio data encoded by the far-end device 10 b is decoded and sent to the signal processing unit 13. The communication unit 15 transmits / receives input / output data including encoded audio data to / from the far-end device 10b via the communication line 23 based on a predetermined digital data communication protocol.

Next, audio processing by the signal processing unit 13 will be described in detail.
First, as a first embodiment, a signal processing unit that performs audio processing by dividing the entire frequency range of an audio signal into a frequency band for selecting a signal component for sound emission and a frequency band for selecting a signal component for sound collection explain.

FIG. 3 is a diagram illustrating a configuration of the signal processing unit according to the first embodiment of this invention. The signal processing unit 30 is incorporated in the signal processing unit 13 of the conference terminal shown in FIG.
The signal processing unit 30 according to the first embodiment of the present invention includes an analysis filter bank 31 that divides a sound emission signal into a plurality of frequency bands, a synthesis filter bank 32 that synthesizes the sound signals that have been divided into bands, An analysis filter bank 33 that divides the sound signal into a plurality of frequency bands and a synthesis filter bank 34 that synthesizes the band-divided sound collection signal are provided.

  The analysis filter bank 31 is a sound emission signal dividing unit, and divides the audio signal data input from the audio codec 14 into 128 frequency bands from low to high. Hereinafter, for the sake of explanation, the lowest frequency channel is numbered sequentially as the first channel, and the highest frequency channel is the 128th channel. This band division processing is performed by, for example, “Subband Adaptive Filter Using Completely Reconstructed DFT Filter Bank” by Watanabe Kazunobu et al. (Electronic Information and Communication Society, August 1996, Vol. J79-A No. 8 pp. 1393). The DFT filter bank described in 1393) is used. The process of dividing the band, performing signal processing after downsampling, and recombining is called multirate signal processing. As a method of band division, various methods such as a QMF filter bank in addition to the DFT filter bank are known depending on applications. In the embodiment, a case where a DFT filter bank is used will be described. However, band division may be performed by another method. Further, as a method other than the filter bank, a method in which transformation from time domain to frequency domain and inverse transformation such as Fourier transformation is defined can be used. The DFT filter bank is divided into analysis and synthesis functions. It is known that the voice data divided by band in the analysis can be re-synthesized into the original voice data by the synthesis filter bank. Although the original signal and the recombined signal may be slightly different depending on the method, the present invention can be configured so as not to have an essential influence.

  The synthesis filter bank 32 is a sound emission signal synthesis unit. Among the 128-channel audio signals divided by the analysis filter bank 31, the audio data from which even-numbered channel signal components are removed by a band selection unit (not shown). , And the components of the entire band are synthesized to generate one audio signal. The audio signal is output to the speaker 21a via the D / A converter 11.

  The analysis filter bank 33 is a collected sound signal dividing means, and the audio signal data input from the A / D converter 12 is banded into a 128-channel frequency band from low to high, similar to the analysis filter bank 31. To divide. The analysis filter bank 33 is configured the same as the analysis filter bank 31.

  The synthesis filter bank 34 is a collected sound signal synthesis means. Of the 128-channel audio signals divided by the analysis filter bank 33, the audio data from which the odd-numbered channel signal components are removed by the band selection means (not shown). , And the components of the entire band are synthesized to generate one audio signal. The audio signal is transmitted to another device via the audio codec 14.

  The band selection means selects a frequency band (channel) in which each audio signal is included so that the audio signal emitted from the speaker 21a and the frequency component of the audio signal collected by the microphone 22a do not overlap. . Here, odd-numbered channels (1, 3,..., 127) with numbers assigned from 1 to 128 in order from the low range are selected for sound signals to be emitted, and even-numbered channels (2, 4 and 4) are selected. ,..., 128) are selected for audio signals picked up from a microphone. That is, the sound signal to be emitted uses the signal component of the frequency band corresponding to the odd-numbered channel, and the signal component of the frequency band corresponding to the even-numbered channel is removed. In addition, the audio signal to be picked up uses the signal component of the frequency band corresponding to the even-numbered channel by removing the signal component of the frequency band corresponding to the odd-numbered channel. In this way, by separating the channels, it is possible to prevent the emitted sound signal from being superimposed on the collected sound signal.

The sound processing executed by the signal processing unit 30 will be described using a flowchart.
First, processing of an audio signal emitted from a speaker (hereinafter referred to as speaker audio processing) will be described. FIG. 4 is a flowchart illustrating a speaker audio processing procedure of the near-end device including the signal processing unit according to the first embodiment. In the far-end device 10b, sound processing is performed in the same procedure.

[Step S01] The encoded voice data from the far-end device 10b is received by the communication unit 15 via the communication line 23.
[Step S02] The audio data received in step S01 is decoded by the audio codec 14, and, for example, digital audio data of 32 KHz sampling 16-bit straight PCM is generated. This digital audio data is sent to a signal processing unit 30 constituted by a DSP.

  [Step S03] The signal processing unit 30 performs band division processing by the analysis filter bank 31 on the input audio data. Here, the DFT filter bank is used to divide sound data to be emitted input from other devices into 128-channel frequency bands from low to high.

  [Step S04] Of the 128 channels, the signal component of the channel assigned to the sound signal to be emitted (in this case, the odd-numbered channel) is directly sent to the synthesis filter bank 32 and assigned to the sound signal to be emitted. The signal component of the missing channel (here, even-numbered channel) is not output to the synthesis filter bank 32. That is, the even-numbered output in the 128 frequency bands of the analysis filter bank 31 is set to 0 and output to the synthesis filter bank 32. Thereby, the signal component of the even-numbered frequency band of the sound signal to be emitted is removed.

  [Step S05] The synthesis filter bank 32 receives the audio data from which the signal component of the frequency band (in this case, the even-numbered channel) to which the sound signal to be emitted has not been assigned has been removed, and the signal component of the entire band is obtained. The synthesized audio signal is sent to the D / A converter 11.

[Step S06] The D / A converter 11 converts the sound output sound signal synthesized by the synthesis filter bank 32 into an analog sound signal, and outputs the analog sound signal to the speaker 21a.
[Step S07] The analog audio signal acquired from the D / A converter 11 is amplified by an amplifier and emitted from the speaker 21a.

  By executing the above speaker audio processing procedure, the signal component of the even-numbered frequency band divided into 128 channels is removed from the audio signal input from another device and emitted from the speaker 21a. After being synthesized with the audio signal, it is output from the speaker 21a.

  Next, audio processing of an audio signal collected from a microphone (hereinafter referred to as microphone audio processing) will be described. FIG. 5 is a flowchart illustrating a microphone sound processing procedure of the near-end device including the signal processing unit according to the first embodiment. In the far-end device 10b, sound processing is performed in the same procedure.

  [Step S11] The speech of the video conference attendee on the near-end device side and the sound emitted from the speaker 21a are collected by the microphone 22a. The collected sound is converted into digital sound data of 32 KHz sampling 16-bit straight PCM by the A / D converter 12. The converted audio data is output to the analysis filter bank 33.

  [Step S12] The analysis filter bank 33 receives the audio data generated in step S11 and divides it into 128-channel audio data in the same manner as the analysis filter bank 31 for the sound emission signal.

  [Step S13] Of the 128 channels, the signal component of the channel (here, the even-numbered channel) assigned to the sound pickup signal opposite to the sound output signal is sent to the synthesis filter bank 34 as it is, and the sound to be emitted The signal component of the channel assigned to the signal (here, the odd-numbered channel) is not output to the synthesis filter bank 34. That is, in the 128-channel frequency band of the analysis filter bank 33, an odd-numbered output opposite to the sound signal to be emitted is set to 0 and output to the synthesis filter bank 34. Thereby, the signal component of the odd-numbered frequency band of the collected audio signal is removed.

  [Step S14] The synthesis filter bank 34 receives the audio data from which the signal component of the frequency band to which the collected audio signal has not been assigned (here, the odd-numbered channel opposite to the sound output signal) is removed, The signal components of all the bands are combined into one audio signal and sent to the audio codec 14.

[Step S <b> 15] The audio codec 14 encodes the audio signal input from the synthesis filter bank 34 into a predetermined code and sends the encoded signal to the communication unit 15.
[Step S16] The communication unit 15 transmits the encoded audio data to the far-end device 10b via the communication line 23.

  By executing the above processing procedure, the signal component of the odd frequency band divided into 128 channels is removed from the audio signal collected by the microphone 22a and transmitted to the far-end device 10b. After being synthesized into one audio signal and encoded, it is transmitted to the communication unit 15.

  As described above, the audio signal emitted from the speaker 21a and collected from the microphone 22a holds only the signal component in the frequency band corresponding to the odd-numbered channel. Therefore, by removing the signal component of the odd-numbered channel from the collected sound signal, the signal component derived from the sound signal emitted from the speaker 21a is removed from the collected sound signal. Become. That is, the echo component is removed from the collected sound signal.

  As described above, in the first embodiment, the frequency components of the sound emitted from the speaker and the sound collected by the microphone are divided into even-numbered and odd-numbered frequency bands so as not to overlap. In addition, it is possible to realize a two-way simultaneous call while suppressing echo and howling.

  In the above description, the channels are divided into even-numbered and odd-numbered channels. However, this can be realized if the frequency components of the microphone and the speaker do not overlap each other. In addition, various ways of dividing are possible.

The channel for microphone sound processing and the channel for speaker sound processing may be divided into two or more frequency bands instead of being alternately divided into even and odd numbers. For example,
Channels selected for microphone audio processing: 1, 2, 5, 6,..., 125, 126
Channels selected for speaker audio processing: 3, 4, 7, 8,..., 127, 128
As such, it may be alternately divided into two.

In addition to a method of simply selecting a channel to be selected in advance, there is a method of dynamically selecting a channel according to audio characteristics.
In the audio processing according to the present embodiment, since some components of the audio are removed, the sound quality may be affected. For example, a method using an auditory masking effect is known as a method of reducing the influence on hearing when removing some components for audio compression.

  FIG. 6 is a diagram showing the characteristics of an audio signal. The figure is a graph in which the output to be output to the speaker for a certain time is processed by the analysis filter bank 31, the 128 channels of the filter bank are arranged from low to high on the horizontal axis, and the power is plotted on the vertical axis. .

  In this figure, there is a characteristic with peaks in the 15th, 60th, and 78th channels. In this case, channel selection that prevents frequency components of speaker audio processing and microphone audio processing using the auditory masking effect from overlapping is performed as follows, for example.

Channels selected for speaker audio processing: 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 51, 54, 57, 60, 63, 66, 59, 72, 75, 78, 81, 84, 87, 90, 93, 96, 99, 102, 105, 108, 111, 114, 117, 120, 123, 126
Channels selected for microphone audio processing: Channels 1 to 128 excluding the above In other words, for speaker audio processing, select every 2 channels including the 15th, 60th, and 78th peaks, and for microphone audio processing, otherwise Select the channel. The selection method using the auditory masking effect is not limited to the above-described two channels and can be adjusted to an arbitrary value. In addition to the method of using a peak at a certain temporary point, the method of detecting the peak of a time-averaged component is also possible for detecting the peak of the frequency component of speech.

  In the above description, the filter bank is used to divide the frequency components so that the frequency components of the microphone audio processing and the speaker audio processing do not overlap each other. It is also possible to use a method using Fourier transform to perform the above. In this case, the analysis filter banks 31 and 33 may be 128-point Fourier transform, and the synthesis filter banks 32 and 34 may be inverse transforms corresponding to the Fourier transform. In addition to the filter bank and Fourier transform, the time-frequency transform can be performed using a discrete cosine transform (DCT) or a wavelet transform.

In the first embodiment, the frequency component is removed and set to 0. However, it is also possible to select a small value according to the environmental noise other than 0.
Next, a second embodiment will be described. In the first embodiment, the entire frequency range of the audio signal is divided into a frequency band for selecting a sound signal component and a frequency band for selecting a sound signal component. Then, it is assumed that such voice processing is performed in a part of the frequency band range, and voice processing using an adaptive filter is performed in other frequency bands.

  FIG. 7 is a diagram illustrating the configuration of the signal processing unit according to the second embodiment. In addition, the component of the processing function which the signal processing part in 2nd Embodiment has is the same as the component of 1st Embodiment shown in FIG. Therefore, the functions of the second embodiment will be described using the reference numerals of the components shown in FIG.

  In the second embodiment, it is assumed that the band division by the analysis filter banks 31 and 33 is 256 channels, the first channel outputs the lowest frequency component, the number is assigned in order, and the highest frequency component filter is selected. The 256th channel. The first to 128th channels are subjected to audio processing using the adaptive filters 41, 42, 43,..., 44, and the 129th to 256th channels are the same as in the first embodiment. The same processing as shown is performed. The reason why the audio processing using the adaptive filter is performed for the first channel to the 128th channel is that, in general, many audio signals are included in the low frequency band.

  For the 129th to 256th channels, the signal components of the even-numbered channels are removed from the signal components of the channels divided by the analysis filter bank 31 and sent to the synthesis filter bank 32 as speaker sound processing. Further, as microphone sound processing, the signal components of the odd-numbered channels are removed from the signal components of the channels divided by the analysis filter bank 33 and sent to the synthesis filter bank 34. By performing the above processing, the same effects as those of the first embodiment can be obtained for the frequency band to which the 129th channel to the 256th channel belong.

  Next, an audio processing procedure according to the second embodiment will be described. FIG. 8 is a flowchart illustrating an audio processing procedure of the signal processing unit according to the second embodiment. Note that description of the same processing procedure as in the first embodiment is omitted.

Speaker audio processing will be described.
[Step S21] The analysis filter bank 31 divides the sound signal to be emitted into 256 channels up to the 256th channel with the first channel as the lowest frequency component.

[Step S22] Of the 256 channels, the even-numbered output is set to 0 for the 129th channel to the 256th channel, and is output to the synthesis filter bank 32.
[Step S23] Among the 256 channels, for the first to 128th channels, the signal components of all the channels are sent to the synthesis filter bank 32, and adaptive filters (processes) corresponding to the respective channels are processed. Part) 41, 42, 43,..., 44, a signal component is sent as a reference sound for adaptive filter processing.

[Step S24] The signal components of all the bands are combined into one audio signal. This audio signal is output from the speaker 21 a via the D / A converter 11.
The microphone sound processing will be described.

  [Step S25] Simultaneously with the speaker audio processing, the audio signal input through the microphone 22a is band-divided into 256 channels by the analysis filter bank 33 having the same configuration as the analysis filter bank 31.

  [Step S26] Contrary to speaker audio processing, odd-numbered outputs are set to 0 for 256th channel to 129th to 256th channels, and output to the synthesis filter bank 34. That is, the components from the 129th channel to the 256th channel include only the audio components of the odd-numbered channel and the even-numbered channel, respectively, in the output sound to the speaker 21a and the input sound from the microphone 22a, as in the first embodiment. However, the same frequency component is configured not to overlap.

  [Step S27] Among the 256 channels, for the first to 128th channels, signal components are sent to adaptive filters (processing units) 41, 42, 43,..., 44 provided corresponding to the respective channels. send.

  [Step S28] The adaptive filters (processing units) 41, 42, 43,..., 44 use the audio signal emitted to the speaker input in step S23 for the first to 128th channels as a reference signal. The voice signal picked up by the microphone input in S27 is processed as an objective signal by an adaptive filter using an LMS (Least Mean Square) algorithm. Since the adaptive filter processing is not directly related to the present invention, a description thereof will be omitted. A pseudo echo is generated by the adaptive filter processing.

  [Step S29] The adaptive filters (processing units) 41, 42, 43,..., 44 are calculated in Step S28 for the first to 128th channels from the audio signal collected by the microphone input in Step S27. The pseudo echo is subtracted and sent to the synthesis filter bank 34.

  [Step S30] In the synthesis filter bank 34, the signal components of the 129th to 256th channels input in step S26 and the components in the entire band of the signal components of the first to 128th channels input in step S29 are synthesized. Thus, one audio signal is obtained. This audio signal is transmitted to another apparatus via the audio codec 14.

  In the second embodiment described above, by combining the configuration of the first embodiment with echo cancellation by an adaptive filter, signal components that have a strong influence on sound quality are processed by the adaptive filter, and the sound quality is affected. The signal component with a small amount is subjected to sound processing by the configuration of the first embodiment. As a result, in the frequency band to which the first embodiment is applied, the amount of calculation can be reduced as in the first embodiment, and therefore an optimal system that takes into account the sound quality and the amount of calculation should be designed. Can do.

Note that the processing from the 129th channel to the 256th channel in the second embodiment is the same as the processing in the first embodiment, and can be modified in the same manner as in the first embodiment.
In addition to the LMS algorithm, various methods are known as adaptive filter methods, and other methods can also be used. Various control methods for improving the performance of the adaptive filter are known, and the performance is improved by applying the adaptive filter to the adaptive processing (step S28).

  In multi-rate signal processing using a DFT filter bank or the like, since frequency conversion is performed, a method of down-sampling the filter output to reduce the calculation amount is known. Particularly for the present embodiment, it is possible to reduce the amount of calculation by this method in the adaptive filter processing.

  In the first embodiment and the second embodiment described above, the method of removing some components has been described in detail so that the frequency components of the sound of the microphone and the speaker do not overlap. . In this configuration, the frequency component removed from the sound of the microphone is a component included in the sound emitted from the speaker, and the frequency component removed from the sound emitted from the speaker is the portion of the sound collected by the microphone. This is a frequency component that is not removed, and has a contradictory relationship. When the near-end device and the far-end device are connected in exactly the same configuration, the frequency component removed from the far-end device microphone becomes the frequency component from which the sound emitted from the near-end device speaker is not removed. The sound output at the near end has already been removed at the far end, and a situation occurs in which nothing can be heard. Several methods for solving such problems are described below.

As the first sound processing, the microphone sound can be transmitted to the far-end device by interpolating the removed component with the component not removed. Details of the interpolation method will be described below.
In the first embodiment, it is described in detail that the odd-numbered channel among the 128-channel outputs is sent to the synthesis filter bank 34 from the analysis filter bank 33 as 0. In step S13 of the microphone sound processing according to the first embodiment shown in FIG. 5, the removed odd channel is replaced with the data of the adjacent even channel that is not removed. For example, the first channel to be removed is the same data as the second channel. Similarly, the third channel configures the data of the fourth channel, the fifth configures the sixth, and so on up to the 128th channel. Other than this interpolation operation can be realized with the same configuration as that of the first embodiment.

As the second sound processing, when there is a removed component in the sound sent from the far-end device, the component is complemented and output from the speaker.
Details of the interpolation method will be described below. In the first embodiment, it is described in detail that the even-numbered channel of the 128-channel output from the analysis filter bank 31 is sent to the synthesis filter bank 32 as 0. After the process of step S03 in the speaker sound process of the first embodiment shown in FIG. 4, it is determined whether there is a removed frequency component. In this method, the power of the filter output for a certain period of time is integrated, and if it is below the threshold, it is determined that this frequency component has been removed. If it is determined that it has been removed, it is replaced with the data of the adjacent odd channel. For example, when it is determined that the component of the second channel is removed, the data is the same as the data of the first channel. After this determination and operation, the process proceeds to step S04. Except for the above-described determination and operation, it can be realized with the same configuration as that of the first embodiment.

  Further, frequency components other than the frequency components removed from the microphone sound of the far-end device are set as frequency components to be removed from the microphone sound of the near-end device. As described above, the frequency component of the sound emitted from the speaker and the frequency component of the sound collected by the microphone are configured in a trade-off relationship. Therefore, if this relationship is reversed between the near-end device and the far-end device, no sound is output when the component removed from the far-end device's microphone sound is emitted from the near-end device speaker. Does not happen. In step 04 in the speaker sound processing of the first embodiment shown in FIG. 4, it has been described that the output of the even-numbered channel is set to 0. The configuration is as follows. For the output from step 03, the sum of the powers of the even-numbered channels and the power of the odd-numbered channels for a certain period of time are integrated. The above values are compared, and an even or odd channel group having a small value is set to 0. In step S13 in the microphone sound processing shown in FIG. 5, when the even channel is set to 0, the odd channel is set to 0. When the odd channel is set to 0, the even channel is set to 0. Except for the above-described determination and operation, it can be realized with the same configuration as that of the first embodiment. In the above description, the presence or absence of the removed component is determined from the power of the filter bank output. However, the component that has been removed by the far-end device and the near-end device is sent as a control signal to the other device, and the control signal A configuration to control is also possible.

  By performing any of the above processes, for example, the output audio signal component that occurs when the frequency component removed by the microphone of the near-end device is selected as the component to be output from the speaker of the far-end device has already been generated. The problem of no signal component being eliminated can be avoided.

It is a conceptual diagram of the invention applied to embodiment. It is the figure which showed the structure of the video conference system of embodiment of this invention. It is the figure which showed the structure of the signal processing part of the 1st Embodiment of this invention. It is the flowchart which showed the speaker audio | voice processing procedure of the near end apparatus containing the signal processing part of 1st Embodiment. It is the flowchart which showed the microphone audio | voice processing procedure of the near end apparatus containing the signal processing part of 1st Embodiment. It is the figure which showed the characteristic of the audio | voice signal. It is the figure which showed the structure of the signal processing part of 2nd Embodiment. It is the flowchart which showed the audio | voice processing procedure of the signal processing part of 2nd Embodiment.

Explanation of symbols

1 ... Sound emission signal dividing means, 2 ... Sound collection signal division means, 3 ... Band selection means, 4 ... Sound emission signal synthesis means, 5 ... Sound collection signal synthesis means

Claims (12)

In a voice processing device that performs voice processing when a full-duplex call is made in a loudspeaker call system including a speaker and a microphone,
Sound emission signal dividing means for dividing a sound emission signal obtained from another device and output from the speaker into a plurality of frequency bands;
A sound collection signal dividing means for dividing the sound collection signal input from the microphone into the plurality of frequency bands;
The predetermined frequency band range including the entire range of the plurality of frequency bands is divided into the frequency band for selecting the sound emission signal and the frequency band for selecting the sound pickup signal, and is selected for each frequency band. Band selection means for removing signal components of the sound emission signal or the sound collection signal of the frequency band that was not present,
Sound emission signal synthesizing means for synthesizing the sound emission signal that has been band-divided and from which the signal component of the frequency band not selected by the band selection means has been removed;
Sound collection signal synthesis means for synthesizing the sound collection signal that has been band-divided and from which the signal component of the frequency band not selected by the band selection means has been removed;
A speech processing apparatus comprising: The sound emission signal dividing means and the sound collection signal division means separate the sound emission signal and the sound collection signal into a plurality of frequency bands by performing a conversion process from a time domain to a frequency domain including a Fourier transform process. To
The speech processing apparatus according to claim 1. The sound emission signal dividing unit and the sound collection signal division unit separate the sound emission signal and the sound collection signal into the plurality of frequency bands by performing multi-rate signal processing using a predetermined filter bank.
The speech processing apparatus according to claim 1. The band selection means sequentially assigns numbers to the frequency bands included in the predetermined frequency band range, and removes signal components of one of the frequency bands, which are even-numbered or odd-numbered, for the sound emission signal that has been band-divided. And removing the other without removing the signal component of the even-numbered or odd-numbered frequency band obtained by removing the signal component of the sound emission signal with respect to the sound-collected signal that has been band-divided, without removing the other. The frequency band in which the sound emission signal and the sound collection signal are included does not overlap.
The speech processing apparatus according to claim 1. Whether the band selection means selects the odd-numbered or even-numbered frequency band for removing the signal component for the sound emission signal and the collected sound signal divided by the band depends on the other party of the full-duplex call. It is determined according to a combination that does not overlap with the selection of the frequency band for removing the signal component of some other device.
The speech processing apparatus according to claim 4. The band selecting unit obtains a predetermined frequency band from the other device according to a level of a signal component in the frequency band of the sound emission signal obtained from the other device and divided into bands by the sound emission signal dividing unit. It is determined whether or not the signal component is removed, and when the frequency band from which the signal component is removed is detected, the detected frequency band is selected as the frequency band from which the signal component of the sound emission signal is removed.
The speech processing apparatus according to claim 1. The band selection unit is configured to release the signal component of the sound emission signal in the frequency band having the sound characteristics according to the sound characteristics of the sound output signals input from the other device. Determining the frequency band for selecting a sound signal;
The speech processing apparatus according to claim 1. The band selection means dynamically performs the selection of the frequency band for selecting the sound emission signal according to the characteristics of the sound.
The speech processing apparatus according to claim 7. The band selection means performs voice processing using an adaptive filter for a part of the frequency band range of the frequency band that is divided, and the sound emission signal of the predetermined frequency band for the remaining frequency band range and the Perform audio processing to remove the signal component of the collected sound signal,
The speech processing apparatus according to claim 1. The band selection means selects the frequency band to be processed using the adaptive filter according to the characteristics of the voice of the sound emission signal input from the other device.
The speech processing apparatus according to claim 6. The band selection means uses the signal component of the collected sound signal of the frequency band from which the signal component is not removed for the collected sound signal obtained by dividing the band and removing the signal component of the predetermined frequency band. Interpolating the signal component of the collected sound signal in the frequency band from which
The speech processing apparatus according to claim 1. The band selecting unit obtains a predetermined frequency band from the other device according to a level of a signal component in the frequency band of the sound emission signal obtained from the other device and divided into bands by the sound emission signal dividing unit. It is determined whether the signal component has been removed, and when the frequency band from which the signal component has been removed is detected, the signal component is used by using the signal component of the sound emission signal in the frequency band from which the signal component has not been removed. Interpolating the signal component of the sound emission signal in the frequency band from which
The speech processing apparatus according to claim 1.

Images